I thought I had been running a 64bit Firefox for years. So I wasn't? Or is this about finally doing a 64-bit Windows build? Probably since Moz Corp is entirely focused on Windows and treats Linux as a red headed stepchild.
I am a Firefox dev and a Linux user. Mozilla is definitely not focused on Windows, in fact many of us devs use Linux (I am posting from Ubuntu right now), and many of the rest use OS X. Windows is in the minority.
There are 64-bit builds available from our build system, but we don't promote them. The reason is that we don't spend as much effort on QAing 64-bit builds, we have limited resources and are focused on the standard (32-bit) builds for the most part.
There are some good reasons for 64-bit browsers, for sure, but AFAIK none of the major browsers make that a priority. For example, there is a 64-bit IE9, however it ships with a hobbled JavaScript engine (without JITs), so clearly they don't intend it very seriously.
In any case, given that Firefox is open source, anyone can build a 64-bit version. I believe several Linux distros ship a 64-bit Firefox, for example. There used to be some problems with running 32-bit Flash in it, but I have heard that is workable now too.
I know it's been said before, but this new release system is fucking retarded.
I'm this close to dumping Firefox on every machine I touch.
I think it's silly to dump both Chrome and Firefox because of their release systems (which are identical - both release a new major version number every 6 weeks).
I guess you can use other browsers than Chrome and Firefox. But those other browsers release new versions with new features very rarely. Is the *version number* enough of a reason to not use Chrome and Firefox? I don't think so - even though I thought it was silly when Chrome started with it, and when Firefox decided to do it as well.
But it seems amazingly inferior to a platform native PDF reader, on any platform imaginable. It will be slower the native x86/ARM code by far, and won't integrate well with the desktop environment.
Regarding speed, two things: First, this will spend most of its time in calls to the browser's Canvas API, which all browsers implement in C++. So it isn't clear that it should be significantly slower than a native implementation. Second, even if this were in 100% JavaScript, that is just around 5X slower than C++ these days. Rendering PDFs might be plenty fast enough at that speed, since you typically render once then show it for a long time. In other words, this isn't something like a game engine that needs to push 60fps.
What's with this trend recently to build everything on fundamentally sucky technologies?
There are several advantages to implementing this using Web technologies. The first is, as you mentioned, it's a good way to find browser bugs, and it also helps find areas to improve the Canvas API. Another important reason is that this implementation will be much more secure than any other existing PDF implementation, because so much work has gone into proper sandboxing of JavaScript. And we know how many vulnerabilities keep popping up in existing PDF readers like Adobe's. It will be much more secure to use this new PDF implementation than the existing ones.
While this is impressive, it has been done before (and better): GWT Quake
I think that Quake demo is awesome! I'd just like to mention though that this Doom demo is very different from a technical standpoint, and I think both are interesting:
The Quake demo compiled Java to JavaScript using GWT, the Doom demo compiles C through LLVM into JavaScript using Emscripten.
The Quake demo uses WebGL to render, the Doom demo translates a 100% software renderer. It's much more challenging to get good performance with a software renderer in JS, especially given that the original renderer was heavily optimized for the CPUs of the day (for example, it uses fixed-point math).
The Quake demo was a major effort, with rewriting and fixing. The Doom demo is a straightforward port, no new code (only a few tiny tweaks), took only a week to do. (Btw, speaking of the timetable, sorry for the sound quality - I just spent a few hours on that part, and I had never used the Audio Data API before.)
So... no music, and the sound effects are like a half second delayed from the action, AND they sound really REALLY bad.
I agree, the sound is terrible, sorry about that. I don't know much about how sound stuff works, that is the best I could do in a few hours of hacking something together with the Audio Data API.
If it's under the GPL and LGPL, it's going to be a rough case Attachmate would be making, considering that it's open licensed and they just kicked the team to the curb.
The FOSS code is not a concern here.
The issue is the proprietary code that Miguel et al worked on in Novell, the Android and iPhone runtimes. That is owned by Attachmate now, and this new startup contains exactly the same coders, who are intentionally going to write the exact same product from scratch - they will be 100% "source compatible" with the old runtimes.
So legalities are a reasonable concern. Even if no code is copied, the same people writing the same product - immediately after writing it the first time - may lead to basically the same code being written. It might be hard to prove no code was copied even if none was. Lawsuits are filed for much less.
Of course, this only matters if Xamarin is a big success - no one sues a failure. Time will tell.
It is very awkward for a distro that bundles Chrome to use the system WebKit, since Chrome's entire approach is opposed to that. Chrome updates silently and often and if it used the system WebKit, Chrome could not update itself at all. You would be frozen with an older version of Chrome, and Google stops supporting them very quickly - Google's goal is update every one to the new versions.
However, if used on something like a netbook or tablet, it would make a lot of sense, since the browser would be one of the first things opened on the device.
I don't get it, why does loading a copy of WebKit in the login manager make sense? That you open a browser soon later doesn't matter, since the by far most common browsers, Firefox and Chrome, would not benefit from that in any way: Firefox since it doesn't use WebKit, Chrome since it has its own copy - you will have 2 copies of WebKit in memory at that point, each taking >25MB!
Bet WebKit's squalid bulk didn't go into the 5KLoC vs 50KLoC size difference.
From the docs it looks like WebKit might be optional in LightDM. But if WebKit is used on Ubuntu, then I don't see why loading a 25MB HTML engine (+ it's dependencies!) is a good thing. WebKit is pretty bulky. And this is a login manager we're talking about!
And it's not like WebKit will be loaded later anyhow. Even if you do use a WebKit based browser, that is most likely Chrome, which bundles it's own copy of WebKit - so no sharing with apps that use the system WebKit like LightDM.
So why is the solution to roll your own, rather than fix LLVM or develop a more direct competitor to it? All of the issues mentioned so far seem to be with the current implementation, not with the fundamental idea.
The goals are somewhat different. LLVM is great for optimizing statically typed code, so it will do e.g. licm that takes into account that kind of code. The IR that would be useful for a JS engine would be higher-level, so it could do licm on higher-level constructs.
The issue is sort of similar to why dynamic languages don't run very fast on the JVM or.NET. They have very efficient JITs, but they are tuned for the level that makes sense for languages like Java and C#, not JavaScript or Python or Ruby. As a consequence, no dynamic language on those platforms comes close to modern JS engines, LuaJIT, etc. Some people seem to assume that translating Python into Java, then JITing that on the JVM will be fast - but it isn't.
So there isn't really something to be 'fixed' in LLVM here - it is very good at what it does. It's just different than what JS engines need.
> Can you please give some more details about how to reproduce it:
I'm not the person you are replying to, but here is one way for me with FF4 under Ubunu 11.04.
Go to mail.google.com and log into it. Keep the page open, but close your network connection (e.g disconnect wireless on your laptop to save power). Go away for a while. Next morning, Firefox4 will be using over 1.5 GB memory and the machine will be swapping its brains out.
Pretending there are no such bugs in FF when people have been complaining about the leaks for years is a rather poor reflection.
I never pretended there are no such bugs - I am sure there are. Thank you for giving some steps to reproduce, I will try this!
The memory cache caches some types of items, but not images, in that way. Take a look at about:cache to see more details of what it currently stores, if you want.
For images, the important cache is probably the page cache (bfcache), which stores entire pages so they can quickly be reloaded. You can change the amount of pages it caches with browser.sessionstore.max_total_viewers (set it to 0 to not cache any, for example).
How about making FF 6 or 7 better able to restrict leakage?
We are working very hard on that, from various directions - from better tools to report how much memory is being used, to eventually running tabs in separate processes so their memory use can be contained, and also allowing Jetpack plugins that are likewise separated so they cannot cause leaks.
The fact that plugins variety is even a problem must then be Mozilla's design flaw.
There is definitely a tradeoff here - Firefox is more customizable and flexible than other browsers. That makes it more vulnerable to leaks from addons and so forth. In other words, it is much easier to prevent these problems when you greatly restrict what your browser and its addons do.
As I said above, we are working on some technical improvements for these matters, which should hopefully be available later this year. The tradeoff should improve at that point.
Meanwhile, it is possible to use Firefox's customization to change the behavior, if you want - stop if from caching pages and so forth.
So the next thing is to start closing tabs but I can't do that right now because I'm working on stuff but I will try later.
I would definitely suspect that one of your tabs is guilty here, if you have no addons or plugins enabled. Hopefully if you try that later, you'll find which is responsible.
If not, and you still see this with a single tab, new profile, no addons or plugins, then that is... extremely interesting. At that point, a screencast would be the perfect thing here, good idea! Would be great if you can do that, and if so, please include the following in the screencast:
Show how you create a new profile and use that
Show Help->About Firefox to make sure we see the version number
Show about:buildconfig too (shows the specific build info for your local copy)
Show that no plugins or addons are enabled
Show two windows, one showing memory, and the other Firefox, and do some typing to show the problem you describe
The behavior you describe should definitely not be happening. Hopefully this can help narrow it down.
I am definitely not dismissing that! If an old profile causes buggy behavior, then that is a bug in Firefox, I fully agree.
The problem is practical, though - this happens on a very small number of user profiles, and users usually don't want to (for totally understandable reasons!) file a bug with their profile attached. So it is very, very hard for us Firefox devs to figure out the problem, since we can't debug it locally on our machines.
In the long term, we are doing a lot of work on about:memory, which right now shows some useful info, but we want it to show a lot more. Then users with too much memory being used will be able to report bugs with information that should help us a lot in figuring out the problem.
So please, I love firefox, I love it's open-ness, the add-ons, the customize-ability, and even many of the new(er) features (the awesomebar is, in fact, awesome), but I hate the memory waste! Even adding a hidden config option to enable 'reclaim memory on tab-close' would be much appreciated!
There is such an option: browser.sessionhistory.max_total_viewers:) Set it to 0, and pages are not cached anymore. See here for more details.
Many people run linux on older, low-end hardware especially because of it's reputation as a low-resource, light-weight operating system. Firefox seems to be building on expectations that are vague or simply not true.
We ship with a default behavior that we think and hope is good for most users. We do user surveys and so forth, to check if we are right, and I think we usually do pretty well (but nobody is perfect, obviously!). I definitely agree that users on low-end hardware have different needs for caching and so forth, and that's why Firefox lets you customize the caching behavior, as I mentioned earlier with that pref. You can also change how much memory is used for the disk and memory caches (search for browser.cache.disk and browser.cache.memory in about:config, to see the relevant options).
I agree this isn't a good solution - it's just a way to narrow down the problem.
If the profile does turn out to be the problem, then we can turn to investigate the profile itself, assuming the user is ok with that (by filing a bug and attaching the profile).
If you see memory usage change that quickly - 2MB per second! - then this should be easy to figure out. Disable Firebug for a few minutes and see if that fixes it. If not, then try closing page by page, until you see that the constant 2MB increases stops. Slowly disable and close stuff, until you are left with just a window with Slashdot and nothing else - no addons or plugins even. Then close Slashdot;) by browsing to about:blank. If you still get 2MB increases every second, that is bizarre.
I would expect that disabling Firebug, or closing one of the web pages you have, will fix the problem. Since you have such a quick memory increase, it should be fairly easy to find out.
About addons: There is no way to see directly how much memory an addon uses - it is tightly integrated with the core browser. Jetpack addons can do that, but something like Firebug will likely never be a Jetpack addon (it needs tight integration).
I suspect that a profile can, in very rare cases, cause this. We save a lot of stuff in the profile, which is useful, but also complicated - there are likely some rare corner cases that hit a very small number of people.
It's simple to find out if this is your issue: Create a new profile, browse with that for a few days, see how it feels. Also worth doing is disabling addons and plugins in the new profile.
If you see that, then you might have plugins or addons installed systemwide (for example, Skype installs such plugins, and many others). Please try to disable or uninstall all plugins and addons, they might cause such problems. For myself, I do those 3 steps you mention every day, both as a normal user of Firefox and as a developer. I have not seen anything similar. If I did, it would be my #1 priority bug to fix.
In any case, if you are pleased with Chrome, then I'm happy you found a browser that works for you. The whole point of the web is to have multiple compatible clients, that users can pick from - so pick the one you like best.
The websites and behavior sound pretty normal to me. So I would suspect AdBlock (which is known to cause leaks in some cases, at least one issue is fixed in Firefox 4.0.1 IIRC), possibly also Flash. I would try disabling them both for a few days and see how that changes things. If the problem remains, I would try a new profile. If the problem still remains after that, then it should be reproducible at that point, maybe we can get more specific with the details (which sites, # of refreshes, etc.), and I'll try to reproduce here.
Slashdot Mirror
Can you point to such a Firefox 64bit build?
Sure, Mozilla's ftp server has them here (the ones with x86_64). Those are for FF8 (Nightly build).
I thought I had been running a 64bit Firefox for years. So I wasn't? Or is this about finally doing a 64-bit Windows build? Probably since Moz Corp is entirely focused on Windows and treats Linux as a red headed stepchild.
I am a Firefox dev and a Linux user. Mozilla is definitely not focused on Windows, in fact many of us devs use Linux (I am posting from Ubuntu right now), and many of the rest use OS X. Windows is in the minority.
There are 64-bit builds available from our build system, but we don't promote them. The reason is that we don't spend as much effort on QAing 64-bit builds, we have limited resources and are focused on the standard (32-bit) builds for the most part.
There are some good reasons for 64-bit browsers, for sure, but AFAIK none of the major browsers make that a priority. For example, there is a 64-bit IE9, however it ships with a hobbled JavaScript engine (without JITs), so clearly they don't intend it very seriously.
In any case, given that Firefox is open source, anyone can build a 64-bit version. I believe several Linux distros ship a 64-bit Firefox, for example. There used to be some problems with running 32-bit Flash in it, but I have heard that is workable now too.
I know it's been said before, but this new release system is fucking retarded.
I'm this close to dumping Firefox on every machine I touch.
I think it's silly to dump both Chrome and Firefox because of their release systems (which are identical - both release a new major version number every 6 weeks).
I guess you can use other browsers than Chrome and Firefox. But those other browsers release new versions with new features very rarely. Is the *version number* enough of a reason to not use Chrome and Firefox? I don't think so - even though I thought it was silly when Chrome started with it, and when Firefox decided to do it as well.
But it seems amazingly inferior to a platform native PDF reader, on any platform imaginable. It will be slower the native x86/ARM code by far, and won't integrate well with the desktop environment.
Regarding speed, two things: First, this will spend most of its time in calls to the browser's Canvas API, which all browsers implement in C++. So it isn't clear that it should be significantly slower than a native implementation. Second, even if this were in 100% JavaScript, that is just around 5X slower than C++ these days. Rendering PDFs might be plenty fast enough at that speed, since you typically render once then show it for a long time. In other words, this isn't something like a game engine that needs to push 60fps.
What's with this trend recently to build everything on fundamentally sucky technologies?
There are several advantages to implementing this using Web technologies. The first is, as you mentioned, it's a good way to find browser bugs, and it also helps find areas to improve the Canvas API. Another important reason is that this implementation will be much more secure than any other existing PDF implementation, because so much work has gone into proper sandboxing of JavaScript. And we know how many vulnerabilities keep popping up in existing PDF readers like Adobe's. It will be much more secure to use this new PDF implementation than the existing ones.
While this is impressive, it has been done before (and better): GWT Quake
I think that Quake demo is awesome! I'd just like to mention though that this Doom demo is very different from a technical standpoint, and I think both are interesting:
So... no music, and the sound effects are like a half second delayed from the action, AND they sound really REALLY bad.
I agree, the sound is terrible, sorry about that. I don't know much about how sound stuff works, that is the best I could do in a few hours of hacking something together with the Audio Data API.
However, without such basic controls as strafe, this demo is not playable.
Actually strafe works, hold down alt.
If it's under the GPL and LGPL, it's going to be a rough case Attachmate would be making, considering that it's open licensed and they just kicked the team to the curb.
The FOSS code is not a concern here.
The issue is the proprietary code that Miguel et al worked on in Novell, the Android and iPhone runtimes. That is owned by Attachmate now, and this new startup contains exactly the same coders, who are intentionally going to write the exact same product from scratch - they will be 100% "source compatible" with the old runtimes.
So legalities are a reasonable concern. Even if no code is copied, the same people writing the same product - immediately after writing it the first time - may lead to basically the same code being written. It might be hard to prove no code was copied even if none was. Lawsuits are filed for much less.
Of course, this only matters if Xamarin is a big success - no one sues a failure. Time will tell.
It is very awkward for a distro that bundles Chrome to use the system WebKit, since Chrome's entire approach is opposed to that. Chrome updates silently and often and if it used the system WebKit, Chrome could not update itself at all. You would be frozen with an older version of Chrome, and Google stops supporting them very quickly - Google's goal is update every one to the new versions.
However, if used on something like a netbook or tablet, it would make a lot of sense, since the browser would be one of the first things opened on the device.
I don't get it, why does loading a copy of WebKit in the login manager make sense? That you open a browser soon later doesn't matter, since the by far most common browsers, Firefox and Chrome, would not benefit from that in any way: Firefox since it doesn't use WebKit, Chrome since it has its own copy - you will have 2 copies of WebKit in memory at that point, each taking >25MB!
Bet WebKit's squalid bulk didn't go into the 5KLoC vs 50KLoC size difference.
From the docs it looks like WebKit might be optional in LightDM. But if WebKit is used on Ubuntu, then I don't see why loading a 25MB HTML engine (+ it's dependencies!) is a good thing. WebKit is pretty bulky. And this is a login manager we're talking about!
And it's not like WebKit will be loaded later anyhow. Even if you do use a WebKit based browser, that is most likely Chrome, which bundles it's own copy of WebKit - so no sharing with apps that use the system WebKit like LightDM.
I guess I remembered wrong then. Or perhaps it was only last year that a stable GCC came out with it? In any case, sorry for being wrong.
So why is the solution to roll your own, rather than fix LLVM or develop a more direct competitor to it? All of the issues mentioned so far seem to be with the current implementation, not with the fundamental idea.
The goals are somewhat different. LLVM is great for optimizing statically typed code, so it will do e.g. licm that takes into account that kind of code. The IR that would be useful for a JS engine would be higher-level, so it could do licm on higher-level constructs.
.NET. They have very efficient JITs, but they are tuned for the level that makes sense for languages like Java and C#, not JavaScript or Python or Ruby. As a consequence, no dynamic language on those platforms comes close to modern JS engines, LuaJIT, etc. Some people seem to assume that translating Python into Java, then JITing that on the JVM will be fast - but it isn't.
The issue is sort of similar to why dynamic languages don't run very fast on the JVM or
So there isn't really something to be 'fixed' in LLVM here - it is very good at what it does. It's just different than what JS engines need.
Thank you! I can reproduce this, and it definitely looks very bad...
I filed bug 654028, and as I promised before, I will investigate this myself. If you want, you can subscribe to the bug to get updates.
> Can you please give some more details about how to reproduce it:
I'm not the person you are replying to, but here is one way for me with FF4 under Ubunu 11.04.
Go to mail.google.com and log into it. Keep the page open, but close your network connection (e.g disconnect wireless on your laptop to save power). Go away for a while. Next morning, Firefox4 will be using over 1.5 GB memory and the machine will be swapping its brains out.
Pretending there are no such bugs in FF when people have been complaining about the leaks for years is a rather poor reflection.
I never pretended there are no such bugs - I am sure there are. Thank you for giving some steps to reproduce, I will try this!
The memory cache caches some types of items, but not images, in that way. Take a look at about:cache to see more details of what it currently stores, if you want.
For images, the important cache is probably the page cache (bfcache), which stores entire pages so they can quickly be reloaded. You can change the amount of pages it caches with browser.sessionstore.max_total_viewers (set it to 0 to not cache any, for example).
How about making FF 6 or 7 better able to restrict leakage?
We are working very hard on that, from various directions - from better tools to report how much memory is being used, to eventually running tabs in separate processes so their memory use can be contained, and also allowing Jetpack plugins that are likewise separated so they cannot cause leaks.
The fact that plugins variety is even a problem must then be Mozilla's design flaw.
There is definitely a tradeoff here - Firefox is more customizable and flexible than other browsers. That makes it more vulnerable to leaks from addons and so forth. In other words, it is much easier to prevent these problems when you greatly restrict what your browser and its addons do.
As I said above, we are working on some technical improvements for these matters, which should hopefully be available later this year. The tradeoff should improve at that point.
Meanwhile, it is possible to use Firefox's customization to change the behavior, if you want - stop if from caching pages and so forth.
So the next thing is to start closing tabs but I can't do that right now because I'm working on stuff but I will try later.
I would definitely suspect that one of your tabs is guilty here, if you have no addons or plugins enabled. Hopefully if you try that later, you'll find which is responsible.
If not, and you still see this with a single tab, new profile, no addons or plugins, then that is... extremely interesting. At that point, a screencast would be the perfect thing here, good idea! Would be great if you can do that, and if so, please include the following in the screencast:
The behavior you describe should definitely not be happening. Hopefully this can help narrow it down.
I am definitely not dismissing that! If an old profile causes buggy behavior, then that is a bug in Firefox, I fully agree.
The problem is practical, though - this happens on a very small number of user profiles, and users usually don't want to (for totally understandable reasons!) file a bug with their profile attached. So it is very, very hard for us Firefox devs to figure out the problem, since we can't debug it locally on our machines.
In the long term, we are doing a lot of work on about:memory, which right now shows some useful info, but we want it to show a lot more. Then users with too much memory being used will be able to report bugs with information that should help us a lot in figuring out the problem.
So please, I love firefox, I love it's open-ness, the add-ons, the customize-ability, and even many of the new(er) features (the awesomebar is, in fact, awesome), but I hate the memory waste! Even adding a hidden config option to enable 'reclaim memory on tab-close' would be much appreciated!
There is such an option: browser.sessionhistory.max_total_viewers :) Set it to 0, and pages are not cached anymore. See here for more details.
Many people run linux on older, low-end hardware especially because of it's reputation as a low-resource, light-weight operating system. Firefox seems to be building on expectations that are vague or simply not true.
We ship with a default behavior that we think and hope is good for most users. We do user surveys and so forth, to check if we are right, and I think we usually do pretty well (but nobody is perfect, obviously!). I definitely agree that users on low-end hardware have different needs for caching and so forth, and that's why Firefox lets you customize the caching behavior, as I mentioned earlier with that pref. You can also change how much memory is used for the disk and memory caches (search for browser.cache.disk and browser.cache.memory in about:config, to see the relevant options).
I agree this isn't a good solution - it's just a way to narrow down the problem.
If the profile does turn out to be the problem, then we can turn to investigate the profile itself, assuming the user is ok with that (by filing a bug and attaching the profile).
If you see memory usage change that quickly - 2MB per second! - then this should be easy to figure out. Disable Firebug for a few minutes and see if that fixes it. If not, then try closing page by page, until you see that the constant 2MB increases stops. Slowly disable and close stuff, until you are left with just a window with Slashdot and nothing else - no addons or plugins even. Then close Slashdot ;) by browsing to about:blank. If you still get 2MB increases every second, that is bizarre.
I would expect that disabling Firebug, or closing one of the web pages you have, will fix the problem. Since you have such a quick memory increase, it should be fairly easy to find out.
About addons: There is no way to see directly how much memory an addon uses - it is tightly integrated with the core browser. Jetpack addons can do that, but something like Firebug will likely never be a Jetpack addon (it needs tight integration).
I suspect that a profile can, in very rare cases, cause this. We save a lot of stuff in the profile, which is useful, but also complicated - there are likely some rare corner cases that hit a very small number of people.
It's simple to find out if this is your issue: Create a new profile, browse with that for a few days, see how it feels. Also worth doing is disabling addons and plugins in the new profile.
If you see that, then you might have plugins or addons installed systemwide (for example, Skype installs such plugins, and many others). Please try to disable or uninstall all plugins and addons, they might cause such problems. For myself, I do those 3 steps you mention every day, both as a normal user of Firefox and as a developer. I have not seen anything similar. If I did, it would be my #1 priority bug to fix.
In any case, if you are pleased with Chrome, then I'm happy you found a browser that works for you. The whole point of the web is to have multiple compatible clients, that users can pick from - so pick the one you like best.
The websites and behavior sound pretty normal to me. So I would suspect AdBlock (which is known to cause leaks in some cases, at least one issue is fixed in Firefox 4.0.1 IIRC), possibly also Flash. I would try disabling them both for a few days and see how that changes things. If the problem remains, I would try a new profile. If the problem still remains after that, then it should be reproducible at that point, maybe we can get more specific with the details (which sites, # of refreshes, etc.), and I'll try to reproduce here.