Slashdot Mirror
Building All the Major Open-Source Web Browsers
An anonymous reader writes: Cristophe de Dinechin, long-time software developer, has an interesting article on the processes involved in building the major browsers. From the article:
"Mozilla Firefox, Chromium (the open-source variant of Chrome) and WebKit (the basis for Safari) are all great examples of open-source software. The Qt project has a simple webkit-based web browser in their examples. So that's at least four different open-source web browsers to choose from. But what does it take to actually build them? The TL;DR answer is that these are complex pieces of software, each of them with rather idiosyncratic build systems, and that you should consider 100GB of disk space to build all the browsers, a few hours of download, and be prepared to learn lots of new, rather specific tools."
"Mozilla Firefox, Chromium (the open-source variant of Chrome) and WebKit (the basis for Safari) are all great examples of open-source software. The Qt project has a simple webkit-based web browser in their examples. So that's at least four different open-source web browsers to choose from. But what does it take to actually build them? The TL;DR answer is that these are complex pieces of software, each of them with rather idiosyncratic build systems, and that you should consider 100GB of disk space to build all the browsers, a few hours of download, and be prepared to learn lots of new, rather specific tools."
Of course, if you use a source-based distribution, it's easy, but that's not the point. If you download the source and build it yourself, you'll have to learn a bit about how they manage the build environment. You'll have to handle being sure all the dependencies are installed. You'll have to figure out any configuration options.
That's exactly what people who make distributions do. If you want to see how complicated the build is for any piece of software, just look at how complicated the build scripts are for various distributions. I expect you can find these for the binary distributions. With Gentoo, just look at the ebuild file.
So it's simply emerge --update world* but I always groan when I see Chromium has a new version, which it does almost weekly, because it takes approximately FOREVER to compile. Firefox and Seamonkey take a long time, sure, but Chromium, yikes, come back tomorrow.
*Actually, it's: emerge -v --deep --newuse --with-bdeps=y --update world 2>&1 | tee update-world-deeper-`date +%F`.log, but that's just me.
The system cannot find the path specified.
I guess those of us using Windows still aren't going to be contributing much any time soon. :-(
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
File it under "stuff that matters".
A lot of arguments for open source are based on things which people outside the project could in principle accomplish, but in practice seldom do. So it's reassuring at least that an experienced developer can build the two most popular browsers from scratch. It means the arguments aren't hollow. I've seen closed source projects that were purchased by companies, only to find out that getting them to build on any computer but the one it was developed on is a serious engineering challenge.
That the process of building these browsers from scratch is somewhat arcane will come as no surprise to any experienced developer. But that it's not so arcane that it's impractical to figure out is good news.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
Are you a frequent contributor? No? Okay, well shut the fuck up and let Bennett weigh in on this.
There are some really great distros that are based on said sources:
Pale Moon is an open source alternative to FireFox:
http://www.palemoon.org/
SRWare Iron is a Chromium project:
http://www.srware.net/en/softw...
C:\> choco install VisualStudioExpress2013WindowsDesktop
http://www.pcworld.com/article...
This space for rent.
Unfortunately, like too many OSS projects, Mozilla seems to think it will have the only cygwin instance on my system. It therefore assumes it's OK for me to just reconfigure the entire universe according to its preferences, redefine all my paths so the MozillaBuild version of everything takes precedence over anything else that's already installed, and so on.
In reality, I have various other tools installed that bridge the Windows and Linux worlds, including things fundamental to using various version control systems and other everyday needs. As much as I'd like to support Mozilla and be willing to contribute a bit here and there, I'm not going to compromise the development machine I also use to earn my pay cheque just to get their esoteric build system to work.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
I have to say, as what people here would probably regard as only an amateur programmer, the setup of a build environment and initial compiles suck up way, way more time than they need to.
It's extremely frustrating. C is a wonderful language that I love to program in. There are compilers on every platform. There are cross-platform libraries for anything I need and my favourite IDE is cross-platform.
But actually getting basic shit to work is an absolute nightmare. And that's the stuff that I'm familiar with. It involves far too much pissing about with paths, switches, versions, system-level installations, scripts, extra options and all kinds of voodoo to make what I know works work how it should. And I often have to replicate it for every new project.
Fuck knows what it's like when you go multi-language, when you go to more obscure platforms, and when you're not familiar with things or am using someone else's code without the benefit of knowledge of their build environment.
I consider it the weak-point of open-source. Like science, one of the big pluses of open-source is reproducibility. You got working code? Good. Then so do I. All I need to do is make it compile. The problem is that "All" in that sentence.
There are languages that encapsulate more into the language but even that causes problems. I've fought with Java environments where the JRE and JDK differ and are in multiple versions, have architecture differences, and require explicit paths, and all kinds of nonsense. Python can be a fucking nightmare on Windows. Even shit like Cygwin interferes with ITSELF on Windows systems.
I don't know what the solution is, but I can't help but feel it's outside the language - it's in the system design and OS interaction. Operating systems just aren't designed with source being a big part of the end product.
Maybe the solution is one of the ideas I constantly toy with (and almost certainly has been done in the 60's on some archaic architecture) - an executable format that incorporates the source as well as the binary. When the executable moves to a new platform, the binary and source visibly differ and the binary can be recreated from the source. To do that requires complete encapsulation of all source, options, compilers and everything else into the executable.
But I'm sick at getting something as standardised as SDL working on the three big platforms (Windows, Mac, Linux) in using one of the largest open-source IDE's (Eclipse) utilising an industry-standard compiler (GCC). It should NOT be as hard as it is.
Idiosyncratic builds is not limited to just browsers and is probably the biggest problem faced by Open Source projects today.
I use open source tools daily and yet with 20 years development experience I have yet to fine one open source project that straightforward to build.
If only someone would come along and write a new, unified make system.
This has been done. A thousand times over.
And now we have a thousand unified make systems.
Maybe, but unless you condone software piracy, that would require either buying a second Windows licence (unless you have one of the SA schemes that covers using Windows as both host and guest OS) or running something like Linux as your guest OS and figuring out the cross-compilation issues (if that's possible).
With today's software and licensing landscape, I just don't think setting up a custom VM for every project you work on is viable, nor that imposing burdens on that kind of level is the way to encourage skilled but casual/irregular contributors to help your project.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
Unfortunately, in this case that '????' is something like "develop several years' worth of cross-compilation infrastructure so you can build on your guest OS but run on the host".
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
It's your fault for using Windows, the platform where building cross-platform software is always black magic.
Yes, silly me, using the same platform for building and testing as approximately 100% of my customers do.
Sadly, the platforms that we geeks like to use aren't particularly relevant to a discussion about software for a mainstream audience, because that mainstream audience isn't running Linux and most of it isn't running OS X either.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
That the process of building these browsers from scratch is somewhat arcane will come as no surprise to any experienced developer. But that it's not so arcane that it's impractical to figure out is good news.
it sounded very impractical. FTFS: "these are complex pieces of software, each of them with rather idiosyncratic build systems, and that you should consider 100GB of disk space to build all the browsers, a few hours of download, and be prepared to learn lots of new, rather specific tools."" I'd say it's good that open source browsers are not impossible to build. But this is no apt-get or whatever people say.
Fair enough. I would agree that the strategy you're describing could be an option in a software development shop where volume licensing is routine, and that given that ability it could well make sense to set up different standardised VMs for working on different projects.
Unfortunately, the average developer who might contribute to Firefox in their spare time isn't likely to have those kinds of professional-grade agreements in place for their own private projects, so I still don't think VMs are a general solution to the original problem that an onerous build system might discourage contributions from otherwise willing volunteers.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
(cmake is probably the best, since it's more portable than autoconf).
As a user of autoconfed packages, I find autoconf superior to cmake. Packages built with autoconf have standardized mechanisms for uninstallation (a cmake package may generate an install-manifest file, an uninstall target, or none of the two), to specify where to put documentation, for cross-compilation, and to fine-tune the build and the installation. With cmake, I can't even tell the package where to install libraries (most packages will allow you to do it, but each package has a different standard about the way to be told); with autoconf, I can even specify a sed to be run on the name of the installed binaries (useful if different packages provide different implementations of the same binary) and still have the installed package work. Also, with cmake packages building both static and dynamic libraries at the same time is usually impossible.
Moreover, modern autoconf scripts are (relatively) easy to debug and patch when they don't work; cmake scripts are more scattered and they're written in an obscure mainframish language.
That said, I imagine that using autoconf on non-posix systems might be less funny.
If you don't need the VM for long term, you can download an evaluation version of Windows that expires after a bit. There's currently a 8.1 evaluation that's good for 90 days, and a technical preview for 10 that's good until (I think) April. You can also just reuse your Vista or later Windows disk, skip typing in the key, and it'll run for a few days before demanding a key though I don't know if this would technically be a license violation. Finally, with some versions of Windows 7 you can install XP mode for free, which gives a Windows XP VM that you can play around with.