Building All the Major Open-Source Web Browsers

An anonymous reader writes: Cristophe de Dinechin, long-time software developer, has an interesting article on the processes involved in building the major browsers. From the article:

"Mozilla Firefox, Chromium (the open-source variant of Chrome) and WebKit (the basis for Safari) are all great examples of open-source software. The Qt project has a simple webkit-based web browser in their examples. So that's at least four different open-source web browsers to choose from. But what does it take to actually build them? The TL;DR answer is that these are complex pieces of software, each of them with rather idiosyncratic build systems, and that you should consider 100GB of disk space to build all the browsers, a few hours of download, and be prepared to learn lots of new, rather specific tools."

Comments

Build

[Anonymous Coward]

cd /usr/ports/www/firefox && make install clean
cd /usr/ports/www/chromium && make install clean
cd /usr/ports/www/webkit-gtk3 && make install clean

Wasn't that hard.

Re:Build

[Anonymous Brave Guy]

C:\Users\ABG>cd /usr/ports/www/firefox && make install clean
The system cannot find the path specified.

I guess those of us using Windows still aren't going to be contributing much any time soon. :-(

Re:Build

[SampleFish]

You have to install the compiler repos just like a Linux user would.

You also might be interested in Cygwin:
http://cygwin.com/

and Windows Services for UNIX:
http://www.microsoft.com/en-us...

Re:Build

[Anonymous Brave Guy]

Unfortunately, like too many OSS projects, Mozilla seems to think it will have the only cygwin instance on my system. It therefore assumes it's OK for me to just reconfigure the entire universe according to its preferences, redefine all my paths so the MozillaBuild version of everything takes precedence over anything else that's already installed, and so on.

In reality, I have various other tools installed that bridge the Windows and Linux worlds, including things fundamental to using various version control systems and other everyday needs. As much as I'd like to support Mozilla and be willing to contribute a bit here and there, I'm not going to compromise the development machine I also use to earn my pay cheque just to get their esoteric build system to work.

Re:Build

[John Bokma]

1) install VirtualBox or similar
2) create a FreeBSD VM
3) ????
4) Profit

Re: Build

[Anonymous Brave Guy]

Maybe, but unless you condone software piracy, that would require either buying a second Windows licence (unless you have one of the SA schemes that covers using Windows as both host and guest OS) or running something like Linux as your guest OS and figuring out the cross-compilation issues (if that's possible).

With today's software and licensing landscape, I just don't think setting up a custom VM for every project you work on is viable, nor that imposing burdens on that kind of level is the way to encourage skilled but casual/irregular contributors to help your project.

Re:Build

[Anonymous Brave Guy]

Unfortunately, in this case that '????' is something like "develop several years' worth of cross-compilation infrastructure so you can build on your guest OS but run on the host".

Re: Build

[tobenemo32]

Is it a full moon or are you cranky for other reasons? I think there are better way of expressing opinions than the F word.

Re: Build

[cavreader]

VM's make excellent DEV and QA testing environments and can be used as many times as needed under most development only licensing options. VM's are good in production environments but this is where distribution licenses need to be taken into account. During development and testing VM's provide the ability to reset the entire environment back to the pre-testing state. Testing applications can end up creating and leaving behind invalid registries, assemblies, old installs, and event logs that make debugging errors more time consuming. Being able to revert back to a pristine environment for the next round of testing can be helpful especially when testing installation processes and application configuration changes that can effect the base OS level.

Re: Build

[Anonymous Brave Guy]

I'm well aware of the advantages of using VMs for creating a controlled environment. Unfortunately, that doesn't make my previous observations about licensing any less true.

I don't know what you mean by "development only licensing options" as far as Windows is concerned. There are different rules if you use Software Assurance to sort out your licensing, but that's the only exception to the one VM/one licensed copy rule as far as I'm aware. If you know better, please cite accordingly, as I'm sure plenty of us would like to hear about it.

Re:Build

[Anonymous Brave Guy]

It's your fault for using Windows, the platform where building cross-platform software is always black magic.

Yes, silly me, using the same platform for building and testing as approximately 100% of my customers do.

Sadly, the platforms that we geeks like to use aren't particularly relevant to a discussion about software for a mainstream audience, because that mainstream audience isn't running Linux and most of it isn't running OS X either.

Re: Build

[Anonymous Coward]

Realistically speaking, almost nobody is going to cough up $100-$200 to the Great Redmond Overlord every time they want to throw together a VM, and almost nobody is going to stop themselves from setting up a VM because they don't want Microsoft's stockholders to be denied their ivory backscratchers. This includes people who already paid for a Windows license.

Re:Build

[dryeo]

Building Firefox (the only one I have experience with) will never work that way unless there has been a lot of patching to the official source. Firefox isn't designed to be installed with make install and install isn't even a target and building means "make -f client.mk"
From client.mk

# Build a mozilla application.
#
# To build a tree,
# 1. hg clone ssh://hg.mozilla.org/mozilla-central mozilla
# 2. cd mozilla
# 3. create your .mozconfig file with
# ac_add_options --enable-application=browser
# 4. gmake -f client.mk
#
# Other targets (gmake -f client.mk [targets...]),
# build
# clean (realclean is now the same as clean)
# distclean

Re:Build

[dryeo]

How hard is it to write a script to adjust your path, open a cmd window and type in mozenv.cmd?

Re: Build

[cavreader]

I wasn't disputing your observations. I just added my observation of being able to use development only licences across multiple VM's or desktops provides a good app testing environment without the hassle of dealing with licensing costs in the development area. The MS development licenses I require are provided and managed through an MSDN subscription in conjunction with an MS Partner agreement which expands the number of developer only licenses. I have also obtained libraries built by non-MS vendors with the most recent example being libraries providing PLC and OMNI comm services. These libraries also came with an unlimited development only license from the vendor with any licensing costs only charged when distributing the applications that use these libraries. The cost of obtaining generous developer only licences more than makes up for the MSDN and MS Partner related costs.

Re:Build

[Anonymous Brave Guy]

If it were just one little thing like a path that needed to be customised, obviously it would be trivial to do so. Unfortunately, with the kind of project that has a whole custom build system and wants its exact required version of everything under the sun installed, it is rarely so simple. For example, the consequences of having two different versions of a VCS installed on your system could be horrible if there had been a change in the internal source repository representation from one to the other and you somehow wound up running the wrong one even once.

Re: Build

[Anonymous Brave Guy]

Fair enough. I would agree that the strategy you're describing could be an option in a software development shop where volume licensing is routine, and that given that ability it could well make sense to set up different standardised VMs for working on different projects.

Unfortunately, the average developer who might contribute to Firefox in their spare time isn't likely to have those kinds of professional-grade agreements in place for their own private projects, so I still don't think VMs are a general solution to the original problem that an onerous build system might discourage contributions from otherwise willing volunteers.

Re:Build

[dryeo]

According to https://developer.mozilla.org/... you just need to make sure that c:\mozilla-build comes before cygwin on the PATH and run something like start-shell-msvc2013.bat (various bat files depending on msvc version). Pretty simple.
Mozilla doesn't require exact versions of everything, it just needs working versions of everything and if you're worried about problems with Mercurial, just use your default Python 2.7 and corresponding hg.exe.
Best to start by following their exact instructions then pruning the files in \mozilla-build and testing. generally most of the tools can be replaced. There are exceptions like autoconf213 which is totally incompatible with newer autoconfs.
For most tools it doesn't matter the exact version as long as the required features are supported, which in general means fairly up to date and non-buggy like some versions of sed. The configure script even does some checks for bad versions.
Best practices are frequent backups so if you do screw a repository with using the wrong VCS version you can recover and getting in the habit of using scripts to setup your environment.
It is a bit more time consuming and works best at the command prompt so may not be worth it for you but not that hard.

Re:Build

[Anonymous Brave Guy]

No doubt it is possible set it all up and get it working eventually. I've worked on projects with much more complicated set-ups than what you describe that build on a lot more different platforms than Firefox, and once you've figured out the details things generally run OK.

That said, there are quite a few statements even on the page you linked to (which is also the one I'd read last time I looked into this) that suggest requirements and dependencies beyond a single PATH update. An obvious example is that prohibiting spaces in the path to the source directory means some people won't be able to work in the standard user data directory on their Windows system, and installing outside that directory might mean more work to configure back-ups, anti-virus (not scanning build outputs to avoid killing performance), and so on.

The problem here is not what is possible, just the time and risk required to get there. This isn't helped IMHO by the lack of transparency in Mozilla's documentation for contributors; I'm all for having a systematic, one-stop tool like MozillaBuild to automate the process, but you still have to explain what that process is so the developer knows what MozillaBuild is going to do to their system.

If someone wants to become a regular contributor and doesn't have a lot of other constraints on their development PC, all of this might not be a big deal, but if every OSS project came with the same barrier to entry, the OSS world would fall apart tomorrow.

Re:Build

[Anonymous Brave Guy]

I use VMs all the time, but they won't solve this particular problem for most people. Please read the other posts before you reply, as we've already covered this elsewhere in the thread.

Re: Build

[toddestan]

If you don't need the VM for long term, you can download an evaluation version of Windows that expires after a bit. There's currently a 8.1 evaluation that's good for 90 days, and a technical preview for 10 that's good until (I think) April. You can also just reuse your Vista or later Windows disk, skip typing in the key, and it'll run for a few days before demanding a key though I don't know if this would technically be a license violation. Finally, with some versions of Windows 7 you can install XP mode for free, which gives a Windows XP VM that you can play around with.

Re:Build

[amaurea]

The grandparent's example was not made up. It was how it's actually done on FreeBSD. Invoking make there downloads the sources, patches them, and then invokes the standard build procedure you quoted. So yes, somebody had to do some work to wrap the build process in a uniform interface, but it's no more work than what's needed for maintaining binary packages like Debian's apt packages. And once that work is done, anybody else using that operating system distribution can do it as simply as the grandparent showed.

Re: Build

[Anonymous Brave Guy]

Yes, in a sense I am too lazy to compile it. I have several businesses to run, family commitments, and so on. That leaves limited time for helping with volunteer efforts, and I choose to spend it helping projects where I can make what seems like the most valuable contribution in that time.

Sometimes that includes doing things for FOSS projects, but right now Mozilla's projects are not among them. The point is that they might have been if they had a better build process on Windows and/or better documentation about that process. While I probably have been relatively busy in recent years, somehow I doubt I'm the only potential contributor who has been deterred by the barriers to getting started. Writing illiterate, profanity-laden posts at me on Slashdot isn't going to help anyone, but improving the situation I've highlighted might.

Re:Build

[descubes]

You are not building the latest, only whatever patched version exists in your ports. And did you actually run the commands and verify that your machine is powerful enough to do it?

Re:Build

[crackerjack155]

Using the windows program Sandboxie allows you to easily do everything you have mentioned. It lets you create different sandboxes for different things.

Re: Build

[cavreader]

One more observation. If a company wants to increase their market share in both applications along with the corresponding hardware the applications run under they should go out of their way to make the barrier for developers as low as possible. The more developers they can attract means more applications are created for a particular OS or environment resulting in an increase in OS sales and overall system buy in. VB is probably the top reason of how MS gained it's overall dominance in the 90's. Regardless of the merits or flaws of VB it opened the door for new developers who didn't have to know and use C/C++or other lower languages to build applications. Windows development went from the developers implementing dialog classes and class templates to dragging a form icon and corresponding controls into their work space along with it's defined events and properties. Of course at the time you had to use Windows to take advantage of this magical developing capability in the eyes of those making corporate IT decisions.

Easy with Gentoo

[crow]

Of course, if you use a source-based distribution, it's easy, but that's not the point. If you download the source and build it yourself, you'll have to learn a bit about how they manage the build environment. You'll have to handle being sure all the dependencies are installed. You'll have to figure out any configuration options.

That's exactly what people who make distributions do. If you want to see how complicated the build is for any piece of software, just look at how complicated the build scripts are for various distributions. I expect you can find these for the binary distributions. With Gentoo, just look at the ebuild file.

Re:Easy with Gentoo

[abhi_beckert]

That's exactly what people who make distributions do. If you want to see how complicated the build is for any piece of software, just look at how complicated the build scripts are for various distributions.

I think it's a bad idea to rely on distributors to do this work. With my projects, I always try to make the build process as simple as possible.

If somebody is thinking about maybe contributing to the project, I want it to be completely painless.

It doesn't always work out, especially with third party dependencies, but I try to keep the build process as simple as checkout source code, and build it. No configuration at all. Obviously you can configure stuff, but there should be a default that will work for almost everyone.

Re:Easy with Gentoo

[crow]

I agree that it should be easy. My point wasn't to rely on distributors for building. It was that the work has already been done for major projects like the ones listed, so you can get a sense of their build complexity by looking at what they had to do.

For example, here are some ebuild sizes:
gcc-4.9.1 1556 bytes
mythtv-0.27.4 9796 bytes
firefox-33.0 11698 bytes
libreoffice-4.3.1.2 18473 bytes
chromium-40.0.2194.2 18610 bytes
netbeans-ide-8.0 29367 bytes

That's a rough approximation of complexity. Sure, you can have a long but simple ebuild with comments and trivial stuff. You can hide lots of complexity in an eclass file. But it's a starting point.

As you can see, a project that uses a typical autoconf setup like gcc, while a very large project, has a very simple ebuild script. Meanwhile, Chromium and LibreOffice are quite complex. I don't know much about netbeans, but it's the largest ebuild in Portage right now. Other large ebuilds include openldap, ghc, php, and ati-drivers.

I run Gentoo

[NoNonAlphaCharsHere]

So it's simply emerge --update world* but I always groan when I see Chromium has a new version, which it does almost weekly, because it takes approximately FOREVER to compile. Firefox and Seamonkey take a long time, sure, but Chromium, yikes, come back tomorrow.

*Actually, it's: emerge -v --deep --newuse --with-bdeps=y --update world 2>&1 | tee update-world-deeper-`date +%F`.log, but that's just me.

Re:I run Gentoo

[SlashdotOgre]

Yeah, I'm in the same boat; it's approaching the point where I'm debating just unmerging the damn thing as I mostly use FF. It's gotten to the point that I've masked* Chromium and am now only updating it monthly when I manually unmask it. I'm on a fairly recent laptop CPU (i5-3230M) and building Chromium takes so long it reminds me of emerging gnome2 back when I had a Pentium3 800MHz.

* for non-Gentoo users, masking a package basically hides it from future updates. You can mask specific versions or anything going forward.

Re:I run Gentoo

[reve_etrange]

I thought there were options to install pre-compiled versions of the packages. Are they not all available, or am I just mistaken?

(Obviously I'm not a Gentoo user. Mostly Debian).

Re:I run Gentoo

[Anonymous Coward]

The entire point of Gentoo is literally to compile the entire system on your system, custom-built for your hardware, for maximum efficiency for your specs. For instance, you might have the latest Intel i7 Whizbang with the added XYZ super-fast instruction set, but it'll be many years before that ends up being compiled by default into binaries. There are still binaries being built that are 386 compatible, for instance! Even when you get those instructions compiled in a lot of time there's conditional code that turns it on and off. When compiled specifically for the hardware, you can set all the compiler flags to enable all this stuff, and disable safeguards you don't like or want. Although it'll segfault or worse on hardware that isn't yours, the binaries are compiled specifically to use your hardware advantages to the max. Plus, it's distribution-wide. While this will work with other distros, you have to build them largely from scratch in most cases to attain this. So Gentoo is a distro based on that compilation process.

Re:I run Gentoo

[morgauxo]

I use Gentoo too. Most things compile in minutes. Some things such as browsers can take hours. Big Desktop environments like KDE/Gnome and also LibreOffice can take over a day each. Some people can't imagine how a Gentoo user can stand it. It's really not a problem though.

The build process never prompts you for anything, all options are chosen before it starts. It also does not require a GUI. (There are GUIs available if you want one) It's just pretty simple commands like (emerge chromium). Emerge handles the whole process, downloading source, compiling, installing and does all the dependencies too.

So... the answer to long build times is easy. Just start the build in a screen session. Then you can close the window and forget about it. It will be done when it is done, feel free to continue using your computer in the mean time. Or, not. You CAN walk away. If you are concerned about the build process running in the background slowing down your computer when you are using it for other things that's no problem either. You can just raise the nice level. It might take longer to finish but you never even have to know that the build is running.

Re:I run Gentoo

[morgauxo]

"The entire point of Gentoo is literally to compile the entire system on your system, custom-built for your hardware, for maximum efficiency for your specs."

Not necessarily. It's also good just for general customization that doesn' t necessarily HAVE to be about hardware fitting.

I used to use Gentoo because I liked to turn on every feature of every package that I possibly could without causing conflicts. There were all sorts of features that could be en/disabled at compile time in various desktop environments and applications that the leading distros tended to turn off. They probably did it to reduce dependencies. I used to use Gentoo so that I could run a monster bloated system and have every bell and every whistle. At the time I didn't usually completely optimize for my hardware because I changed my hardware too often.

Now I go almost completely the other way. I know what features I use and don't. My hardware is old and doesn't get upgraded so often. Now my Gentoo is configured very lean so that my processor and RAM go towards the main functions I am working on and not so much the eye candy. It is optimized for my hardware but not because my hardware is new like you suggested! I'ts more because my hardware is old! Optimization makes my old computer work similar to a new one running normal pre-compiled software.

Gentoo is also great for dependency issues. There is no such thing as "ebuild hell". You almost never need to find exactly X version of libY to install an application because the application just gets compiled specifically against the libraries you have installed. That doesn' t mean much if you are normally just running a major distro and install everything from that distros package manager but if you ever try to install a program downloaded from the internet somewhere, not built for your specific version of your distro... it makes a HUGE difference! Things just work!

Actually... I change my answer.. counterintuitive as it seems, I use Gentoo because it Just Works!

Re:I run Gentoo

[morgauxo]

"I thought there were options to install pre-compiled versions of the packages."

Gentoo does support that although the repositories only have binaries for a few really big packages like web browsers and desktop managers that would otherwise take a long time to compile. Those are probably the things you benefit the most from compiling though.

You can set up third party repositories. I think there might be some pre-compiled ones out there. I think there is even a way to set up one build server that builds packages for all your other computers. I've never done this myself.

Re:I run Gentoo

[morgauxo]

I just run my updates in screen terminals and I have turned portage's nice level up a bit so I can keep doing other things. It finishes when it finishes, I don' t watch it. I don't have Chromium though and have pretty much gotten rid of everything that updates constantly and is big like that.

I do groan when I see libreoffice on the list though. But.. again, it runs in the background so no real time loss to me.

Re:I run Gentoo

[penix1]

I too use Gentoo and have since 2003. I started using it because it was the only one that had all my hardware working "out of the box" so to speak. Besides, it is a really good way to learn Linux under the hood.

Having said that, you can get some really messed up crap especially if you setup your use flags or compiler options wrong. What I do hate about Gentoo is the seemingly random masking of packages that knocks out other packages that are working just fine. Yes, you can unmask them but that is a PITA. So care needs to be taken when updating. Don't do it willy nilly neigh...

Re:I run Gentoo

[SlashdotOgre]

Pre-compiled binaries do exist as ebuilds in portage for some very large apps (i.e. libreoffice, firefox, seamonkey, etc.) however they are not very common (only ~100 ebuilds out of ~17K available on my laptop running unstable aka ~amd64) however there's another option called BINHOST that lets you take prebuild packages on one system and distribute just the binaries to other clients.

There are both public and private binhosts, however Gentoo doesn't officially provide any so you're somehwhat using them at your own risk. It's actually pretty easy to set up your own binhost, and if you are doing anything in scale, it's definitely the way to go (especially if you have standardizes hardware).

The big issue with using binhost, and at least part of the reason why it's not popular (and why you want standard hardware), is because you have to sacrafice optimizations to do so. Unless all the client systems have the same CPU, you have to go with the least common denominator when it comes to optimizations (aka CFLAGS). i.e. if one of your clients is a Core 2 Duo and doesn't support sse3 or newer, you can't build any packages with that CFLAG without risking broken packages on the C2D system. Additionally you have to sacrafice customization with binhosts as all your builds will have the same USE flags.

As both optimization and customization are both features that often attract people to Gentoo, the lack of binhosts and minimal formal binary builds makes a lot of sense.

Re:I run Gentoo

[morgauxo]

Potering? Is that you?

Re:What is the significance here?

[hey!]

File it under "stuff that matters".

A lot of arguments for open source are based on things which people outside the project could in principle accomplish, but in practice seldom do. So it's reassuring at least that an experienced developer can build the two most popular browsers from scratch. It means the arguments aren't hollow. I've seen closed source projects that were purchased by companies, only to find out that getting them to build on any computer but the one it was developed on is a serious engineering challenge.

That the process of building these browsers from scratch is somewhat arcane will come as no surprise to any experienced developer. But that it's not so arcane that it's impractical to figure out is good news.

Re:Qupzilla

[tibit]

Heck, the whole process to build Qt and its browser-du-jour (webkit then, chromium now) amounts to configure followed by make, and that's whether you're on Windows or Unix.

Re: So the STL:DR version...

[Anonymous Coward]

Let's hear from a frequent contributor first

Re:What is the significance here?

[recoiledsnake]

Is it just me or isn't there hardly an content in that blog-post? What is the significance?

The significance is that Open Source doesn't automatically mean not bloated. Just see Mozilla, Open/Libre Office etc.

Building should not be complex.

[jd]

There's software for auto-detection of necessary libraries (cmake is probably the best, since it's more portable than autoconf).

If you've the source tree, then you should require one single platform-dependent package containing cmake, gnu make, curl or wget, grep, cut and associated libraries, along with a text file containing a list of dependencies, where to get them and where to put them.

Your build system then scans for everything needed. If you've got it, it uses it. If you don't, it fetches the source, builds it and installs it.

This is not rocket science. Gentoo has been doing something similar for a very long time, so has Perl, so has Cygwin and Cygwin-based packages like OSGEO4W.

Yes, it's slow. Yes, it means the browser maintainer has to have a text editor. Yes, it's going to be as painful and agonizing as installing X11R4 or GateD. I did both. On a 386SX-16. Uphill. Both ways. In the snow. If you can't write your code properly to begin with, get off my lawn!

Re:Building should not be complex.

[Anonymous Coward]

If only someone would come along and write a new, unified make system. Perhaps "systemm" would be a nice name. As it would replace a lot of outdated and clunky UNIX tools, it could, with time, also replace all compilers (systemm.gcc, systemm.clang, etc), file system management (systemm.mv, systemm.cp, and systemm.rm) and date/time functions (systemm.time).

Re:Building should not be complex.

[abhi_beckert]

If only someone would come along and write a new, unified make system.

This has been done. A thousand times over.

And now we have a thousand unified make systems.

Re:Building should not be complex.

[peppepz]

(cmake is probably the best, since it's more portable than autoconf).

As a user of autoconfed packages, I find autoconf superior to cmake. Packages built with autoconf have standardized mechanisms for uninstallation (a cmake package may generate an install-manifest file, an uninstall target, or none of the two), to specify where to put documentation, for cross-compilation, and to fine-tune the build and the installation. With cmake, I can't even tell the package where to install libraries (most packages will allow you to do it, but each package has a different standard about the way to be told); with autoconf, I can even specify a sed to be run on the name of the installed binaries (useful if different packages provide different implementations of the same binary) and still have the installed package work. Also, with cmake packages building both static and dynamic libraries at the same time is usually impossible.

Moreover, modern autoconf scripts are (relatively) easy to debug and patch when they don't work; cmake scripts are more scattered and they're written in an obscure mainframish language.

That said, I imagine that using autoconf on non-posix systems might be less funny.

Re:Building should not be complex.

[menkhaura]

Great idea! And in time it could be seamlessly integrated with the init system, providing an accross-the-board synergetic user experience!

Re:So the STL:DR version...

[Lunix Nutcase]

Are you a frequent contributor? No? Okay, well shut the fuck up and let Bennett weigh in on this.

People who do this

[SampleFish]

There are some really great distros that are based on said sources:

Pale Moon is an open source alternative to FireFox:
http://www.palemoon.org/

SRWare Iron is a Chromium project:
http://www.srware.net/en/softw...

Re:People who do this

[Anonymous Coward]

SRWare Iron is not a Chromium project - it's a closed-source fork of the open-source Chromium codebase. I find it interesting that there are people who are concerned enough about privacy to switch away from Chrome/Chromium, but are still happy to use a closed-source browser which has had unknown modifications done to it.

http://www.insanitybit.com/201...

apt-get: bad command or file name

[tepples]

So let me open an elevated command prompt:

C:\>apt-get install visual-studio
"apt-get" is not an operable program [yadda yadda]
C:\>yum install visual-studio
"yum" is not an operable program [yadda yadda]

What's the command to download and install a desktop application package (in this case Visual Studio Express) from Microsoft's repository?

Re:apt-get: bad command or file name

[recoiledsnake]

C:\> choco install VisualStudioExpress2013WindowsDesktop

http://www.pcworld.com/article...

Re:apt-get: bad command or file name

[tepples]

Discussion continues here: Windows 10 Gets a Package Manager for the Command Line

Re:Well, no kidding

[ledow]

I have to say, as what people here would probably regard as only an amateur programmer, the setup of a build environment and initial compiles suck up way, way more time than they need to.

It's extremely frustrating. C is a wonderful language that I love to program in. There are compilers on every platform. There are cross-platform libraries for anything I need and my favourite IDE is cross-platform.

But actually getting basic shit to work is an absolute nightmare. And that's the stuff that I'm familiar with. It involves far too much pissing about with paths, switches, versions, system-level installations, scripts, extra options and all kinds of voodoo to make what I know works work how it should. And I often have to replicate it for every new project.

Fuck knows what it's like when you go multi-language, when you go to more obscure platforms, and when you're not familiar with things or am using someone else's code without the benefit of knowledge of their build environment.

I consider it the weak-point of open-source. Like science, one of the big pluses of open-source is reproducibility. You got working code? Good. Then so do I. All I need to do is make it compile. The problem is that "All" in that sentence.

There are languages that encapsulate more into the language but even that causes problems. I've fought with Java environments where the JRE and JDK differ and are in multiple versions, have architecture differences, and require explicit paths, and all kinds of nonsense. Python can be a fucking nightmare on Windows. Even shit like Cygwin interferes with ITSELF on Windows systems.

I don't know what the solution is, but I can't help but feel it's outside the language - it's in the system design and OS interaction. Operating systems just aren't designed with source being a big part of the end product.

Maybe the solution is one of the ideas I constantly toy with (and almost certainly has been done in the 60's on some archaic architecture) - an executable format that incorporates the source as well as the binary. When the executable moves to a new platform, the binary and source visibly differ and the binary can be recreated from the source. To do that requires complete encapsulation of all source, options, compilers and everything else into the executable.

But I'm sick at getting something as standardised as SDL working on the three big platforms (Windows, Mac, Linux) in using one of the largest open-source IDE's (Eclipse) utilising an industry-standard compiler (GCC). It should NOT be as hard as it is.

idiosyncratic is understatement

[Martin Spamer]

Idiosyncratic builds is not limited to just browsers and is probably the biggest problem faced by Open Source projects today.

I use open source tools daily and yet with 20 years development experience I have yet to fine one open source project that straightforward to build.

Re:idiosyncratic is understatement

[EETech1]

I've always been curious about nixos.

Has anyone ever used it?

http://nixos.org/

I used to build and use Firefox nightlies on OS X

[93 Escort Wagon]

Funny thing is, I don't actually remember why - other than there was some feature that wasn't available in the default build back then (Keychain integration, perhaps?)

Anyway, there were a number of libraries and other packages that had to be added to the system in order for the build process to succeed... but I don't remember it being all that difficult to do. Time consuming, yes - but not difficult.

Re:What is the significance here?

[ramorim]

OK. And how about built one from scratch?

Re:What is the significance here?

[Noah Haders]

That the process of building these browsers from scratch is somewhat arcane will come as no surprise to any experienced developer. But that it's not so arcane that it's impractical to figure out is good news.

it sounded very impractical. FTFS: "these are complex pieces of software, each of them with rather idiosyncratic build systems, and that you should consider 100GB of disk space to build all the browsers, a few hours of download, and be prepared to learn lots of new, rather specific tools."" I'd say it's good that open source browsers are not impossible to build. But this is no apt-get or whatever people say.

Lightweight Entry Missing

[pinkushun]

The Surf webkit browser source is 96kb (788kb with the full commit revision history).

Good luck with your bloat.
surf.suckless.org

Re:Lightweight Entry Missing

[messymerry]

Thanks for the comment about the Surf browser. I am really disillusioned about the current state of Internet Browsers in general. They have become tools of the marketing asshats and spyware in their own right. It's appalling what the industry is getting away with and just how far they are willing to go for a buck. It turns out nerds are not paragons of integrity...

Re:Lightweight Entry Missing

[descubes]

Interesting. Actually, the Qt web browser is small too (but Qt itself is big). Same thing here, it uses WebKit behind the scene.

Unfortunately, does not build on OSX:

surf.c:9:10: fatal error: 'gtk/gtk.h' file not found
  #include
                    ^
  1 error generated.

Re: Well, no kidding

[jd]

I absolutely agree.

The solution necessarily involves three mechanisms:

* Determining what is present
* Fetching what is absent
* Isolating everything that is build-specific

The "tradition" established by CPAN, CPyAN and CTAN is that source packages should specify dependencies - not only the software name, but the range of versions permitted. Archives should then permit requests for specific versions.

Isolation (such as by root jailing) deals with file path issues, software interactions, etc. All the build system should be able to see is the software the build system needs. Nothing more.

Determining what is present is more complex. The software must see what options exist that are compatible and then, through build flags, defaults and user queries, determine which of those options to actually use. If you're using Windows and have GnuWin32, Cygwin and MinGW installed, it's no good just asking if Gnu build tools are to be used.

Installs and deinstalls are platform-specific, it's best if generic package installers for each platform took care of the databases and links. If you can find any. Seems to be a shortage in useful tools.

Re:What is the significance here?

[jafac]

If there isn't a well-documented, repeatable build process, then it's not "engineering" AT ALL.

Not a Gentoo user here but...

[JoSch1337]

...last time I tried

        apt-get source --build iceweasel
        apt-get source --build chromium
        apt-get source --build webkitgtk

it worked perfectly fine - what's the issue?

Re:What is the significance here?

[descubes]

Well, as the author of the blog post, I did not intend this post to go on Slashdot. It was just a note-to-self about the various commands and build steps involved. Because they are, well, complicated.

In the grand scheme of things, the significance behind this is that I plan to open-source Tao, the 3D document description language invented by Taodyne, with the intent to be able to integrate it into at least one web browser. More details about why Tao would make a difference for the web here: http://www.taodyne.com/shop/de... (and in the follow up article). More details about how it works for a developer here: https://www.youtube.com/watch?....