Use x86 Boxes to Compile Mac OS X Binaries

IceFox writes "While working on the KDE on Darwin project I have only had one Mac to do development with. At the same time I have been playing around with distcc for Linux/x86 compiling. Combining the two projects I built a Mac OS X cross-compiler (for Linux/x86) and have created the DistccPPCKnoppix distribution. DistccPPCKnoppix is a 46MB Knoppix distribution based on distccKnoppix; with it you can use your extra x86 computers to build Linux/x86 or Mac OS X/PPC binaries. It might not be as shiny as an Xserve cluster, but it is a heck of a lot cheaper."

Comments

This is news?

[shamino0]

MacOS X has always used gcc as the main compiler. And GCC has always supported cross-compiling. I would be far more surprised if you couldn't compile Apple's version of GCC to cross-compile from Windows/Linux/Solaris/whatever.

I suppose it's worth noting that someone's distributing such a cross-compiler, but the most newsworthy aspect is that it took somebody this long to get around to doing it.

Re:This is news?

[Ranger Rick]

The reason it's news is that GNU has no support for Mach-O, Apple has their own assembler and linker. Making them work on linux is a real pain in the ass.

Re:This is news?

[torpor]

Not any more though, and I have to say that having this new distcc-centric Knopix distro around has turned it from being 'a pain in the ass' to 'unbelievably easy, cheap, and fast!' ...

oh dear. linux is unstoppable. you'd think that, by now, the point where linux is becoming 'The Development Environment of Choice' for compiling stuff for Other Operating Systems, well ... how can I put this: c'mon Microsoft, roll your own distro, for cryin' out loud!!!

Works in reverse, too...I think!

[cbiagini]

Well, I'd like to say that I think it's really cool that a Mac developer is so enthusiastic about his project that he's going this far to get the product out.

But check out this excerpt from gcc's man pages:

-arch arch
Compile for the specified target architecture arch. The allowable
values are i386 and ppc. Multiple options work, and direct the
compiler to produce ``fat'' binaries including object code for each
architecture specified with -arch. This option only works if
assembler and libraries are available for each architecture speci-
fied. (APPLE ONLY)

Aww man...all my moderation's going to be undone in this discussion now. Well, to the person who said something funny up there: Good job.

Re:Works in reverse, too...I think!

[cremes]

Specifying '-arch i386' on MacOS X doesn't work though. OSX doesn't ship with fat libraries, so it isn't possible to generate an x86 executable using the gcc switches. To make this work, you need to recompile several system frameworks and libraries and install them on your box. This is hairy. I saw instructions on how to do this about 2 years ago somewhere...

Compiling ppc on x86 or x86 on ppc is actually a bit easier using OpenDarwin. You can run this fat on your hardware and emit fat executables just as the man page suggests.

cr

Re:Works in reverse, too...I think!

[lpontiac]

pecifying '-arch i386' on MacOS X doesn't work though. OSX doesn't ship with fat libraries, so it isn't possible to generate an x86 executable using the gcc switches.

The Darwin 7.x installation CD you can get from Apple (also linked from opendarwin.org) does have fat binaries for everything, although MacOS X proper doesn't.

Well, a slight correction - everything in /usr/bin in the Darwin 7.x is either a script or a fat binary, with one sole exception - distcc. Methinks Apple doesn't like the idea of Mac developers buying a Mac for every desk, then a pile of cheap x86's to make builds ten times faster :)

fruits

[lethe1001]

It might not be as shiny as an Xserve cluster, but it is a heck of a lot cheaper.

Um, yeah, an Xserve is a piece of hardware, and a C-compiler is a piece of software. apples and oranges.

you could install distcc on your Xserve cluster, nnd it (the software) would still be cheap, and it (the hardware) would also be shiny.

Um, i guess the point is that if you want distributed compiling for MacOSX, heretofore the only option was Xserve cluster, and now linux/x86 cluster is also an option. ok ok....

Re:fruits

[Anonymous Coward]

apples and oranges

Actually the article was about apples and pcs.

Re:fruits

[geoffspear]

Xcode will let me do distributed compiling on whatever random Macs happen to be on my network, without buying an Xserve cluster.

Re:fruits

[AnonymousKev]

Actually the article was about apples and pcs.

So he should have said apples and lemons?

Re:fruits

[buckhead_buddy]

Caveats: Xcode's distributed compiling is fantastic. It was an unexpected gift and I know a number of people who have networks to make great use of it. But...

There is latency added to the compile process by having to distribute things out over the network. I think there was an Apple white paper or tech note that talked about these issues in more detail.

It needs a speedy network. Apple engineers recommend Gigabit Ethernet but 100baseT will due in a pinch. They don't see much speed savings over 10BaseT or Airport.

It needs fast processors. G5's are the recommended platform. Be wary of sub-Gigahertz G4's. Forget the G3's.

The distributed compilation feature is a GREAT addition to XCode, but I still see a place for this project among those who are really on a tight budget (like me) but who still want to support Macs when feasible.

Cool, I like this!

[Cecil]

This is great! I do all my development on my 12" Powerbook G4, since I prefer to code in (and for) Mac OS X. That is all good, but it certainly isn't the fastest thing in the world, and I dread having to compile something large like Tex or Mozilla on it. If I was able to get it to do distributed compiles on the numerous Athlons I've got sitting around, it would be awesome.

Re:Cool, I like this!

[gabebear]

I have an old P166Mhz laptop I use for with a Athlon XP 2100+ via distcc. It's ridiculous how much faster it can compile stuff.

I wonder how hard it will be to get distcc going inside Xcode?

Re:Cool, I like this!

[Ranger Rick]

XCode uses distcc for it's distributed compiles; the problem is it's an old version and the patch is no where near applying cleanly to modern distcc. Might be easier to just use something to advertise distcc on rendezvous manually.

How it should be

[MrIrwin]

Ferrari use Fiat vans to ship car parts around.

Mac is about the desktop, why not use cheap bland hardware for backend muling.

And that is how it should be, we should be able to select the type of hardware for the job in hand, the software should run wherever it is required.

Re:How it should be

[gsfprez]

>why not use cheap bland hardware for backend muling?

sometimes, you're a small business and you don't want to pay someone $xxxxxx to run your backend mules. You don't know how to grok Linux email and file servers because you're a parts manufacturer, or a solar panel business, or a gift basket maker...

With Mac OS X Server and a $300 iMac off eBay and prudent backups, you could run email for 40 people and have your own webserver off your home DSL with no problem.

I only know this because i did it. They know jack shit about running computers. I did the scary part and installed the server for them (i hit "okay" a lot) and then showed them...

see this window? This is where you add users... this window shows you that your email server is running just fine.

They call me every so often to tell me that things are fine. And i get to not answer dumbass questions and watch the playoffs on ESPN.

They own their IT... their IT does not own them.

With Linux, if you're going to have geeks around, then yes, its cheaper and does everything i just mentioned at a MUCH lower cost. But if you're not a geek, and you want services like email and web servers for small business, and you want rock solid performance (not Windows shitty performance) then you're wanting Mac OS X Server.

Mac OS X Server is the greatest boon to small business i've ever seen. I have helped 4 small businesses (friends) do this - and they are all running just fine without me - and they have all the stability that they would get from a competent Linux install and a in-house geek running it.

Mac OS X Server can get a lot of low-ball geeks fired... OR it can help low-ball geeks make a lot of money serving a lot more customers. Its all in how you think about it.

Re:How it should be

[Lars T.]

Too bad I can't afford the admission cost. Then you have no place in running a business. "With Mac OS X Server and a $300 iMac off eBay"

One more thing...

[b1t r0t]

Now all we need is for it to be compatible with Xcode.

Re:One more thing...

[IceFox]

Probably the same way that I do it under Linux. I alias make to make -j 32 in my bash init file and in the top of my path I have gcc g++ etc all point to distcc (just like the distcc man page gives as an example). So Xcode would just be calling make (really make -j 32) which calls g++-3.3 (really distcc). -Benjamin Meyer

Re:One more thing...

[snowtigger]

It's almost compatible already. XCode uses standard GNU gcc, g++, etc. for compiling.

In XCode, the equivalent of a Makefile is called project.pbxproj (it's a text file) and the equivalent of make is a command called pbxbuild.

So everything you need to do is to create the utility that parses project.pbxproj and compiles.

Of course, make with a Makefile works perfectly fine, but XCode doesn't create those files for you.

Re: One more thing...

[aqsalter]

And one more thing...

PBTOMAKE -- Xcode .pbproj to Unix Makefile

??

Why not, I hear you ask??

Re:One more thing...

[douthat]

I beleive he was talking about making it compatible with XCode's built-in "distributed build", which works automatically via Rendevous and doesn't require any mucking around with command lines. (i.e. Just press the "Build" button)

From Apple

Distributed build Anyone could guess that bringing multiple processors to bear on a build would make it go much faster, but Xcode lets you act on the obvious solution. With the Rendezvous-enabled distributed build feature it's easy to simply farm out your build by distributing compile workload across idle desktop machines or, better, deploy a dedicated Xserve build farm to do in minutes what would take hours on any single machine.

The iso download

[IceFox]

Hehe I learned my lesson before. I'll be putting the 46MB iso up later tonight, but if someone wants to host it you can e-mail me or im (aim: icefox2) me and I can get you the file.

-Benjamin Meyer

Re:Yessss

[NivenHuH]

You would think more people would love to work with Objective-C.. it really is a nice language =)

Apple GCC vs. GNU GCC

[WatertonMan]

I thought Apple's gcc was slightly different than the regular gcc - especially in terms of PPC optimizations. If true, then the code output won't be quite the same, will it?

Re:Apple GCC vs. GNU GCC

[tyrione]

That changes with the very recent release of GCC 3.4 and moreso when GCC 3.5 is released.

Re:Apple GCC vs. GNU GCC

[TomorrowPlusX]

Any chance apple's Objective-C++ patches will *ever* be integrated into mainstream GCC?

I simply could *not* get my development work done ( at least, not gracefully ) without Objective-C++, and that's a bit hampering considering I'm a believer in The Right Tool for the Right Job.

Some things are better written in C++ and some things are better written in Objective-C. Sometimes both will be needed for one project, and if i have to write vanilla C glue between them I'll be rather unhappy. Sure, C glue-code works, but you tend to lose polymorphism, and you end up with a bunch of typedef'd pointer types, which while acceptable, give me vietnam-style flashbacks to GTK programming, and I don't like that.

I love developing on Mac OS X, mainly because I love Cocoa. I'd love to be able to port my work to GNU Step, but without Objective-C++ it's simply not going to happen.

Nice job

[-tji]

This is a cool concept. I would think that most Mac users have at least one fast/cheap PC laying around. I know I've got a few of them. You might as well use them for some extra horsepower.

What would be really cool is an x86 agent for iMovie, iDVD, and Final Cut - where it could offload all of the number frunching of MPEG2 compression or other video conversion to some cheap PC's in the basement. I would love to have my 450MHz G4 Cube as the front end, and my Athlon 64 doing the heavy lifting behind the scenes.

Sparc...

[Roogna]

Now when is someone going to post binaries for cross-compiling to Mach-O PPC from Sparc's. I've got a bunch of Sun's here I could be using. Granted, I know I could do this myself. But gcc takes so bloody long to compile ;)