Facebook Rewrites PHP Runtime For Speed

VonGuard writes "Facebook has gotten fed up with the speed of PHP. The company has been working on a skunkworks project to rewrite the PHP runtime, and on Tuesday of this week, they will be announcing the availability of their new PHP runtime as an open source project. The rumor around this began last week when the Facebook team invited some of the core PHP contributors to their campus to discuss some new open source project. I've written up everything I know about this story on the SD Times Blog."

Comments

is this being used now?

[Anonymous Coward]

Is this what they're using on the newly redesigned site? Because if so, it's pathetically slow. Facebook is one of those places that with every attempt to "improve" things somehow manages to make it worse and worse. They're a perfect candidate for a Microsoft buyout.

Re:is this being used now?

[Daengbo]

Try the "Lite" version. It's much faster, and doesn't have that annoying chat bar.

Re:is this being used now?

[Anonymous Coward]

Thanks for the tip. It looks like there are some big things that it's missing, but it works well for the basics. For those trying it out, there's a setting that puts a link/bar at the top of the window that lets you easily switch between the lite and full versions at any time.

Re:is this being used now?

[amn108]

Microsoft isn't exactly known for speed either.

Re:is this being used now?

[Anonymous Coward]

Microsoft isn't exactly known for speed either.

Context, buddy. What I'm saying is that Facebook tried to improve it, but ended up making it slower. Microsoft does the same thing a lot of them. Ergo, Facebook makes a good candidate for a Microsoft buyout because the two are very similar in that regards. Got it?

Re:is this being used now?

[ehrichweiss]

I have to agree fully here. The lite version is missing some things that I like but I can live without them for the speed tradeoff. Only problem I have is that the toolbar for Firefox only points at the main site, otherwise it keeps the bloat way down.

Re:is this being used now?

[amn108]

:'-(

Re:is this being used now?

[rel4x]

Facebook has a few problems. Overuse of ajax combined with this absolutely bizarre habit of including dynamic javascript at random points in the script. These lead to slower runtimes, especially with older browsers where (upon encountering a JS file) they completely stop doing everything else to execute it.

Re:is this being used now?

[pvera]

After using it for a few minutes I got a link to a preference that makes lite the default site. And you have the option to keep a bar on top of the screen that lets you toggle between the two sites.

Re:is this being used now?

[scumm]

While I agree in general, we're talking about a PHP interpreter/compiler(maybe) here, so the closest comparison would be MS's language offerings, which are actually rather good. MSVC has considerably better optimization than GCC, for instance.

Re:is this being used now?

[ehrichweiss]

Thanks. I just found that and am now using it. This makes the site that much more bearable.

Re:is this being used now?

[Anonymous Coward]

There doesn't seem to be a way to confirm/request new friendships. Did they think this feature would be superfluous to anyone with knowledge of the "lite" version?

Re:is this being used now?

[DrGamez]

*whoosh*

Re:is this being used now?

[that this is not und]

MSVC runs dirt slow on my sparc hardware running NetBSD, however. When you target a single arch. it's pretty easy to optimize for it.

Re:is this being used now?

[Anonymous Coward]

Your being very cynical in my opinion.
I congratulate facebook for its speed despite the pressure its servers are on/

Re:is this being used now?

[Anonymous Coward]

especially with older browsers where (upon encountering a JS file) they completely stop doing everything else to execute it

It could be worse... your unpatched old insecure browser could also be getting you infected with malware, moron.

You’re complaining about something that’s the result of your own stupidity.

Re:is this being used now?

[dino213b]

Actually, they are. The new PHP contribution is NOT efficiency-related. Instead, they built privacy controls into PHP. However, the problem is that by default it's set to open.

PHP versus

[Anonymous Coward]

OK, so can we put the "PHP is comparable to JSP and ASP.NET" to rest now?

Screw PHP, I write everything in C

[Anonymous Coward]

PHP is for lazy developers. I develop my webapps in C and I even wrote my own httpd to improve performance.

Re:Screw PHP, I write everything in C

[biryokumaru]

C is for lazy developers. I develop my webapps in JWASM and I even wrote my own httpd to improve performance.

Re:Screw PHP, I write everything in C

[eclectro]

I develop my webapps in C and I even wrote my own httpd to improve performance.

C is for lazy coders if you ask me. I hand code and tune assembly language programs on an altair 8800 flipping switches on the front panel. As all real programmers should do.

Re:Screw PHP, I write everything in C

[sopssa]

JWASM is for lazy developers. I develop my webapps in machine code and I even wrote my own internet to improve performance.

Re:Screw PHP, I write everything in C

[TheRaven64]

Meh. I write my code in Smalltalk and if it's not fast enough then I hack on the compiler and runtime until it is.

Re:Screw PHP, I write everything in C

[aBaldrich]

Programmable chips are for lazy developers. I even invented a new architechture to improve performance.

Re:Screw PHP, I write everything in C

[biryokumaru]

Servers are for lazy developers. I develop my webapps in my head and I even deliver the pages manually to improve performance.

Re:Screw PHP, I write everything in C

[deniable]

Excuse me, I'm still waiting for the last page and it's almost bed time.

Re:Screw PHP, I write everything in C

[AikonMGB]

I don't know why this has not yet been linked, but... obligatory XKCD reference

Aikon-

Re:Screw PHP, I write everything in C

[Anonymous Coward]

Screw you, I invented time to allow you to wait for increased performance.

Re:Screw PHP, I write everything in C

[Anonymous Coward]

Coding is for lazy people. I create my programs with ICs.

Re:Screw PHP, I write everything in C

[Anonymous Coward]

Webapps are for lazy developers. I sip my coffee, causing a multiply entangled photon to collapse and resolve the location of countless electrons throughout the universe, spurring various exotic species of butterflies to flap their wings a twitch differently, which in turn subtly alters the flow of the viscid gaseous matter in Earth and various other planets, affecting the touch organs of living matter that can feel moving fluid, which messenger or nervous impulses relay to their minds, creating customer-tailored web content for my clients.

Re:Screw PHP, I write everything in C

[AmberBlackCat]

Every time I telnet to your brain, I keep getting something about Chuck E Cheese...

Re:Screw PHP, I write everything in C

[Galactic Dominator]

I don't know why this has not yet been linked

Just taking a shot in the dark here, but I'll attempt an answer. The reason no one else linked to it is because you're the only one who considers it obligatory. Slashdot regulars will know that this type of thread happens on a near daily basis and with all due respect to xkcd there is simply no need to make another tired attempt at karma whoring.

Re:Screw PHP, I write everything in C

[BRSQUIRRL]

Steve Gibson, is that you? :)

Re:Screw PHP, I write everything in C

[noidentity]

Software is for lazy developers. I develop my webapps in hardware with Verilog and I even wrote my own httpd to improve performance.

Re:Screw PHP, I write everything in C

[jopsen]

PHP is for lazy developers. I develop my webapps in C and I even wrote my own httpd to improve performance.

I've actually done that...a few years ago... With on a chip 1024 byte ram... And I wrote my own fairly hacked tcp/ip stack!!! :)
FYI, this was a web 2.0 app, the "index.htm" page included a javascript file from a server on the internet, that using document.write() printed html and included images... Apart from "index.htm" page the chip only served to AJAX requests :)

Re:Screw PHP, I write everything in C

[rpetre]

I'd like to point out that long before xkcd there was userfriendly, and that in my circle we still like to and this sort of joke by saying "magnets" and giggle. The "Edward Lorenz, the butterfly and the chaos theory" punchline seems a bit forced (unless you go for the 'M-x butterfly' twist to make the emacs guy get the attention ;) )

Re:Screw PHP, I write everything in C

[S.O.B.]

Nice try. I rewrote the universe to include php and httpd in the kernel.

Reboot in 3...2...

Re:Screw PHP, I write everything in C

[mr_lizard13]

mr_lizard13 likes this

Re:Screw PHP, I write everything in C

[mldi]

I don't know why this has not yet been linked

Just taking a shot in the dark here, but I'll attempt an answer. The reason no one else linked to it is because you're the only one who considers it obligatory. Slashdot regulars will know that this type of thread happens on a near daily basis and with all due respect to xkcd there is simply no need to make another tired attempt at karma whoring.

Harsh. First time I saw that XKCD toon, so I kindly thank grandparent for their "karma whoring".

Re:Screw PHP, I write everything in C

[Anonymous Coward]

And I modded you all down.

Re:Screw PHP, I write everything in C

[troll8901]

You win.

Re:Screw PHP, I write everything in C

[Rufty]

You've got an httpd? Why not static link in the server code and the TCP/IP stack, then you can construct the page straight to the packet buffers...

Re:Screw PHP, I write everything in C

[fuzzix]

I'd like to point out that long before xkcd there was userfriendly, and that in my circle we still like to and this sort of joke by saying "magnets" and giggle. The "Edward Lorenz, the butterfly and the chaos theory" punchline seems a bit forced (unless you go for the 'M-x butterfly' twist to make the emacs guy get the attention ;) )

XKCD is occasionally amusing, UF never was.

Also, inodes? They're talking about DOS... sheesh!

Re:Screw PHP, I write everything in C

[K. S. Kyosuke]

I even invented a new architechture to improve performance.

Chuck Moore, is that you?

Re:Screw PHP, I write everything in C

[Tumbleweed]

Machine code is for lazy developers. I develop my tubeapps on paper and I even built my own system of tubes to improve performance.

Re:Screw PHP, I write everything in C

[Tumbleweed]

I'd like to point out that long before xkcd there was userfriendly, and that in my circle we still like to and this sort of joke by saying "magnets" and giggle.

The best part of that comic is the Amiga calendar on the wall behind them. :)

Re:Screw PHP, I write everything in C

[aylons]

Computers are for lazy developers. I project ASICs for my webapps and I even built my own modem to improve perfomance.

Re:Screw PHP, I write everything in C

[bytesex]

So long as you can balance the blocking waits on your solid state calls with the sending of a TCP/IP data packet, that must be doable. I like to have an OS for that, though.

Re:Screw PHP, I write everything in C

[quickOnTheUptake]

you must be *REALLY* new here

Re:Screw PHP, I write everything in C

[DMUTPeregrine]

Assembler is for lazy coders if you ask me. I do all my coding in Brainfuck or Malbolge.

Re:Screw PHP, I write everything in C

[jc79]

Doesn't emacs have a command for that?

Re:Screw PHP, I write everything in C

[Anonymous Coward]

Switches are for lazy coders if you ask me. Real programmers use butterflies.

Re:Screw PHP, I write everything in C

[jc79]

Goddammit, 6 hours late to the party. Note to self - scroll down before posting blatantly obvious link.

Re:Screw PHP, I write everything in C

[BikeHelmet]

Machine code is for smart x86 developers. I'm stupid, so I develop my webapps in Java. I got a 20x performance boost over PHP!

Re:Screw PHP, I write everything in C

[Anonymous Coward]

C is for lazy coders if you ask me. I hand code and tune assembly language programs on an altair 8800 flipping switches on the front panel. As all real programmers should do.

I've done that. It was incredibly cool.

Given the choice, I'll never do it again. (except maybe to show off my leet old skool skillz...)

Re:Screw PHP, I write everything in C

[anerdburger]

Flipping Switches is for lazy programmers, I hand code then wire wrap my code and implement in hardware as all real programmers should do. (It really maximizes performance as well !!!)

Re:Screw PHP, I write everything in C

[mkosmul]

Actually programming in your head is for lazy developers. I didn't write anything, I only proved that my algorithm is correct.

Re:Screw PHP, I write everything in C

[that this is not und]

You're hitting the firewall. That's the decoy page you're seeing.

Telnet is blocked in inetd.conf, you'll have to ssh.

Re:Screw PHP, I write everything in C

[that this is not und]

Also, inodes? They're talking about DOS... sheesh!

We're impressed that you figured that out by the third frame, seeing as there was blatant mention of DOS in the first frame and DOS references were in all three frames.

Re:Screw PHP, I write everything in C

[supernova_hq]

Too late, I wrote it as a module and modded it in during runtime.

Re:Screw PHP, I write everything in C

[shutdown -p now]

Nice try. I rewrote the universe to include php ... in the kernel.

That explains a lot about the universe...

Re:Screw PHP, I write everything in C

[RichiH]

Machine code is for lazy developers. I develop my webapps in VHDL and I even wrote my own Layer 2 protocols to improve performance.

Re:Screw PHP, I write everything in C

[xouumalperxe]

We're impressed that you figured that out by the third frame, seeing as there was blatant mention of DOS in the first frame and DOS references were in all three frames.

We're unimpressed that you didn't realise that inodes are a feature of UNIX-like filesystems, and are not present in the FAT filesystem. Please hand in your geek card.

Re:Screw PHP, I write everything in C

[LanMan04]

Hey, writing a web server is pretty freaking easy. Here's one I did in Java for grad school. Copy away!

--------------------

import java.io.*;
import java.net.*;
import java.lang.*;
import java.util.*;

public class HttpServer
{

static String proto = ""; //should be HTTP/
static String errorMsg = ""; //output to terminal
static boolean errorFlag = false; //error flag
static String origUri = ""; //orig uri2
static String uri2 = ""; //uri2
static String fileType = ""; //file type
static String contentType = ""; //content header stuff
static String headerMessage = ""; //header message
static String errorHTML = ""; //error HTML page
static int errorNum = -99; //error code (404, 500, etc)
static String errorMessage2 = ""; //File not found, etc
static int portNum = -99; //port num to run on
static long fileLength = -99; //file length in Bytes

public static void main (String args[]) {

String versionStr=""; //the full HTTP version string
String versionNum=""; //version number
String frontTok=""; //first token in line
String method=""; //HEAD or GET
String responseHead=""; //response header
String statusMessage=""; //status msg for terminal
String debugMessage=""; //echo of request line
boolean hostMode=false; //make sure only one Host: line
boolean gotMethod=false; //make sure only one method is called

try
{
//check if there's a portNum argument, if not, quit
if (args.length > 0)
{
portNum = Integer.parseInt(args[0]);
}
else
{
System.out.println("Usage: java httpServe port number");
System.exit(1);
}

//time to create our server socket
ServerSocket serverSocket = new ServerSocket(portNum);

//infinite loop, waiting for connections
while(true)
{
System.out.println("Waiting for connection on port "+ portNum);
Socket client = serverSocket.accept();

BufferedReader httpIn = new BufferedReader (new InputStreamReader(client.getInputStream()));
PrintStream httpOut = new PrintStream(client.getOutputStream());

hostMode=false;
gotMethod=false;
errorFlag=false;
method="";
debugMessage="";
statusMessage="";
errorMsg="";
errorNum=200;

Re:Screw PHP, I write everything in C

[TemporalBeing]

Hey, writing a web server is pretty freaking easy. Here's one I did in Java for grad school. Copy away!
<snip>

Sorry - it took down the network on the first page load. Please use a better performing language.

Re:Screw PHP, I write everything in C

[LanMan04]

Heh, I agree. I was required to be written in Java. :)

Re:Screw PHP, I write everything in C

[clone53421]

It’s not all bad... you were so late to the party that you didn’t even get the redundant mod.

Re:Screw PHP, I write everything in C

[Anonymous Coward]

Slashdot regulars will know

that +1 Funny doesn't give any karma?

Re:Screw PHP, I write everything in C

[mkarcher]

So that's how the Avatar bionet worked...

High performance in scripting languages?

[BadAnalogyGuy]

At some point, if you are lucky enough, you will require extremely high performance from your web pages. You start out coding HTML in Notepad and move on to Perl CGI then on to PHP with scripting embedded right in the generated HTML. All the time you gain programming crutches at the expense of processing speed, and for a while this is a great tradeoff.

But one day you start having server hiccups because your scripts can't keep up with your traffic. Sites like Amazon have already run into this and have moved away from scripting languages and back to system languages. Running applications directly on the CPU instead of relying on a runtime to translate (at best) bytecode into machine instructions means maximizing CPU cycles.

So I wonder what longterm benefit there is in improving the language runtime.

Re:High performance in scripting languages?

[sakdoctor]

1. Static HTML
2. PHP
3. ???
4. Rewrite the PHP runtime

Truth is, that step 3 involves a whole load of steps where 90% of the problem will be database bound. Complied languages are not going to the magic solution in a real world situation.

Re:High performance in scripting languages?

[Anonymous Coward]

It's not just websites. I don't know what the fascination is with scripting languages on the Linux platform or with FOSS in general, but it results in slow programs with flaky UIs.

I like to use refurbished/recycled machines; which means that I'll have an old P4, 512M RAM and a slow bus. Many times, applications written in a scripting language, whether it be Perl, Python, PHP, or whatever, will hang often and then start working. I can always tell. That's why I knew Open Office was written in something like C when others on the Net were stating it was written in some such interpreted language.

Re:High performance in scripting languages?

[sopssa]

Exactly, and it's not like there is so much heavy processing cpu wise. Facebook probably has calculated that they can get enough performance out of recoding the runtime (even 1% is large enough for site as large as facebook). While doing that they also create a faster runtime that everyone can use. Everyone moving to write their sites in C/C++ doesn't make any sense.

Also a lot of the site structure can be cached in memcache or accelerating proxies like squid, so you aren't actually interpreting PHP code lots of the time. Facebook also did a lot of work towards memcache, because they are mostly a DB heavy site, not CPU.

Re:High performance in scripting languages?

[Anonymous Coward]

Not true. When you start using memcache (which FB heavily relies on), the latency of data retrieval drops down below the latency to execute PHP opcode.

PHP has a big problem, it executes scripts from scratch on every request. Every request has to load over and over again same configuration data, even if it comes from memcached -- there is TCP latency involved.

Applications that use regex-based URL mapping to controllers suffer heavily in PHP because on each request the interpreter has to compile the regex and then do the matching.

Instead of preparing all the app specific configuration once upon startup, and use that as process-local data.

Re:High performance in scripting languages?

[Anonymous Coward]

Lol wut? You must be a computer science genius!

I like to use refurbished/recycled machines; which means that I'll have an old P4, 512M RAM and a slow bus.

So its obvious you don't do any of this professionally then?

Re:High performance in scripting languages?

[gbjbaanb]

Complied languages are not going to the magic solution in a real world situation.

whilst that's perfectly true, its only true to a point. Lots of people run eaccelerator or apc on their PHP sites, simply to improve performance. If these pre-compile caches didn't do anything for performance then you'd be totally right, but as they do, you've got to appreciate that replacing the PHP with a compiled language will make a significant difference.

as always, don't guess where performance problems live, measure them. Often you'll be surprised, especially as load increases.

Re:High performance in scripting languages?

[amn108]

Check your facts better next time please.

PHP does not have to execute scripts from scratch on every request. APC cache API transparently caches JIT-compiled PHP script intepretations and these are run instead upon request.

Apache does not have to compile regular expression for mapping each URL request, when you specify RewriteRule directives in virtual server or server context it compiles them (or however else it wishes to cache these) on server startup and that is it. During the entire lifetime of the server, the specified rules are no longer interpreted.

Other than that I agree that the current style of server infrastructure we are "enjoying" can be improved at least 100-fold, database access including.

* Truly persistent (across HTTP requests) user memory is a good step in the right way - extend the lifetime of all scripted objects to persist across entire application lifetime (i.e. forever - servers are not supposed to die)

* Many database requests are so primitive that these could bypass TCP socket layer and benefit from extra speed at the cost of no longer being asynchronious. Most PHP developers use blocking database query APIs anyway, even though non-blocking calls are available.

* Compile scripts and run from compiled cache, preferrably as machine code. Think about environment - all those quad-core datacenters wasting cycles, because programmers are supposedly very expensive. Well, they are not that expensive that when a good team get together they cannot re-think this. They can. If nothing, they should worry about the environment too - it is not cheap.

* Offer asynchronious calls where these make sense (i.e. where they can benefit from hardware parallelization). Those devs who know how to make use of them will be happy to do speed up their applications.

In short, cache everything, whenever you can. Memory is cheap. Cache SQL query strings, cache script compilation output, cache, cache, cache.

Re:High performance in scripting languages?

[amn108]

You are however, dead on on the loading application configuration (including libraries!) once as process local memory, the way "normal" client software functions.

Re:High performance in scripting languages?

[Anonymous Coward]

You should check your facts first.

I did not say PHP had to PARSE each script per request -- which would happen without an opcode cacher. It has to EXECUTE the script, opcode cached or otherwise, from scratch on each request.

So there is no way for PHP to hold application data in memory between requests, except by using shared memory or a memcache. Other languages like Python, Java, etc... allow you to instantiate your classes and their data upon startup, and then call methods per request, without having to instantiate all the classes over and over again in each request.

Of course, sometimes you have to, but PHP does not even allow you to pre-instantiate those classes that do not have to.

So on each request, the application has to load its configuration, even if it is stored as PHP array in some config.inc.php. It has to re-evaluate the arrays (construct the array, build hash keys, etc...) even when opcode cached.

Granted, certain important extensions do keep pools of resource handlers between requests, like PDO, memcached, etc...

Also, I did not mention Apache. I said (PHP) applications that map URL patterns to controllers -- aka central index.php that patches URL to controllers and their methods. That is different from Apache rewriting that maps URL patterns to PHP scripts.

Re:High performance in scripting languages?

[Anonymous Coward]

1. Static HTML
2. PHP
3. ???
4. Rewrite the PHP runtime

Truth is, that step 3 involves a whole load of steps where 90% of the problem will be database bound. Complied languages are not going to the magic solution in a real world situation.

I think that It depends upon where the bottlenecks are. Traffic that is reads only can be very parallel and shouldn't be "database" bound. There are ways to get very high performance (in terms of transaction throughput) for write traffic as well. It sounds to me like they want to get rid of an obscene number of PHP servers. This would be profit (cooling, power, hardware, licenses).

Those interested may wish to look at all the previous work on compilers (e.g. hotspot java), or carve out another solution by identifying and coding the heavy weight transactions in a more efficient way altogether.

Re:High performance in scripting languages?

[Lennie]

Facebook added to memcache the ability to use UDP instead of TCP. They also changed MySQL so one replication-command from one datacenter to the next would also invalidate what is in memcache on that location.

At some point they have so much traffic from their webservers to their backendsystems, they saturated their internal network and were dropping UDP.

That's the kind of problems/scale they deal with, I'm surprised PHP wasn't their biggest bottleneck before (they did some work on PHP already, but not something like this).

After all Facebook is the second site after www.google.com-search page (which handles 'just' one task) and Google has pretty much a custom-build platform.

Re:High performance in scripting languages?

[Anonymous Coward]

That's the kind of problems/scale they deal with, I'm surprised PHP wasn't their biggest bottleneck before (they did some work on PHP already, but not something like this).

It is easy to add new front end servers, they all connect to the same pool of backend resources. Backend coordination is the most fragile point in the life of a request. You can't just add backends, which they realized with memcached, adding more did not help.

However, if they used a faster language, which is the point of TFA, they would have to use LESS front end servers and with that spare the money.

Because, the backend is already working as fast as possible -- in terms of programming language used -- native on CPU. I guess the backend could get faster only if they reimplemented MySQL and memcached, taking out the functionality that is not required, or adding critical new functionality. Oh, wait... they did.

So that leaves removing PHP from the chain.

Re:High performance in scripting languages?

[Anonymous Coward]

I'm probably just a "clueless noob", but wouldn't it make sense to do something along the lines of pre-compiling included php scripts in the way that some SQL databases do with stored procedures? So you get the benefit of the (apparently) productive PHP language, but can also reap some of the benefit of moving away from JIT compiling to a full compiler for some of the more used scripts?

Re:High performance in scripting languages?

[Anonymous Coward]

Who in their right mind codes in notepad when gvim is free and just as lightweight? Do you enjoy pain or something?

Re:High performance in scripting languages?

[CAIMLAS]

My understanding is that one of the major problems with PHP for something like facebook - as opposed to a language like Perl or Python - is that unlike other languages, PHP does not manage SQL database session connections. This results in those professional "oops, we fucked up" messages, timeouts, and other fun stuff like that which wouldn't happen (as regularly) if they'd designed things well.

So if they're fixing that in their PHP rewrite, I'd say they're fixing half the problem they've got. The other half of the problem - slowness - can be gradually fixed by reverting parts of their software to system-level (C) programming, but the database issues need to be fixed ASAP.

Re:High performance in scripting languages?

[FlyingBishop]

If you take another look at the article, an update suggests that's exactly what Facebook has done.

Re:High performance in scripting languages?

[elnyka]

It's not just websites. I don't know what the fascination is with scripting languages on the Linux platform or with FOSS in general, but it results in slow programs with flaky UIs.

Slow programs with flaky UIs can be obtained with any languages - just a matter of looking at the graphical C/C++ applications written in the 90s will show that this is true. The reasons behind whether an application runs well enough are varied and complex and cannot be pigeonholed into simple "it's the language" explanations. There are architectural issues that many people, scripting languages or not, do not take into account which makes systems flop.

I like to use refurbished/recycled machines; which means that I'll have an old P4, 512M RAM and a slow bus.

What kind of applications, how many users and what type of usage patterns are we talking about when referring to this type of platform? What type of application/system architecture are we talking about here?

Many times, applications written in a scripting language, whether it be Perl, Python, PHP, or whatever, will hang often and then start working. I can always tell.

Can you quantify that statement? I've seen applications written in Java or C/C++ doing the same. Any software engineer worth his salt knows that erratic/subpar performance behavior is not just a function of programming language.

That's why I knew Open Office was written in something like C when others on the Net were stating it was written in some such interpreted language.

So that's how you knew it was written in C? By eye-balling its performance? I would have thought that looking at the developers' documentation and source code would have been a much more viable way to measure it.

Anecdotes as form of evidence win again.

Re:High performance in scripting languages?

[whydna]

Sites like Amazon have already run into this and have moved away from scripting languages and back to system languages

You know Amazon's running on Perl Mason, right? http://www.masonhq.com/?AmazonDotCom ;-P

Re:High performance in scripting languages?

[thetoadwarrior]

I'm sure a lot of intensive stuff is done in a system language but Amazon still uses Perl. Google use Perl and Python through their sites.

There's no need to to use a system language for everything. Facebook is probably using PHP on its own and that's just not wise for a site like that.

Re:High performance in scripting languages?

[thetoadwarrior]

P4 CPUs start at 1.3 ghz. I think my machine is probably similar to yours aside from having 1 gig of ram. I've not have any issues and I rather have loads of options as opposed to a handful of finely tuned options. Like websites, doing the heavy lifting in C and some of the lightweight stuff in Python is acceptable and speeds up development quite a bit.

Re:High performance in scripting languages?

[Eil]

What you call a crutch, most developers call an enormous time saver. The web moves so fast that you simply cannot afford to take forever developing it just so that the code executes efficiently. Sure, PHP, Perl, Python, and Ruby are slower than C or C++. But for at least the last decade, hardware has been cheap enough that it makes a lot more business sense to just throw more servers at the problem than it does to delay your product launch for a year and/or double your programming staff while you make everything "perfect" in a lower-level language. Most of the problems around making web apps scalable to millions of concurrent users have been solved or will be in the near future. (CDNs, memcache, load balancers, etc.) When you find bottlenecks, you rewrite those specific parts using a better design or a lower-level language. If your developers are any good, they will have modularized the code, making such upgrades relatively painless. Trying to optimize the entire codebase for performance before it's even out in the wild ensures that it will never get there.

Re:High performance in scripting languages?

[Anonymous Coward]

You start out coding HTML in Notepad and move on to Perl CGI then on to PHP with scripting embedded right in the generated HTML

For web apps, I went from REXX to perl to PHP and now, BACK to perl.

PHP is great for templates, when combined with CVS and a filesystem, it's pretty much a self contained content manager. It's great for producing websites.

REXX was horrible for web apps, by the way :-)

But it is NOT even 1/2 as good as perl for web applications. (I've used both for years now..) if you're serious, you should consider perl or even Java, but never PHP. (perl + FastCGI if you're disciplined and experienced enough not to shoot yourself in the foot with it, java if you'll be having a team of programmers and can spare the CPU resources) Even python or ruby.. heck, ANYTHING but PHP.

PHP was initially designed as a quick and dirty template system, later retro-fitted with security hacks and limitations appealing to ISP's of cheap hosting packages... and it really shows.

I know this runs counter to the current trends, people seem to think PHP is a step above the others, mainly because everyone else is using it.

Everyone else uses it because the McDonald's of web hosting providers love it. (it is easier for an ISP to impose limitations on PHP)

The fact that facebook would even consider PHP makes me question how experienced they were when they started the project, perhaps the managers making the decisions had no real experience in coding? (I see that a lot as a freelancer, people gauge quality based on popularity and trends)

Re:High performance in scripting languages?

[Anonymous Coward]

Gosh, no. Unless the code is written in a way which is very difficult to optimize (ex. a variable's type changes a lot unpredictably; lots of unpredictable lookups by name), high-level languages should be faster than C/C++ when run through a proper optimizer. Facebook is doing the right thing by trying to make the language implementation faster which saves everyone from having to re-write code in lower-level languages, and for well-written code, it could eventually reach the point where only carefully hand-optimized C would beat the PHP code's performance. High-level languages just have a bad reputation because all of the current implementations suck.

Re:High performance in scripting languages?

[MichaelSmith]

Facebook added to memcache the ability to use UDP instead of TCP. They also changed MySQL so one replication-command from one datacenter to the next would also invalidate what is in memcache on that location.

At some point they have so much traffic from their webservers to their backendsystems, they saturated their internal network and were dropping UDP.

Now thats just dumb. We send trace information by UDP but we accept that some of it will get lost under high load. It is a way of shedding load in fact. If you send business data over UDP you can't act surprised when it winds up in the bit bucket.

Re:High performance in scripting languages?

[Lennie]

Their are not sending business data.

Have a look at some of their presentations, then you know what I'm talking about.

The scale they are at, they have a lot of machines with memcached running. And PHP has to connect to them, it has to connect to many and just the overhead of TCP was even to much for them. So they switched to UDP.

The issue of packetloss was actually an other problem, it's because they are sending that much data in their internal network, they saturated it at some point. Yes, it might have meant that UDP-packets would be lost, but it doesn't really matter. Because TCP wasn't even able to cope before that, it was already to slow and with the packetloss would have been useless as well.

Re:High performance in scripting languages?

[bill_mcgonigle]

Facebook added to memcache the ability to use UDP instead of TCP...That's the kind of problems/scale they deal with, I'm surprised PHP wasn't their biggest bottleneck before

They also did [mumble] to the memcache logger so that it wasn't a bottleneck anymore. I'm told they were able to get rid of 60 logging servers with their fix.

It's a reasonable engineering approach to first work on the stuff that can't be fixed by throwing obscene amounts of hardware at it, then fix the stuff that can.

Re:High performance in scripting languages?

[lonecrow]

Well and doesn't PHP compile on first use like ASP Classic does? In ASP every call to a script after the first is essentially running a compiled program.

Re:High performance in scripting languages?

[Anonymous Coward]

they could also write better pages which doesn't requires dozens hits to the backend to be displayed, there is simply too much uncoordinated javascript.

Re:High performance in scripting languages?

[DaVince21]

Oh, come on. Slashdot runs on Perl scripts, don't start telling me that's a system language now.

Re:High performance in scripting languages?

[Anonymous Coward]

My company is developing a new (compiled) platform for web apps.

Interestingly, we are already from 10x to 20x faster that PHP although most optimization are yet to do.

Compiled services are better, and there is no value to interpreted services. What is required is very quick compilation (and to be honest, we're working on this but not yet there).

(For the curious: http://mlstate.com )

Re:High performance in scripting languages?

[nullchar]

I really wish PHP was more like a Java application server -- where it would perform like you said: "allow you to instantiate your classes and their data upon startup, and then call methods per request, without having to instantiate all the classes over and over again in each request."

Not having a shared memory pool (without memcached) and having to re-instantiate the same classes on every load really pains me.

Take the task of initializing a logger in PHP -- you create the "object", which loads its configuration of where to write the log file, and then inside your page you call log.debug() or log.error() or whatever, which writes to the log. Unfortunately, every time your PHP page is requested, it does the same startup tasks each time! (e.g. new Logger() writes "application started" each time to the log.) What a waste. If only mod_php would somehow keep those objects in memory, referenced from your page that lives in memory, and you could call the same log.info() method over and over again without creating a new instance of the logger.

I would definitely use PHP if it behaved like an application server.

Re:High performance in scripting languages?

[amiran]

Check your facts better next time please.

PHP does not have to execute scripts from scratch on every request. APC cache API transparently caches JIT-compiled PHP script intepretations and these are run instead upon request.

True, but "compiled" in this context means opcode, not machine code as in Java/C# JIT.

Assembler is High Performance

[Murdoch5]

Don't starting talking about high performance and then naming languages that don't have the chance to deliver. What you really need to do is just program the entire web page in Assembler and then your going to have speed and performance that can't get any faster. If your developers are noobs and can't use real languages and there just Object Oriented kids who can't work on memory and need to access everything through abstracted methods, then fire them and get in some embedded developer who know speed = good code and good languages. If you don't want to use assembler then use good old C!

You want speed use languages that can deliver and don't try to rewrite slow scripting languages to do the job of the trusted old methods, assembler and C.

Re:Assembler is High Performance

[erroneus]

Actually, that isn't necessarily true. It might be true in a linear sense, but when it comes to juggling different threads and the like, assembly language as I knew it wasn't all that capable of describing the process all that well. Assembly language is a go-cart with rocket boosters.

I would truly like to see an assembly language revival though. I truly would. It would be a return to sensibilities in programming. It would be a return to being careful with memory usage with improved focus on small efficient programming. It would be a really good thing. I just don't see it happening.

The purpose for these more complex languages is about being able to more symbolically describe the processes to be executed by the machine. Assembly language was some of the worst about that -- if there wasn't a very detailed set of comments for nearly every line of code, it would be nearly impossible to follow in source. These more complex languages will always have their place and purpose. Trying to make them more efficient is a good and useful thing. Now if we were talking about writing the PHP interpreter in assembler, I'd say you had a winner compromise.

Re:Assembler is High Performance

[Anonymous Coward]

I believe you mean "assembly."

Re:Assembler is High Performance

[Anonymous Coward]

You are kind of person I would never hire. Period.
Assembly doesn't mean speed. there is only potential not more. It has become increasingly difficult to develop in assembly as the architecture and complexities involved (drivers, APIs, hardware, devices, underlying os, etc.) has evolved so much. Decent optimizing compilers nowadays out perform hand written assembly. Further more in commercial software development time is pretty much the most important factor. If you could do in 1 day same thing that would take 2 weeks with assembly? The choice is clear. Not to mention concerns about portability, maintainability, extendibility, ..

So really everything considered you should seriously rethink your ideals about developers because any seriously minded person just laughs at statements like yours.

Sorry, but true.

Re:Assembler is High Performance

[Anonymous Coward]

People, this comment is a joke or written by a moron. It is not insightful and whoever modded it up should be ashamed.

Re:Assembler is High Performance

[neoform]

If your developers are noobs and can't use real languages and there just Object Oriented kids who can't work on memory and need to access everything through abstracted methods, then fire them and get in some embedded developer who know speed = good code and good languages.

Yeah, us assholes with +10 years experience developing web applications are just a bunch of "noobs" because we never needed to learn Assembler or C. Enjoy spending two weeks developing what would take me an hour.

Re:Assembler is High Performance

[noidentity]

What you really need to do is just program the entire web page in Assembler and then your going to have speed and performance that can't get any faster.

For a given amount of time, in almost all cases you can deliver a faster end program by writing in C and possibly optimizing some parts in asm, rather than writing everything in asm.

Re:Assembler is High Performance

[twrake]

php v perl v C v asm is a waste of time.... oh, I forgot this is SLASHDOT!

Test how it runs then optimize the most used sections and repeat until you have the best results you can afford.
Don't waste time avoid pieces of code that don't matter find out the most used sections by testing, optimize those portions.
Perhaps this is how it should be done in the real world perhaps this what Facebook is doing. Who cares what management thinks we are the coders of slashdot...

resistance is futile or V/I ?

Re:Assembler is High Performance

[Anonymous Coward]

Your customer base must be insignificantly small with an attitude like that.

Re:Assembler is High Performance

[noidentity]

I can't figure out whether you're agreeing with me (code for clarity, in language that allows most clarity, then optimize small sections as necessary) or disagreeing, or saying that nobody actually does it like this.

Re:Assembler is High Performance

[javacowboy]

I'm a Java developer with 10 years of experience developing enterprise grade server applications. We use Java, like the majority of Fortune 500 companies, because a Java app can be maintained with a development team greater than 1 coder, common memory coding errors and behaviours is avoided, a large API library prevents us from having to re-invent the wheel constantly, and the JVM is battle-tested in large deployments.

But, no, I guess I'm just a kid who doesn't know how to code.

Re:Assembler is High Performance

[TheLink]

Same goes for writing in a "scripting language" and optimizing some parts in C.

Re:Assembler is High Performance

[jc42]

I've been involved in a number of projects that were prototyped in a scripting language (usually perl or python) and then rewritten in C for performance, with the disappointing result that the C code ran slower. I've also seen the same with C -> assembly a few times.

The explanation is fairly straightforward. The low-level-language experts (including me a few times) may have known their language well, but they'd never looked into the perl/python/cc code to learn the algorithms used there. It turned out that the implementers of perl/python/cc have developed some rather sophisticated algorithms for some of the time-wasting operations (e.g. table lookup) that were unknown to the asm programmers.

If they'd recoded the same algorithm used in the interpreters and compilers for the higher languages, they'd probably have won the contest, because there's no doubt that there's still some wasted cpu time in the higher languages' code. But, as others have pointed out, very often the algorithm used is a better predictor of speed than the language used.

So high-level vs. low-level language is a bit of a bogus distinction. The actual speed of the code is a combination of the algorithm and the efficiency of the implementation of the language. And some languages have several implementations with different efficiencies.

Re:Assembler is High Performance

[twrake]

This is SLASHDOT! Agreement is futile!

Actually you were the closest to my thoughts as I read down the thread. The right way is code AND optimize with the least amount of resources, that is if you are paying programmers to do a job. There are trade off in most choices including maintenance and the cost of keeping brains who know how your system is coded which is a lot higher for keeping asm updated.

Unless I can find a expressive asm routine or a real need to cut processor cycles C is preferred IMHO. I have coded and debugged at "switch panels" we have higher level languages is because of the labor involved with direct machine interaction.

And not enough people or companies do it the right way, they just pay and waste programming brains, and because of this program language wars will just go on and on. Pointless because the method of code,test, measure and optimize is the economic/engineers way to work.

Re:Assembler is High Performance

[BitZtream]

assembly language as I knew it wasn't all that capable of describing the process all that well

You do realize that regardless of what language you use, it pretty much ALL makes its way down to assembly at some point ... right?

It may not be output to a .asm file, but asm is just machine code thats slightly easier to read, and thats what the processor runs.

If you can do it in C/C++, C#, Basic, Python, Lua, Java, you name it ... you can do it in asm.

Just because you don't know how to do it in asm doesn't mean its impossible, it most certainly IS possible.

You can make 'functions' in assembly that are EXACTLY like those that your C compiler produces, its not even hard. You can make C++ compatible classes in ASM for that matter, though it'd be dumb to not reuse the work that compilers already do for you and optimize that output as needed.

ALL languages, from ASM to Java are just preprocessors, code libraries and code generators for machine code.

Writing a PHP interpreter in ASM would most certainly be retarded. The optimizations you'd get would make so little difference considering that the overhead of dealing with 'lazy' programmers outweighs any optimization you're going to get over C by so much that its not even worth talking about. Yes, you'll probably be able to save some CPU cycles here and there, but if you're to that point, where you need that kind of optimization, you'll almost certainly be better off by not using PHP and going to something lower and removing the extra unneeded overhead of the things that PHP does that you don't really need.

Re:Assembler is High Performance

[coolsnowmen]

The low-level-language experts (including me a few times) may have known their language well, but they'd never looked into the perl/python/cc code to learn the algorithms used there. It turned out that the implementers of perl/python/cc have developed some rather sophisticated algorithms for some of the time-wasting operations (e.g. table lookup) that were unknown to the asm programmers.

Yes, yes, 1000X yes.

I've seen this in Matlab versus C++a couple times. The matlab people implemented some very fast algorithms that were way faster than some of the things I came up with.

Re:Assembler is High Performance

[Murdoch5]

And yet Assembler would still out perform it under 100% of all cases.

Re:Assembler is High Performance

[Murdoch5]

If you had read one of my reply' I did clearly stat there are good desktop level programmers, just I've noticed alot of them are "Hey I have 10GB of memory lets see if we can use it all".
If your a good Java programmer then thats awesome and I don't mean to call you a bad programmer. I'm talking about the programmers who pick the wrong language for the wrong job and use memory like it's going out of style.

Bascily you'd know if I was talking about you in the post, there are those who are good programmers even in high level languages but more so alot of them are bad. I'm not centering you out saying your not a good programmer and the fact you can defend your self means you probley can hold your own.

Over all the point was don't try to turn PHP into a high server load language and try to rewrite te run time to go for speed, pick a language that has speed on it's side and go from there.

Re:Assembler is High Performance

[Anonymous Coward]

You do know that there's more to programming than "it's possible to make something that works", right?

Re:Assembler is High Performance

[MichaelSmith]

very often the algorithm used is a better predictor of speed than the language used.

Particularly when you move from development to operating on real data at real world load.

Re:Assembler is High Performance

[TheLink]

Yeah, and that's why I suggested people write the whole thing in a faster to develop language first, then try to optimize parts of it.

Then if some bright spark thinks he's some hotshot programmer, he'll waste less of everyone's time failing to optimize small parts of it than if he tried to write the whole thing.

While strictly speaking slower algorithms were causes of many performance problems in code I've had to speed up, they were closer to being candidates for "thedailywtf" than what people would normally call "inefficient algorithms".

Making 300 database queries EVERY time a single webpage is loaded is not a good thing.

Trying to get info on a connection by looking in a file in /proc/ for up to 20 seconds for every connection is not a good idea. Especially when it's done in a way where it can't handle other connections during those 20 seconds.

Re:Assembler is High Performance

[Civil_Disobedient]

Sure, after the ten years you spend writing it.

Re:Assembler is High Performance

[clone53421]

Whoosh.

Sounds like a fork...

[bogaboga]

...though traditional forks do not get started after "friendly" meetings. But it still sounds like one; which is not a very good thing in my opinion.

What they (Facebook) should have done is to combine resources with the PHP folks, then later release a "new" PHP version with this new engine.

This would be dubbed progress by the majority here.

By the way, where are the stats that show how wanting the current PHP engine's speed still is? I want to see some serious comparison.

Re:Sounds like a fork...

[Anonymous Coward]

What they (Facebook) should have done is to combine resources with the PHP folks, then later release a "new" PHP version with this new engine.

And which version would that be? PHP 7? Coming to your nearest web server in 2021!

C'mon. 5.3 was supposed to be out in fall 2009. It came out late spring 2010. It will be another year (from that date) until all the distros start supporting it in their stable repos. PHP 6 won't be ready for year(s) to come. What "new" PHP version are you talking about?

The only obvious thing is to screw it and fork on your own.

Better yet, drop that crap of PHP altogether, but I guess FB has invested too much into PHP already so rewriting everything in another lang (Python for instance) would be too expensive.

Or maybe rewriting the application in a better performing language might be cheaper than running thousands of servers for PHP now, because it would reduce the number of required front-end application servers greatly, which translates to money spared to invest into rewriting in a new language.

Re:Sounds like a fork...

[Smask]

Wow! My first time-traveler spotting!

Re:Sounds like a fork...

[lbft]

It depends. This new interpreter may be better in some specific circumstances but worse in others, making it unsuitable for most people and highly useful for others. That's the sort of situation where you want to keep both the old and the new and everyone's happy.

Re:Sounds like a fork...

[Xest]

It depends how they've handled it.

It could be that they've dropped a lot of the idiotic legacy crap that makes PHP such a mess, and so their new engine simply doesn't have a lot of this cruft in, which is fine for them if they've avoided the legacy stuff, but which would be useless in the mainstream version as people require backwards compatability.

It could alternatively just be optimised towards certain tasks at the expense of slow downs in others so not a one size fits all improvement.

So is it a fork or isn't it?

[erroneus]

Sounds like Facebook rewrote PHP and then invited PHP core developers to adopt it as their core development platform? I can't imagine that went over all that well... probably hit a number of them in the pride region. And the article said it is to be released as open source, but failed to mention the license. Will this be some sort of twisted "FriendFace Public License" or some perversion?

This is not what is meant when a party contributes to an open source project. "Here, I rewrote it for you. It's better. Now just throw away everything else you've done and use this." Really?

Re:So is it a fork or isn't it?

[maxume]

They tend to work with the Apache Software Foundation:

http://developers.facebook.com/opensource.php

I would imagine the licensing for this will be similar.

And it is probably best described as an implementation; if they start adding language features not present in other PHP interpreters, then it becomes a fork (lots of languages have multiple implementations, for instance, C, C++, C#, Java, Python, Perl, Ruby, etc.).

Re:So is it a fork or isn't it?

[Lennie]

I think it will use the same license as regular PHP, as it's based on regular PHP. Of the projects they adapted they all used the same license.

Re:So is it a fork or isn't it?

[maxume]

Makes sense.

Re:So is it a fork or isn't it?

[Tumbleweed]

Sounds like Facebook rewrote PHP and then invited PHP core developers to adopt it as their core development platform? I can't imagine that went over all that well... probably hit a number of them in the pride region. And the article said it is to be released as open source, but failed to mention the license. Will this be some sort of twisted "FriendFace Public License" or some perversion?

You can only get the new engine if you get to level 100 in Vampire Wars, or if you get 10 windmills and adopt a stray black kitten in Farmville.

Re:So is it a fork or isn't it?

[DaVince21]

I dunno man, the article states they invited those guys over two years ago to discuss the problems with PHP, and then went to improve it.

Re:So is it a fork or isn't it?

[DragonWriter]

Sounds like Facebook rewrote PHP and then invited PHP core developers to adopt it as their core development platform? I can't imagine that went over all that well...

Not sure why it wouldn't, unless its something particular about the PHP core team. YARV and Rubinius started life as third-party ground-up reimplementations of Ruby, and YARV was adopted as the main Ruby interpreter for 1.9, and there has been lots of talk about Rubinius replacing YARV in the future.

Re:So is it a fork or isn't it?

[Lennie]

It seems I was right about the license:

http://developers.facebook.com/news.php?blog=1&story=358

(details about what they have released)

HyperPHP, or HPHP

[hkz]

According to that article posted recently about Facebook's master password being 'Chuck Norris', the project is indeed a compiled PHP that goes by the name of HyperPHP, or HPHP. It will supposedly lower the load on the servers by 80% and speed up things 5x, according to the unnamed source in the original blog post.

Re:HyperPHP, or HPHP

[Anonymous Coward]

And if it was writting in C they could turn off 27.000 machines. Whats the point ?

Dinosaur

[Anonymous Coward]

May be presumptuous, but, if it is better and liberally licensed; one would be foolish not to use it. This happens in all industries, someone re-invents the wheel and does it better, faster, cheaper with newer techniques, and the dinosaur quickly disappears. If the article is correct, Facebook is giving the dinosaur a chance to avoid extinction.

Misleading Summary (surprise!)

[Anonymous Coward]

From TFA: UPDATE: After sifting through the comments here and elsewhere, I'm inclined to agree with the folks who are saying that Facebook will be introducing some sort of compiler for PHP.

Not a fork. Not as newsworthy as implied.

Re:Misleading Summary (surprise!)

[Anonymous Coward]

Did they have a play with the likes of Eaccelerator (http://eaccelerator.net/) first? (DNRTFA)

Re:Misleading Summary (surprise!)

[paziek]

If thats so, then they are reinventing wheel, since there is already PHP compiler available, with is also open source: http://www.roadsend.com/home/index.php

Re:Misleading Summary (surprise!)

[gmuslera]

Maybe the round wheels don't fit well in what they are doing, how they are using PHP, as dynamic language the approach that was taken in roadsend could bump against some core practice in Facebook and thats why they must use another approach to fit into that scheme.

Compile

[hey]

Would be nice of there was an option to compile it to say .phpc files like Python. Would be a nice thing for Perl too.

Re:Compile

[Linker3000]

http://eaccelerator.net/

"eAccelerator is a free open-source PHP accelerator, optimizer, and dynamic content cache. It increases the performance of PHP scripts by caching them in their compiled state, so that the overhead of compiling is almost completely eliminated. It also optimizes scripts to speed up their execution. eAccelerator typically reduces server load and increases the speed of your PHP code by 1-10 times. "

Was revealed 3 weeks ago by insider

[diretalk]

This PHP compiler item was revealed three weeks ago by a Facebook employee. Read at http://therumpus.net/2010/01/conversations-about-the-internet-5-anonymous-facebook-employee/?full=yes

Re:Was revealed 3 weeks ago by insider

[gyepi]

Moreover, was featured on Slashdot a week ago! But then again the Chuck Norris part of the story captivated more the ./ crowd.

One man effort

[hey]

So there is one guy at Facebook doing this PHP rewrite. It must be possible to figure out who he is. Have they hired any high profile PHP developers?

Re:One man effort

[larry bagina]

If the PHP developers (high profile or otherwise) were capable of producing something like this, PHP wouldn't be the turd that it is. I'd guess he's coming at it with more experience in VMs (java, .net, etc) or compiler theory than PHP development.

Re:One man effort

[bill_mcgonigle]

He's so nerdy you have to talk to him about basketball with references to graph theory.

I suppose we'll find out from the README in a couple days anyway, that might be easier.

Re:One man effort

[Anonymous Coward]

Nobody will believe this until tomorrow after the event, but http://www.facebook.com/hzhao - another reason to call it HPHP.

Re:One man effort

[Anonymous Coward]

I know, RTFA is hard, here let me help you:

One night at a Hackathon a few years ago (see Prime Time Hack), I started my first piece of code transforming PHP into C++.

And then he goes on to say:

Needless to say, it took longer than that single Hackathon. Eight months later, I had enough code to demonstrate it is indeed possible to run faster with compiled code. We quickly added Iain Proctor and Minghui Yang to the team to speed up the pace of the project. We spent the next ten months finishing up all the coding and the following six months testing on production servers. We are proud to say that at this point, we are serving over 90% of our Web traffic using HipHop, all only six months after deployment.

Written by the author of the article linked in the summary: Haiping Zhao

VM's

[MattBD]

Would a language that runs in a VM, like Java, Scala or C#, be faster? After all, Twitter rewrote their backend in Scala and they seem to have gotten better performance.

Re:VM's

[BadAnalogyGuy]

How are you differentiating between a virtual machine and a runtime?

Re:VM's

[MattBD]

Really I mean a runtime, I guess. I do think the JVM is somewhat misnamed.

Re:VM's

[Anonymous Coward]

the Zend Engine is a VM, and PHP uses it since version 4

Re:VM's

[shutdown -p now]

Would a language that runs in a VM, like Java, Scala or C#, be faster?

The general question about the VM is meaningless, since PHP has its own VM as well. For three languages you've listed, which use 2 different runtimes - JVM and CLR - the answer is "yes", because both runtimes generate native code at runtime (JIT), and then run that. JVM is better optimizing and has smarter GC, where CLR has some lower-level primitives which can be used to perform manual optimizations.

Re:VM's

[Xest]

The JVM is indeed a virtual machine, there's certainly nothing wrong with that name, as is the CLR.

To give you a better understanding of the meaning, you have to imagine a theoretical machine that can execute Java bytecode (or in the case of .NET, CIL bytecode) directly. The JVM and .NET CLR are both still virtual machines because they are simply software implementations of such a theoretical machine. In this respect you'll note that the concept is actually all that different to the idea of software like VMWare which is effectively just a Virtual Machine that runs x86 machine code.

I'm not aware of any examples off by heart, but I wouldn't be suprised if there is hardware out there that directly executes Java bytecode nowadays in fact, rather than it being a mere theoretical machine.

I guess the point is to realise that a machine doesn't necessarily have to be a piece of actual physical hardware though, but can be a theoretical concept like the Turing machine is. A virtual machine is just an implementation of such a concept, whether it's real or theoretical. The actual output of execution for example either a full blown operating environment as in VMWare, or a single app as in the JVM is really irrelevant, both are still virtual machines at the end of the day.

The only caveat is to remember that not all theoretical machines can be implemented, so although a theoretical machine can have a VM, not all theoretical machines could have virtual machine implementations. An example reason for this might be a theoretical machine that has memory bounds far beyond that of a real machine (i.e. infinite memory), and so which could never be implemented in software.

How about...

[Hurricane78]

...rewriting your site, to use a real language, instead?

I had to use PHP for 4 years, and I’d rather die than to do it again. (Same thing with the Internet Explorer.)
Get yourself a real language. One that makes sense! One with an actual spec. One that makes sense! (Has to be said twice!)
Even Python would make more sense. Java would be a professional choice. And if you want to get futuristic, I’d recommend Haskell. ^^
Everything is better than PHP. (Ok, except perhaps Intercal/Malbolge/Piet. Perhaps...)

Re:How about...

[Anne Thwacks]

I wrote assembler for near on 20 years. I am rather fond of PHP. It can P*ss futher than Fortran4, and its significantly faster than a dead slug.

Three sources of scripting language inefficiency

[tepples]

I don't know what the fascination is with scripting languages on the Linux platform or with FOSS in general, but it results in slow programs

Speed of development is faster in a scripting language, and in developed countries, below a certain scale, throwing hardware at it is cheaper than throwing programmers at it. The point of the article is that Facebook is above that scale, and programmers to write a new PHP interpreter have become cheaper than adding hardware+power+cooling.

with flaky UIs.

Citation needed. True, the often use a different widget set from the rest of the desktop (e.g. Tk from Tcl and Python and Swing from Java), but the popular widget sets also have scripting language bindings. how can one really tell the difference between a wxWidgets or GTK app written with Python vs. C++?

I like to use refurbished/recycled machines; which means that I'll have an old P4, 512M RAM and a slow bus.

Do these use more electric power than, say, an Acer Aspire Revo? The power consumption of a Pentium 4 and the power to remove the heat it generates can become an issue, especially for a server that's turned on 24/7.

Many times, applications written in a scripting language, whether it be Perl, Python, PHP, or whatever, will hang often and then start working.

There are three causes for this, and you can distinguish them with 'top' or 'Task Manager' or something else that can count CPU time and page file accesses:

• Swapping: More dynamic languages tend to use more general data types, which incur memory overhead. For instance, they might use UTF-16 strings instead of 8-bit strings with an assumed encoding. Or they might use double-precision floating point instead of short integers for large arrays. This might cause a program to run out of RAM and fail over to the disk more often.
• Garbage collection: This covers ways of determining which resources are no longer in use by any active part of the program. Python, Objective-C, and lately C++ primarily use an incremental garbage collection method called reference counting, which keeps track of the number of things that "know about" an object. But some other language interpreters use only tracing garbage collection, which in the naive form causes the application to pause and make a list of all live objects once memory allocations exceed some amount. This will cause a CPU spike.
• Blocking: A lot of the APIs available to programs in scripting languages don't have well-known non-blocking versions. For example, a host name lookup might freeze the program until it finishes. The only way to work around these is to start a thread. This will cause a pause with 0 CPU and 0 disk.

Assembly language revival

[tepples]

I would truly like to see an assembly language revival though.

The revival is here. It is called NESdev. It's just not for making PC programs because the processes to be executed on a PC tend to be much more complex.

Now if we were talking about writing the PHP interpreter in assembler, I'd say you had a winner compromise.

You'd have to do so over and over for x86, x86-64, ARM, PowerPC, and other architectures on which PHP runs. That's why these interpreters are most often written in C, with occasional reference to the assembly language code that the compiler generates. Really the only time you have to handle assembly in a PC application is when you're implementing a just in time compiler, and it's becoming the fashion to let LLVM do that for you.

Re:Assembly language revival

[epine]

Really the only time you have to handle assembly in a PC application is when you're implementing a just in time compiler, and it's becoming the fashion to let LLVM do that for you.

That's an interesting combination of overstating and understating the case.

For one thing, your favourite C/C++ compiler likely contains a hand optimized memcpy() routine, down to assembly if it exposes a worthwhile gain, or coded in C with or without intrinsics if it doesn't. Many C/C++ compilers contain hand-optimized floating point routines, even more so in the embedded world. Plus there are many performance libraries out there to handle the heavy lifting in multimedia, mathematics, and encryption, some of which are vendor tuned to the n'th degree. It's been a while since I've used an Intel library, but this is likely one of the breed:

Intel MKL

As for LLVM, I'd say it's more than fashion. The differences in performance characteristics from one micro-architecture to another are nightmares to cope with at the assembly language level. The average tablet computer these days could probably play Kasparov to a draw, and there are still macho programmers out there who think they can do register assignment and live range analysis better than your compiler? Dude, if you've got that much talent, roll up your sleeves and fix the freaking compiler. Hopefully LLVM will solve that old problem of first having to swallow the gcc ast syntax enzyme.

Tautology #1: I can beat my computer at chess => your chess computer sucks (or it's running on your wristwatch).

Tautology #2: I can beat my compiler at coding a non-trivial loop => your compiler sucks.

Unless your goal in life is to win rigged competitions, LLVM is a lot more than a fashion statement.

Re:Assembly language revival

[nacturation]

The revival is here. It is called NESdev.

Sounds interesting, but developing for the Nintendo Entertainment System is a bit outdated, don't you think? :)

PHP is slow (check), now what....

[Ritz_Just_Ritz]

Why not just stash your farm of slow php systems behind some heavy duty caching appliance(s)?

Something like aicache might fit the bill.

Re:PHP is slow (check), now what....

[jimicus]

Why not just stash your farm of slow php systems behind some heavy duty caching appliance(s)?

Something like aicache might fit the bill.

When your application is with each iteration generating more content dynamically than it was before (and you want to continue down that route), the benefit of caching starts to drop quite quickly.

Re:PHP is slow (check), now what....

[slim]

Facebook does as much caching as it can - I mean, they're not daft. They're probably the world's greatest experts on large scale MySQL + memcached.

But sometimes cached data isn't good enough. Facebook users expect their statuses, messages and comments to reach their friends within seconds.

Re:PHP is slow (check), now what....

[mebrahim]

This is scored 5-Interesting? Am I at Slashdot?

Complexity and cost of embedded approach

[tepples]

If your developers are noobs and can't use real languages and there just Object Oriented kids who can't work on memory and need to access everything through abstracted methods, then fire them and get in some embedded developer

Embedded developers tend to 1. work on smaller, more focused systems, and 2. charge more. For one thing, a module inside Facebook deals with data types more complex than those in the firmware of a car engine's microcontroller. And below a certain scale, the money you save by hiring noobs (and taking the tax credit for recent graduates if available) can pay for throwing more hardware at the problem.

Re:Complexity and cost of embedded approach

[Murdoch5]

true, but on the other hand most of what I've seen is that the embedded developers can program the higher level languages with more care anyway. So throw a bunch of embedded developers at a sever level project and you'll probley end up with a faster solution.

In no way I'm I trying to imply there aren't great desktop level programs because I know a few that are top notch. On the other hand it is cheaper to hire the dime a dozen desktop guys and coops and just get it working and never speak of it again.

Re:Complexity and cost of embedded approach

[CptPicard]

true, but on the other hand most of what I've seen is that the embedded developers can program the higher level languages with more care anyway

My impression tends to be that the best overall programmers are those with a solid understanding of algorithmics theory, programming language design in general (meaning they have had exposure to all kinds of solutions), and most interestingly, tend to have an understanding of functional programming. The true programmer gods I have come across have always been Lispers, almost without exception.

On the other hand, I never understood what is supposedly so educational and intellectually important in things like assembly. If one only learned that, it wouldn't still mean that one could actually use it for anything... it's just "manipulate state in registers and RAM by making use of extremely rudimentary basic operations". The transformations into machine code from higher-level program solution descriptions are much more consistently handled by a compiler than a human, and as that is manual, automatable work, it may be more important to study compiler construction... (which Lisp is pretty good for)

Re:Complexity and cost of embedded approach

[Anonymous Coward]

Personally, understanding assembly is fulfilling - I want to have some idea what is going on under my magic, higher-level keystrokes. I do agree with the programming languages statement, and found that I looked at things differently after writing a small (and pretty bad) interpreter; it was one of the best learning experiences of my life.

As for
 

The true programmer gods I have come across have always been Lispers, almost without exception.

I have found that more like Haskell. But that may be dating you =P. Even some of the gods write somewhat shitty functional code, though, in terms of style (algorithms are usually solid)

Re:Complexity and cost of embedded approach

[non0score]

The kinds of programming gods I know of (ones whose only concern is speed within the memory constraints) all know quite a few languages.

But when it gets down to it, they will hand roll their own assembly code. Now you may think that compilers could do all this for you...but then that just shows how naive that assumption is. Compilers actually do a pretty shitty job. Just think of this simple C code:

uint8_t foo = function_param_1; // function_param_1 is something passed into the current function scope
foo++;

Now, what do you think the compiler for a 32-bit RISC processor would do? Most likely involving something to do with loading the memory values into a register and do some add immediate. And then it will do another AND against 0x000000ff. Why? Because the compiler doesn't know what foo is, and will need to mimic the wrapping behavior on an 8-bit variable. But clearly if you, the programmer, know that you'll never get 255 as function_param_1, then that extra AND is totally useless.

Basically, the problem is that the compiler never really knows what you're intending to do. It also never knows what you're expecting as the input so it always has to use the safest route. Sometimes you can substitute an even pipe instruction for an odd pipe instruction. And when your inner loop involves only 10 instructions and runs a billion times per second (no need to go into loop unrolling for now), shaving that extra instruction off and/or rebalancing the odd/even pipes so that you can run it in
And for those who don't know assembly and what algorithms to use...well, they've got a lot to learn anyway.

Re:Complexity and cost of embedded approach

[CptPicard]

IMO Haskell fulfills that role just as well... I do find Lisp to be more general-purpose -- in particular the syntax-tree transformations that Lisp macros are are somehow the most "general" form of programming I've come across... the whole thing is, in the end, an extensible compiler in itself on top of a minimally small set of maximally expressive primitives.

This is also why the "writing of interpreter" is an educational experience. I used to somewhat dismiss the statement from likes of Paul Graham about how you should be programming in terms of creating specialized languages, but then I got the religion, and talked to a really smart hobbyist programmer linguist, too, at length...

Re:Complexity and cost of embedded approach

[CptPicard]

I have actually been talking at length to these people whose only interest is in speed within tiny constraints (in the context of "in what order should one teach things when teaching a future insightful coder"), and they always leave me with the impression that that really is the only thing they know -- at least that is how loudly they condemn higher-level programming without giving me much reason to assume they know what they're talking about... for example "closure" doesn't seem to mean much ;)

Assembly is a specialized programming formalism and domain in a similar style as J2EE is. It requires a specific knowledge set, but it is possible to be a just code monkey in both of them.

And yes, I do know asm, just never saw it as much more than a simple sequence of state manipulations... E. Dijkstra said that "our tools influence our thinking", and I have personally been influenced by pretty much everything else.

But then again I'm a somewhat philosophical CS type -- my interests are not only in speed in tiny constraints :p

Is compiled PHP even possible?

[RJabelman]

PHP is a weakly typed language, so for any given operation, the interpreter will have to check the types of the operands and then figure out which operation(s) on the CPU to call to solve it. Also, as it's dynamic, the operand may not even exist yet.

So, even if you did write a compiler for PHP, instead of the PHP interpreter doing the type checking and figuring out what to do, you'd have to compile in some runtime checks to implement the same logic that's currently in the interpreter for every single operation. This doesn't sound to me like it'll be significantly faster (Although I'll freely admit it's just a gut feeling.)

So, a question to the room - if it's even possible, is there any advantage in compiling a dynamic weakly typed language to native code?

Re:Is compiled PHP even possible?

[TheRaven64]

I've written a static compiler for Smalltalk, so I'm pretty sure it's possible for PHP.

Re:Is compiled PHP even possible?

[RJabelman]

Ah, that's interesting. What was performance like compared to the interpreted version?

Re:Is compiled PHP even possible?

[BadAnalogyGuy]

http://groups.csail.mit.edu/mac/classes/6.001/abelson-sussman-lectures/

Worth a look.

Re:Is compiled PHP even possible?

[TeknoHog]

PHP is a weakly typed language, so for any given operation, the interpreter will have to check the types of the operands and then figure out which operation(s) on the CPU to call to solve it. Also, as it's dynamic, the operand may not even exist yet.

I think Python has basically the same problem. Compiled .pyc files are actually bytecode, like Java. They save the time of the initial interpretation, but do not make the actual execution faster. Then there is also Psyco.

Re:Is compiled PHP even possible?

[maxume]

Python (and Java) compile source files into an intermediate byte code that a virtual machine executes. The byte code does not have to be run through a parser, so overall execution is faster.

Re:Is compiled PHP even possible?

[kobaz]

'Overall execution' being faster is a misnomer. The change to bytecode only improves startup time, but what counts is runtime speed. Many of the modern interpreted languages: php, perl, python, ruby, etc, have a just in time compiler that will internally bytecodeify your raw source before execution. Assuming a straight non-optimizing JIT, The actual executing part, running the application will have the same speed running raw source or bytecode.

In the case of an optimizing JIT, then you gain some speed in runtime. But many implementations of said modern scripting languages lack the optimizing part.

Re:Is compiled PHP even possible?

[FlyingGuy]

Not just possible but my WAG is that probably not that hard. Yes PHP is dynamic and variables are typed at runtime, but the logic analyzer and parser would figure all that out and then the resulting machine code would be type appropriate.

Internally all interpreters of loosely typed languages have to figure out what the they types are before it can perform the requested operation. One way to speed up loosely type languages is to remove the looseness and force variable and type declaration. This simplifies matters greatly as the interpreter now knows exactly what do do with the operation and does not waist valuable cpu cycles figuring it all out beforehand.

Re:Is compiled PHP even possible?

[maxume]

Python has strong, dynamic typing. The dynamic part makes it difficult to compile.

Projects like Cython and Shedskin each compile a less dynamic subset of Python.

Re:Is compiled PHP even possible?

[CptPicard]

Yes PHP is dynamic and variables are typed at runtime, but the logic analyzer and parser would figure all that out and then the resulting machine code would be type appropriate.

The whole problem is that when typing is dynamic, your variable in your program can have some arbitrary type at runtime, and you can't figure that out in the general case without executing the actual program. Static code analysis goes only so far in this matter.

Re:Is compiled PHP even possible?

[multipartmixed]

> So, a question to the room - if it's even possible, is there
> any advantage in compiling a dynamic weakly typed language to
> native code?

You don't have to get to native code reap the benefits.

I work on a server-side javascript platform; it pre-compiles the JavaScript to an intermediary token representation. The benefit is measureable, especially with large scripts. The cost is also negligible for trivial scripts.

Actually, that reminds me, that is the same mechanism that Firefox uses to "fast load" extensions.

Re:Is compiled PHP even possible?

[TheRaven64]

The interpreter we have uses direct AST interpretation, which is pretty slow. On a simple test program (a parser), it took 0.96 seconds of CPU time in the interpreter, 0.023 seconds in JIT-compiled code (pretty primitive so far, doesn't use any profiling info) and something a bit less than that in statically compiled code. Since running that benchmark, I've made a few improvements to the compiler, so it's probably a bit faster now.

For a recursive Fibonacci calculation, implementing the same algorithm in C, Objective-C and Smalltalk took 2.35, 6.60, and 5.69 seconds, respectively (calculating fib(30) 100 times), with about a 50% variation margin on successive runs. The Smalltalk version was not always faster than the ObjC version (it was most of the time; not sure why, probably some weirdness with the Smalltalk code happening to line up with cache boundaries better), so it's safe to consider them roughly the same speed.

It's worth noting that the Smalltalk version, unlike the C and Objective-C versions, will never suffer from integer overflow. Tweaking the benchmark so that it computes fib(47) in the three versions, the timing results are: 50, 130, and 280 seconds, respectively.

The difference is that the Smalltalk version generates the correct answer, while none of the others do. Personally, I'd rather have slow code generating the correct answer, but maybe that's just me. It is, of course, possible to write code in C that would check for overflow (in this case it's relatively easy, you can just test whether the sign bit flipped because you're just adding two positive integers), but returning something that is either an integer or an arbitrary-precision value in C is a bit harder and you'd end up with at least four times as much code to make the C version, and a lot more potential for bugs.

By the way, calculating fib(47) with a sensible algorithm in Smalltalk takes a tiny fraction of a second, highlighting the fact that good algorithms are usually more important than good compilers.

The compiler targets the (GNU) Objective-C ABI, so Smalltalk and Objective-C classes can be used interchangeably (you can, for example, subclass an Objective-C class with Smalltalk and then call the Smalltalk methods from Objective-C). Some of the improvements I've recently made to the Objective-C runtime mean that the compiler can now emit code to do polymorphic inline caching and speculative inlining. It doesn't yet do either, but in benchmarks these reduce the cost of a message send to within a hair of the cost of a C function call. For most uses, Objective-C is already fast enough, so I'll probably only implement these as a profile-driven optimisations and enable them for hot code paths where the message sending overhead is actually important.

I'm giving a talk about this at FOSDEM in the GNUstep developer room next weekend.

Re:Is compiled PHP even possible?

[Anonymous Coward]

PHP makes your wang hard?

Re:Is compiled PHP even possible?

[siride]

Nope. PHP doesn't work that way. Take a function that takes a single parameter $x. That parameter $x can be of any type. You could have three callers, one passing an int, one passing a string, and one passing an array. The function code itself cannot predict that, so the code in the function must be generic enough to handle all three (and more). Since PHP doesn't allow multiple functions with the same name, to implement overloading, you make a method which checks the types of parameters, for example, and then changes its behavior based on that information. That has to be done at runtime, period.

Now, if they made typing a bit more strict in PHP, then yes, the compiler could figure out all types during JITing or whenever. But PHP as it stands now is not strict enough. Personally, I'd be in favor of making typing a bit more strict. The looseness really doesn't buy you that much and causes a whole lot of headaches (a lot of runtime checks in the PHP itself, to say nothing of what the runtime itself is doing).

Re:Is compiled PHP even possible?

[FlyingGuy]

I understand what you are saying; however does not:

function foo($x){
if($x == 72.6) return 100.00 ;
return -1 ;
}

upon feeding it a string or an array() it return -1 if not cause a runtime error ( even if it is silent) as an interpreter would have to treat anything besides an actual numeric type as boolean false. Would not the RTL treat it the same?

Re:Is compiled PHP even possible?

[thasmudyan]

No, if it's a string containing the characters "72.6", it will silently convert it to float and return 100. (Which is actually pretty nice)

Assembler? Really?

[Atmchicago]

Assembly language isn't platform-independent. It's really easy to screw up and hard to optimize. And it's not much faster than C/C++. The issue at hand is balancing the cost of writing the code with the cost of running it. I don't see how the cost of writing and maintaining software in assembly language will ever compete with the costs of C/C++, potential speed increases and all. Object-oriented languages make small performance sacrifices in return for much greater maintenance, and that's how it should be. Scripting languages take this even further, and for these large websites have lost their advantage. The only time assembly will prevail is when we return to incredible memory constraints, but even embedded systems pack tons of memory now so I don't see that being an issue.

Facebook's architecture is the problem, not PHP

[QuietLagoon]

Facebook is dependent upon a scripting language, and then Facebook complains about speed? Facebook's real problem here is their short-sightedness. Instead of putting a band-aid on the current architecture via some sort of acceleration for PHP, Facebook should re-architect their site with a structure and language that is capable of handling the current and future loads.

.

I am sure we will be hearing all about how successful this project is, but is the auccessful application of a band-aid really the long-term solution Facebook needs?

Re:Facebook's architecture is the problem, not PHP

[Anonymous Coward]

Do you even build websites?

Re:Facebook's architecture is the problem, not PHP

[the eric conspiracy]

If you read the article, it sounds like it might be more than just an accelerator.

Re:Facebook's architecture is the problem, not PHP

[pmontra]

I'm no PHP fan but I won't be surprised if FB decided that optimizing the interpreter and investing resources in new functionality is a better business decision than investing in a giant rewrite of what they have now. That would effectively stop them for many months in the best case, or double their costs as a team keeps adding features to the PHP architecture and another one plays catch-up in another language. But maybe they also have some plan to rewrite some core components in a faster language, like twitter did porting the backend tasks from Ruby to Scala.

We could say that they started with the wrong technology but using PHP Zuckerberg was able to deliver what turned out to be a successful product back in 2004. Had he wrote it in Java he could have missed a window of opportunity and people could be using some different social network now. Same logic applies to twitter's choice of Ruby, which by the way they still use for the frontend. Many recent interpreted languages (I'm thinking about Ruby) trade execution speed for speed of coding and delivering products. Many products totally fail and many others don't get so successful to need optimizations so IMHO speed of delivery is the key factor: deliver, get customers, get money and only then we'll think about making our servers run fast.

Ah... If only FB's new interpreter could access instance variables without that redundant $this-> construct that clutters all OO PHP code...

Re:Facebook's architecture is the problem, not PHP

[QuietLagoon]

Yes. I've built some, and I've architected a few more. Facebook is in reactionary mode with regards to their website performance. They need to get proactive.

Re:Facebook's architecture is the problem, not PHP

[kobaz]

Instead of putting a band-aid on the current architecture

But that's exactly how you run a successful system.

1) Design product to meet needs of your audience
2) Design the implementation that you think will handle the load the best (with lots of load testing and simulations to make sure it meets expected demand)
3) Build product
4) Watch it behave in the wild... Realize that actual demand is considerably higher than expected demand and will continue to grow
5) Performance slows with more users... you need a solution that will the push the date of catastrophic overload further into the future, to buy time to work on *really* fixing the problem
6) Migrate to a new or adjusted architecture that will solve this current problem
7) Go to step 4

Facebook is on phase 5. You sound like scripting languages are the bane of slow products. Yet in reality, the main bottleneck is generally the database. If facebook rewrote everything in C or some other non-scripting language, not only would it be an incredibly long process, but the the end result would be far less beneficial than if they revamped their existing technologies and worked to up database performance. There is no ultimate solution for scaling a product. You need to be constantly adjusting your strategies, implementations, and systems to cope with resource usage.

Re:Facebook's architecture is the problem, not PHP

[CptPicard]

The key architectural performance issues in large web apps like Facebook are about scalability by clustering and parallelism and caching... usage of proper higher-level languages helps in this (think how pure-functional programming removes shared state and Google's mapreduce for example), while using a lower-level language may give a speedup on single individual machines but makes the architectural problems harder to tackle.

Re:Facebook's architecture is the problem, not PHP

[CyDharttha]

I think Facebook's Thrift is worth mentioning during this discussion, as well as the other items listed under their 'open source projects'.

Re:Facebook's architecture is the problem, not PHP

[Abcd1234]

By rewriting their entire codebase, a codebase that has invested it in likely hundreds of thousands, if not millions of manhours in debugging, testing, etc, thus representing millions of dollars worth of IP.

Yeah. Good call. Clearly you're *way* smarter than the guys running Facebook...

BTW, where's your website that's used by millions of users daily?

Yeah. Thought so.

Re:Facebook's architecture is the problem, not PHP

[QuietLagoon]

By rewriting their entire codebase, a codebase that has invested it in likely hundreds of thousands, if not millions of manhours in debugging, testing, etc, thus representing millions of dollars worth of IP.

.

Sometimes you have to do what you have to do. The near-continuous stream of attempts to improve Facebook's performance is the telltale. At some point, Facebook has to stop putting lipstick on the pig, and face the real problem - they have outgrown the architecture of their website. It won't be the first time a start-up has outgrown its codebase.

Facebook needs to come up with a new architecture, and stop trying to fix the limited tools used by the current architecture.

One does not need to be smarter, only to have a more objective view of the problem and possible solutions, and be less emotionally attached to the current solution.

Re:Facebook's architecture is the problem, not PHP

[raju1kabir]

Facebook has the ability to rapidly deploy changes, something they would lose if they started using the interbank-transfer-system COBOL development model.

You talk about "future loads" as if they are a known quantity. This whole web world thrives on the idea that you can quickly react to changes in the market, user preferences, and so on. For Facebook it makes much more sense to remain nimble and put some effort into increasing the efficiency of the tools that allow them to do so.

Re:Facebook's architecture is the problem, not PHP

[Abcd1234]

Facebook needs to come up with a new architecture, and stop trying to fix the limited tools used by the current architecture.

You have absolutely *no* idea what the sources of Facebook's performance problems are. None. You're simply guessing based on your personal biases.

Facebook, on the other hand, has benchmarked their code and determined that by optimizing the PHP engine, they can achieve substantial performance gains. Who the hell are you to decide that they're wrong and you're right?

One does not need to be smarter, only to have a more objective view of the problem and possible solutions, and be less emotionally attached to the current solution.

Yes, you, with your baseless guessing, are clearly being the more objective one...

Re:Facebook's architecture is the problem, not PHP

[QuietLagoon]

Facebook, on the other hand, has benchmarked their code and determined that by optimizing the PHP engine, they can achieve substantial performance gains.

.

Sounds more and more like a reactionary band-aid to me. Looking for and finding the bottlenecks in the current codebase is a tactical solution to performance issues, not a strategic one.

Who the hell are you to decide that they're wrong and you're right?

That's good. Attack the messenger. It just underscores the weakness of your position.

Re:Facebook's architecture is the problem, not PHP

[Abcd1234]

Sounds more and more like a reactionary band-aid to me. Looking for and finding the bottlenecks in the current codebase is a tactical solution to performance issues, not a strategic one.

And if such an approach is sufficient to achieve their performance goals, *who cares*?

Your argument seems to be that they should rearchitect their entire solution because, well, you say so. My response to that is, no, if they can achieve their performance goals through other means, then it's smarter to keep the existing codebase around, as rebuilding it from scratch is an absurd waste of time in that context.

And, again, you have absolute no idea what their performance problems are, and are utterly clueless about their underlying architecture. Meanwhile, the Facebook folks claim they can achieve *substantial* performance gains by building a new PHP engine. So either you're wrong, are they are. And given they know a hell of a lot more about their codebase and performance issues than you do, I think it behooves you to assume they know what they're doing.

Frankly, your arrogance is pretty astounding, here. Like a remarkable number of Slashdotters I've encountered, you seem to believe that, with the mere trickle of information you have, you can better judge how Facebook should run their business than they can, which is incredibly absurd. Then again, you are, apparently, the god of all software architects, with the necessary omniscience to judge Facebook's entire software architecture and codebase without having ever looked at a line of their code. Good for you! With powers like that, you should be able to make some big bank fixing everyone's scalability problems.

Re:Facebook's architecture is the problem, not PHP

[mr_mischief]

The objective view is the one taken by the shareholders. They don't give a rat's ass about engineering pride so long as the site pulls the same revenues with lower expenses. An 80% reduction in frontend servers for a company using tens of thousands of servers for one guy's yearly pay sounds like a good investment. Even if it's only for a couple of years while they do a top-to-bottom rewrite, it might just pay for a good portion of that rewrite you'd like.

hackity hack

[Anonymous Coward]

its why its the way it is
now facebook
will faceplant

Re:hackity hack

[Anonymous Coward]

*facepalm*

Re:Three sources of scripting language inefficienc

[Anonymous Coward]

Flaky UIs - click on a button and nothing happens. Or things not drawing properly. These are my observations.

A refurb machine is about a third the cost of a new machine - even cheaper than the cheapo Acers.

Interesting list of possible causes. I don't care what the reasons are - scripting languages are not appropriate for large applications with GUIs. And as far as time savings for development is concerned, there are these things called "frameworks" - like gtk+ and Qt - that can speed up development to be just as fast as the scripting languages.

Nice.

[IANAAC]

I was using touch.facebook.com on my N800, but I like the look/feel of this better.

Thanks!

Re:Nice.

[xaosflux]

m.facebook.com is the lightest of all (designed for the most basic mobile browsers) Though I still haven't gotten it to render in Lynx :/

Re:Assembler? Really?

[Anonymous Coward]

Assembly language isn't platform-independent. It's really easy to screw up and hard to optimize.

You wouldn't necessarily use it for your app, but having developers understand what's going on at a lower level, can help even when using higher level languages.

Perhaps this won't help with developers making VB-written corporate apps, but at the scale of Facebook, knowing a larger part of the whole stack helps you makes things run better. And since you don't necessarily know where you're working in your career, covering it in school (or doing it yourself, if self-taught), could be useful.

Even if you don't ever use it, personally I think knowledge and exploration are their own rewards, regardless of any "useful" benefit.

Re:Three sources of scripting language inefficienc

[amn108]

with flaky UIs.

Citation needed.

Run Eclipse.

Gotten

[heffrey]

What the hell kind of word is gotten?! Can't you people learn the language?!!

Re:Gotten

[Aladrin]

http://wiki.answers.com/Q/Is_gotten_correct_grammar

Can't you?

Re:Gotten

[Snarf You]

I'm guessing you haven't gotten laid recently.

Re:Gotten

[Happy Finish]

I guess he might have forgotten.

Re:Gotten

[Mitchell314]

I didn't got it.

Re:Gotten

[maxume]

"Can you not?" would probably be better style, even though it would probably trip up more readers.

Re:Gotten

[heffrey]

Grammar snobs have sex too!

Re:Gotten

[heffrey]

Yeah my position is pretty indefensible isn't it! Still, that past participle just sounds awful to my British ears! I just haven't gotten used to it yet!

Re:Gotten

[Anonymous Coward]

Haha you got bitchslapped... bitch ahhahahaha

JavaScript

[tepples]

Flaky UIs - click on a button and nothing happens. Or things not drawing properly.

I've seen buttons do nothing and redraws fail even in compiled programs.

A refurb machine is about a third the cost of a new machine

By "cost", do you include or exclude the cost of power and cooling? And do you include or exclude the cost of replacing failed components? Capacitors die.

scripting languages are not appropriate for large applications with GUIs.

One scripting language has a huge deployment advantage over everything else: ECMAScript. It interacts with Document Object Models exposed by various runtime environments, and it's sandboxed so that users can more or less safely run a program without getting an administrator to install it. You might know it as JavaScript (ECMAScript + HTML DOM) or ActionScript (ECMAScript + SWF DOM). Or would you rather go back to ActiveX, where the web site sends the equivalent of a compiled DLL to each user, which runs with the user's full privileges and doesn't run on anything but a convicted monopolist's operating system?

Re:JavaScript

[drinkypoo]

One scripting language has a huge deployment advantage over everything else: ECMAScript.

This is a nice lead-in to my point, though; Facebook is one of those websites that make it look like Firefox is murdering your machine. In reality, it's sites which misuse Javascript. (Sorry, but ECMAScript is just too unwieldy. I wouldn't say it. Too bad, because it desperately needs renaming.) If I leave Slashdot open all night, nothing bad happens. If I leave a long Facebook tab open all night, I have to close it before I can use my browser in the morning, even on my shiny new 4GB RAM system, let alone on my 1GB machines. If they want to improve the user experience, they should try cleaning up their crap Javascript.

Re:JavaScript

[Pharmboy]

What is worse is some of the flash based games. Cafe World uses about 350mb to 450mb of ram, with Chrome, IE or Firefox. Leave that open and it is leak city. On my new system, it is unplayable using IE (8GB ram, quad processor, q9550, etc.) It seems to me that much of the features and games are just sloppily written, rushed to market, and prone to simply not work at all sometimes. Yes, that is Zynga, not Facebook, but they both suffer from the same problem: everything requires checking in with the server for even a single click. The bandwidth it uses is also ridiculous.

Re:JavaScript

[asdf7890]

If I leave Slashdot open all night, nothing bad happens. If I leave a long Facebook tab open all night, I have to close it before I can use my browser in the morning, even on my shiny new 4GB RAM system, let alone on my 1GB machines. If they want to improve the user experience, they should try cleaning up their crap Javascript.

I've noticed similar things, though in the general case I thing Firefox (much as I like it, it is still my browser of choice) and/or its Javascript sub-system may be to blame, at least in part, here. The key difference between slashdot and most facebook pages it that slashdot does nothing unless you activley use it, but most facebook pages perform automatic updates to refresh relative posting times (the "x minutes/hours ago") that are displayed and to check for (& display if found) new posts. I suspect that the issue is one of memory fragmentation due to the constant allocation and deallocation of resources for new/changed objects (in the DOM structure, the JS environment's structures, or both) which over time makes the job of the garbage collector and memory allocation routines harder and harder. Once it seems to get into a state where this is an issue updating, navigating too, or navigating from, any relatively complex page or opening/closing a tab/window incurs a noticeable lag during which the processor (well, one core there-of as FF is resolutely single threaded for the time being unless you count certain development trees) is busy).

This is most noticeable on a slower system such as an Atom based netbook. Once the issue is noticeable simply closing FF (with "close and save" so open windows/tabs are remembered) and restarting it brings things back to a better level of responsiveness.

Having said that, I can't blame just internal memory fragmentation in FF for the FaceBook "pop-out" chat page, which always seems to consume 100% CPU time on my netbook (well, 50% - but on a single core hyper-threaded CPU like the Atom running a thread-unaware app like FF 50% is 100% of one virtual core which is effectively 100% as the chip doesn't really have two cores but most performance counters see that it has). This I suspect it due to FB chat updating its display *far* more often that it needs to resulting in excess re-draw operations, which make little difference with a quick desktop CPU but are much more apparent on a power-conservation-over-performance optimized arrangement. It gets worse over time (until restarting the browser), if I'm right due to growing memory fragmentation, but it consumes that sort of CPU time even on my Atom based machine even if it is the second window/tab opened since FF was started clean.

Re:JavaScript

[sog_abq]

I'm not entirely certain the web developers entirely to blame here either. If I accidentally leave pandora on overnight at work, my firefox will be hosed when I get back in the morning. However, using opera I do not have this problem. This also holds true for facebook, cnn, slashdot, and other web sites as well. It seems as though opera is doing something smarter such that my whole system isn't hosed by the web page. This is why I'm starting to use opera more than firefox at work. (ubuntu 9.10 x64)

Re:JavaScript

[tepples]

Yes, that is Zynga, not Facebook, but they both suffer from the same problem: everything requires checking in with the server for even a single click.

How is that any different from any other multiplayer video game such as Quake III: Arena?

Re:JavaScript

[Pharmboy]

Its a FLASH game. It isn't downloading 350mb of data, it's using 350 mb of ram to run a few megs worth of program.

Re:JavaScript

[clone53421]

If I leave a long Facebook tab open all night, I have to close it before I can use my browser in the morning, even on my shiny new 4GB RAM system

Odd; I haven’t experienced that.

Writing the single biggest bottleneck in asm

[tepples]

If you could do in 1 day same thing that would take 2 weeks with assembly? The choice is clear.

Unless the two weeks of hand-tweaking the assembly language code of your program's single biggest bottleneck would reduce your program's system requirements so that twice as many users can use it. Such a case is reportedly common in video game development, where the increased revenue is often worth it.

Not to mention concerns about portability

"Portability" has more than one meaning. There's portability of the code, or its suitability for execution in multiple environments whose hardware isn't compatible. For this, you can keep a fallback implementation of each asm module in C. That's useful for running test cases such as whether the asm version still works correctly or whether it's worth continuing to maintain. The other kind of portability is the ability to run on small, battery-powered devices. These tend to have underpowered CPUs to save manufacturing cost and increase battery life, and the code to run on these CPUs must be extremely efficient in order for the application to be responsive. Go try to make a software 3D renderer on a handheld device with a 16.8 MHz ARM CPU and tell me you don't need assembly anymore.

They should spend more on the upload tool

[crossmr]

That thing is a broken buggy piece of garbage. Any time I go out to an event or something and want to upload anything more than half a dozen photos, it inevitably blows up on random photos for no reason (completely fresh off the camera unedited photos). I have to babysit the upload and instead of just hitting select all and letting it go, I end up having to upload it in chunks of 5 photos at a time.

Re:They should spend more on the upload tool

[stimpleton]

Modern users demand upload progress feedback. Which the HTML spec cannot do. The solution is a bevy of hoary hacks on the server end, usually by using a cache or tmp file callback. The value is then read periodically from the client as a javascript page load in an iframe.

For PHP this is the APC Cache module. You send an id with your file upload form then "Load that page using that ID" till the progress gets to 100%. According to the docs the module can poll at a period of "0 seconds" meaning as fast as possible. This halves upload speed.

On the client end, the old HTML way(no feedback) was a simple form with a submitted page. If you arrive at the submit page then the upload worked. The new way is 50-60k of javascript, which is a collection of fragile code. Yahoo's GUI upload for example. Try moding their code and your GUI *will* fail. The file may or may not upload.
br Given the modern web is *all about* uploading user submitted media, I am amazed there arent headlines "Mozilla forgets everything and rebuilds file upload in partnership with Apache...then thinks about HTML 5"

Re:They should spend more on the upload tool

[stimpleton]

"This halves upload speed.

Correction: This doubles the upload time. Changing the value to poll at say 1-5 second seems to introduce unrealibility, yet halves the time taken to upload".

Re:They should spend more on the upload tool

[corychristison]

Your files are too large. Facebook scales them down and the larger the file the more RAM and CPU cycles it takes to scale it down.
Just scale your photos down to 1024x768 or close to that depending on your aspect ratio and you shouldn't have any issues with multile files. It will also be much faster as your files will be tiny. My camera takes photos at 10MP an the file size is usually around 5MB. Scaling them down brings them closer to 200kB.

Re:They should spend more on the upload tool

[raju1kabir]

Just scale your photos down to 1024x768

Scale your photos down to 604x453, which is the size Facebook displays them at, and you will get to control the sharpness and image quality.

Upload at any other size, and Facebook will re-sample them with some very cheap algorithm and apply aggressive compression and they will look like ass.

Try it, you'll be amazed how much better your photos suddenly look.

I normally use "convert -strip -sharpen 0.3 -quality 85 -geometry 604x604" before uploading - it just takes a second, and makes a huge difference.

Re:They should spend more on the upload tool

[RajivSLK]

Just a heads up, there are other tools that can upload to facebook.... I use iPhoto on my Mac; just select the album I want to upload and click "upload to facebook". It's been a year since I've used the facebook uploader and you're right it's terrible.

Re:They should spend more on the upload tool

[larry bagina]

Flash has functionality to do file uploads with progress feedback. However, it's somewhat crippled in that it doesn't let you return data back, just that it worked or didn't work.

Re:They should spend more on the upload tool

[JohnnyBGod]

Modern users demand upload progress feedback. Which the HTML spec cannot do.

Flickr seems to manage...

Re:They should spend more on the upload tool

[crossmr]

That makes it slow. that doesn't make it blow up. I have no idea what makes it blow up. I'll select a group of photos, It'll be going along and then it'll "fail" on one. Sometimes, I remove that one, and it'll upload the other 20 photos just fine.

Yet there was no difference between them.

Upload that one all by itself and it'll just fail.

Re:They should spend more on the upload tool

[mini me]

Modern users demand upload progress feedback. Which the HTML spec cannot do.

True, but the browser is fully capable of providing this information within its own interface. It is amazing to me that none of the mainstream browsers provide this information feedback to the user.

Re:They should spend more on the upload tool

[mr_mischief]

It doesn't need to be 50-60k of code. It just happens to be that way because the developers get into the habit of using the same big JS library for everything. It's understandable, really, because that keeps development time down.

When it becomes a problem, though, the UI and class libraries can be broken out into smaller bits and pieces. There's no need to have animation, server-side file browsing, sorting, and a lot more in an upload progress bar. All the progress bar really needs on the client side is a request for a document, as it's really simple to calculate an upload status from the server side.

The only parts of your JS library you really need loaded client-side all the time are the cross-browser compatibility bits to account for stupid browsers with stupid implementations of the DOM or the HTTPRequest object.

This is one place where some companies could make their customers happier with just some minor tweaking to how they load their JS libs onto the client side and how fancy they get in rendering what are really basic UI updates.

Re:They should spend more on the upload tool

[Golthar]

Actually Chrome does; I'm using it to deploy large WAR files and it gives me a nice percentage on the left bottom of my browser

Re:They should spend more on the upload tool

[clone53421]

...except the facebook picture upload widget is written in Java, which means your argument is wholly irrelevant.

Re:They should spend more on the upload tool

[clone53421]

I’d mod you the rest of the way up if I could.

Re:They should spend more on the upload tool

[clone53421]

I’ve seen the same thing. Something about their resize / resample algo seems to make it blow up on certain images. Resizing them manually does seem to help (presumably it takes the load off their tool, or just randomly usually fixes whatever it was that was odd about the picture so as to screw up their algorithm).

It always fails after uploading the image, so if I had to take a shot in the dark I’d say it’s either trashing the picture or it’s writing a malformed file (or some JPEG variant that the server fails to open), and when the server gets the garbage or malformed image file it decides it’s not an image and tells you the upload failed.

Old news

[hmckee]

Not that surprising if you've read the book "The Accidental Billionaires". They specifically mention that there is one person dedicated to writing a PHP compiler and compiling all Facebook PHP.

Also, I don't understand why they don't use one of the currently available PHP compilers, phc or Roadsend. It's possible they started their initiative earlier, but they should have announced it and possibly prevented some duplicate work.

Resin Quercus

[parryFromIndia]

Caucho Resin has a mostly pluggable replacement for PHP which is written in Java. It adds web friendly features to PHP like distributed sessions and load balancing. Given the JVM JIT is already plenty fast and the benchmarks show that Java/PHP beats regular PHP handily - I wonder if Facebook considered using it at some point.

Re:Resin Quercus

[parryFromIndia]

Here is the URL in case people are interested in checking this out - http://www.caucho.com/resin-4.0/doc/quercus.xtp .
In summary:
It is OpenSource, 100% Java and it brings all the advantages of using a JVM to PHP - performance (JIT), Safety, Scalability (clustering/load balancing), quality tools (Development, Profilers). One can use most of the Java technologies in PHP to ease development even further - XA Transactions, JNDI, Connection pooling, object caching for example.

Besides, improving performance of this pure Java PHP implementation ought to be easier than improving the PHP runtime. (Java6 onwards the available tools to debug and optimize Java applications have made significant progress. jmap/jhat , easy heap dumps on OutOfMemory, Object Query Language etc. already come bundled with the JVM and then there are Eclipse and NetBeans GUI profilers.)

Also worth checking out Dr. Cliff Click's extensive Java vs. C performance blog post - http://blogs.azulsystems.com/cliff/2009/09/java-vs-c-performance-again.html .

Re:Resin Quercus

[julesh]

Caucho Resin has a mostly pluggable replacement for PHP which is written in Java. It adds web friendly features to PHP like distributed sessions and load balancing. Given the JVM JIT is already plenty fast and the benchmarks show that Java/PHP beats regular PHP handily - I wonder if Facebook considered using it at some point.

The main problem is lack of support for some of the quite popular extensions that are widely used in PHP apps. Just looking over the list of what's supported and what isn't, I find quite a few items missing that would be needed to support PHP projects I've worked on over the years, including raw cookie suport, unix filesystem integration functions (readlink and umask), call_user_method, the pspell module, the imap module, the mailparse module, the gnupg module, posix shared memory support, a rather large chunk of the stream handling functions, and by the looks of it the curl integration.

Re:Resin Quercus

[parryFromIndia]

The POSIX shared memory support should be unnecessary for the Java implementation - I think its required due to the multiprocess nature of Apache/mod_php? Also since you can call Java code from PHP for this implementation, finding or writing equivalent Java libraries for the missing pieces should not be that difficult. Plus there is always JNI - you can, for example wrap around libcurl in Java API and call the underlying libcurl using JNI?

It's not exactly plug-in replacement if you use a lot of that stuff - but it should be still worthwhile to invest in creating replacements for that stuff as a one time thing.

Re:Resin Quercus

[Billly Gates]

Resin still exists?

I have not heard about it in a long time.

Akamai sucks

[Turmoyl]

If Facebook really wants to speed up the customer experience all they need to do is remove Akamai from their content delivery network (CDN). That's where my browser is always stuck in a Waiting status when I notice a connectivity issue.

Re:Akamai sucks

[Eil]

As I understand it, Akamai peers with ISPs and backbone hubs, so the problem could be within your ISP's network or their upstream provider. If you did a couple of traceroutes, I'd bet that you would see fewer hops to the CDN node than to facebook.com or any other random address.

It's worse than weeks vs. months

[Anonymous Coward]

You are saying that it takes weeks to use assembly for something that you can code in an hour? That's gross understatement.

When assembly was used more in PCs, processors and programs were far simpler. Today, 4 cores is the standard even in desktop PCs! Things such as multithreading, etc. are essential basics of modern programming languages and can be handled exrtremely easily. However, properly designing, coding, debugging, testing and documenting those things on Assembly takes far more time than you implied. And then they are very platform dependant and need to be redesigned, modified and tested, etc. when you want to change servers... Or if you want to have different types of servers running the same code... It's a nightmare! Not to mention that it is harder to find developers for that, you need to pay them more, etc...

And how much you really win doing that on Assembly instead of using C or even C++? Very little! I am sure that the difference is notable in a system at the size of Facebook but they have obviously decided that it isn't worth it even for them.

php and threading!

[daveb1]

php and threading! ... um nevermind nothing to see here. Mean while efforts to get python running nicely without the GIL etc. are still going on without press.

Re:Three sources of scripting language inefficienc

[siride]

Eclipse isn't written in a scripting language.

What the heck version of PHP were they using?

[mgkimsal2]

From that article:

PHP is an example of a scripted language. The computer or browser reads the program like a script, from top to bottom, and executes it in that order: anything you declare at the bottom cannot be referenced at the top.

This was true in PHP3, but since PHP4, even declaring functions at the bottom of a file, they were still available at the start of a file execution. Everything got compiled in to an intermediate stage before execution.

Re:What the heck version of PHP were they using?

[clone53421]

Functions aren’t declared in PHP. If they’ve been defined, they exist. (Vs. C, where you can define a function, but until you declare it with a function prototype you can’t use it.)

The scripting explanation is needed for those incredibly dense people who somehow think that if they set $x = 5 at the end of their script, it should be equal to 5 everywhere. Variables don’t work that way, and scripting languages don’t work that way.

Freelance PHP Programmers?

[Anonymous Coward]

Anyone want to help me out with starting a PHP project? If so, please post your contact information here. I just need to consultation and perhaps initial coding.

"Java" and ".Net" as an inspiration? No, thank you

[Max_W]

If "Facebook" buys PHP and makes it similar to ".Net", i.e. kills it, it would be really sad.

Speed of "Java" and ".Net"? Is it a joke? "Java" hangs all the time and the ".Net" code to do a simple task is so convoluted that it is just ridiculous.

Re:"Java" and ".Net" as an inspiration? No, thank

[Abcd1234]

Speed of "Java" and ".Net"? Is it a joke?

No, it's not.

"Java" hangs all the time

No it doesn't.

and the ".Net" code to do a simple task is so convoluted that it is just ridiculous.

No, it's not.

Honestly, you really have no fucking clue what you're talking about, do you?

Re:Three sources of scripting language inefficienc

[lucifron]

Flaky UIs - click on a button and nothing happens. Or things not drawing properly. These are my observations.

Unresponsive UIs tend to be written by naive/ignorant developers who don't understand threading.

Re:"Java" and ".Net" as an inspiration? No, thank

[Anonymous Coward]

almost all java speed problem that I have seen were caused by developer ignorant of the java memory model
or by admin unqualified to administer (performance tuning is a part of administration in my book) an application server.
If you know when to use different garbage collector, you applications will never froze.

This will end badly

[LittleWebFoot]

Rewriting a managed runtime is difficult, costly, and time consuming. And at the end of the day, the odds that they will beat the performance of the CLR or JVM is super small. At best, they may match the performance of either of those runtimes, but at an enormous cost. Open sourcing it won't make a difference - it's already open source and has been for years. A much better investment would be changing the site to use a different technology that was faster. There are lots of options. The goal is to improve performance, not embark on a fun science project.

Re:This will end badly

[Anonymous Coward]

at the end of the day, the odds that they will beat the performance of the CLR or JVM is super small

These VM's are primarily designed to execute bytecode generated from compiled languages with static type systems. PHP is a dynamic language and is badly in need of a runtime like the ones mentioned in this blog post.

If I was looking to port something I'd written in PHP to a static language, I'd be looking at Google's go or at Vala (via LibSoup). At the end of the day, the odds that the JVM or CLR ever beat optimized native code are super small ;)

Re:"Java" and ".Net" as an inspiration? No, thank

[Max_W]

"Java" is a creation of "Sun" Co. We all use "Open Office" of the "Sun". It is free.

But did you try to switch on French or German language spell-checking in "Open Office"? I always think when I do it that their chief engineers should be really stupid. Indeed a little boy is needed to say that the king is naked.

As for ".Net" I do not trust it either. The creator of this language does not want the Internet to become what it can become, because it will damage their core business. Their natural interest is to slow down the Internet, at least for a year or two more (trillion dollars more). That is why the "Internet Explorer" is 4 times slower than other browsers. That is why ".Net" is so confusing, slow and obfuscating.

Oh, yes. They have 5000+ strong marketing department, which succeeded to convince you otherwise. No surprise, they spend billions on it each year.

But if the PHP will be bought out, as the MySQL has already been, it would mean the end of the Internet as we know it. Because this is the good open technology that just works.

You wanted the truth, here is the truth.

Re:"Java" and ".Net" as an inspiration? No, thank

[RoboProg]

Agreed. The JVM does an decent job, and C# has some pretty nice syntactic conveniences.

I ran a string whacking benchmark (posted on my website, roboprogs.com: "... faster ..." page), and, for a sequential task, Perl cleaned up handily. However, for the commonly available languages on *nix, Java outshone everything else for dealing with threads. Keep in mind this was a naive use of threads as well: fire up for a single task, rather than having a thread pool that looped over requests from a queue. Java still did a good job managing it.

(though I do wish Java had more alternatives for quick and dirty coding at times - weakly typed options, delegate/function pointer type stuff -- still need to look more into jRuby, Scala, Groovy, et al)

Re:"Java" and ".Net" as an inspiration? No, thank

[raju1kabir]

"Java" hangs all the time
No it doesn't.

I think one of you is talking about client-side Java in web browsers (which is the worst thing in the universe) and one is talking about server-side Java.

Java Implementation

[Anonymous Coward]

I was hoping this would be a fully open version of the PHP interpreter implemented in Java, maybe with a bytecode compiler.. Much like Quercus or Project Zero.

Re:"Java" and ".Net" as an inspiration? No, thank

[Anonymous Coward]

Come on. I've used Java. I always have to install some crap on my browser, usually about 12 MB of it. Then it doesn't work.

At work there's software that pops up from IE. This always crashes. Heaps of out software is written in Java because the developers are lazy and stupid. It always crashes, uses inconsistent controls, and just doesn't feel write. Trying to open up two of the same app will crash the computer.

Java is useless. It might work in a lab environment, but in the real world it is a buggy, naggy, ugly crutch for useless programmers.

Re:Three sources of scripting language inefficienc

[amn108]

And why is that important exactly?

Its UI is sluggish, and the whole thing is a living example of leaking abstraction gone wrong.
Might as well be written in PHP.

Whoosh Re:is this being used now?

[Anonymous Coward]

with every attempt to "improve" things [Facebook] somehow manages to make it worse and worse. They're a perfect candidate for a Microsoft buyout.

Microsoft isn't exactly known for speed either.

Yeah...

Re:Three sources of scripting language inefficienc

[siride]

There is a lot of crappy software written in C/C++. It's easy to make sluggish UIs and bad abstractions. But that's not the point. The fact of the matter is, Eclipse is not written in a scripting language, it is written in a compiled (or what might as well be compiled) language with strong typing and a slightly more dynamic runtime than what you get with C++ and comparable speed. So bringing up Eclipse as an example of the problems with scripting languages is just plain invalid.

Re:"Java" and ".Net" as an inspiration? No, thank

[idontusenumbers]

I think you're comparing to a poorly written Java application instead of a good Java application or Java itself. Applications can hang and crash in any language.

Re:"Java" and ".Net" as an inspiration? No, thank

[Anonymous Coward]

this is just contradiction!

Re:Three sources of scripting language inefficienc

[amn108]

I did not bring Eclipse up because it is scripted. I brought it up because it has flaky UI. The parent poster needed citation. I believe the amount of abstraction layers involved in Eclipse code in combination with the cost of the Java primitives used by each of these layers, are what bring Eclipse to such crawl. This is a potential problem with any scripted language as well - if you interpret something, the cost of primitives is high, and stacking APIs on top of each other is going to cost you (and your users) plenty. C/C++ get away not because the software is written better, but because the primitives cost less. And here is where I get back at the Eclipse issue - if Java program has this kind of performance, and it is written by a good team, I believe scripted language software is to expect the same set of problems. You need to make sure that if you do abstract things in your project, you understand the cost involved with each additional such layer. Also, you should not pay for features you do not use (the zero overhead principle.) Native code is the true God. You can start with whatever script you desire, but make sure you can reduce it to something that can run fast. The problem is thus not necessarily in languages (most of them offer the same superset of needed features), but in implementations.

Re:Assembler? Really?

[metamatic]

Assembly language isn't platform-independent. It's really easy to screw up and hard to optimize. And it's not much faster than C/C++.

I wouldn't call C or C++ platform-independent either. Not without a huge amount of work and lots of precautions.

Re:"Java" and ".Net" as an inspiration? No, thank

[nacturation]

As for ".Net" I do not trust it either. The creator of this language does not want the Internet to become what it can become, because it will damage their core business. [...] You wanted the truth, here is the truth.

.NET is a runtime environment, not a language. Examples of languages that run within the .NET environment are Assembly, C, C++, Haskell, Lisp, Perl, Python, Scala, and many many others. Speaking of the truth, you might want to do your homework before claiming you know it.

Re:Assembler? Really?

[that this is not und]

even embedded systems pack tons of memory now so I don't see that being an issue.

Talk to Microchip, STMicro, and Freescale about that. They sell billions of micros per year with less than 1K of general purpose read/write memory. I've written apps for chips with 24 bytes of RAM.

Re:Assembler? Really?

[thechao]

Hold on, there, Sparky. Assembly, C/C++, your-favorite-compiled-language ... are all just as "fast" as each other. They're all dependent upon the compiler for "speed". I recently moved to Big Company where we do Serious Embedded Programming. The old-skool assembly programmers were always bitching that "compilers were no good" and that to get real speed you had to "program to the iron." Here's what I observed: they redesigned the algorithms when moving to assembly. So, basically, they had algorithmic improvements. This was generally true except, in certain cases, when it was impossible to completely annotate the "hot" variables and how register spill/fill should be performed. In this case I suppose we are just dealing with a lack of expressiveness, and not any sort of fundamental problem with "high level" (or lower level) languages.

Re:Assembler? Really?

[Billly Gates]

In today's modern processors you wont gain much performance in assembly. A core2duo simply reads the x86 instructions and converts them to risc and much of the optimizations happen at the compiler and during execution on the fly. You can always gain some speed but its nowhere near what it could do just a decade ago.

What also needs to be taken into account is the costs and time to rewrite years of development work from scratch. Sunken costs drive accountants crazy and threaten the job of any IT manager.

Instead of starting from scratch its better to use tens of millions of dollars of existing code.

Its nice from an engineer perspective but facebook is a corporation and money needs to come first and foremost.

Also assembler can crash a system and freeze it. The point of switching to NT or Unix was the point of stability of using c api's that are managed rather than using Windows95 which had assembly code that could freeze your computer.

Missing the forest for the trees

[baptiste]

250 comments later and all we get is arguments about what Facebook 'should' have done to resolve specific architectural problems none of us really know. Facebook is growing like wildfire. It has it's issue, some big. Many features are 'bad' (ever tried to run pop out chat? Jeebus my CPU cried for mercy) But it's quickly becoming THE go to site for millions of people. So Facebook has growth beyond their current ability to scale and they decide that rewriting PHP is a possible answer. The agree to open source it. Isn't this *exactly* what makes FOSS so great? Everyone benefits from the efforts of those using the code for their needs. Will this rewrite mean a global replacement to PHP's current implementation? I doubt it. But it may be just what is needed for many other sites with growing user bases and less $$$ for HW. Again, this is a bad thing because... If some random guy in a basement had done this, he'd be a borderline hero. But because a large corporate entity did, it's suspect and bad. I for one look forward to seeing what they really did and hearing from the PHP developers who attended the meetings as to what they are really doing, what types of bottlenecks they found, and what ideas they had to resolve them. Will they be 100% right? Doubt it. But in the end a large corporation is contributing back to the community, and potentially in a big way if their rewrite is widely usable. This. Is. A. Good. Thing.

Funny how it all works out...

[hesaigo999ca]

Way back when I was first programming PHP came out and all was miraculous, everyone jumped on that band wagon saying it was the fastest you could get, because it was like running c on your server. Well now 6 generations later, we see the bogged down version of an
older lighter creature turned monster. How did this happen? I remember /. covering so many years worth of PHP stories for all the version and patches and upgrades and next version, how it was doing this and that, we had to take this out, need more security,...and oh yeah the doosy, going from one version to another broke so much code because it was no longer backwards compatible...4 to 5 i think....anyways, now I am seeing that the monster has gotten so out of control, that someone has decided to castrate it and make it a little lighter....problem is those balls only weight a few ounces....you need either a much older version of itself...which would be insecure, or a completely new product (ruby?) to go the distance with pure speed.

"Don't hate the coder......, hate the language"

unlock

[JumpSocial]

If they would just unlock my account I'd say they of a great site... I find that the client side is slow but I've never really had any issues with the back end speed.

Re:Three sources of scripting language inefficienc

[Anonymous Coward]

Appropriate libraries can reduce or eliminate the advantage that scripting languages have. I don't know of any web template libraries for c or c++ but I'm sure they exist.

10 years of one language does not make good coders

[Anonymous Coward]

But, no, I guess I'm just a kid who doesn't know how to code.

Don't worry, with 10 years of java devel you're most likely an adult who doesn't know how to code.

If you haven't written production code in at least ten languages, you're still a beginner. You can cut that down to three languages if one of them is LISP and one of them is COBOL, but personally I recommend against the short painful route. No single language is enough to make one a good programmer, and very similar languages aren't much help either.

Languages vs. Programmers vs. Hardware

[BugHappy]

1. Some bet on the Language (asm
2. Others bet on the Programmers (whatever the language an idiot remains an idiot).
3. The rest bet on the Hardware (servers are cheap, unlike humans -well, Facebook disagree this time).

All those points make sense -but presented in isolation they miss the real thing:
what can happen when you have C (asm will be only 20-30% faster), experienced coders, and powerful hardware?
Answer: you redefine the performance standards.
Why would it make sense, be needed, or only be desirable?

Because we will soon not have any other choice.