Slashdot Mirror
NASA Runs Competition To Help Make Old Fortran Code Faster (bbc.com)
NASA is seeking help from coders to speed up the software it uses to design experimental aircraft. From a report on BBC: It is running a competition that will share $55,000 between the top two people who can make its FUN3D software run up to 10,000 times faster. The FUN3D code is used to model how air flows around simulated aircraft in a supercomputer. The software was developed in the 1980s and is written in an older computer programming language called Fortran. "This is the ultimate 'geek' dream assignment," said Doug Rohn, head of NASA's transformative aeronautics concepts program that makes heavy use of the FUN3D code. In a statement, Mr Rohn said the software is used on the agency's Pleiades supercomputer to test early designs of futuristic aircraft. The software suite tests them using computational fluid dynamics, which make heavy use of complicated mathematical formulae and data structures to see how well the designs work.
But a bit of googling shows that there's still more than enough justification to call it the best programming language for physics simulations.
So... there will be Fortran programmers out there. I'd suspect, though, given that it's maintained a niche in high-end physics simulation, that anyone who would program in Fortran at the level required here currently has a job doing just that, and won't have time for a major side project with an unknown probability of paying off.
C++, but Fortran's pretty much the same for all but a few cases... where it's markedly better. Apparently it's also easier to learn and has a few other practical advantages.
So the answer is that there ISN'T a replacement yet. Fortran is the programming language of choice for large physics simulations at present.
"If you can make my simulation code run 10,000 faster, I'll give you Fifty Five Thousand dollars!"
1. Run it on better hardware.
2. Re-write the compiler to optimize this code in the best way possible.
3. Re-write the code so it provides optimal input to the compiler.
4. Come up with a new algorithm.
5 and beyond: Left as an exercise to the reader.
Assuming any improvements from #1 and #2 don't "count" for this contest, that leaves you with 3 and 4.
Unless the code is brain-dead there is no way you'll get anywhere close to 10,000 improvement JUST by #3. You MIGHT get it with a combination of #3 and #1 and/or #2 vs. just #1 and #2 alone. That is to say, changes in hardware and compilers may give an opportunity to re-factor the code to get huge improvements vs. un-modified code on new compilers and new hardware.
The big win will be in #4, but only if there are better algorithms out there or someone can come up with one. As with re-factoring the code, changes in hardware and corresponding changes in compilers may turn an algorithm that was inefficient in the 1980s into something that is best-in-class today.
5 and beyond are open-ended and the sky is the limit.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
"This is the ultimate 'geek' dream assignment,"
Actually it sounds like what I call "work."
"First they came for the slanderers and i said nothing."
I entered the contest and made some modifications but it runs well over 10,000 times faster. Disqualified again! >:(
Anons need not reply. Questions end with a question mark.
They want a 10000 times speed up. They would not ask for such if the capability to do that was just lying on their hardware, idle. They either already have the GPU cores to make the calculations, and those are not in use - or they at least expect the same approach, but to make use of multiple-cores that are similarly idle. Otherwise asking for that kind of improvement is simply idiotic. (I think that for this price it is idiotic anyway, they should proper fund that thing).
-><- no
Pleiades is just a slurm cluster. You can deploy new hardware, push your processes to be queued to the new hardware and link your run time to use cuda enabled libraries no problem.
Switching link time and link time hooks is pretty easy on Pleiades. It's the same way you would switch between python versions, different fortran run times (intel and gnu), different mpi runtimes (openmpi/sgi enabled stuff).
Source: worked on Advanced Vehicle Make and had to run cfd code on Pleiades.
Um, towards the bottom of the article it mentions "10 times faster" which is probably achievable and probably what they actually are looking for. They even mention that rewriting an inner loop, shaving a few milliseconds off will give a substantial speed increase.
Way back I worked at Alliant Computer Systems (https://en.wikipedia.org/wiki/Alliant_Computer_Systems) which was a hardware/software system that would automatically unroll loops in Fortran code and run them in parallel on up to 8 processors at a time (and each of the 8 also had vector hardware). It was very fast on the right Fortran code... Hardware support for concurrency control when one iteration of the loop depended on a computation from another iteration. All done in custom hardware which was killed (like many supercomputer companies) by the advent of high performance microprocessors.
Jesus tits.
Both of you should google 'Pleiades supercomputer specs'. 100 nodes of 10,000 have GPUs. They are currently at near 100% utilization. The do have enough GPU equipped nodes for someone to prove the speedup and for them to start slapping in GPUs and upgraded power supplies.
Alternatively: Spin off the numeric parts to SSE. Use fixed point math routines.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
You don't know how tax brackets work. At least in the USA.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
They want a 10000 times speed up.
To be fair, NASA did not say that. They said a 10 times speed up. They also said they are hoping that maybe it could be sped up even more, perhaps even by a factor of a thousand. So the journalist took the "10" and the "thousand" and multiplied them together. Then that inflated made-up number was copy-pasted into the summary.
WTF is up with slashdot?
Not even a link to the actual contest page. Not that anybody in their right mind would touch it, I might signup. Just to look at how bad it was, perhaps to laugh/cry a little. Bet I've seen worse. Bet it doesn't use calculated GOTOs (Fortran 'feature' Goto IntVar, where IntVar contains a line number).
I bet hidden at the bottom is a linear problem (LP) solver that bangs on sub simulations, getting those results to converge across cells faster would be another trick worth trying. Just understanding how the aero problem gets approximated to an LP would likely take longer than the contest duration though.
This whole contest is bait for the retired old bastards that wrote the thing in the first place. They think one of them might have a trick or two, still up his sleeve.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
Old FORTRAN's default statically allocated arrays are one less indirection to access.
Also, FORTRAN does not allow pointer aliasing, so memory accesses can be cached in registers. That is a barrier to optimization in C/C++. C has the "restrict" keyword which helps, but it is rarely used and is not supported by old compilers.
$55,000 is much cheaper than hiring developers to do it. It's akin to companies having a contest for a new marketing logo, and the winner get's their work used. The compensation is a line on a resume.
Where I work (we have tons of legacy Fortran code), generally C++. However, the real answer is not to bother. If you have a working Fortran compiler, the old code works just fine. "Porting" it to another language does nothing for you but potentially introduce new bugs. At best you'd just have the same functionality you did before (as that is the goal of a port), but we all know the best never happens. Why go through a really expensive effort that will do nothing for you but make your product worse?
However, we do all brand new work in C++, so it will slowly take over.
I suspect most COBOL houses will tell you the same vs C# (or Java or whatever your current language du-jour is).
What would you tell a mechanic who wants $400 to change your perfectly functional alternator with a new one that works exactly the same, but uses the latest in alternator technology, so mechanics will enjoy working on it more? I know I'd tell him if the old one breaks and can't be fixed, put a new one in. Otherwise, I have better uses for my $400, tyvm.
Don't listen to him, he'll be stone dead in a minute.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
System Architecture
Manufacturer: SGI
161 racks (11,472 nodes)
7.25 Pflop/s peak cluster
5.95 Pflop/s LINPACK rating (#13 on November 2016 TOP500 list)
175 Tflop/s HPCG rating (#9 on November 2016 HPCG list)
Total CPU cores: 246,048
Total memory: 938 TB
2 racks (64 nodes total) enhanced with NVIDIA graphics processing units (GPUs)
184,320 CUDA cores
0.275 Pflop/s total
1 rack (32 nodes total) enhanced with Intel Xeon Phi co-processors (MICs)
3,840 MIC cores
0.064 Pflop/s total
Operating Environment
Operating system: SUSE® Linux®
Job scheduler: Altair PBS Professional®
Compilers: Intel and GNU C, C++ and Fortran
MPI: SGI MPT
Full specs here.
Sounds like stone soup to me. CUDA cores, Phi coprocessors, SGI interconnects, Linux OS because nothing else in the whole wide world could talk to all of that...
Ick. Keep your prize money.
Weaselmancer
rediculous.
When you cross into a new bracket, you don't pay that rate on all your income. Only the income above the cutoff.
Some payroll programs are _stupid_, it is possible to get a raise and see a smaller check. But at the end of the year, you get a bigger refund, all other things being equal.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
Actually, it is Fortran. Maybe it was FORTRAN when FUN3D was originally written, but now it's most certainly just Fortran.
Or FORTRAN because early keyboards couldn't do lowercase letters?
The problem with FORTRAN was the punch card formats popular for FORTRAN was BCDIC which has no encoding for lower case and the machine FORTRAN was designed for (IBM704) used a 6-bit BCD code (6 of which were packed into a 36-bit word).
The FORTRAN-77 language still required UPPERCASE keywords, but most compilers (including ratfor) accepted lower-case keywords, but by Fortran-90, lower-case keywords were officially supported and thus in tribute to that long awaited change, the official capitalization of the language changed.
Since the FUN3D project was started in the late '80s, it started in FORTRAN-77, but 2001 they converted to Fortran-90. The latest versions require a Fortran-2003 compiler
The original punch-key machines of the day for punch-card FORTRAN didn't have lower case because the early cards didn't support it. On the other hand computer keyboards being essentially retasked electric typewriter keyboards generally supported lower case from the get go...
Yep, this problem has JavaScript written all over it.
Sometimes, years after you finish a job (that you never really liked the solution to) a better solution comes to you.
Also: In the 1980s they didn't have a 10,000 node cluster, Multicore CPUs, SSI, Nvidia, ASICs, FPGAs or Gigabytes of RAM. They barely had math coprocessors and blitters.
If at the end of a job, you don't know how to do a it better, that proves you haven't learned anything since you started.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
I thought: ah, a fun challenge.
Then I read the last sentence in the BBC article:
>The sensitive nature of the code means the competition is only open to US citizens who are over 18.
They just don't get it.
You still don't know how tax brackets work.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
When you try to download their software, you are taken to this page which at the bottom contains the follow text:
By accessing and using this computer system, you are consenting to system monitoring, including the monitoring of keystrokes. Unauthorized use of, or access to, this computer system may subject you to disciplinary action and criminal prosecution. [emphasis mine]
A keylogger for using your website? Microsoft hasn't even thought of that yet!
-- Political fascism requires a Fuhrer.