Slashdot Mirror

The guy is a legit one-name wonder.

Another difference is that we are talking about supercomputers here.

No. This figure is probably based on the actual rates that the supercomputing facility charges to research projects. For example, look at:
http://www.nersc.gov/users/acc...

Dunno about "top secret", but the DoE puts a huge amount of computing resources into physical simulation. Check out some of the NERSC projects (GTC, for example).

This machines are most likely goign to be the replacement of the ones we already have. NERSC is presenting the projects that are run on its computing infrastructure on its web site [1]. You can see on the first page the project that are currently running jobs and what they are doing. For instance this project [2] is about designing artifical photosynthetic cells. If you are interested just check the project they are funding.

[1] https://www.nersc.gov/
[2] https://www.nersc.gov/science/energy-science/artificial-photosynthesis-i-design-principles-for-light-harvesting/

This machines are most likely goign to be the replacement of the ones we already have. NERSC is presenting the projects that are run on its computing infrastructure on its web site [1]. You can see on the first page the project that are currently running jobs and what they are doing. For instance this project [2] is about designing artifical photosynthetic cells. If you are interested just check the project they are funding.

[1] https://www.nersc.gov/
[2] https://www.nersc.gov/science/energy-science/artificial-photosynthesis-i-design-principles-for-light-harvesting/

as someone with access to some national supercomputers, how fast can hopper mine bitcoins? Only one way to find out.
$ qsub xminebitcoin_mpi.hopper.pbs
PBS Job Id: 8005323.hopper02

You mean like this?

There is no DEADBEEF in the first 4 billion digits of pi. but there is a DEADBABE.

You mean like this?

There is no DEADBEEF in the first 4 billion digits of pi. but there is a DEADBABE.

You mean like this?

There is no DEADBEEF in the first 4 billion digits of pi. but there is a DEADBABE.

Not sure if I like this method of measuring processor usage since a project that took a million hours in 2001 wouldn't take a million hours in 2010 but that's what's in the article.

Usually the value of a cpu-hour depends on the current cluster hardware. At NERSC I think they revalue the cpu-hour annually when they allocate time for the next year. http://www.nersc.gov/hypermail/all-announcements/1013.html http://www.nersc.gov/nusers/accounts/mpp-charging.php

Not sure if I like this method of measuring processor usage since a project that took a million hours in 2001 wouldn't take a million hours in 2010 but that's what's in the article.

Usually the value of a cpu-hour depends on the current cluster hardware. At NERSC I think they revalue the cpu-hour annually when they allocate time for the next year. http://www.nersc.gov/hypermail/all-announcements/1013.html http://www.nersc.gov/nusers/accounts/mpp-charging.php

If climate modeling is so open, why is a login needed and "Ultimate permission to utilize this data is reserved by NCAR. "?
US DOE Coupled Climate Model Data Archives

A nice port-scanning secviz realtime animation was mentioned (I think here?) back in 2003. See this paper (images and animations are at the bottom) from DOE/LBNL/NERSC.

Now, the CX1 really is Cray in name only. Don't make the mistake of thinking of Cray as a maker of itty bitty clusters. Oak Ridge has a >30,000 core Cray XT4, NERSC has an almost 20,000-core XT4, and of course Red Storm has over 26,000 cores.

Umm, that is Cray in name only. The real Cray (Seymour) designed from scratch computers which had performance as their only goal, and which shattered the performance standards of their peers. He did not cobble together existing parts in basically the same way a half dozen other companies do. Now, you can argue whether something like the old Cray is even possible anymore, let alone economically viable, but you shouldn't pretend that the current Cray is in the same league.

Are you trying to imply that Cray the company is "in name only?" Because that's not at all the case.

It's true that Cray was a shadow of its former self after Tera bought it, but many of the Tera executives have left, and some of what Cray Research used to be has re-emerged.

Now, the CX1 really is Cray in name only. Don't make the mistake of thinking of Cray as a maker of itty bitty clusters. Oak Ridge has a >30,000 core Cray XT4, NERSC has an almost 20,000-core XT4, and of course Red Storm has over 26,000 cores.

Use this search engine and you can report all of the positions in the first few billion decimal places that string (or any other) appears.

NERSC's primary resource is another Cray XT4.

In fact, a lattice QCD problem was one of the model problems for the Track 1 proposals. Proposers had to "provide a detailed analysis of the anticipated performance of the proposed system on the following set of model problems...A lattice-gauge QCD calculation in which 50 gauge configurations are generated on an 84^3*144 lattice with a lattice spacing of 0.06 fermi, the strange quark mass m_s set to its physical value, and the light quark mass m_l = 0.05*m_s. The target wall-clock time for this calculation is 30 hours." Full details here.

This is a Big F-ing Problem that does in fact require Big F-ing Computers to solve. To meet the target time would require at least a petaflop of sustained performance; hence the inclusion of this problem in the call for proposals. The other model problems came from CFD and molecular dynamics, and there was a wide range of smaller required problems as well.

Now, none of this explains how these machines will really be used, or to what end. Nevertheless, I can vouch for such large machines being used under heavy load to solve very large problems. Poke around any of the national supercomputing labs' websites, and you should be able to find at least plenty of news releases, if not papers.

Here are some quick samples:

• ASC at Lawrence Livermore National Lab (home of BG/L, Top500 #1)
• NCCS at Oak Ridge National Lab (home of Jaguar, Top500 #2)
• Sandia National Lab (home of Red Storm, Top500 #3)
• NERSC at Lawrence Berkeley National Lab

http://pi.nersc.gov/cgi-bin/pi.cgi?word=0000&forma t=hex

The real trick would be setting up cues so you don't lose track of where you are.

Also cool is http://pi.nersc.gov/, which allows you to seach digits of pi (converted to letters) for words and such.

One other tiny "motherhood and apple pie" good practice I thought of is that some (most?) compilers have flags you can turn on to generate various runtime checks (eg: http://www.nersc.gov/vendor_docs/ibm/vac/compiler/ ref/ruoptchk.htm ). They usually aren't all that useful, but on the other hand they are "free", they can be used at least during nightly regression-testing without any extra effort.

How about The Spinning Cube of Potential Doom?

Dunno about server, but I was wondering about a new desktop...
http://www.orionmulti.com/products/specs_ds96
- Performance 230 GFlop peak, 110 GFlop sustained (Linpack) -

What else ??? ah, yes, an ugly as hell server I wouldn't mind this winter : http://www.iwill.net/product_2.asp?p_id=90
Dunno if many people are into rack server modding, but you cannot make it more ugly than it is on the outside....

The inside can host 16 Opteron 800 cores and 128Gig of Ram, which make it pretty hot - both senses.....

For the Network War room, what about a nice visual representation of network attacks/activity, using "The Spinning Cube of Imminent Doom" which is both impressive and easy to explain...:
http://www.nersc.gov/nusers/security/TheSpinningCu be.php
(maintenance of servers going on, use the cache, luke...

Personnaly I like people that forego LCDs and such and directly use a nice and silencious video projector for general informations (say a Sanyo Z3).
Even more if you show your skills at system management using "Doom, the Aftermath"http://www.cs.unm.edu/~dlchao/flake/doom /after.html

Yeah, follow my words, I can garantee your customers will be impressed 8)

It's no big secret that many of the most powerful supercomputers are not shown off to the press. For example, in regards to the comment about cryptography, the NSA reportedly has enormous supercomputers that are never shown to the press. And of the ones that are, few are connected to the outside world. I believe right now the fastest is the Seaborg computer at the Berkeley Lab.

The most pressing open question about is whether it is a normal number -- whether any digit block occurs in the expansion of just as often as one would statistically expect if the digits had been produced completely "randomly". This must be true in any base, not just in base 10. Current knowledge in this direction is very weak; e.g., it is not even known which of the digits 0,...,9 occur infinitely often in the decimal expansion of .

Bailey and Crandall showed in 2000 that the existence of the above mentioned Bailey-Borwein-Plouffe formula and similar formulas imply that the normality in base 2 of and various other constants can be reduced to a plausible conjecture of chaos theory. See Bailey's web site for details.

ScienceNews article (2001) on Randomness of Pi's digits

Interesting work from Johan on Testing the a-periodic randomness of and comparing it with a Quantum Mechanical source.

But are the digits truely random ? In 1996, NERSC Chief Technologist David H. Bailey, together with Canadian mathematicians Peter Borwein and Simon Plouffe, found a new formula for pi. This formula permits one to calculate the n-th binary or hexadecimal digits of pi, without having to calculate any of the preceding n-1 digits. This formula was discovered by a computer, using Bailey's implementation of Ferguson's PSLQ algorithm

It's not a very good one, as it "inflates" the number instead of compresses it.

pi search

Use pi search to find the number. The index into pi is the compressed value.
the 'PPB' algorithm

The PPB algorithm can extract the data without having to calculate pi.

The problem is that with any random or psuedo random sequence, the probability for finding any large amount of data is tiny, and the index would be larger than the actual data.

It inflates the info :(

Here's the fixed links:

http://www.nersc.gov/

http://www.intel.com/technology/infiniband/

If all you want to do is search for mystic stuff inside the number, you don't need the CD with its measly 1.7bi digits.

Save your bandwidth and just go here to search within 4bi digits.

While I was graduate student in the 1980's there were a bunch of the most powerful Cray supercomputers at Los Alamos running this OS.

No. They were running the Cray Time Sharing System, as mentioned in this article:

The Center acquired a Cray 1 in 1978 and soon became known as an innovator in the management and operation of supercomputers. We converted our 7600 operating system, utilities, and libraries to the new machine, creating the Cray Time Sharing System (CTSS) -- the first timesharing system for a Cray -- and demonstrating that the machine could be used interactively. CTSS was subsequently adopted by nine other computer centers.

Different CTSS - "Cray", not "Compatible".

I think there was even a variant, LTSS that was run at Livermore.

That appears orignally to have been an OS for Control Data machines, according to this article, although it says it was moved to the Cray-1 as well:

LLL has a long history of pushing the state of the art in high speed scientific processing to satisfy the prodigious raw processing requirements of the many physics simulation codes run at the laboratory. The high speed, often few of a kind computing engines (For example, Univac-1, 1953, Larc, Remington Rand, 1960, Stretch, IBM, 1961, 6600, CDC, 1964, Star-100, CDC, 1974, Cray-1, Cray Research, 1978) utilized at LLL are usually purchased before mature operating system software is available for them [22]. The very early operating systems implemented at LLL were quite simple and were usually coded in assembly language. By the time of the CDC 6600 (1965), however, they were becoming more comp1ex timesharing systems. By 1966 it was decided to write the operating system for the 6600 in a higher level language. This decision made it easier to transfer that system (dubbed LTSS, Livermore Time Sharing System) to new machines as they arrived: CDC 7600, CDC Star-100, and the Cray-l.

Right now there are a lot of file systems that do somehing not all that different than what Sun is proposing. The project I am on is evaluating them as we speak for a center wide filesystem. I've had the fun (no sarcasm, honestly) of setting up a number of different onces and helping to run benchmarks and tests against each. All of them have strengths. Every single one of them has some nasty weaknesses.

If you are looking for an open source based cluster file system, Lustre is what you want. It's supported by LLNL, PNNL, and the main writers at ClusterFS Inc. It's a network based cluster FS. We've been using it over GigE. However, we've found that there needs to be a ratio of 3:1 for data server:clients for a ratio. Wehave only used one metadata server. Failover isn't the greatest. Quotas don't exist. it also makes kernel mods (some good and bad) to do a mild fork of the linux kernel (they put them into the newer kernels every so often). It only runs on Linux. Getting it to run on anything else looks...scary.

GPFS runs on AIX and Linux. Even sharing the same storage. It runs and is pretty stable. it has the option to run in a SAN mode or network based FS. In the latter form, it even does local discovery of disks via labels so that if a client can see the disks locally it will read and write to them via FC rather than to the server. It, however, is a balkanized mess. It requires a lot more work to bring up and run: there is an awful lot of software to configure to get it to run (re: RSCT. If you haven't had the joys of HATS and HAGS, count yourself very, very lucky).

ADIC's StorNext software is another option. This one is good if you are interested in ease of installation, maintanence, and very, very fast speeds (damn near line speed on Fibre channel). I have set this one up for sharing disks in less than two hours from first install to getting numerous assorted nodes of different OS's to play together (Solaris, AIX, Linux). It freakin on virtually everything from Crays to Linux to Windows. It's issues seem to be scaling (right now doesn't go past 256 clients) and it has some nontrivial locking issues (righting to the same block from multiple clients, and parallel I/O to the same file from multiple clients if you change the file size).

There are some others that are not as mature. Among them are Ibrix, Panasas, GFS, and IBM's SANFS. All of them are interesting or promising. Only SANF looks like it runs on more than Linux though at this point. Our requirements for the project I am on are to share the same FS and storage instance among disparate client OSes simultaneously. This might not be the same for others though and these might be worth a look. Lustre dodges this because its open source and they're interested in porting.

This seems like common sense. Shouldn't all network admins be doing this anyway?

Yes of course. You should spend an hour a day in silent contemplation of the "The Spinning Cube of Potential Doom".

BTW, If you think common sense is common, your sample size is to small.

Let me give a little commentary about what's in the sample cube pic. (BTW, does anybody have a mirror of the animation?)

We have a 3 dimensional cube shown on a 2 dimensional display, so the image can be a little confusing. Every dot represents a connection attempt to a machine at the conference, presumably mostly laptops being used by attendees. Successful connections are shown in "white" supposedly, but on my display they look gray. The colored dots are all unsuccessful connections, connection attempts where the machine did not respond. The presumption is that the vast majority of these are attacks and scans.

The left to right access represents the IP address of the machine at the confernece being attacked. Back to front is the IP address of the machine doing the attacking, from out on the internet. Bottom to top is the port number. To aid in viewing, the unsuccessful connections are shown in a color that represents the port, i.e. their height in the cube. That's all the color means. Red and orange are at the bottom for low numbered ports, then through yellow, green and blue in the middle ports, up to purple and back to red at the top for high number ports.

Now let's take a look at the picture. The main feature that jumps out is that most of the dots are colored; there are a lot more attacks than successful connections. Presumably these laptops are not hosting many legitimate servers. Second, we see that most of the dots are orange, meaning that they are attempts to connect on low numbered ports. That makes sense, as most services listen on standard low numbered ports of 1024 or less, or a bit more. That's why we see so many orange dots. Those are attempts to connect to web servers, mail servers, various Windows services that are known to be vulnerable, etc.

Another feature of the orange dots is that they are largely clustered towards the back, which would mean that the attacks are coming from Internet addresses which are relatively low in the address range. Looking closely, I make it out to be about 1/4 of the way from the back to the front, which would correspond to IP addresses of around 64.X.X.X. If we look at the first field of IPV4 addresses, ARIN (North America) has 24, then 63-70; APNIC (Asia/Pacific) has 60-61; RIPE (Europe) has 62, then 80-84, and all of them go on up from there. I'm not sure of the worldwide distribution of IP addresses but I suspect that accounts for the fact that many of the attacks and scans are coming from the 60-80 range or so, on the graph. There's another cluster of IPV4 address assignments in the 198-222 range, and that corresponds to a weak cluster of orange dots near the front of the cube, at the bottom.

Another feature we can see is some vertical structure in the blue and cyan dots, especially to the left and the right. These represent port scans, where a particular host machine is making connection attempts to a series of port numbers on a particular target machine. Such scans show up as vertical lines. Here we don't have a full line but only aligned dots, so we may be missing some packets, or the scan may be accessing only selected ports.

Well, that's about as far as I can go with my analysis. But you can see that if you had a real-time display of the last N minutes or seconds of activity, it would show you a visual picture of scans into your network. Probably be pretty hypnotic. Of course I'm not sure it makes sense to pay somebody to stare at it all day... you'd probably want to run a sped-up version at the end of the day and see if anything untoward leaped out.

See the cube in action here.

You spin me right round baby right round

Wonder if they've got one of these monitoring DOS attacks now that they've been posted on Slashdot.
Here's the 31 meg AVI if you want to make it spin faster.

IBM has put out documentation inspired by Adams' before: like this

Pretty cool move from Big Blue.

The national energy research scientific computing center (NERSC: http://www.nersc.gov ) is actively working on creating computing clusters capable of a petaflop within the next five years (I can't seem to find a reference on their website, but as I understand it, this is one of their oeprating goals.) NERSC facilities are used extensively to model the kinds of processes involved in this sort of fusion as well as others (The z-pinch, for example, http://zpinch.sandia.gov/ ) I'm sure groups are using this facility to do computation for magnetically confined fusion, and certainly all theses tasks are being worked on by other groups. The Princeton Plasma Physics Lab does much work in all these fields, and their website has links to many other sites devoted to plasma and fusion ( http://www.pppl.gov )

The home page at Cray for the Cascade project.

There are some interesting PDFs there. Chew, mull, and consider.

Also consider what Horst Simon, head of NERSC said here too.

IBM does have their own super-dooper compilers, the newest of which supports "...32-bit and 64-bit architecture and optimization technologies are supported.". It has gone through 6 revisions like this; the only issue is it is AIX only, hence Apple isn't using it.

But, the IBM AIX compiler does pump out some seriously good code. GCC, on the other hand, is well known for being a bit of a slow poke; the reason it is used everywhere isn't its quality, but its price.

Interesting to note is that #3, #6, and #8 are all linux clusters. All three of which are at Livermore.

Cray's X1 also debuted, but it was much lower @112. However, it ought to be noted, that the examples out so far are only 60 processors at tops. As soon as the money gets ponied up, prolly at ORNL, they'll be waaaay up towards the top. My guess is, if all goes as planned, they'll be at #15 by year's end.

What I find exciting these days is actually the High Productivity Computing Systems Effort, the Blue Planet or Blue Gene. These are a little ways off from being on the Top500 list yet though. :D

I do wish there were more SC companies doing hardware development in the US. I love Cray, but a single vendor smacks of eggs in one backet syndrome...So, geeks, if ya wanna start a startup with a design, go for it...Betcha the NSA (aka Cthuhlu of HPC) would be happy to sponsor ya...;)

Mod this guy up. He's really telling the truth!

Loosely coupled clusters like PDSF are great for work like what the high energy physics people do, like SNO.

However, somethings work better on vector architectures such as climate models and fusion work: there is a reason why the Spanish Met troops bought a Cray. Additionally, some chemistry, many fusion and several other codes work best on vector architectures.

There guys presented their global warming work where at my job. They've developed their climate code though as a parallel one. See here. One of the places that they have been running is on seaborg, an IBM RS/6000 with over 6k and near 7k processors.

Interestingly, the PCM guys presented what they wanted for an uber'puter. While it had massive amounts of storage, it was also a 500 *PETAFLOP* SUSTAINED PERFORMANCE machine.

*clickety clack* That'd be something like 166,666,666 Athlons. IDK of any interconnects that handle that. Can you imagine being an admin? Better hope you're good on rollerblades zipping to and fro replacing those oh-so-reliable commodity disks and CPUs...even if you have a .05% failure rate, that's still too damn much. As an admin, that'd be a huge waste of time. It'd also wreck havoc on the guys running stuff.

Or is that what grad students are for? To attempt such a silly thing and then admin it? ;)

Seriously tho. To get from here to their, we're going to need some exotic techs...not just more 'attack of the killer micros'.

Mod this guy up. He's really telling the truth!

Loosely coupled clusters like PDSF are great for work like what the high energy physics people do, like SNO.

However, somethings work better on vector architectures such as climate models and fusion work: there is a reason why the Spanish Met troops bought a Cray. Additionally, some chemistry, many fusion and several other codes work best on vector architectures.

There guys presented their global warming work where at my job. They've developed their climate code though as a parallel one. See here. One of the places that they have been running is on seaborg, an IBM RS/6000 with over 6k and near 7k processors.

Interestingly, the PCM guys presented what they wanted for an uber'puter. While it had massive amounts of storage, it was also a 500 *PETAFLOP* SUSTAINED PERFORMANCE machine.

*clickety clack* That'd be something like 166,666,666 Athlons. IDK of any interconnects that handle that. Can you imagine being an admin? Better hope you're good on rollerblades zipping to and fro replacing those oh-so-reliable commodity disks and CPUs...even if you have a .05% failure rate, that's still too damn much. As an admin, that'd be a huge waste of time. It'd also wreck havoc on the guys running stuff.

Or is that what grad students are for? To attempt such a silly thing and then admin it? ;)

Seriously tho. To get from here to their, we're going to need some exotic techs...not just more 'attack of the killer micros'.

NERSC has some information about pi and you can search through the first 4 billion digits. The server serial number doesn't code into their input form (numeric or letters only, not a combination), and it's restricted to a 10 character input anyway. The encoding could be converted to hex, but the odds of finding it in the first 4 billion digits are very slim indeed.

It makes brute forcing valid serial numbers almost look attractive.

Ian.

Floating point arithmetic is not the same as real arithmetic -- but neither is it fuzzy. Well-behaved floating point, like that specified by the IEEE 754 standard, actually allows you to simulate higher precision. Doug Priest's thesis explains how such simulated high precision works in detail, and software packages by , Yozo Hida, and others let you use fast high-precision arithmetic from languages like C++ and Fortran.

The widely used 1 + epsilon model of arithmetic is useful for proving algorithms are accurate, but it is only a model. Some computations in floating point are exact; for instance, the difference of any two positive floating point numbers that are within a factor of two of each other is computed exactly -- see Doug Priest's appendix on computer arithmetic in P&H.

It ought to be noted that the SNO guys that did all the hardwork to find out what was going on with the neutrinos used PDSF, a large linux cluster used in a batch farm configuration. The Japanese observatory that verified the work also used PDSF, as I understand.

The PDSF guys got a lot of thank yous and praise for the help they gave in building, running, and growing their cluster. PDSF as a result has been getting a lot of kudos from the NERSC management. With any luck that will translate into better backing.

At any rate, I thought I'd include them since /. readers like to hear how Linux is used IRL science.

It ought to be noted that the SNO guys that did all the hardwork to find out what was going on with the neutrinos used PDSF, a large linux cluster used in a batch farm configuration. The Japanese observatory that verified the work also used PDSF, as I understand.

The PDSF guys got a lot of thank yous and praise for the help they gave in building, running, and growing their cluster. PDSF as a result has been getting a lot of kudos from the NERSC management. With any luck that will translate into better backing.

At any rate, I thought I'd include them since /. readers like to hear how Linux is used IRL science.

If you're in a time wasting mood, you can try these:

Search for a string of numbers in the first 100 million decimal digits of pi. Try your birthday, or whatever.

Search for a char or hex string in the binary representation of pi. Find your name in pi, woohoo!

More pi time wasting stuff.

Note though: the SGI press release states that it only scales to 512 processors. it looks like they are having problems scaling beyond that. It is probably having to do with the interconnect and SSI approaches they are taking (at a guess).

That means that you will see a peak of around 5 teraflops. The density is impressive for that speed. The peak performance and scalability is not. Speaking from the Supercomputing world that is. It is something to be proud of (for SGI), but if they want to take the SC world by storm they need to scale higher. The high end of machines that will be ordered over the next 5 years are going to be in the 100+ teraflop range for peak performance. (re: Blue Planet)

While most of the market does not care about the very high end systems - they can't afford them - they ARE excellent PR. Bragging rights can go a long way.