Slashdot Mirror
Export Controls on Beowulf?
Gary Franczyk writes, "The United States government has tightly controlled the export of "supercomputers" to certain other nations (i.e., China, Pakistan, India, etc.) for quite some time. Sun has had to deal with this numerous times when selling their equipment. How will the U.S. government handle the fact that now anyone with access to large numbers of PCs can create a "super-computer" cluster? I'm sure that the government is using Beowulf to do nuclear simulations right now... Who says that other nations cannot do the same? " Interesting thought. I'm not aware of any export controls on Beowulf, but with the U.S.'s views on cryptography, how will it be before such draconian views extend to any powerful computing technology? Is it even possible for the U.S. to restrict Beowulf in any way?
Yep, that should keep the cat in the bag. It worked for cryptography, after all. Them furriners don't have kryptography, 'cause of our export controls.
Oh, well.....
See what I've been reading.
Apple advertised its G4 range a while back as being too powerful to export to unstable countries because the US government classified it as a munition.
Off the record, however, Apple staff here in the UK told me that the USG didn't specifically restrict the G4, as there were other, more powerful systems that were already available on the general market that were more suited to tinkering with things beyond man's ken and other dictatorish type things.
I'm assuming they were referring to things like Beowulf and various other Unix machines.
IMHO the restrictions applied to older machines, back when men wore plaid and a gigaflop meant not having to say you were sorry...
Golf; a good walk spoiled. -Mark Twain
I'm with the Technical University Munich, and the Leibniz Supercomputing Center next door is getting a new Big Box in March, which will then be the most powerful computer in Europe. The peak transfer rate between its nosed is 10 GIGABytes per second, IIRC. At the moment, thay're still installing the cooling units (the thing will consume about 600 Kilowatts!).
The illegal we do immediately. The unconstitutional takes a little longer.
--Henry Kissinger
how will it be before such draconian views extend to any powerful computing technology
You mean "powerful computing technology" such as the Playstation 2? If the U.S. govt can restrict it in any way, they probably will.
Not that it is likely to make much of a difference by now. Just as anyone who wants DeCSS can get it, anyone who wants Beowulf would probably be able to get it. There are no border checks on the Internet.
~~~~~~~~~
auntfloyd
Assuming that say Iraq won't be able to buy a pile of Alphas, or K7's if they so wish is utter bullshit. The US may ofcourse choose not to export such computers to those countries, but what is to stop some person from buying the vary same boxes in some other country and shipping them to Iraq then ?
It's not as if all countries have export-regulations equally silly as the US.
I guess you haven't been hanging around Slashdot long enough. This came up and was resolved nearly two years ago.
--
Here is the result of your Slashdot Purity Test.
Linux MAPI Server!
http://www.openone.com/software/MailOne/
(Exchange Migration HOWTO coming soon)
Barcelona, Belgium, Japan, Cambridge Uni (the n hundred year version), Italy and Korea are just some of the places running Beowolf clusters. Its already out there, stop the shipping from the US of PCs and the software and it will be picked up from one of these places outside the US.
:-), Windows and Sunny Delight.
Ma: Where is the horse ?
Pa: She's bolted.
Ma: Well you'd better bolt the door now.
Things the US should stop exporting: McDonalds (eat British, eat Burger King
An Eye for an Eye will make the whole world blind - Gandhi
OK, we all know that the US government owns the world, and that what it says goes, right?
But, how exactly would it prevent Beowulf ending up in the hands of 'unsavoury' regimes?
When, for example, AMD is manufacturing Athlons in Germany, and the code for Beowulf must be mirrored all over the world already, what does the US govt expect to be able to do?
If they wanted to resrict large quantities of CPU's making it to these regimes, it would have to ban US chip manufacturers from building fabs abroad (not terribly feasible, legally). Even if they managed that, there would be nothing to stop foreign firms manufacturing chips.
Then what would they do about the mirrored versions of the code? Shut down every Linux related FTP site in the world 'just-in-case'?
This smells far too much like paranoia, and fuss over nothing, it ain't gonna happen, so why worry about it?
--
Listening for the sound of the coming rain...
Yes, and the easter bunny visits my house and leaves golden eggs on my porch.
Try the following: ""The United States government has enacted legislation that attempts to tightly control the export of "supercomputers" to certain other nations (i.e., China, Pakistan, India, etc.)". Even then you mislead people, simply because of the word "export". The vast majority of the required parts are not made in the US. (Is there a single necessary part where all possible components that could be used are manufactured in the U.S.?)
To keep a product like that in the hands of the U.S. only would require the creating corporation agreeing to do the following.
- Keeping all manufacturing in the U.S. at U.S. wages.
- Refusing to patent the technology.
- Keeping a very expensive security lid on the entire facility.
- Not releasing any details that would allow anyone with the resources of China to come up with an equivilent technology
Yeah. Right.In a way, it seems silly to refuse to sell certain nations supercomputers when we still hire their citizens to work on our supercomputers...
-----
No Zen is good zen
However, most nuclear tests these days seem to be for shows of strength (France and the India/Pakistan tests spring to mind), so it is actually more dangerous, in my view, to develop and test nuclear technology using supercomputers, than to develop and test them in "the open", since open testing is a good deterent to other countries.
Perhaps there should be a clause in the GPL, that GPL'd software can't be used to bring about armageddon. OK, that won't work since: a) it violates the Open Source Definition, and b) Emacs would have to be removed from all sites;) - but at least require any nuclear technology developed under Linux be released GPL, maybe have nuke.soureforge.net. This would actually be cool, perhaps VA Linux could fund tests of the open-source nukes on some random place (off the top of my head - Redmond?), if an angry penguin running at you at 100mph is scary, what'll an angry penguin with a nuclear warhead be like?
Sorry about the incoherence of the above post, it's been a long day (and it's only half-way through as well)
--
You'll probably find the story in the Slashdot archives. People were mirroring the Red Hat CD and the Beowulf archives on every part of the globe, within an hour of the story breaking on this site. (I'm not joking! If there's any "wild exageration" it is more likely that of one of an hour being far longer than it actually took.)
About two, maybe three, weeks later, the Beowulf site was back up and running. Almost certainly monitored, though. This was definitely munitions, according to someone with the clout to push a NASA site around.
IIRC, though, Beowulf is really not much more than some finer tuning for the network drivers, PVM, MPI, and some freebie cluster management software. Most of the tuning was for the 2.0.x kernels and has since been incorporated into the main tree. PVM and MPI are freely downloadable, and there are later versions than on the Beowulf site. There are also lots of cluster management packages around, now, as well. Beowulf, IMHO, has ceased to be the specific patches/bundle released by AMES, and has become any collection of boxes, configured to act as a single, multi-node, supercomputer.
And, yes, export of supercomputers is VERY restricted. Apple can't export any G3-based computers (though whether anyone in the rest of the world is upset by this is anyone's guess), and it's unlikely that newer-generation processors from other companies will qualify for export, either.
(Personally, I suspect an overclocked, supercooled SMP K7 board would exceed the limits by quite a substantial margin.)
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Let's go through this real quick:
That's all there is to it, as far as I know. I should add that many "Freedonias", during the cold war, used the exact same procedures to illegally acquire hardware they were not allowed to buy... There are even tales of the (old) USSR acquiring Cray machines, when these were the "crown jewels" of US computing. Commodity hardware has just made this 100 times more simple...
The right to offend is far more important than the right not to be offended. (Rowan Atkinson)
Apple can't export any G3-based computers (though whether anyone in the rest of the world is upset by this is anyone's guess)
:)
Gee whiz, in that case the iMac that has sat in my desk since October 1998 is just a figment of my imagination, right? I've been dreaming about it all along, eh?
Apple is forbidden to directly export one specific model: the G4/500MHz, which exceeds 1 GFlops and is therefore subject to "supercomputer" regulation. But iMacs, iBooks, "big" G3s and more recently "big" G4s can be found all around the world, including here in Brazil. (Okay, so they're all priced like supercomputers... but that's not the issue
To the editors: your English is as bad as your Perl. Please go back to grade school.
While the Beowulf patches come out of NASA, there's a whole bunch of stuff out there which isn't written in the US at all. For example, the best session clustering technology for Linux is MOSIX which is put out by Hebrew University in Isreal. To my knowledge this isn't export restricted at all, and is released as a set of patches against the main kernel tree. Anyone with basic System Administration skills could set up a Mosix cluster pretty quickly.
e ms.html">Yahoo Batch Queuing Page" for a short list of many popular packages.
If you're less interested in interactive clustering and need computational load balancing instead, there's a whole slew of batch queuing packages available from GNU QUEUE to the many derivatives of NQS out there. Here's the a href="http://www.cmpharm.ucsf.edu/~srp/batch/syst
I don't think the US government could stop any nation from purchasing commodity hardware manufactured from around the world, installing a basic Linux or BSD distribution, and setting up a batch queue or other type of basic cluster. Never mind that a sufficiently serious government could just up and write their own... in my department at BBN (Speech and Natural Language Processing) we use an internally written batch manager which is surprisingly simple... all written in C.
All the restrictions in the world cannot alter the intent of another nation. If you treat someone (or a nation) with suspicion, interpret their every move as hostile, and generally bully their citizens, is it surprising that negative attitudes form? Now that the Cold War is over, perhaps a more enlightened foreign policy could be forged based on a more consistent set of principles?
In my mind, the export of "munitions" like Beowulf pales in comparison with past doctrine now being slowly exposed such as military training given to the Indonesian special forces which rebounded badly in the Timor Separation. Certain Latin American countries have no particular appreciation of the US "aid" that they received in the past either (Panama, Niguargua, Columbio, Haitii, etc).
The steadiest water stream can erode the strongest rock. A consistent message of rule by law, respect for human and property rights, and civil society will do more in the long-term in altering societial values than bombing the living daylights out of people whenever they step over a dimly perceived line. By watching TV, the Chinese have gained respect for western police reading people's rights when arrested (not a common occurance for a society slowly emerging from fuedal warlord times). By consistently demonstrating the virtues of a open society, with the free exchange of ideas, even when that could put us in a perceived position of vulnerability (with a decent armored cluebat hidden out of sight just in case), moral authority can be maintained. Even though many people do not share RMS views, they do respect his passion for sticking to his principles. Leadership, especially in the global setting, should be more setting an example, not trying to blugeon or bully people into following your lead blindly. When a giant, walk softly and bend down to listen sounds like a good analogy to get along with normals.
The Beowulf example is like trying to stick a finger in a dam when the whole ediface is changing, about as useless as patenting the click when everyone is moving on beyond the mouse. Similar technology exists in Isreal and could be duplicated given enough time. Given the basics of enough food (a recent Nobel winner proved that most starvation resulted from bad distribution systems, rather than absolute lack of food) and some decent shelter, the average citizen from other countries are much more interested in sitting down and knocking back a few beers than in the posturing antics of self-opinioned leaders on a media crusade. And ultimately it is the average citizen that benefits from openess when they can vote with their feet.
If they can afford to build a Beowulf then ship them the CDs and invite them to join the GNU revolution.
LL
This is not the case with other problems. With those, you need high-bandwidth, low-latency communication between the nodes, and that is something you simply don't get out of Ethernet, which, actually does have exponential speed losses when the network comes close to saturation.
The illegal we do immediately. The unconstitutional takes a little longer.
--Henry Kissinger
Beowulf is nothing more than a bunch of PCs, connected via medium speed (100mbps to 1gbps) networking, running one or more of a couple particular software packages. Both pvm and mpich
have been readily available, to the whole world, for a long time. MOSIX has also been available for a long time (though not on Linux)
In short, you can't restrict Beowulf because there's very little there to restrict. For example, mpich based parallel programs can very easily be recomiled on FreeBSD, and the performance is better. (yes, FreeBSD does have better networking. See http://www.cs.duke.edu/ari/trapeze/ip/ for some real benchmarks.)
Besides, MPI is a documented protocol, so even if implementations were banned, it shouldn't be that hard to re-implement from the protocol descriptions.
ERROR: Null
Many problems do not parallelize well. For instance, to my admittedly limited knowledge no parallel version of the fast Fourier transform algorithm (which serves as the backbone of many spectral and pseudospectral codes) is known which does not require a prohibitive amount of interprocessor communications.
At the risk of being overly pedestrian, let me try tackling your differential equations question: Communications latency issues can crop up even if you have just a single equation to solve. Let's imagine, for the sake of discussion, that you wish to understand the propagation of heat on a metal plate, and you have a differential equation that describes the process. Conceptually, you might imagine solving this problem on a parallel computer by breaking the metal plate into a bunch of smaller regions, and asking each processor to compute heat flow on an individual region, as in the following, where a plate is broken into 9 regions:
OOO
OOO
OOO
You can see that every region borders other regions, and herein lies the difficulty: To compute how heat propagates in any one region, say the top left corner, you have to have information from each of the neighboring regions. (In the case of the top-left corner, it'd be the center-left and top-center domains). In solving differential equations, this information is called the "boundary conditions." Each time step would require sending a considerable amount of information among the processors in order to handle the boundary conditions. To use a real-world analogy, if communication latency is high, then many of the processors will end up waiting for the information they need, much like workers in a bureaucracy that has an inefficient internal mail service. In "real" supercomputers you pay big bucks for fast communications, and problems that are communications-intensive will naturally perform better on these machines than on Beowulf or Appleseed clusters. Alternatively, problems whose algorithms require few messages to be passed among processors (many Monte Carlo algorithms have this feature) may run very efficiently on a Beowulf or Appleseed cluster, where communications latency is high.
Parallel computing seems to be largely an exercise in economics. Any parallel algorithm with a nonzero number of messages to be passed will necessarily run at something less than 100% efficiency. Just how far below 100% depends on the nature of the algorithm and the machine/cluster it is running on.
exerpt from Category 4 - Computers
b. "Digital computers" having a "composite theoretical performance" ("CTP") exceeding 2,000 million theoretical operations per second (Mtops);
c. "Electronic assemblies" specially designed or modified to be capable of enhancing performance by aggregation of "computing elements" ("CEs") so that the "CTP" of the aggregation exceeds the limit in b.;
How the US Government thinks it could ever enforce this with Beowulf is beyond me...
Name 1 - 1 task - that requires a supercomputer that can't be broken down into nodes well
Sure, I can name several. Just some examples: Weather simulation. Ocean simulation. Molecular simulations. Simulation of astronomical bodies. All of which are very real problems.
In short, any problem which is not trivially parallel will get a much poorer speedup on a NOW (network of workstations) versus a real supercomputer. Many of the problems above will generate many MB/s of data per processor (60 - 200 MB/s is not uncommon).
What you fail to realize is that many problems run for many iterations, and for each iteration you need to distribute the global dataset to all worker nodes. Take the Barnes-Hut program, for example. In that program, each node get a set of close-by astral bodies (stars and planets), and calculates their new positions for the next time step. To do that you need the positions of all other stars. For the next time step, you need to a) collect the calculated positions from all worker nodes, and b) distribute them back for the next iteration. When trying to run that on a NOW, you will very soon find that doubling the size of the cluster will not give any speedup at all, since they will spend most of the time chatting with each other on the network. On a supercomputer, you can run many more worker nodes before this happens.
> Ideally, you have a compiler that takes care of using all the nodes and distributing the code.
> If you have to hand-code all that, it just takes too long and is error-prone (debugging
> distributed code is a really ugly task). Something like High Performance Fortran.
Are you aware that there are HPF compilers for Linux being used in Beowulf clusters?
Check out:
VastHPF
PGHPF
There are other commercial products, plus some educational type compilers.
- Darren
--
-- Fnord.
If you take a look at the hype on www.beowulf.org shutdown, and Important Beowulf notice stories. Now, I know there was great debate over what *really* happened, but one of the possibilities that was a concern was exactly what you are asking about..
-- I'm the root of all that's evil, but you can call me cookie..
Now, I'm CERTAINLY no expert on the subject, but I must beg to differ. Weather simulation *CAN* be broken out to run on a distributed machine, which is proven by the fact that several government entities have awarded contracts to build them to just such machines!
I also believe that perhaps you are unfamiliar with a good cubic beowulf setup. Typically, each machine would have 4 Gigbits of bandwidth to transfer data, i.e., 4 Gigbit ethernet cards, no more then 2 hops from any given machine. Using this method, transfering the datasets is fairly trivial. Granted, *THIS METHOD DOESN'T SCALE AS WELL* if you use the same programming techniques as those one would use on a supercomputer scale. However, this can be overcome if you take this into consideration, aka, clusters of 9 cluster machines. Each machine would really only be talking to one of 9 machines, with the cluster controller talking with the other controllers. It's just a different way of looking at the problem.
-- I'm the root of all that's evil, but you can call me cookie..
I spoke to Michael Reagan on the air about this very subject about 8 months ago. Some people (like mike) just don't get it. What Mike was talking about was the Clinton administrations easing of computer export restrictions to countries like China.
The Genie is out of the bottle. Beowulf, actually allows one to build faster computers for LESS money. If you buy a "million dollar" super-computer and you spend a million dollars on nodes for a Beowulf cluster, typically the Beowulf if going to be capable of doing the work faster.
I'm not too worried about China getting nuclear technology through eased export restrictions, I'm worried about well funded terrorist groups getting access to nukes, Bin Laden's boys or that death cult in Japan are far scarier to me than China.
LK
"Hi. This is my friend, Jack Shit, and you don't know him." - Lord Kano
Wow, someone who knows what they're talking about....
:)
I deceive people well.
If I understand correctly, you are describing how a "surface-to-volume" ratio goes up as the volume elements get larger, thus allowing individual processors to spend more time crunching numbers and less time waiting for boundary data. This is indeed true, and this is precisely the kind of balancing act one has to perform to compute efficiently in parallel. As you've demonstrated, the same algorithms may be more efficient on some machines than on others, but based on my (albeit limited) experience in computational physics, optimization almost always seems to boil down to how one reduces the number of messages that have to be passed in order to perform the task. This seems to be the single most important factor in the scalability of numerical calculations (how much speedup is gained by increasing the number of processors).
Disclaimer: While I have some experience in parallel computing, I am by no means an expert in this field, and I suggest you read some of the other excellent posts in this thread to hear from the real experts.
The problem is that each particle's position needs to be sent to all the other nodes for them to do their calculations. So taking your assumption that each particle takes 34 bytes and assuming the simulation has a million particles in it, you need to send at least 34 MB of data through the network after each calculation. So assuming the calculation takes a while, say 10,000 clock cycles (on a 500Mhz machine) then you need to do 50,000 updates a second. This comes out to about 1,700,000 MB/s of bandwidth. Even if the calculation takes 1,000,000 clocks, you still need at least 17GB/s bandwidth. A beowulf cluster may have a 1 Gb/s bandwidth over gigabit ether but that leaves you short by a factor of 136.
Using a beowulf for the example above means you can only do 30 calculations per second even though you could theoretically do 500 to 50,000 depending on the amount of clocks the calculation takes.
Now suppose you had a supercomputer that had a bandwidth of 10GB/s (gigabyte/s). Then you could do about 300 calculations per second even if your processors were running at 300MHZ instead of 500MHz. In cases like this, the I/O bandwidth determines how fast the simulation will go and a beowulf cluster would not be better.
"When you sit with a nice girl for two hours, it seems like two minutes. When you sit on a hot stove for two minutes, it
The web page says nothing about nuclear simulations. Its a few molecular dynamics/astrophysics/pde applications.
"When you sit with a nice girl for two hours, it seems like two minutes. When you sit on a hot stove for two minutes, it
You break the problem up, and solve parts of it. *Then* figure out the parts that inter-relate.
In the case of the above, you also need to take into consideration that you could simply have the math equations needed sent to the cluster, with one primary machine just putting it all together. Example:
10 to the 10th can be thrown accross 5 machines, calculating 10 to the second. This a very simplistic example, and in real life, wouldn;t work quite right, but in the case of very math intensive applications, some of the equations *can* scale to that extent. It is also able to be processed in parallel, simply becouse you can be working on several equations at the same time.
-- I'm the root of all that's evil, but you can call me cookie..