In the case that Mozilla is being rejected, they already believe that the only clients they need to support are IE. Quite frankly, when you consider the mass internet population as a whole, IE is pretty much the only browser around anyways. Mozilla, Opera and other superior browsers account for a very small corner of the browserspace. It's sad, but it's true, and if I were a bank suit in charge of online banking, coming from a traditional background, I'd probably support only IE as well, it makes sense to them. Of course if I were the real me of today I would support open standards and interoperability, but that's not the point here:)
You would have thought thise question would have been rejected by the editors wouldn't you? Yet they commonly reject pointers to truly insightful projects. Are the editors just as dumb as the people who ask these questions, or are they just trying to cater to a dumbed down crowd to get more viewership? Inquiring minds want to know.
Market forces drive IDE drives to be built as cheaply as possible while still having the right buzzwords to make consumers believe they're faster than their competitor. RPMs higher than 7200 still don't register with the mass populace, so it's not yet a factor.
SCSI hard drives are all about top-end performance. That's why some SCSI drives cost $1,500 for the same capacity as a $150 IDE drive. It's about being able to reliably move the platter at twice the speed of IDE, and having the correct drive logic and buffer memory to make it useful in the real world, getting very high MTBF numbers, etc..
Comparing typical IDE drives versus high-end SCSI (or FC for that matter) drives is like comparing small asian economy cars with the contenders in the F1 racing series. They have entirely different goals.
Often times the web interface is actually standards-compliant enough that Mozilla works fine, and it's just a matter of their detecting your browser id string and rejecting you out of hand. You can fake this string to make them believe you're running IE, and the problem is solved.
Now if, on the other hand, they're using IE-isms like ActiveX, you're back to square one.
The use of the term "thread" in the statements you quote refers more to a "thread of execution" than specifically to a thread in a multi-threaded app. Their statements were intended to (and do) apply to multiple threads in one multithreaded app, or several seperate multithreaded apps, or several seperate processes. The things they say about true parallelism are basically the same things you could say of SMP vs UP. They built a "half-smp" inside a single processor. Another way to think of it is that they are allowing threads of execution that would otherwise be completely blocked to make opportunistic use of parts of the instruction pipeline that the main active process isn't using.
In any case (up, smp, up-ht, smp-ht), one should code multi-threaded for apps that can benefit from parallelism, and don't bother for those that don't. And in any case, the os will do it's best in the given hardware context to satisfy your needs.
This is one of the best all-around security articles I've read in a long time. If even 10% of the world's programmers read this and take it to heart, the world will be a measurably better place.
The article doesn't make any comparison between this and other (read x86 linux clusters) solutions. Do the x86 clusters have a problem scaling as well as xserves? I've heard of several-thousand node x86/linux clusters, so I would guess not, but I don't really know. Also no mention of $$/{MIPS/FLOPS/Whatever}, which would be nice to compare against an x86 cluster as well.
I've only tried reiserfs and xfs for a few days each, for the most part I've stuck to ext3 in recent days. I've hard-crashed (pull the plug type of thing) several different machines with ext3 while filesystem write activity was going on and never had a problem. Based on my time with ext3, my limited experience with reiserfs/xfs, and reading lots of lkml, I think ext3 is the safest choice at this point in time, even if it's not neccesarily the best performance.
While the shielding may protect your PC from environmental EMF at times, and that may be beneficial, the primary purpose is to keep the immense EMF inside your computer from leaking out. Aside from the debatable long term health effects sitting next to it, it can and will interfere with other electronic devices in the vicinity. Building a computer with absolutely no EMF shielding also means NSA Tempest vans can sniff you from much further away:)
As a side note, I've noticed that my company-supplied cellphone (PoS Motorola i1000plus Nextel phone) emits some serious wattage, which I would suspect could induce errors if it was around an unshielded computer. If it sends/receives any calls/pages/data while within about 5-6 feet of a traditional CRT, it causes the CRT to flicker. Without a slightler larger range of any sort of speaker, it actually induces blips of white noise on the speakers.
If you undertake a project to build a funky case out of plastic/glass/stone/wood/whatever, be sure to take the EMF into account. If it's something opaque like wood, this is easily remedied by liing the inside of the case with a faraday cage (wire mesh of the right size, basically). I'm not sure what you should do for clear materials - perhaps leaded glasses have some emf shield properties? not really sure on this.
Pro: You could hook eth1 of all the machines to switch1, and eth2 all to switch2, etc... and use independant switches for each parallel communication path, and it owuld give better performance than running it all into one switch and would still work.
Con: N-way parallelism in this fashion leads to multiplying your chance of total network failure by N (4 way means you've now got 1/4 the normal MTBF on your network cards/switches/etc). Of course the biggest con is that to go 8 way would require a lot of PCI slots. I think some PCI card vendors make dual ethernet cards, but I'm not sure if any are linuxable. Sun makes a quad fast ethernet PCI card for their boxes, maybe the driver from spaclinux can work with a few changes putting the card in an x86 box?
Picture the following scenario (we'll use 4x parallelism as an exmaple, you could go with whatever number you fancy):
You have an alternate protocol, lets call it PIP (Parallel IP), and you code for it in your network stack. It acts like IP and normal TCP/UDP will run on top of it. On a per-interface basis you decide that your public card talks IP and the 4 cards to the private beowulf network talk PIP.
In some config file, you define that eth1-eth4 are part of a PIP group with a single IP address and four MAC addresses. You probably have to define all the beowulf nodes in a central file that you distribute to all the nodes (so they know the mac/ipaddrs of the remote nodes).
The PIP stack takes each would-be IP frame, and instead of encapsulating it in an ethernet frame bound for the single mac-address of the recipient IP address, it split the packet into four chunks with unique serial number 34534568 and sequence numbers 1-4 in an extremely short PIP header that goes on top (inside the ethernet framing). You lookup the 4 MACs for the destination IP (PIP) address, and you send packet #1 out your MAC#1 destined for their MAC#1, etc...
On the receiving side, the PIP stack looks for all four sequence numbers to reconstruct packet 34534568. When and if it receives all four, they are reconstructed and passed upwards. After some minimal timeout value (a few milliseconds at most?) if only some of the sequence numbers have been received, toss the packet out and let TCP (or the app in case of UDP) deal with the lost packet, after all IP isn't gauranteed delivery.
I'm of course leaving out lots of little implementation problems, but that's what you get in a 5-minute slashdot idea:) The nice thing about a plan like this is that it's transparent to your Layer-2 switches (layer-3 switching wouldn't understand the packets though), and it's transparent to UDP and TCP applications.
You might also want to handle ICMP in some custom fashion, like sending a ping down all four mac-pair-paths and telling the ping program you got a valid echo when/if you see all four replies.
The downside of course is that it's not really gauranteed to give any latency decrease due to the problem of lost packets and waiting for all four to sync up and whatnot - although it would almost definitely result in an overall bandwidth increase for a tcp stream.
Overrated my ass, but back to the topic - in response to the responses..
Oracle OPS (or the new RAC) kill MS SQL clustered servers on reliability/scalability/performance. If you've got some big corporate database, and you've got DBA worth their stuff that don't need cute graphics, go Oracle by all means. From the sound of it though, this guy has a small database backing a website, not some enterprise monster. For a small webserver database like most of these are, I think PostgreSQL is the best choice bar none when you consider its cost and simplicity. I only included MySQL as an alternative in hopes of pre-quenching and MySQL vs PostgreSQL flames.
The problem is that we don't know the content. If it's just his pr0n collection he's worried about, then he wouldn't even mention or ask about the crypto. Lots of people use their home network to run small businesses, or to remotely work at large business. In both cases, their can be corporate secrets at stake, and therefore both corporations and individuals determined to get them.
Consider also that he will presumably use this for some years if he deploys it. If he uses it for 2-4 years, that adds up (from back to 1998 when it took $250k and 56 hours) to 3-4 Moore's Law cycles. That means assuming that hardware/R&D costs don't drop (and they do), for $250k it will now break in 3.5-7 hours near the end of his use of it. Since it was a scaled solution, that also means (again assuming no drop in hardware/R&D cost drop) for $15-30k it can be broken in the original 56 hours. Or if the attacker is on a real budget and the data isn't all that time sensitive, they could spend a month cracking it for $300 or so.
Great comment except for the EXCELLENT SQL Server part:) Then again I guess if he's stuck on windows for server software, it's the best shot he's got.
My advice: convince your obviously failing employer that Windows really is just a client platform, and go Linux/Apache/{Perl|PHP}/{Postgresql|MySQL} or some solution along those lines.
Anyone who would bother cracking DES can surely grab powerline-transmitted data from your house. Either by a tap on your waterpipes (getting the nosieit induces in the house's ground), tapping the power at the transformer (far enough away you might not notice at night, but close enough to get the signal), or drive-by EMF pickup (data on your whole house's power lines probably emits even better than the EMF that Tempestt picks up frmo PCs and monitors from outside on your street).
56-bit DES is very weak these days. While it might keep a non-technical neighbor at bay, the very idea of sniffing your traffic off the power circuit would probably have kept them at bay anyways. If you do anything at home that's the least bit sensitive, I wouldn't entrust it to 56-bit DES.
Facts from when the EFF machine broke the DES challenge (it was 56-bit DES like you're about to use):
Time to crack key: 56 hours Total cost of machine, including R&D: US$250K
The R&D was extensive, and would probably cost less now that people have an example to follow. Also, this was done in 1998, so there's been four intervening years to make it faster and cheaper.
However, the NSA is somewhat of an exception to this rule. It is widely known that they are the largest employer of mathemeticians worldwide, compared to any other governmental or private organization, including universities. Therefore, widespread solid peer-review of cryptography can actually happen *inside* the NSA without making anything public to the outside world and they would still get decent results. Add on that the NSA's cryptographers and mathematicians tend to be about a decade ahead of the public/academic world, and it all adds up to the NSA not needing to follow the conventional cryptography peer review mantra.
Deadlines are the death of all good software. If the business guys need it done faster than it can be done right, that's their problem, not mine. Plan ahead:)
Still, that just means you need several racks, and the problem against isn't solid-state specific. Check out EMC and Hitachi for better storage density in a normal array, but they may not offer a stock solid-state option (although I bet you could retrofit it with drives yourself). In any case, managing a large amount of storage, and managing solid-state storage, are two different things, both of which are pretty standard question with standard answers.
I will say one thing - if you're thinking of SANing all of this storage to a cluster of servers - don't buy the existing market leaders' products. Cisco is on the verge of coming out with their second-gen SAN switches that are in catalyst chassis and support virtualization (read: veritas software-raid) inside the switch, making for a very nice solution.
Slashdot Mirror
In the case that Mozilla is being rejected, they already believe that the only clients they need to support are IE. Quite frankly, when you consider the mass internet population as a whole, IE is pretty much the only browser around anyways. Mozilla, Opera and other superior browsers account for a very small corner of the browserspace. It's sad, but it's true, and if I were a bank suit in charge of online banking, coming from a traditional background, I'd probably support only IE as well, it makes sense to them. Of course if I were the real me of today I would support open standards and interoperability, but that's not the point here
You would have thought thise question would have been rejected by the editors wouldn't you? Yet they commonly reject pointers to truly insightful projects. Are the editors just as dumb as the people who ask these questions, or are they just trying to cater to a dumbed down crowd to get more viewership? Inquiring minds want to know.
Market forces drive IDE drives to be built as cheaply as possible while still having the right buzzwords to make consumers believe they're faster than their competitor. RPMs higher than 7200 still don't register with the mass populace, so it's not yet a factor.
SCSI hard drives are all about top-end performance. That's why some SCSI drives cost $1,500 for the same capacity as a $150 IDE drive. It's about being able to reliably move the platter at twice the speed of IDE, and having the correct drive logic and buffer memory to make it useful in the real world, getting very high MTBF numbers, etc..
Comparing typical IDE drives versus high-end SCSI (or FC for that matter) drives is like comparing small asian economy cars with the contenders in the F1 racing series. They have entirely different goals.
Often times the web interface is actually standards-compliant enough that Mozilla works fine, and it's just a matter of their detecting your browser id string and rejecting you out of hand. You can fake this string to make them believe you're running IE, and the problem is solved.
Now if, on the other hand, they're using IE-isms like ActiveX, you're back to square one.
The use of the term "thread" in the statements you quote refers more to a "thread of execution" than specifically to a thread in a multi-threaded app. Their statements were intended to (and do) apply to multiple threads in one multithreaded app, or several seperate multithreaded apps, or several seperate processes. The things they say about true parallelism are basically the same things you could say of SMP vs UP. They built a "half-smp" inside a single processor. Another way to think of it is that they are allowing threads of execution that would otherwise be completely blocked to make opportunistic use of parts of the instruction pipeline that the main active process isn't using.
In any case (up, smp, up-ht, smp-ht), one should code multi-threaded for apps that can benefit from parallelism, and don't bother for those that don't. And in any case, the os will do it's best in the given hardware context to satisfy your needs.
fuck off
This is one of the best all-around security articles I've read in a long time. If even 10% of the world's programmers read this and take it to heart, the world will be a measurably better place.
virtually everything you mentioned can be plugged up with LDAP one way or another.
The article doesn't make any comparison between this and other (read x86 linux clusters) solutions. Do the x86 clusters have a problem scaling as well as xserves? I've heard of several-thousand node x86/linux clusters, so I would guess not, but I don't really know. Also no mention of $$/{MIPS/FLOPS/Whatever}, which would be nice to compare against an x86 cluster as well.
What's next? A private hospital suing to shutdown a government-run free public health clinic because it's competing with them?
I've only tried reiserfs and xfs for a few days each, for the most part I've stuck to ext3 in recent days. I've hard-crashed (pull the plug type of thing) several different machines with ext3 while filesystem write activity was going on and never had a problem. Based on my time with ext3, my limited experience with reiserfs/xfs, and reading lots of lkml, I think ext3 is the safest choice at this point in time, even if it's not neccesarily the best performance.
While the shielding may protect your PC from environmental EMF at times, and that may be beneficial, the primary purpose is to keep the immense EMF inside your computer from leaking out. Aside from the debatable long term health effects sitting next to it, it can and will interfere with other electronic devices in the vicinity. Building a computer with absolutely no EMF shielding also means NSA Tempest vans can sniff you from much further away
As a side note, I've noticed that my company-supplied cellphone (PoS Motorola i1000plus Nextel phone) emits some serious wattage, which I would suspect could induce errors if it was around an unshielded computer. If it sends/receives any calls/pages/data while within about 5-6 feet of a traditional CRT, it causes the CRT to flicker. Without a slightler larger range of any sort of speaker, it actually induces blips of white noise on the speakers.
If you undertake a project to build a funky case out of plastic/glass/stone/wood/whatever, be sure to take the EMF into account. If it's something opaque like wood, this is easily remedied by liing the inside of the case with a faraday cage (wire mesh of the right size, basically). I'm not sure what you should do for clear materials - perhaps leaded glasses have some emf shield properties? not really sure on this.
Another thought on the pros and cons
Pro: You could hook eth1 of all the machines to switch1, and eth2 all to switch2, etc... and use independant switches for each parallel communication path, and it owuld give better performance than running it all into one switch and would still work.
Con: N-way parallelism in this fashion leads to multiplying your chance of total network failure by N (4 way means you've now got 1/4 the normal MTBF on your network cards/switches/etc). Of course the biggest con is that to go 8 way would require a lot of PCI slots. I think some PCI card vendors make dual ethernet cards, but I'm not sure if any are linuxable. Sun makes a quad fast ethernet PCI card for their boxes, maybe the driver from spaclinux can work with a few changes putting the card in an x86 box?
Picture the following scenario (we'll use 4x parallelism as an exmaple, you could go with whatever number you fancy):
You have an alternate protocol, lets call it PIP (Parallel IP), and you code for it in your network stack. It acts like IP and normal TCP/UDP will run on top of it. On a per-interface basis you decide that your public card talks IP and the 4 cards to the private beowulf network talk PIP.
In some config file, you define that eth1-eth4 are part of a PIP group with a single IP address and four MAC addresses. You probably have to define all the beowulf nodes in a central file that you distribute to all the nodes (so they know the mac/ipaddrs of the remote nodes).
The PIP stack takes each would-be IP frame, and instead of encapsulating it in an ethernet frame bound for the single mac-address of the recipient IP address, it split the packet into four chunks with unique serial number 34534568 and sequence numbers 1-4 in an extremely short PIP header that goes on top (inside the ethernet framing). You lookup the 4 MACs for the destination IP (PIP) address, and you send packet #1 out your MAC#1 destined for their MAC#1, etc...
On the receiving side, the PIP stack looks for all four sequence numbers to reconstruct packet 34534568. When and if it receives all four, they are reconstructed and passed upwards. After some minimal timeout value (a few milliseconds at most?) if only some of the sequence numbers have been received, toss the packet out and let TCP (or the app in case of UDP) deal with the lost packet, after all IP isn't gauranteed delivery.
I'm of course leaving out lots of little implementation problems, but that's what you get in a 5-minute slashdot idea
You might also want to handle ICMP in some custom fashion, like sending a ping down all four mac-pair-paths and telling the ping program you got a valid echo when/if you see all four replies.
The downside of course is that it's not really gauranteed to give any latency decrease due to the problem of lost packets and waiting for all four to sync up and whatnot - although it would almost definitely result in an overall bandwidth increase for a tcp stream.
Anyways, food for thought...
Overrated my ass, but back to the topic - in response to the responses..
Oracle OPS (or the new RAC) kill MS SQL clustered servers on reliability/scalability/performance. If you've got some big corporate database, and you've got DBA worth their stuff that don't need cute graphics, go Oracle by all means. From the sound of it though, this guy has a small database backing a website, not some enterprise monster. For a small webserver database like most of these are, I think PostgreSQL is the best choice bar none when you consider its cost and simplicity. I only included MySQL as an alternative in hopes of pre-quenching and MySQL vs PostgreSQL flames.
The problem is that we don't know the content. If it's just his pr0n collection he's worried about, then he wouldn't even mention or ask about the crypto. Lots of people use their home network to run small businesses, or to remotely work at large business. In both cases, their can be corporate secrets at stake, and therefore both corporations and individuals determined to get them.
Consider also that he will presumably use this for some years if he deploys it. If he uses it for 2-4 years, that adds up (from back to 1998 when it took $250k and 56 hours) to 3-4 Moore's Law cycles. That means assuming that hardware/R&D costs don't drop (and they do), for $250k it will now break in 3.5-7 hours near the end of his use of it. Since it was a scaled solution, that also means (again assuming no drop in hardware/R&D cost drop) for $15-30k it can be broken in the original 56 hours. Or if the attacker is on a real budget and the data isn't all that time sensitive, they could spend a month cracking it for $300 or so.
Great comment except for the EXCELLENT SQL Server part
My advice: convince your obviously failing employer that Windows really is just a client platform, and go Linux/Apache/{Perl|PHP}/{Postgresql|MySQL} or some solution along those lines.
Anyone who would bother cracking DES can surely grab powerline-transmitted data from your house. Either by a tap on your waterpipes (getting the nosieit induces in the house's ground), tapping the power at the transformer (far enough away you might not notice at night, but close enough to get the signal), or drive-by EMF pickup (data on your whole house's power lines probably emits even better than the EMF that Tempestt picks up frmo PCs and monitors from outside on your street).
1 - the $250K and 56 hours was four years ago. I can't give a good estimate for 2002, but I bet it's closer now to $100K and 20 hours.
2 - I specifically said it would keep the neighbor out, so why are you using the neighbor as an argument.
3 - It's likely that some organizations already have the ~100K equipment and amortize it's use over time.
4 - It's likely that a lot more people are being surveilled by the government for less probable cause than in the past.
5 - If he works for any sort of major company, and works from home, then his data might mean something to corporate spies as well.
My advice is well placed given the unknowns.
56-bit DES is very weak these days. While it might keep a non-technical neighbor at bay, the very idea of sniffing your traffic off the power circuit would probably have kept them at bay anyways. If you do anything at home that's the least bit sensitive, I wouldn't entrust it to 56-bit DES.
Facts from when the EFF machine broke the DES challenge (it was 56-bit DES like you're about to use):
Time to crack key: 56 hours
Total cost of machine, including R&D: US$250K
The R&D was extensive, and would probably cost less now that people have an example to follow. Also, this was done in 1998, so there's been four intervening years to make it faster and cheaper.
However, the NSA is somewhat of an exception to this rule. It is widely known that they are the largest employer of mathemeticians worldwide, compared to any other governmental or private organization, including universities. Therefore, widespread solid peer-review of cryptography can actually happen *inside* the NSA without making anything public to the outside world and they would still get decent results. Add on that the NSA's cryptographers and mathematicians tend to be about a decade ahead of the public/academic world, and it all adds up to the NSA not needing to follow the conventional cryptography peer review mantra.
Deadlines are the death of all good software. If the business guys need it done faster than it can be done right, that's their problem, not mine. Plan ahead
Oops, I was off by 10^3 when I read his post
Still, that just means you need several racks, and the problem against isn't solid-state specific. Check out EMC and Hitachi for better storage density in a normal array, but they may not offer a stock solid-state option (although I bet you could retrofit it with drives yourself). In any case, managing a large amount of storage, and managing solid-state storage, are two different things, both of which are pretty standard question with standard answers.
I will say one thing - if you're thinking of SANing all of this storage to a cluster of servers - don't buy the existing market leaders' products. Cisco is on the verge of coming out with their second-gen SAN switches that are in catalyst chassis and support virtualization (read: veritas software-raid) inside the switch, making for a very nice solution.
I would use vi, perl and postgresql, but that's just me