The Rice connection almost certainly has to do with Peter Druschel and Pastry (for which the other PI seems to be Antony Rowstron of Microsoft Research, interestingly enough). I'm not totally sure of the ICSI connection, but they seem to be closely affiliated with UCB and I know that Ion Stoica works in these areas. OceanStore, CFS/SFS, Pastry, Kademlia - it's definitely a pretty good collection. A lot of the top people in DHT/DOLR (Distributed Hash Table, Distributed Object Location and Routing) research are involved, and I'd love to know how they plan to converge their various efforts toward a common solution.
The InfoWorld article describes a secure distributed storage system, not just plain old messaging connectivity. There aren't too many such beasts around; usually it's more of a "distributed, secure, usable - pick two" kind of thing. Some of the projects that approach the goal of combining all three actually seem to sharing the IRIS award - i.e. OceanStore at Berkeley and various projects at NYU. I don't know off the top of the head how ICSI and Rice fit in, but I'm about to go check their sites because I'll bet it's interesting.
Over the years, XML has...become an integral part of virtually every component within an enterprise application and developer tools that we use everyday.
Only if you define "enterprise" as "web". XML is making inroads in some enterprise applications, but there are still vast swaths of that territory where XML remains irrelevant.
I'm not saying routing dynamic mesh network is impossible, it's just very hard
Yes, it is, and I'd have to say that I don't think the routing is "there" yet. By far the best resource I've found on the topic is Perkins's Ad Hoc Networking which describes several types of protocols. Some of the work on overlay networks (e.g. Resilient Overlay Networks at MIT) and P2P is also applicable, though much of it tends to underestimate the importance of geographic locality and little of it deals with issues like routing-related power consumption.
Address space is not the real problem. The devices need addresses no matter whether you're using "lily pads" or more traditional means. The real problem problems are things like routing (not just in a mesh but in a dynamic mesh just for extra fun) and power management (bad to kill your batteries forwarding other people's packets).
You're right that storing both values on the card forces you to assume that the sensor is not compromised and is reporting actual observed (rather than recorded) speckle patterns, and that's a bad thing. On the other hand, I don't think your suggestion really protects the vendor either, because the bank is still not authenticated to the vendor. Maybe we both need to go think about this some more.
Your analysis seems right on target to me. Any system that's not challenge/response is vulnerable to replay by anyone who can intercept the messages involved, and this system only allows for a limited number of challenge/response exchanges.
I think the "validation server" approach might be problematic, though, since it allows new avenues for compromise. It might actually be better to store challenge/response pairs on the card itself, such that each use of a pair also erases that pair. Each card would then be good for a finite number of non-repeatable transactions, with server communication only necessary to "recharge" the card with new pairs. If the storage on the card and the challenge space are quite large, this is something the consumer would only have to do every N months or years, so it might actually be a decent convenience/security tradeoff even if recharging requires going to a service center or something.
Because stealing the speckle pattern does you no good. You need to create a device that makes that pattern, when light is shone through it and an inaccessible air gap onto a sensor. You can't just lay something on top of the sensor itself because, in any even half-way sensible design, you couldn't get to the sensor itself without disabling the entire reader.
I actually think this idea is extremely clever, but I don't know if I'd consider it a method of encryption. Even if you had an LED grid representing cleartext on one side, so you could read the "ciphertext" speckle pattern on the other side, how do you decrypt that? What kind of resolution, frequency and loss ratio are we talking about? This seems like it might be a really good authentication mechanism, where a known input will only be converted to a known output in the presence of a unforgeable secret, but I don't see how it can work for encryption where the input varies.
It is indeed possible that Dr. Simonyi had such a special arrangement, but he doesn't just get rights to the ideas because he developed them (what this post's great-grandparent seems to assume). Exclusive ownership of intellectual property by the employer rather than the actual inventor is so commonplace that it's the exceptions which are noteworthy.
He didn't work on the business side of the company. He was a hard core geek, thinks, codes, and collects a check.
...all of which means precisely nothing. Nada. Zilch. Top-level (and often even mid-level) technical staff sign the exact same employee agreement as the business-side folks, and the part about "all your patents are belong to us" is there specifically for them. Dr. Simonyi might indeed have had some extra-special agreement, but if so it's because he's Simonyi and not because he's a geek instead of a suit.
That's so totally wrong that I hardly know where to begin. Patents have both inventors and owners, with only the latter really meaning anything legally. It's standard practice throughout the industry for employee agreements to require that ownership of patents be turned over to the company, so all the actual inventor(s) get is their names on the patent and maybe a bronze plaque if the company's feeling generous (which they weren't for my patent). There's very little Simonyi can do about it; the employer almost invariably holds all the cards.
And where is the law or precedent that would be used to prosecute an ISP for caching things that are illegal for other than copyright reasons? Common-carrier laws have been around for a long time, and have been used many times to shield the people who provide a communications medium from culpability for what is transmitted over that medium. The reason I mentioned the DMCA is that, in the specific case of copyrighted material, it attempts to supplant existing law and precedent; in cases other than copyright, that law and precedent is still valid and likely to protect the ISP as a common carrier.
Then again, law moves in mysterious ways. Are you aware of a recent case that would lead to a different conclusion?
The cite was already provided, but here it is again. The relevant claims from the AAP pages themselves are as follows:
on a dual Pentium III Linux 2.4 system the STM is 25% faster than the dexter MPM, which uses a similar MPMT architecture with pthreads [http://aap.sourceforge.net/stm.html]
the patches increase the performance of Apache/1.3.x up to 900% (10x) on Irix, up to 70% (1.7x) on Linux 2.2, and up to several hundred percent on Linux 2.4 [http://aap.sourceforge.net/faq.html]
AAP consists of two components: the STM and the Quick Shortcut Cache (http://aap.sourceforge.net/mod_qsc.html). According to AAP's own documents, STM accounts for a 25% performance gain. 25% divided by 700% is less than 4% of the total performance gain attributable to STM; the remaining 96% attributable to QSC definitely qualifies as "the vast majority" to me.
Your lie is not in claiming that STM is faster than the pthreads-based MPM. It might be, though 25% in "informal" and unspecified tests conducted by an advocate of one system don't exactly prove that beyond doubt. Your lie, rather, is in claiming that ST is "several times faster" based (apparently) on the 700% mostly-QSC number, even after the true significance of that number has been pointed out.
Let's try this one more time. You're the one claiming ST has good SMP scalability. I have expressed reasons for doubt, but the burden of proof nonetheless remains on you no matter how you try to evade or distract. It's "put up or shut up" time. Do you, or do you not, have any actual evidence to back up your claim? Any answer that doesn't include the numbers (and references to how they were derived) is just wasting my time.
Actually, it's a really good idea and IMO an area where I believe not enough work has been done. Which is not to say no work has been done. There are several projects aimed at finding the best way to determine network distance, and several more that seek to use that information to create more optimal connection topologies. I don't have my links to that stuff handy right now, but if you send me email I should be able to dig them up.
Part of the problem is that many P2P networks have dependencies on some model or other of the higher-level abstract topology through which they route search queries, and it can be difficult to map (for example) a hypercube onto the actual IP-network topology. Lacking a good solution to that problem of mapping one topology onto another, many P2P developers punt; they try to minimize hops through the overlay network, and vaguely hope that by doing so they'll make up for the extra hops through the underlying network. In many cases it even seems to work, because the search algorithms that operate on the higher-level topology can be extremely efficient.
Nonetheless, if someone could figure out a way to reconcile those advanced search algorithms with a more "reality-based" topology, that would be great. If you think you have ideas on how to do that, by all means explore them. The more the merrier.
The article itself was kind of ho-hum, but the following part of the Slashdot intro caught my attention:
I doubt that developers of those free p2p applications have gave much thought to efficiency.
Again...wow. One would need to search far and wide, even on Slashdot, to find another example of such absolutely astonishing cluelessness. Timothy has obviously never talked to a P2P developer in his life. Sometimes it seems like efficiency is just about the only thing P2P developers think about, unless someone's on a security/anonymity rant. Little things like robustness or usability get short shrift because so much of the focus is on efficiency. Hundreds of papers have been written about the bandwidth-efficiency of various P2P networks - especially Gnutella, which everyone who knows anything knows is "worst of breed" when it comes to broadcasting searches.
It's unfortunate that the most popular P2P networks seem to be the least efficient ones, and doubly unfortunate that so many vendors bundle spyware with their P2P clients, but to say that P2P developers don't give much thought to efficiency is absurd. They give a lot more thought to efficiency than Slashdot editors give to accuracy, that's for damn sure.
Unfortunately, if we cache and then illegal material is downloaded, we can be held responsible for that material.
That's not really true. Even under the DMCA you can qualify as a safe harbor and avoid that liability. The requirements are too burdensome for an invididual, and that IMO is only one flaw among many in the DMCA, but for an ISP that employs full-time staff it's entirely doable and many ISPs have done it. You'll need to find another excuse.
Zeldman already addressed this point in the article (which you should have read):
Often, non-standards-compliant sites work in yesterday's browsers because their owners have invested in costly publishing tools that accommodate browser differences by generating multiple, non-standard versions tuned to the biases of specific browsers and platforms. This practice taxes the dial-up user's patience by wasting bandwidth on code forking, deeply nested tables, spacer pixels and other image hacks, and outdated or invalid tags and attributes.
At the same time, these multiple versions squander the site owner's bandwidth at a cost even the bean counters may be at a loss to calculate. The bigger the site and the greater its traffic, the more money gets wasted on server calls, redundancies, image hacks, and unnecessarily complex code and markup.
He's absolutely right. Reliance on browser-detecting back-end scripts is the truly broken approach. If you're using scripts to deal with browser differences (as opposed to using them for content that is truly dynamic) your site is broken. You're replacing what should be a one-time charge for having the site designed right with a recurring charge to support the additional compute capacity abused by the scripts.
It might seem like the obvious approach, but it's also the wrong approach. What the/. majority believes does not even closely resemble truth in many cases.
I was recently a victim of this problem. A machine at my former hosting provider (JTLnet, and they were already my former hosting provider before this incident) got infected by an email worm, and started propagating to everyone in that machine's address book - which seems to've included their entire customer-contact list. Being a modern email worm, it picked one address from that address book to spoof as the source of the messages, and I was the "lucky" guy so I ended up getting all the bounce messages.
There's a lot more to the story, but it's mostly about JTLnet and it's not their faults that are relevant here. The more interesting story is the part played by Verizon (my DSL service provider). Here's a major provider to millions of people, and their mail server was set up so it would happily propagate the worm's spoofed emails. A little experimentation quickly revealed that as long as the original FROM line (the SMTP one, not the one in the header) matched my email address the message would go through, regardless of where the connection came from. Unbelievable.
There is the tiniest shred of an excuse, though. I do remember being annoyed when they shut off SMTP access from outside their network entirely, so I couldn't reply to messages received on that account while at work. However, there are other ways to deal with the problem without allowing worms to spoof email through subscribers' accounts. SMTP authentication would be the obvious solution. A web interface for subscribers to specify which hosts could send email through their account would also have stopped the worm in its tracks. There's no excuse for a provider employing that many people to take the cheesy way out.
I was beginning to wonder whether you're a fool or a liar, but I've concluded that you're both. Your misrepresentations of ST's capabilities and your misunderstanding of basic parallel-programming concepts have exceeded my patience for correcting you, and so I bid you good day. Find someone else to teach you what you should already know.
Somehow I cannot see such blindingly obvious reasons.
I'm not an educator. There is a certain level of inability to see the obvious that I am neither qualified nor inclined to address.
ST takes away all burden associated with programming servers using pthreads. Quoting documentation:
The quoted excerpt is only applicable to a single-process single-thread environment - i.e. one that does not allow the programmer to benefit from SMP. It is thus irrelevant.
In that case, standard UNIX IPC facilities can be used. In addition to that, there is a way to synchronize different processes so that only the thread accessing the shared resource will be suspended (but not the entire process) if that resource is unavailable. In the following code fragment a pipe is used as a counting semaphore for inter-process synchronization:
So forcing programmers to use explicit IPC and semaphores is your idea of freeing them from the burden of classical multi-process models? Ummm, no. That is the burden of classical multi-process models. That's what the framework should be helping the programmer to avoid. The reason for the error becomes clear if we go back a sentence or two, though:
It is always better to have several separate smaller process-specific resources (e.g., data caches) than to have one large resource shared (and modified) by all processes.
Wrong, wrong, wrong. Sometimes it's better to have separate per-process resources, but by no means always - not even most of the time, in my experience. Anyone who would either make the quoted statement or accept it at face value is simply not qualified to be doing parallel program development, let alone telling others how it should be done.
Slashdot Mirror
The Rice connection almost certainly has to do with Peter Druschel and Pastry (for which the other PI seems to be Antony Rowstron of Microsoft Research, interestingly enough). I'm not totally sure of the ICSI connection, but they seem to be closely affiliated with UCB and I know that Ion Stoica works in these areas. OceanStore, CFS/SFS, Pastry, Kademlia - it's definitely a pretty good collection. A lot of the top people in DHT/DOLR (Distributed Hash Table, Distributed Object Location and Routing) research are involved, and I'd love to know how they plan to converge their various efforts toward a common solution.
The InfoWorld article describes a secure distributed storage system, not just plain old messaging connectivity. There aren't too many such beasts around; usually it's more of a "distributed, secure, usable - pick two" kind of thing. Some of the projects that approach the goal of combining all three actually seem to sharing the IRIS award - i.e. OceanStore at Berkeley and various projects at NYU. I don't know off the top of the head how ICSI and Rice fit in, but I'm about to go check their sites because I'll bet it's interesting.
Only if you define "enterprise" as "web". XML is making inroads in some enterprise applications, but there are still vast swaths of that territory where XML remains irrelevant.
Indeed. You can also be much more sophisticated about it than that. That's why I said it was an interesting problem. ;-)
Yes, it is, and I'd have to say that I don't think the routing is "there" yet. By far the best resource I've found on the topic is Perkins's Ad Hoc Networking which describes several types of protocols. Some of the work on overlay networks (e.g. Resilient Overlay Networks at MIT) and P2P is also applicable, though much of it tends to underestimate the importance of geographic locality and little of it deals with issues like routing-related power consumption.
Address space is not the real problem. The devices need addresses no matter whether you're using "lily pads" or more traditional means. The real problem problems are things like routing (not just in a mesh but in a dynamic mesh just for extra fun) and power management (bad to kill your batteries forwarding other people's packets).
You're right that storing both values on the card forces you to assume that the sensor is not compromised and is reporting actual observed (rather than recorded) speckle patterns, and that's a bad thing. On the other hand, I don't think your suggestion really protects the vendor either, because the bank is still not authenticated to the vendor. Maybe we both need to go think about this some more.
Your analysis seems right on target to me. Any system that's not challenge/response is vulnerable to replay by anyone who can intercept the messages involved, and this system only allows for a limited number of challenge/response exchanges.
I think the "validation server" approach might be problematic, though, since it allows new avenues for compromise. It might actually be better to store challenge/response pairs on the card itself, such that each use of a pair also erases that pair. Each card would then be good for a finite number of non-repeatable transactions, with server communication only necessary to "recharge" the card with new pairs. If the storage on the card and the challenge space are quite large, this is something the consumer would only have to do every N months or years, so it might actually be a decent convenience/security tradeoff even if recharging requires going to a service center or something.
Because stealing the speckle pattern does you no good. You need to create a device that makes that pattern, when light is shone through it and an inaccessible air gap onto a sensor. You can't just lay something on top of the sensor itself because, in any even half-way sensible design, you couldn't get to the sensor itself without disabling the entire reader.
I actually think this idea is extremely clever, but I don't know if I'd consider it a method of encryption. Even if you had an LED grid representing cleartext on one side, so you could read the "ciphertext" speckle pattern on the other side, how do you decrypt that? What kind of resolution, frequency and loss ratio are we talking about? This seems like it might be a really good authentication mechanism, where a known input will only be converted to a known output in the presence of a unforgeable secret, but I don't see how it can work for encryption where the input varies.
Yet another example of why we need a "-1, Wrong" moderation option.
Point taken, and thanks for the info. I hope some kind moderator will take a break from glue-sniffing long enough to mod that up.
It is indeed possible that Dr. Simonyi had such a special arrangement, but he doesn't just get rights to the ideas because he developed them (what this post's great-grandparent seems to assume). Exclusive ownership of intellectual property by the employer rather than the actual inventor is so commonplace that it's the exceptions which are noteworthy.
...all of which means precisely nothing. Nada. Zilch. Top-level (and often even mid-level) technical staff sign the exact same employee agreement as the business-side folks, and the part about "all your patents are belong to us" is there specifically for them. Dr. Simonyi might indeed have had some extra-special agreement, but if so it's because he's Simonyi and not because he's a geek instead of a suit.
That's so totally wrong that I hardly know where to begin. Patents have both inventors and owners, with only the latter really meaning anything legally. It's standard practice throughout the industry for employee agreements to require that ownership of patents be turned over to the company, so all the actual inventor(s) get is their names on the patent and maybe a bronze plaque if the company's feeling generous (which they weren't for my patent). There's very little Simonyi can do about it; the employer almost invariably holds all the cards.
And where is the law or precedent that would be used to prosecute an ISP for caching things that are illegal for other than copyright reasons? Common-carrier laws have been around for a long time, and have been used many times to shield the people who provide a communications medium from culpability for what is transmitted over that medium. The reason I mentioned the DMCA is that, in the specific case of copyrighted material, it attempts to supplant existing law and precedent; in cases other than copyright, that law and precedent is still valid and likely to protect the ISP as a common carrier.
Then again, law moves in mysterious ways. Are you aware of a recent case that would lead to a different conclusion?
The cite was already provided, but here it is again. The relevant claims from the AAP pages themselves are as follows:
AAP consists of two components: the STM and the Quick Shortcut Cache (http://aap.sourceforge.net/mod_qsc.html). According to AAP's own documents, STM accounts for a 25% performance gain. 25% divided by 700% is less than 4% of the total performance gain attributable to STM; the remaining 96% attributable to QSC definitely qualifies as "the vast majority" to me.
Your lie is not in claiming that STM is faster than the pthreads-based MPM. It might be, though 25% in "informal" and unspecified tests conducted by an advocate of one system don't exactly prove that beyond doubt. Your lie, rather, is in claiming that ST is "several times faster" based (apparently) on the 700% mostly-QSC number, even after the true significance of that number has been pointed out.
Let's try this one more time. You're the one claiming ST has good SMP scalability. I have expressed reasons for doubt, but the burden of proof nonetheless remains on you no matter how you try to evade or distract. It's "put up or shut up" time. Do you, or do you not, have any actual evidence to back up your claim? Any answer that doesn't include the numbers (and references to how they were derived) is just wasting my time.
I think the infoAnarchy Wiki covers it better than I could.
Nonetheless, people have obviously thought about it a lot, and timothy is still full of crap for saying otherwise. :-P
Actually, it's a really good idea and IMO an area where I believe not enough work has been done. Which is not to say no work has been done. There are several projects aimed at finding the best way to determine network distance, and several more that seek to use that information to create more optimal connection topologies. I don't have my links to that stuff handy right now, but if you send me email I should be able to dig them up.
Part of the problem is that many P2P networks have dependencies on some model or other of the higher-level abstract topology through which they route search queries, and it can be difficult to map (for example) a hypercube onto the actual IP-network topology. Lacking a good solution to that problem of mapping one topology onto another, many P2P developers punt; they try to minimize hops through the overlay network, and vaguely hope that by doing so they'll make up for the extra hops through the underlying network. In many cases it even seems to work, because the search algorithms that operate on the higher-level topology can be extremely efficient.
Nonetheless, if someone could figure out a way to reconcile those advanced search algorithms with a more "reality-based" topology, that would be great. If you think you have ideas on how to do that, by all means explore them. The more the merrier.
The article itself was kind of ho-hum, but the following part of the Slashdot intro caught my attention:
Again...wow. One would need to search far and wide, even on Slashdot, to find another example of such absolutely astonishing cluelessness. Timothy has obviously never talked to a P2P developer in his life. Sometimes it seems like efficiency is just about the only thing P2P developers think about, unless someone's on a security/anonymity rant. Little things like robustness or usability get short shrift because so much of the focus is on efficiency. Hundreds of papers have been written about the bandwidth-efficiency of various P2P networks - especially Gnutella, which everyone who knows anything knows is "worst of breed" when it comes to broadcasting searches.
It's unfortunate that the most popular P2P networks seem to be the least efficient ones, and doubly unfortunate that so many vendors bundle spyware with their P2P clients, but to say that P2P developers don't give much thought to efficiency is absurd. They give a lot more thought to efficiency than Slashdot editors give to accuracy, that's for damn sure.
That's not really true. Even under the DMCA you can qualify as a safe harbor and avoid that liability. The requirements are too burdensome for an invididual, and that IMO is only one flaw among many in the DMCA, but for an ISP that employs full-time staff it's entirely doable and many ISPs have done it. You'll need to find another excuse.
Zeldman already addressed this point in the article (which you should have read):
He's absolutely right. Reliance on browser-detecting back-end scripts is the truly broken approach. If you're using scripts to deal with browser differences (as opposed to using them for content that is truly dynamic) your site is broken. You're replacing what should be a one-time charge for having the site designed right with a recurring charge to support the additional compute capacity abused by the scripts.
It might seem like the obvious approach, but it's also the wrong approach. What the /. majority believes does not even closely resemble truth in many cases.
I was recently a victim of this problem. A machine at my former hosting provider (JTLnet, and they were already my former hosting provider before this incident) got infected by an email worm, and started propagating to everyone in that machine's address book - which seems to've included their entire customer-contact list. Being a modern email worm, it picked one address from that address book to spoof as the source of the messages, and I was the "lucky" guy so I ended up getting all the bounce messages.
There's a lot more to the story, but it's mostly about JTLnet and it's not their faults that are relevant here. The more interesting story is the part played by Verizon (my DSL service provider). Here's a major provider to millions of people, and their mail server was set up so it would happily propagate the worm's spoofed emails. A little experimentation quickly revealed that as long as the original FROM line (the SMTP one, not the one in the header) matched my email address the message would go through, regardless of where the connection came from. Unbelievable.
There is the tiniest shred of an excuse, though. I do remember being annoyed when they shut off SMTP access from outside their network entirely, so I couldn't reply to messages received on that account while at work. However, there are other ways to deal with the problem without allowing worms to spoof email through subscribers' accounts. SMTP authentication would be the obvious solution. A web interface for subscribers to specify which hosts could send email through their account would also have stopped the worm in its tracks. There's no excuse for a provider employing that many people to take the cheesy way out.
I was beginning to wonder whether you're a fool or a liar, but I've concluded that you're both. Your misrepresentations of ST's capabilities and your misunderstanding of basic parallel-programming concepts have exceeded my patience for correcting you, and so I bid you good day. Find someone else to teach you what you should already know.
I'm not an educator. There is a certain level of inability to see the obvious that I am neither qualified nor inclined to address.
The quoted excerpt is only applicable to a single-process single-thread environment - i.e. one that does not allow the programmer to benefit from SMP. It is thus irrelevant.
So forcing programmers to use explicit IPC and semaphores is your idea of freeing them from the burden of classical multi-process models? Ummm, no. That is the burden of classical multi-process models. That's what the framework should be helping the programmer to avoid. The reason for the error becomes clear if we go back a sentence or two, though:
Wrong, wrong, wrong. Sometimes it's better to have separate per-process resources, but by no means always - not even most of the time, in my experience. Anyone who would either make the quoted statement or accept it at face value is simply not qualified to be doing parallel program development, let alone telling others how it should be done.