Who's to say what's one person's fun and one person's not-fun? Is it better to use up a part of your life fussing over exercise numbers, to use it up playing video games, watching movies, reading books, talking politics in coffee shops, or posting to slashdot? That cyclist you're disparaging may look at your hobbies with similar disdain. I used to have a similar attitude about people who listened to ipods while running, and then I discovered escape pod's podcasts -- and now I'm completely hooked on listening to scifi stories while running. There's a reason this is all called "recreation" and not "activities designed to satisfy Kupfernigk's view of what people should do."
But on that note: An HRM can be a great training tool. I recently picked up a garmin 310XT to use as part of training for a marathon (I haven't raced for several years). Used right, the HRM can remind you to slow the bleep down when you're going too fast for training; the GPS makes it easy to run a target distance on new, unmapped courses -- freeing you from the tiny tyranny of carefully pre-planning long runs. And if you want to take a half an hour afterwords and compare your HR vs. pace vs. a month ago to see if your training plan is effective? More power to you. If you don't find it fun, don't do it.
That's only partly true. While the classification system is not classified, the names of specific compartments or special access programs can be and are classified. A nit, but might as well be accurate.:)
The core perspectives design supports this; we're in the middle of implementing it. If you read the Perspectives paper (it's fairly readable for anyone with a moderate technical background), section 5 (Detecting Malicious Notaries) discusses exactly how you can have other notaries log the observations made by the rest of the network. It's on the list of things to add to the Firefox plugin for the next major version.
(I linked to the HTML version of the paper; the PDF version is prettier.:)
It only helps for a short period of time - say, a day or two. If the attack against the whole site is successful for longer than that, Perspectives will eventually say "okay, that key's been around long enough, it's good." The assumption is that sites will eventually notice the attack -- which may or may not hold.:-) One cool thing about Perspectives is that you can use it to monitor your own site... if Perspectives starts showing a different key than the one you're using, you know you have a problem.
Halfway correct. The Perspectives user can also specify a time period over which the certs must be consistently observed (we don't default to using that right now, because it makes new websites not appear). Using this setting, Perspectives can help avoid short-lived attacks against the connection to the webserver.
The motto behind this is roughly "You can fool all of the browsers some of the time, and some of the browsers all of the time..." - but an adversary who can hijack all connections to a site for a long period of time will defeat Perspectives.
As an academic researcher funded by the NSF and DARPA, among other sources, I'd simply point out that registering a copy of a published paper isn't a particularly onerous burden. NSF requires multiple-page yearly reports; DARPA requires the same on a quarterly basis. The NSF reports already require listing the bibliographic information for every paper published as a result of the research. It's actually very much in a researcher's interest to track these things carefully anyway---it's one way to show that you're doing what you promised with the grant and that your work is having an impact. While I don't publish in PubMed-related areas, I and many others I know in computer science already take care to upload new papers to indexes like CiteSeer. It benefits everyone---including the authors---to have your work more readily available and easy to find via major databases like PubMed.
Correct (and the parent is correct as well). We didn't exhaustively characterize the reasons that the files differed, but most of the cases appeared to be modifications _after_ encoding. We have on the TODO list to test encode the same CD multiple times with the same settings to see if it produces a similar file, but we haven't run the test yet. There were tons of cases of metadata-altered MP3s, many cases of video files with the same video content but different audio or subtitle information, and some cases where the files differed in "seemingly random" (aka, we're not sure why.:) parts of the file for no reason we've figured out yet.
I think that for audio files, having a plugin for the p2p system that separated metadata from song information would probably capture a lot of the benefit we found. It's much harder for video files and for the files that had various weird changes that we don't have a good explanation for.
As other posters have suggested, the techniques we used also work well for software distributions sometimes. We've found around 10-30% similarity between different Linux ISOs and RPMs, *uncompressed* distributions of things like gcc (but not a.tar.gz file), etc. Part of what we really like about the SET technique is that it's able to speed transfers of all of those types of files without needing to have any file-type specific logic.
Similar in spirit - except rsync looks at files on your local hard drive by the same name, so there's only one possible candidate to draw from. SET looks at all of the files that everyone else is currently downloading, so we had to develop a much more efficient technique for locating useful files.
We define chunk boundaries using Rabin fingerprinting. It's a cute trick - not one of our own invention - that is relatively insensitive to insertions and deletions. It was used in some of the other work in this area, such as the Low Bandwidth File System (LBFS). There's a family of work in this area called "shingling" that can also apply to sequence similarity.
Take a peek at the paper - it actually does work, and we demonstrated it. The intuition: people make small changes to files like changing the artist or title in the MP3 header, and then BitTorrent and other systems treat this as a "different" file, when in fact it's 99.9% similar. (Yes, I'm one of the authors.)
Your statement doesn't make a lot of sense. a) Hard drives are a non-repairable system, for all intents and purposes. Therefore, there *is* no repair. MTTF is the only useful metric. b) MTBF = MTTF + the time to repair. Assuming that's zero, then for any useful failure engineering, hard drive MTBF = hard drive MTTF. That's about all you've got if you're expressing the statistic as a single number. The reason that MTBF is a function of time is to cope with the assumption that the system is less reliable after a repair, which doesn't apply in this case.
Now, you can have all sorts of distributions that you draw that mean from, but a mean is a mean.
Or take my answer: make them all available, freely and openly. I'm not having this semester's lectures videotaped, but I did three semesters ago. (The content hasn't changed that drastically, and the videotaping is a bit pricey.) I post the links to the video and the ppt/pdf/ps for the lecture notes immediately after class.
Why not? Your students are adults. I suspect I took a 10% attendance hit because of the videotaping, but the students were almost universal in saying that they liked having the videos around. Watching class at 2x speed is apparently a great trick.
Why bother restricting to just students in your class? Unless it's the cost of bandwidth, I don't get it. Perhaps there are students out there who'd really love to see the material, but wouldn't have the opportunity otherwise. Watching the video is not a substitute for the whole experience of the class (or for attending university); if MIT, CMU, and Stanford are all not worried about it, why should you be?
Before you jump to conclusions, keep in mind that Varghese is one of the leading experts in the design of routing algorithms and similar topics. Take a peek at his industrial impact list; he's not kidding. The fast route lookup algorithm he developed while at WUSTL was, along with some work done concurrently at Lulea university in Sweden, the most major advance in fast route lookup algorithms in about a decade. In other words - don't judge the book by the qualifications of someone who decided to review it.
Though the reviewer is right - many of the algorithms in the book would be useful in software-based routers, though there's room for caution: some of them are encumbered by patents of various sorts, and are recent enough that they'll remain so for a while.
Since people seem to be interested in this, you might also take a peek at the CMU computer networks course, which I put online almost entirely (lecture nodes, video, homeworks, and the programming projects). Click on "Syllabus" to get to the contentful-bits. Feedback is welcome: Srini and I hope that leaving it online will be useful for students and instructors everywhere.
If that's the attitude held by most of your professors as an undergrad, I'm sorry -- for you and for them. I love teaching my undergrad class, and I find it seriously rewarding. We alternate grad/undergrad, so I get a bit of a break from it, and I wouldn't want to teach the same class every semester, but teaching one semester of undergrad per year is awesome.
(I'll freely confess that I'm spoiled at CMU. We have a low student to teacher ratio and a relatively low teaching load because half of our jobs are research. But hey, that's part of the reason to go to a good school, eh?:)
> "What you get in security, you lose in freedom."
One could argue - and many have - that the current Internet does not give you enough of either. Security in the Internet context applies also to the security of the user from eavesdropping or interruption.
Also - please distinguish between government funded projects and research - GENI is research, pure and simple. Right now, there's no blueprint for what the results of this will look like, no deployment plan for rolling out a new, improved Internet. Rather, there's a plan to create a testbed to enable network research to be drastically more innovative and effective than it has been. Like all research, there will be successful projects under this umbrella, and unsuccessful ones. But if you never take the risk, you never have the
chancetomakeamazingdiscoveries. Say what you want about government inefficiency, it's funding like that provided by the NSF, NIH, and (formerly) DARPA, and their corresponding agencies in other countries over the centuries that has made possible some of the greatest advances in the history of humanity.
Why is the Internet less reliable? I think it's a combination of things. First, as you suggested, it's young, and is constantly undergoing massive change. The telephone network had years of relative stasis in which to stabalize. The Internet is still experiencing huge growth in capacity and capabilities, as the network and the connected devices grow by leaps and bounds (c.f., Moore's Law.:).
The second thing is that the Internet is a general -purpose network. People do the damndest things with it. Napster and Bittorrent drastically changed the traffic patterns on the Internet, just like the Web did before them, and like NNTP (news) did before that. Phone engineers had fairly accurate models of call traffic behavior and could plan around those.. at least, until modems came around and people started dialing in to the Internet.
The final thing is one of management complexity. The Internet is a looser confederation of interconnected systems. There are thousands of ISPs in the U.S. alone, compared to the handful of telephone companies. This distributed operation is a powerful thing for innovation - any schmoe with a great new idea (56kbps modems, DSL, cable, wireless, free space optics, carrier pidgeons) can start an ISP. But it also means that the trust model is a little flakier and that you've got to get more people to work properly to make the network stable. There may also be a commodity cost argument in terms of what the average customer is willing to pay for. Particularly when their 911 service doesn't run over the Internet.
Re the ARPAnet: If you look at the Internet inside of one ISP or autonomous system, it's quite a bit more reliable than the end-to-end picture when you're going from your house through two ISPs to some remote service provider. AT&T and its ISP friends will happily sell you a 99.99% uptime SLA on their Internet service, if you pay for it, and I believe they can probably meet it most of the time. The basic packet switching technology works exceptionally well, and can deliver great availability. But things get very complex when you scale that up into the entire Internet and start involving not just the raw links of a single ISP, but many organizations, servers, DNS, etc.
To the posters to shouted "insane!" and "if it's not broken, don't fix it!", a couple of comments.
First off, there are a number of major challenges facing the Internet. The ones that spring immediately to mind are security, management, and availability. To see some of these, compare the Internet to the (good parts of) the telephone network. 911 emergency phone service has roughly 99.99% availability; the Internet is an order of magnitude worse. You can't get a virus over the phone lines, and it's very difficult to create a botnet of 100,000 people to DDoS, say, a hospital's telephone system. Now, that ignores many of the good things about the Internet -- you can create and run fabulous applications that the network designers never envisioned, etc., at least, if you're not running behind a NAT.;)
But wouldn't it be nice to have a network that had the best of all worlds? A network that cost 1/10th as much to manage as it does today? A network where your parents didn't call you up frequently and ask, "It says it couldn't find my DHCP server - what's wrong??" A network where you didn't resort to weird (but clever) hacks like traceroute to try to diagnose problems? Where Scott Richter couldn't create a spam-blasting army of drones? I use Vonage, and I had to dial 911 a few weeks ago to report a fire at the apartment across the street. During part of the conversation, I couldn't hear the operator well enough to understand the questions she was asking. It was a frightening and educational experience.
One of the most important parts of this program is that it's encouraging researchers to not feel constrained to fit into the current design, and is looking at ways to get that deployed in a way that it can gateway to or run on top of the current Internet. There's a big difference between this program and the Internet2, IPv6, etc. It's both higher risk and (hopefully!) higher reward. Internet2 was pretty much "Internet + faster links + some focused researchy bits"; it got co-oped early on because it provided lots of bandwidth to big science, and was too entrenched to try radical new things that (gasp!) might break. GENI is research + interfaces to allow early adopters -- like, say, slashdotters -- to make use of its services. The idea of creating an infrastructure that can safely be used simultaneously for testing out new research prototypes at all levels and running production versions of those services that succeed is a powerful notion that will give GENI a big edge over prior attempts.
It's an exciting proposal, and a scary one. If it gets funded, it could be either the biggest success in networking since the Web, or the biggest flop.
(Disclosure- I'm a networking professor at Carnegie Mellon. This is my field, I've been involved in some of the GENI discussions, and I intend to submit funding proposals to it. I think it'll be one of the best things in years to help academic networking research have a big impact on the real world.)
The last digits, according to the pi searcher, are 315921943469. Now you too can recite them -- just make up a lot of numbers in the middle and hope the judges get bored!
MySQL and data crunching
on
Data Crunching
·
· Score: 2, Insightful
MySQL's lack of support for some of the ANSI SQL features is annoying. But, that said, I do a lot of data crunching on a terabyte or so of Internet measurement data, and MySQL remains my database of choice. In a data-mining application like mine, I need speed and a compact on-disk representation of the data and the indices before anything. Our inserts are batched a couple of times a day; having them fast is important, but having them run concurrently with queries isn't. I don't need transactions, I can deal with table-level locking, and I'm willing to give up a couple of things like nested selects to get that speed.
Given that MySQL is the best fit for some types of data crunching applications, the earlier comment about assuming nested queries has merit.
My requirements arise in a research setting, so perhaps they're less common. Companies like wal-mart can afford big iron on which to do their data mining. Smaller data crunching tasks don't make the same kind of performance demands on their RDBMS. Of course, one thing to consider is that the standard RDBMS model isn't all that well suited to huge-scale data-mining in general, so there may be no silver bullet here for any of us to get religious about yet.
1. In a host that has been connected to a network that does not have an IP address server and is not connected with any network having an IP address server, a method for automatically generating an IP address for the host, without another component of the network being required to transmit, to the host over the network, an IP address of said other component, the method comprising the steps of:
without the host having received over the network any IP address of another component of the network, selecting a valid network identifying value as a network identifying portion of the IP address for the host;
without the host having received over the network said any IP address of another component of the network, generating a host identifying portion of the IP address for the host based on information available to the host; and
testing the generated IP address for the host for conflicting usage by another host on the network and determining that no conflicting usage of the generated IP address exists.
And compare it to RFC1971 and RFC2462, where they define the creation and testing of link-local addresses. The patent seems to cover things outside the scope of IPv6 autoconfiguration, and IPv6 autoconfiguration goes beyond the patent to specify how to do router-based autoconfiguration, but there is a distinct area of overlap at claim 1.
We keep seeing this happen. The patent system is a mess -- and even Microsoft agrees. We can, and should, try to solve this problem through legislative means, but we, as the people and companies that invent and use technology, can and must also take steps ourselves.
I call on Microsoft to
to take the lead in cleaning up the patent mess starting with patent 6101499. Bill Gates has established himself as a leader in philanthrophy through the Gates foundation; Microsoft is surely capable of helping lead the fight to clean up the IP system by example.
David Andersen
Assistant Professor of Computer Science
Carnegie Mellon University
302 hijacks work because Google goes to http://bad.site/ and gets redirected to http://good.site/. It then treats the contents of the bad.site as identical to that of good.site. The effect seems similar to if somebody simply copied an entire page off of your site (I'm not sure if it's actually more serious than this), but it's easier to do because you're just keeping a small table of redirections.
How serious is it? Don't know. It's pretty easy for a webmaster to check for hijacking and have her pages de-hijacked (see aforementioned article). It's probably not as screamingly awful as the threadwatch.org article suggests, but the redirector sites are rather annoying. Several of the comments in the webmaster article suggest that Google has already started moving on the problem.
The author suggests that microsoft must be scraping google b/c the only place _he_ could find the URLs they're requesting was google's cache.
Uh.
Microsoft has been developing their internal search engine for quite a while now. Part of developing a search engine is using it to crawl and creating a large corpus of test data. It's hugely likely that M$ has had a working crawler system for much, much longer than would be indicated by their public announcement. Quite a few people who helped develop Altavista at HP/Compaq/DEC research joined Microsoft Research about two years ago - the kind of people who could write a high-performance crawler in their sleep and wake up feeling refreshed.
That article seems like baseless, uninformed speculation, to put it not-so-politely.
I just finished reading a pretty cool book (Troubleshooting Analog Circuits) written by an engineer from National Semiconductor in 1990. His main job focus was on switching power supplies, and he commented several times in the book about the perils of underestimating the complexity of building one well, even with today's mega-modern power supplies and switching regulator ICs. Today's Dell power supply recall, and a few other recent examples, illustrates that point quite graphically.
Even in our/.'d digital world, analog design is still important. Bummer, since I'm a software person...:)
Slashdot Mirror
Who's to say what's one person's fun and one person's not-fun? Is it better to use up a part of your life fussing over exercise numbers, to use it up playing video games, watching movies, reading books, talking politics in coffee shops, or posting to slashdot? That cyclist you're disparaging may look at your hobbies with similar disdain. I used to have a similar attitude about people who listened to ipods while running, and then I discovered escape pod's podcasts -- and now I'm completely hooked on listening to scifi stories while running. There's a reason this is all called "recreation" and not "activities designed to satisfy Kupfernigk's view of what people should do."
But on that note: An HRM can be a great training tool. I recently picked up a garmin 310XT to use as part of training for a marathon (I haven't raced for several years). Used right, the HRM can remind you to slow the bleep down when you're going too fast for training; the GPS makes it easy to run a target distance on new, unmapped courses -- freeing you from the tiny tyranny of carefully pre-planning long runs. And if you want to take a half an hour afterwords and compare your HR vs. pace vs. a month ago to see if your training plan is effective? More power to you. If you don't find it fun, don't do it.
That's only partly true. While the classification system is not classified, the names of specific compartments or special access programs can be and are classified. A nit, but might as well be accurate. :)
Re: recording the transactions
The core perspectives design supports this; we're in the middle of implementing it. If you read the Perspectives paper (it's fairly readable for anyone with a moderate technical background), section 5 (Detecting Malicious Notaries) discusses exactly how you can have other notaries log the observations made by the rest of the network. It's on the list of things to add to the Firefox plugin for the next major version.
(I linked to the HTML version of the paper; the PDF version is prettier. :)
It only helps for a short period of time - say, a day or two. If the attack against the whole site is successful for longer than that, Perspectives will eventually say "okay, that key's been around long enough, it's good." The assumption is that sites will eventually notice the attack -- which may or may not hold. :-) One cool thing about Perspectives is that you can use it to monitor your own site... if Perspectives starts showing a different key than the one you're using, you know you have a problem.
Halfway correct. The Perspectives user can also specify a time period over which the certs must be consistently observed (we don't default to using that right now, because it makes new websites not appear). Using this setting, Perspectives can help avoid short-lived attacks against the connection to the webserver.
The motto behind this is roughly "You can fool all of the browsers some of the time, and some of the browsers all of the time..." - but an adversary who can hijack all connections to a site for a long period of time will defeat Perspectives.
-Dave (one of the researchers on the project)
As an academic researcher funded by the NSF and DARPA, among other sources, I'd simply point out that registering a copy of a published paper isn't a particularly onerous burden. NSF requires multiple-page yearly reports; DARPA requires the same on a quarterly basis. The NSF reports already require listing the bibliographic information for every paper published as a result of the research. It's actually very much in a researcher's interest to track these things carefully anyway---it's one way to show that you're doing what you promised with the grant and that your work is having an impact. While I don't publish in PubMed-related areas, I and many others I know in computer science already take care to upload new papers to indexes like CiteSeer. It benefits everyone---including the authors---to have your work more readily available and easy to find via major databases like PubMed.
This change is a good thing.
Correct (and the parent is correct as well). We didn't exhaustively characterize the reasons that the files differed, but most of the cases appeared to be modifications _after_ encoding. We have on the TODO list to test encode the same CD multiple times with the same settings to see if it produces a similar file, but we haven't run the test yet. There were tons of cases of metadata-altered MP3s, many cases of video files with the same video content but different audio or subtitle information, and some cases where the files differed in "seemingly random" (aka, we're not sure why. :) parts of the file for no reason we've figured out yet.
.tar.gz file), etc. Part of what we really like about the SET technique is that it's able to speed transfers of all of those types of files without needing to have any file-type specific logic.
I think that for audio files, having a plugin for the p2p system that separated metadata from song information would probably capture a lot of the benefit we found. It's much harder for video files and for the files that had various weird changes that we don't have a good explanation for.
As other posters have suggested, the techniques we used also work well for software distributions sometimes. We've found around 10-30% similarity between different Linux ISOs and RPMs, *uncompressed* distributions of things like gcc (but not a
Similar in spirit - except rsync looks at files on your local hard drive by the same name, so there's only one possible candidate to draw from. SET looks at all of the files that everyone else is currently downloading, so we had to develop a much more efficient technique for locating useful files.
We define chunk boundaries using Rabin fingerprinting. It's a cute trick - not one of our own invention - that is relatively insensitive to insertions and deletions. It was used in some of the other work in this area, such as the Low Bandwidth File System (LBFS). There's a family of work in this area called "shingling" that can also apply to sequence similarity.
Take a peek at the paper - it actually does work, and we demonstrated it. The intuition: people make small changes to files like changing the artist or title in the MP3 header, and then BitTorrent and other systems treat this as a "different" file, when in fact it's 99.9% similar.
(Yes, I'm one of the authors.)
Your statement doesn't make a lot of sense. a) Hard drives are a non-repairable system, for all intents and purposes. Therefore, there *is* no repair. MTTF is the only useful metric. b) MTBF = MTTF + the time to repair. Assuming that's zero, then for any useful failure engineering, hard drive MTBF = hard drive MTTF. That's about all you've got if you're expressing the statistic as a single number. The reason that MTBF is a function of time is to cope with the assumption that the system is less reliable after a repair, which doesn't apply in this case.
Now, you can have all sorts of distributions that you draw that mean from, but a mean is a mean.
Or take my answer: make them all available, freely and openly. I'm not having this semester's lectures videotaped, but I did three semesters ago. (The content hasn't changed that drastically, and the videotaping is a bit pricey.) I post the links to the video and the ppt/pdf/ps for the lecture notes immediately after class.
Why not? Your students are adults. I suspect I took a 10% attendance hit because of the videotaping, but the students were almost universal in saying that they liked having the videos around. Watching class at 2x speed is apparently a great trick.
Why bother restricting to just students in your class? Unless it's the cost of bandwidth, I don't get it. Perhaps there are students out there who'd really love to see the material, but wouldn't have the opportunity otherwise. Watching the video is not a substitute for the whole experience of the class (or for attending university); if MIT, CMU, and Stanford are all not worried about it, why should you be?
Before you jump to conclusions, keep in mind that Varghese is one of the leading experts in the design of routing algorithms and similar topics. Take a peek at his industrial impact list; he's not kidding. The fast route lookup algorithm he developed while at WUSTL was, along with some work done concurrently at Lulea university in Sweden, the most major advance in fast route lookup algorithms in about a decade. In other words - don't judge the book by the qualifications of someone who decided to review it.
Though the reviewer is right - many of the algorithms in the book would be useful in software-based routers, though there's room for caution: some of them are encumbered by patents of various sorts, and are recent enough that they'll remain so for a while.
Since people seem to be interested in this, you might also take a peek at
the CMU computer networks course, which I put online almost entirely (lecture nodes, video, homeworks, and the programming projects). Click on "Syllabus" to get to the contentful-bits. Feedback is welcome: Srini and I hope that leaving it online will be useful for students and instructors everywhere.
If that's the attitude held by most of your professors as an undergrad, I'm sorry -- for you and for them. I love teaching my undergrad class, and I find it seriously rewarding. We alternate grad/undergrad, so I get a bit of a break from it, and I wouldn't want to teach the same class every semester, but teaching one semester of undergrad per year is awesome.
:)
(I'll freely confess that I'm spoiled at CMU. We have a low student to teacher ratio and a relatively low teaching load because half of our jobs are research. But hey, that's part of the reason to go to a good school, eh?
One could argue - and many have - that the current Internet does not give you enough of either. Security in the Internet context applies also to the security of the user from eavesdropping or interruption.
Also - please distinguish between government funded projects and research - GENI is research, pure and simple. Right now, there's no blueprint for what the results of this will look like, no deployment plan for rolling out a new, improved Internet. Rather, there's a plan to create a testbed to enable network research to be drastically more innovative and effective than it has been. Like all research, there will be successful projects under this umbrella, and unsuccessful ones. But if you never take the risk, you never have the chance to make amazing discoveries. Say what you want about government inefficiency, it's funding like that provided by the NSF, NIH, and (formerly) DARPA, and their corresponding agencies in other countries over the centuries that has made possible some of the greatest advances in the history of humanity.
Why is the Internet less reliable? I think it's a combination of things. First, as you suggested, it's young, and is constantly undergoing massive change. The telephone network had years of relative stasis in which to stabalize. The Internet is still experiencing huge growth in capacity and capabilities, as the network and the connected devices grow by leaps and bounds (c.f., Moore's Law. :).
The second thing is that the Internet is a general -purpose network. People do the damndest things with it. Napster and Bittorrent drastically changed the traffic patterns on the Internet, just like the Web did before them, and like NNTP (news) did before that. Phone engineers had fairly accurate models of call traffic behavior and could plan around those .. at least, until modems came around and people started dialing in to the Internet.
The final thing is one of management complexity. The Internet is a looser confederation of interconnected systems. There are thousands of ISPs in the U.S. alone, compared to the handful of telephone companies. This distributed operation is a powerful thing for innovation - any schmoe with a great new idea (56kbps modems, DSL, cable, wireless, free space optics, carrier pidgeons) can start an ISP. But it also means that the trust model is a little flakier and that you've got to get more people to work properly to make the network stable. There may also be a commodity cost argument in terms of what the average customer is willing to pay for. Particularly when their 911 service doesn't run over the Internet.
Re the ARPAnet: If you look at the Internet inside of one ISP or autonomous system, it's quite a bit more reliable than the end-to-end picture when you're going from your house through two ISPs to some remote service provider. AT&T and its ISP friends will happily sell you a 99.99% uptime SLA on their Internet service, if you pay for it, and I believe they can probably meet it most of the time. The basic packet switching technology works exceptionally well, and can deliver great availability. But things get very complex when you scale that up into the entire Internet and start involving not just the raw links of a single ISP, but many organizations, servers, DNS, etc.
First off, there are a number of major challenges facing the Internet. The ones that spring immediately to mind are security, management, and availability. To see some of these, compare the Internet to the (good parts of) the telephone network. 911 emergency phone service has roughly 99.99% availability; the Internet is an order of magnitude worse. You can't get a virus over the phone lines, and it's very difficult to create a botnet of 100,000 people to DDoS, say, a hospital's telephone system. Now, that ignores many of the good things about the Internet -- you can create and run fabulous applications that the network designers never envisioned, etc., at least, if you're not running behind a NAT. ;)
But wouldn't it be nice to have a network that had the best of all worlds? A network that cost 1/10th as much to manage as it does today? A network where your parents didn't call you up frequently and ask, "It says it couldn't find my DHCP server - what's wrong??" A network where you didn't resort to weird (but clever) hacks like traceroute to try to diagnose problems? Where Scott Richter couldn't create a spam-blasting army of drones? I use Vonage, and I had to dial 911 a few weeks ago to report a fire at the apartment across the street. During part of the conversation, I couldn't hear the operator well enough to understand the questions she was asking. It was a frightening and educational experience.
One of the most important parts of this program is that it's encouraging researchers to not feel constrained to fit into the current design, and is looking at ways to get that deployed in a way that it can gateway to or run on top of the current Internet. There's a big difference between this program and the Internet2, IPv6, etc. It's both higher risk and (hopefully!) higher reward. Internet2 was pretty much "Internet + faster links + some focused researchy bits"; it got co-oped early on because it provided lots of bandwidth to big science, and was too entrenched to try radical new things that (gasp!) might break. GENI is research + interfaces to allow early adopters -- like, say, slashdotters -- to make use of its services. The idea of creating an infrastructure that can safely be used simultaneously for testing out new research prototypes at all levels and running production versions of those services that succeed is a powerful notion that will give GENI a big edge over prior attempts.
It's an exciting proposal, and a scary one. If it gets funded, it could be either the biggest success in networking since the Web, or the biggest flop.
(Disclosure- I'm a networking professor at Carnegie Mellon. This is my field, I've been involved in some of the GENI discussions, and I intend to submit funding proposals to it. I think it'll be one of the best things in years to help academic networking research have a big impact on the real world.)
The last digits, according to the pi searcher, are 315921943469. Now you too can recite them -- just make up a lot of numbers in the middle and hope the judges get bored!
Given that MySQL is the best fit for some types of data crunching applications, the earlier comment about assuming nested queries has merit.
My requirements arise in a research setting, so perhaps they're less common. Companies like wal-mart can afford big iron on which to do their data mining. Smaller data crunching tasks don't make the same kind of performance demands on their RDBMS. Of course, one thing to consider is that the standard RDBMS model isn't all that well suited to huge-scale data-mining in general, so there may be no silver bullet here for any of us to get religious about yet.
Not really. See claim 1 of the patent:
1. In a host that has been connected to a network that does not have an IP address server and is not connected with any network having an IP address server, a method for automatically generating an IP address for the host, without another component of the network being required to transmit, to the host over the network, an IP address of said other component, the method comprising the steps of:
without the host having received over the network any IP address of another component of the network, selecting a valid network identifying value as a network identifying portion of the IP address for the host;
without the host having received over the network said any IP address of another component of the network, generating a host identifying portion of the IP address for the host based on information available to the host; and
testing the generated IP address for the host for conflicting usage by another host on the network and determining that no conflicting usage of the generated IP address exists.
And compare it to RFC1971 and RFC2462, where they define the creation and testing of link-local addresses. The patent seems to cover things outside the scope of IPv6 autoconfiguration, and IPv6 autoconfiguration goes beyond the patent to specify how to do router-based autoconfiguration, but there is a distinct area of overlap at claim 1.
I call on Microsoft to to take the lead in cleaning up the patent mess starting with patent 6101499. Bill Gates has established himself as a leader in philanthrophy through the Gates foundation; Microsoft is surely capable of helping lead the fight to clean up the IP system by example.
David Andersen
Assistant Professor of Computer Science
Carnegie Mellon University
302 hijacks work because Google goes to http://bad.site/ and gets redirected to http://good.site/. It then treats the contents of the bad.site as identical to that of good.site. The effect seems similar to if somebody simply copied an entire page off of your site (I'm not sure if it's actually more serious than this), but it's easier to do because you're just keeping a small table of redirections.
How serious is it? Don't know. It's pretty easy for a webmaster to check for hijacking and have her pages de-hijacked (see aforementioned article). It's probably not as screamingly awful as the threadwatch.org article suggests, but the redirector sites are rather annoying. Several of the comments in the webmaster article suggest that Google has already started moving on the problem.
The author suggests that microsoft must be scraping google b/c the only place _he_ could find the URLs they're requesting was google's cache.
Uh.
Microsoft has been developing their internal search engine for quite a while now. Part of developing a search engine is using it to crawl and creating a large corpus of test data. It's hugely likely that M$ has had a working crawler system for much, much longer than would be indicated by their public announcement. Quite a few people who helped develop Altavista at HP/Compaq/DEC research joined Microsoft Research about two years ago - the kind of people who could write a high-performance crawler in their sleep and wake up feeling refreshed.
That article seems like baseless, uninformed speculation, to put it not-so-politely.
Even in our /.'d digital world, analog design is still important. Bummer, since I'm a software person... :)