I don't want to trivialize the effort in moving hundreds of thousands of users, but IMHO they exercised extremely poor judgement in prioritizing the work.
There are two separate issues here. One is basic connectivity, the other are the bundled ISP services.
Many of us (a small fraction of their users, but more common among the Linux/Unix users) used them solely for connectivity. It's not just elitism either: when you have your own domain(s) and hosting services, you don't have much interest in these bundled services. *Especially* when we consider all ISPs a bit iffy, having been around the block a few times already - some of us have "vanity domains" precisely to avoid this sudden need to change email and web addresses etc.
Yet we spent days without access while someone was busy creating an account we will never use to replace another account we never used. Give us basic connectivity and we're happy - at worst we use the DNS from our hosting account for a few days. But no, we were left in the dark for days.
Of course, most people do use the bundled ISP accounts, but again they have alternative accounts at Hotmail, at the office, etc. Again, give them basic connectivity and DNS services and they'll be able to do a lot, even if they don't have their usual email for a few days. But no, they were left in the dark for days.
The only people this policy served were those refugees from AOL who never looked beyond their own email or web pages. I'm sure there were a few, but I would be surprised if it was more than 10%.
I believe the vast majority of people would prefer to have basic connectivity up within 24 hours, even if it delayed email and web pages for a few additional days, than to be dark for days.
They're intercepting DNS queries to the Excite DNS IP addresses and returning a static value - a record to their transitional site. The name doesn't matter - *everything* resolves to that site.
You didn't specify how you get your DNS addresses. On a Unix/Linux box, it can hide in a lot of places. (DHCP packets,/etc/resolv.conf,/etc/dhcp.conf,/etc/bind/named.conf, etc.)
Checking my settings, I saw that I was using my last known @Home DNS addresses as the 'forwarder' addresses. When I replaced them with 0.0.0.0 (forcing a query against the root servers) I got the right addresses.
So they hijacked their old DNS server addresses (assuming they were operated by Excite), not the root DNS servers... but that would be a trivial change to make. Definitely not something that gives knowledgeable users warm fuzzies.
As I mentioned in a post which crossed in the ether, this "interesting DNS hack" kills any local DNS server, and raises some disturbing long-term questions.
FYI, my AT&T cable (Boulder) came up and everything was fine once I told my Linksys box to use DHCP instead of a static IP address, but everything went to hell after about 15-20 minutes when AT&T HIJACKED THE ROOT DNS SERVERS. Every single address, including attbi.com, resolves to their transition site. I couldn't even bring up their help page.
On the one hand, this is clearly a (feeble) attempt to communicate with their users. How many Windows users do they think are using the root DNS servers?! -- it will primarily hit the people using "unsupported" operating systems.
But this makes the broadband service unusable to those of us running our own local DNS servers precisely because of problems we've had in the past with theirs. Sure, there are workarounds (I can think of several), but in the overall picture they're more hassles to maintain than my current approach.
I couldn't get through the ATTBI number (never any complaints when you don't give the sheep a way to reach a person!), but asked the cable TV person to pass on my... annoyance but temporary acceptance of the situation... and to ask the ATTBI people to call be back with an ETA for when the root DNS servers will be restored.
I fear, deep in my cynical heart, that this is actually an attempt to force everyone to use their DNS servers so they can track our movements and ultimately hijack additional content. E.g., you ask for "www.ford.com" but get a "www.chevrolet.com" interstital. In that case the root DNS servers are never coming back... and I want to close my account as soon as possible.
At least, for now, they aren't blocking the DNS servers of other ISPs. I've still lost some important local functionality, but at least I'm able to get back up.
Unfortunately, it also blocks all Debian users. At least it looks like somebody *finally* packaged ssh2 for woody (ironically, a few days after I last checked for the packages, from the time stamps). Even ssh3 is now listed.
Seriously, I long ago got tired of DNS servers "disappearing" because some bozo forgot (or never knew) that there were some systems set up for static IPs because their support people didn't want to deal with the odd Linux user... and without DHCP you don't get the new IP address for the name servers.
Once I had a basic DNS server running, I took immense pleasure in adding a few authoritative entries. Doubleclick? X10.com? They go straight to my web server (usually) where the browser returns a quick 404 error. Watching the status bar on the dialup line, I'm beginning to suspect that the good performance on my cable modem is as much due to local DNS server as the bandwidth. (For some reason the local server isn't working with the dialup line.)
I was told that we're scheduled to have our service back up on Thursday.
Of course, who knows how long it will take to get to noncompliant drones who aren't using Windows. We cause them too much trouble since we can't follow their predigested scripts (and I'm usually too busy to be willing to lie my way through endless Windows menus instead of taking 15 seconds to edit a text file and restart a server).
This argument assumes that the passage of time is real. It may not be. At the human scale, this is a moot point since our consciousness is predicated on the perception of the passage of time, but at the fundamental level where there's CPT conservation it may be nothing but an illusion that distracts you from the truth.
Restrictions on the time, manner and place of speech have been upheld countless times in meatspace. Your right to speak on the lawn of the courthouse does not give you the right to set up a PA system in a residential area at 3 AM.
This is partly because of the tradeoff between freedom of speech and the right to peaceful enjoyment of personal property and life.
But it's also because "freedom of speech" does not protect the physical act of speaking, it protects the right to express a dissenting view. The majority requires no explicit protection precisely because it's the majority. But the minority, especially the lone dissenter, *does*. That's why some cities have laws requiring that protestors stand in specific "boxes" when they make their speech - it's partly to prevent others from attempting to drown out their voice!
The same thing applies in cyberspace. If you have *no* moderation and attempt to discuss controversial issues, you *will* have an asshole appear who doesn't mind posting hundreds of marginally pertinent responses to drown out "objectionable" content. Just look at alt.scientology (or something like that) sometime. While it's technically true that the original messages are still there, and it's not an exact analogue of the real-world situation where the lone protester may not be heard at all, in practice few people will bother to search for meaningful content and the protester(s) will have succeeded in supressing speech.
It's ironic, but sometimes the only way to guarantee that everyone has a voice is to be willing to silence those who would use theirs as a weapon.
It's easy to implement a crypto filesystem, but hard to do it *right*.
Some quick examples:
1) Is a standard cipher used? (easy, now that libraries are widely available)
2) Is a standard cipher used *correctly*? (e.g., no ECB mode!)
3) Does the same data in two blocks encrypt to the same ciphertext? If not, how are you randomizing them? What happens if you copy an encrypted FS from one media to another, e.g., via backups?
4) How do you detect an incorrect encryption key?
There's then the whole issue of key management, the truly hard part. How do you generate the key from the password? How do you support multiple users on the encrypted file system? (N.B., this is cryptospeek for "how do you prevent disgruntled employees from encrypting your data then walking away?" This usually means secondary and even tertiary keys automatically inserted by the system.) How do you handle system reboots?
Finally there's the mundane. Top of that list - how do you handle backups? Can you back up the encrypted data? Can you deny backups of the unencrypted data?
Maybe you need deniability, but out here in the real world a lot of people should be using encrypted file systems just to ensure that sensitive or confidential information is not exposed to others if the disk is stolen, the cleaning people are bored, etc.
Personally, I don't want my doctor to have deniability about his records regarding me. Or my lawyer. Or my accountant. And most especially not my banker, financial adviser, etc.
In fact, for these people deniability makes a solution look much less attractive. People get *really* nervous when their accountant or lawyer has strong deniability about what the advice they gave you, about where your money went, etc.
You're confusing two different masses. One is the mass of the star that collapses in a type-II supernova. There's a minimal size for such stars (just as there's a minimal size for a star to be a star, instead of a large Jovian body), and IIRC there's also a maximum mass due to various physical constraints.
The other mass is what's crunched into the black hole. If I remember the numbers right, and they haven't been surplanted by more recent research, at "maximum" crunch some of the stellar mass is falling inward at a third of the speed of light, and the maximum density is something like 4 times that of a regular nucleus. This is a *very* hard surface, and anything outside of the maximum crunch will be blown outward. A lot of the matter in the crunch will be blown outward, as the "spring" releases. This is the same force (under Newton's second law) that pushes the matter within this shell inward past the final resisting force and into a black hole.
All of this conspires to mean that only a fraction of the stellar mass will actually end up in the black hole. Far more will end up in the planetary nebula. But it all together and you get the usual figure of about 7 stellar masses as the maximum mass of a black hole created by a single star.
Don't assume that just because you were never bored, ON YOUR HONEYMOON I might add!, that nobody else is never bored either, or never has other reasons to remain connected. Besides the obvious down time in lines, at meals, etc., there's the fact that some people have older relatives near death, younger relatives near childbirth, etc. You can't put your life on hold, and carrying celphones everywhere is not always an option.
As for the kiosk abuse, that's completely irrelevant when you're talking about people using their own wireless devices. Think anyone is going to leave an expensive laptop or PDA lying around? If they keep it with them, then it's easy to identify the person responsible for the images.
Besides meteors and aurora, some people claim to hear sounds from a facility in New Mexico (IIRC).
But the really scary thing, if you're a conspiracy nut, is the HAARP facility in Alaska. Huge power generators designed to manipulate the ionosphere... and do Mind Control on the US population.
I think it's total bullshit. President Bush is a man of outstanding moral standing and would never tolerate anything like that. We should be proud to have him as President, and Ashcroft as Attorney General. They would never do anything remotely questionable.
This could get interesting.... I have an @Home cable modem, but am totally self-hosting. I use the cable modem solely for connectivity, running my own DNS servers, getting mail via MX records at my old ISP, etc.
It should go without saying that @Home customer service has no idea what to do with me. They don't have a mechanism to support a "foreign" email address for customers who neither need nor want the "@Home experience." (It could be worse - USWorst DSL service now requires you agree to the incomparable "MSN experience.") Every service call is a joy because I have no clue what my username or password is, and they can't believe anyone has had an account for years yet never logged in.
Hopefully if things go *splat* I'll get a call... or can get through to them. I'll be severely annoyed if I have to go back to dialup after I finally dropped the second phone line.
Thanks, I couldn't get Joseph Campbell out of my head after discussing with a friend just how much "Phantom Menace" sucked. That's very sad, my copy of "Hero with a Thousand Faces" even has Luke Skywalker on its cover.
You're right, I was so focused on getting his first name down (which seems to annoy the Scientologists for some odd reason) that my fingers were clearly on autopilot later.
But just because we think that this file shouldn't be used for security purposes doesn't mean that some idiot won't come up with this "bright idea." Just because the spec is intended to list directories and files that a robot shouldn't index doesn't mean that someone won't write a robot that actively seeks them out.
The US "Star Wars" program was supposedly thought up by a panel of "hard" SF authors asked to come up with ways to fight the USSR.
Many people (myself included) think it's still SF, and will remain so for a very long time. It's the difference between the "big picture" thinker who comes up with a concept, and the "detailed" thinker who has to actually implement it. In concept, Star Wars is really neat. In practice, it's far too easy to introduce decoys, and by the time you can handle one decoyed MIRV your enemy has smuggled 100 warheads into the country by other means.
If you don't mind a bit of controversy, to put it mildly, you should include dianetics/Scientology.
Were Lafayette Ron Howard and Analog's Editor (Joseph?) Campbell pulling everyone's legs with a fake science and fake religion? Was it just a tax scam? Or was it a legitimate effort that went horribly wrong?
This isn't just an idle question - Scientologists have shut down web sites, even seized computer hardware and essentially destroyed it while the Federal courts did nothing, because they published religious "trade secrets" about the evil god Unix. I mean Xinu. I mean Xenu. (Hmm, makes you think....) They have flooded newsgroups with bogus posts to make it hard to find the on-topic posts. They have bought the top 40-odd places on search engines that provide "sponsored links," to make it hard for the casual browser to find critical sites. They have created "safe harbor" web browsers that protect their members from "objectionable" material.
You don't have to agree with my opinion of Scientologists to see how they're linked to many of the most controversial issues facing us on the 'net. And it all started with a science fiction writer and a magazine editor discussing psychology based on "science," not Freud.
Many years ago on comp.risks somebody actually looked at the contents of a number of robot.txt files - he wondered if they could be used as a quick index into "interesting" files. At the time, erroneous use of the file was still pretty rare... but I'm sure that was a selection effect that is no longer valid.
Bottom line: that standard may be intended for one behavior (robots don't look in these directories), but there's absolutely nothing to prevent it from being used to support other behaviors (robots look in these directories first). If you don't want information indexed, don't put the content on your site. Or at a minimum, don't provide directory indexes and use non-obvious directory names.
Slashdot Mirror
I don't want to trivialize the effort in moving hundreds of thousands of users, but IMHO they exercised extremely poor judgement in prioritizing the work.
There are two separate issues here. One is basic connectivity, the other are the bundled ISP services.
Many of us (a small fraction of their users, but more common among the Linux/Unix users) used them solely for connectivity. It's not just elitism either: when you have your own domain(s) and hosting services, you don't have much interest in these bundled services. *Especially* when we consider all ISPs a bit iffy, having been around the block a few times already - some of us have "vanity domains" precisely to avoid this sudden need to change email and web addresses etc.
Yet we spent days without access while someone was busy creating an account we will never use to replace another account we never used. Give us basic connectivity and we're happy - at worst we use the DNS from our hosting account for a few days. But no, we were left in the dark for days.
Of course, most people do use the bundled ISP accounts, but again they have alternative accounts at Hotmail, at the office, etc. Again, give them basic connectivity and DNS services and they'll be able to do a lot, even if they don't have their usual email for a few days. But no, they were left in the dark for days.
The only people this policy served were those refugees from AOL who never looked beyond their own email or web pages. I'm sure there were a few, but I would be surprised if it was more than 10%.
I believe the vast majority of people would prefer to have basic connectivity up within 24 hours, even if it delayed email and web pages for a few additional days, than to be dark for days.
They're intercepting DNS queries to the Excite DNS IP addresses and returning a static value - a record to their transitional site. The name doesn't matter - *everything* resolves to that site.
/etc/resolv.conf, /etc/dhcp.conf, /etc/bind/named.conf, etc.)
You didn't specify how you get your DNS addresses. On a Unix/Linux box, it can hide in a lot of places. (DHCP packets,
Checking my settings, I saw that I was using my last known @Home DNS addresses as the 'forwarder' addresses. When I replaced them with 0.0.0.0 (forcing a query against the root servers) I got the right addresses.
So they hijacked their old DNS server addresses (assuming they were operated by Excite), not the root DNS servers... but that would be a trivial change to make. Definitely not something that gives knowledgeable users warm fuzzies.
As I mentioned in a post which crossed in the ether, this "interesting DNS hack" kills any local DNS server, and raises some disturbing long-term questions.
FYI, my AT&T cable (Boulder) came up and everything was fine once I told my Linksys box to use DHCP instead of a static IP address, but everything went to hell after about 15-20 minutes when AT&T HIJACKED THE ROOT DNS SERVERS. Every single address, including attbi.com, resolves to their transition site. I couldn't even bring up their help page.
On the one hand, this is clearly a (feeble) attempt to communicate with their users. How many Windows users do they think are using the root DNS servers?! -- it will primarily hit the people using "unsupported" operating systems.
But this makes the broadband service unusable to those of us running our own local DNS servers precisely because of problems we've had in the past with theirs. Sure, there are workarounds (I can think of several), but in the overall picture they're more hassles to maintain than my current approach.
I couldn't get through the ATTBI number (never any complaints when you don't give the sheep a way to reach a person!), but asked the cable TV person to pass on my... annoyance but temporary acceptance of the situation... and to ask the ATTBI people to call be back with an ETA for when the root DNS servers will be restored.
I fear, deep in my cynical heart, that this is actually an attempt to force everyone to use their DNS servers so they can track our movements and ultimately hijack additional content. E.g., you ask for "www.ford.com" but get a "www.chevrolet.com" interstital. In that case the root DNS servers are never coming back... and I want to close my account as soon as possible.
At least, for now, they aren't blocking the DNS servers of other ISPs. I've still lost some important local functionality, but at least I'm able to get back up.
My Boulder modem is back up, at least temporarily. I did have to change my linksys firewall/router from a static IP address to DHCP....
Unfortunately, it also blocks all Debian users. At least it looks like somebody *finally* packaged ssh2 for woody (ironically, a few days after I last checked for the packages, from the time stamps). Even ssh3 is now listed.
Why are you using their DNS server?
Seriously, I long ago got tired of DNS servers "disappearing" because some bozo forgot (or never knew) that there were some systems set up for static IPs because their support people didn't want to deal with the odd Linux user... and without DHCP you don't get the new IP address for the name servers.
Once I had a basic DNS server running, I took immense pleasure in adding a few authoritative entries. Doubleclick? X10.com? They go straight to my web server (usually) where the browser returns a quick 404 error. Watching the status bar on the dialup line, I'm beginning to suspect that the good performance on my cable modem is as much due to local DNS server as the bandwidth. (For some reason the local server isn't working with the dialup line.)
I was told that we're scheduled to have our service back up on Thursday.
Of course, who knows how long it will take to get to noncompliant drones who aren't using Windows. We cause them too much trouble since we can't follow their predigested scripts (and I'm usually too busy to be willing to lie my way through endless Windows menus instead of taking 15 seconds to edit a text file and restart a server).
This argument assumes that the passage of time is real. It may not be. At the human scale, this is a moot point since our consciousness is predicated on the perception of the passage of time, but at the fundamental level where there's CPT conservation it may be nothing but an illusion that distracts you from the truth.
Restrictions on the time, manner and place of speech have been upheld countless times in meatspace. Your right to speak on the lawn of the courthouse does not give you the right to set up a PA system in a residential area at 3 AM.
This is partly because of the tradeoff between freedom of speech and the right to peaceful enjoyment of personal property and life.
But it's also because "freedom of speech" does not protect the physical act of speaking, it protects the right to express a dissenting view. The majority requires no explicit protection precisely because it's the majority. But the minority, especially the lone dissenter, *does*. That's why some cities have laws requiring that protestors stand in specific "boxes" when they make their speech - it's partly to prevent others from attempting to drown out their voice!
The same thing applies in cyberspace. If you have *no* moderation and attempt to discuss controversial issues, you *will* have an asshole appear who doesn't mind posting hundreds of marginally pertinent responses to drown out "objectionable" content. Just look at alt.scientology (or something like that) sometime. While it's technically true that the original messages are still there, and it's not an exact analogue of the real-world situation where the lone protester may not be heard at all, in practice few people will bother to search for meaningful content and the protester(s) will have succeeded in supressing speech.
It's ironic, but sometimes the only way to guarantee that everyone has a voice is to be willing to silence those who would use theirs as a weapon.
It's easy to implement a crypto filesystem, but hard to do it *right*.
Some quick examples:
1) Is a standard cipher used? (easy, now that libraries are widely available)
2) Is a standard cipher used *correctly*? (e.g., no ECB mode!)
3) Does the same data in two blocks encrypt to the same ciphertext? If not, how are you randomizing them? What happens if you copy an encrypted FS from one media to another, e.g., via backups?
4) How do you detect an incorrect encryption key?
There's then the whole issue of key management, the truly hard part. How do you generate the key from the password? How do you support multiple users on the encrypted file system? (N.B., this is cryptospeek for "how do you prevent disgruntled employees from encrypting your data then walking away?" This usually means secondary and even tertiary keys automatically inserted by the system.) How do you handle system reboots?
Finally there's the mundane. Top of that list - how do you handle backups? Can you back up the encrypted data? Can you deny backups of the unencrypted data?
Maybe you need deniability, but out here in the real world a lot of people should be using encrypted file systems just to ensure that sensitive or confidential information is not exposed to others if the disk is stolen, the cleaning people are bored, etc.
Personally, I don't want my doctor to have deniability about his records regarding me. Or my lawyer. Or my accountant. And most especially not my banker, financial adviser, etc.
In fact, for these people deniability makes a solution look much less attractive. People get *really* nervous when their accountant or lawyer has strong deniability about what the advice they gave you, about where your money went, etc.
You're confusing two different masses. One is the mass of the star that collapses in a type-II supernova. There's a minimal size for such stars (just as there's a minimal size for a star to be a star, instead of a large Jovian body), and IIRC there's also a maximum mass due to various physical constraints.
The other mass is what's crunched into the black hole. If I remember the numbers right, and they haven't been surplanted by more recent research, at "maximum" crunch some of the stellar mass is falling inward at a third of the speed of light, and the maximum density is something like 4 times that of a regular nucleus. This is a *very* hard surface, and anything outside of the maximum crunch will be blown outward. A lot of the matter in the crunch will be blown outward, as the "spring" releases. This is the same force (under Newton's second law) that pushes the matter within this shell inward past the final resisting force and into a black hole.
All of this conspires to mean that only a fraction of the stellar mass will actually end up in the black hole. Far more will end up in the planetary nebula. But it all together and you get the usual figure of about 7 stellar masses as the maximum mass of a black hole created by a single star.
Don't assume that just because you were never bored, ON YOUR HONEYMOON I might add!, that nobody else is never bored either, or never has other reasons to remain connected. Besides the obvious down time in lines, at meals, etc., there's the fact that some people have older relatives near death, younger relatives near childbirth, etc. You can't put your life on hold, and carrying celphones everywhere is not always an option.
As for the kiosk abuse, that's completely irrelevant when you're talking about people using their own wireless devices. Think anyone is going to leave an expensive laptop or PDA lying around? If they keep it with them, then it's easy to identify the person responsible for the images.
I fear for the future... and not because of HAARP, or even "gentlemen's C" Yale graduates.
Besides meteors and aurora, some people claim to hear sounds from a facility in New Mexico (IIRC).
But the really scary thing, if you're a conspiracy nut, is the HAARP facility in Alaska. Huge power generators designed to manipulate the ionosphere... and do Mind Control on the US population.
I think it's total bullshit. President Bush is a man of outstanding moral standing and would never tolerate anything like that. We should be proud to have him as President, and Ashcroft as Attorney General. They would never do anything remotely questionable.
This could get interesting.... I have an @Home cable modem, but am totally self-hosting. I use the cable modem solely for connectivity, running my own DNS servers, getting mail via MX records at my old ISP, etc.
It should go without saying that @Home customer service has no idea what to do with me. They don't have a mechanism to support a "foreign" email address for customers who neither need nor want the "@Home experience." (It could be worse - USWorst DSL service now requires you agree to the incomparable "MSN experience.") Every service call is a joy because I have no clue what my username or password is, and they can't believe anyone has had an account for years yet never logged in.
Hopefully if things go *splat* I'll get a call... or can get through to them. I'll be severely annoyed if I have to go back to dialup after I finally dropped the second phone line.
Thanks, I couldn't get Joseph Campbell out of my head after discussing with a friend just how much "Phantom Menace" sucked. That's very sad, my copy of "Hero with a Thousand Faces" even has Luke Skywalker on its cover.
I think that's a reference to the turbine size, 4 mm vs. 4 m diameter, not the entire physical plant.
You're right, I was so focused on getting his first name down (which seems to annoy the Scientologists for some odd reason) that my fingers were clearly on autopilot later.
Um, we aren't disagreeing. Not one bit.
But just because we think that this file shouldn't be used for security purposes doesn't mean that some idiot won't come up with this "bright idea." Just because the spec is intended to list directories and files that a robot shouldn't index doesn't mean that someone won't write a robot that actively seeks them out.
The US "Star Wars" program was supposedly thought up by a panel of "hard" SF authors asked to come up with ways to fight the USSR.
Many people (myself included) think it's still SF, and will remain so for a very long time. It's the difference between the "big picture" thinker who comes up with a concept, and the "detailed" thinker who has to actually implement it. In concept, Star Wars is really neat. In practice, it's far too easy to introduce decoys, and by the time you can handle one decoyed MIRV your enemy has smuggled 100 warheads into the country by other means.
If you don't mind a bit of controversy, to put it mildly, you should include dianetics/Scientology.
Were Lafayette Ron Howard and Analog's Editor (Joseph?) Campbell pulling everyone's legs with a fake science and fake religion? Was it just a tax scam? Or was it a legitimate effort that went horribly wrong?
This isn't just an idle question - Scientologists have shut down web sites, even seized computer hardware and essentially destroyed it while the Federal courts did nothing, because they published religious "trade secrets" about the evil god Unix. I mean Xinu. I mean Xenu. (Hmm, makes you think....) They have flooded newsgroups with bogus posts to make it hard to find the on-topic posts. They have bought the top 40-odd places on search engines that provide "sponsored links," to make it hard for the casual browser to find critical sites. They have created "safe harbor" web browsers that protect their members from "objectionable" material.
You don't have to agree with my opinion of Scientologists to see how they're linked to many of the most controversial issues facing us on the 'net. And it all started with a science fiction writer and a magazine editor discussing psychology based on "science," not Freud.
Many years ago on comp.risks somebody actually looked at the contents of a number of robot.txt files - he wondered if they could be used as a quick index into "interesting" files. At the time, erroneous use of the file was still pretty rare... but I'm sure that was a selection effect that is no longer valid.
Bottom line: that standard may be intended for one behavior (robots don't look in these directories), but there's absolutely nothing to prevent it from being used to support other behaviors (robots look in these directories first). If you don't want information indexed, don't put the content on your site. Or at a minimum, don't provide directory indexes and use non-obvious directory names.