Slashdot Mirror


Archive.org Sued By Colorado Woman

An anonymous reader writes "The Internet Archive is being sued by a Colorado woman for spidering her site. Suzanne Shell posted a notice on her site saying she wasn't allowing it to be crawled. When it was, she sued for civil theft, breach of contract, and violations of the Racketeering Influence and Corrupt Organizations act and the Colorado Organized Crime Control Act. A court ruling last month granted the Internet Archive's motion to dismiss the charges, except for the breach of contract claim. If Shell prevails on that count, sites like Google will have to get online publishers to 'opt in' before they can be crawled, radically changing the nature of Web search."

37 of 797 comments (clear)

  1. Posted notice? by Aladrin · · Score: 5, Insightful

    Did she post the notice properly, though? There are laws about how you must post a 'no trespassing' sign on physical properties.

    Similarly, there are ways to post that notice on your website as well. robots.txt comes to mind. If she didn't bother to post the notice correctly, the case should be just thrown out.

    --
    "If you make people think they're thinking, they'll love you; But if you really make them think, they'll hate you." - DM
    1. Re:Posted notice? by ZakuSage · · Score: 4, Funny

      Don't read this post.

    2. Re:Posted notice? by spellraiser · · Score: 5, Informative

      No, she didn't post the notice properly:

      Her suit asserts that the Internet Archive's programmatic visitation of her site constitutes acceptance of her terms, despite the obvious inability of a Web crawler to understand those terms and the absence of a robots.txt file to warn crawlers away.

      The case should be thrown out, period. She should just have learned her lesson and used a proper robots.txt file next time. If you're going to post stuff on the Internet and don't want it to beb indexed or archived, you should know what you're doing. If you don't, it's your problem. The lawsuit is frivolous and inane.

      --
      I hear there's rumors on the Slashdots
    3. Re:Posted notice? by gEvil+(beta) · · Score: 4, Funny

      OK

      --
      This guy's the limit!
    4. Re:Posted notice? by BJH · · Score: 5, Interesting
      Gotta love it:

      IF YOU COPY OR DISTRIBUTE ANYTHING ON THIS WEB SITE, YOU ARE ENTERING INTO A CONTRACT. SEE COPYRIGHT NOTICE & SECURITY AGREEMENT (READ BEFORE ACCESSING THIS WEBSITE)


      This notice is posted (where else?) on her web site...
    5. Re:Posted notice? by Anonymous Coward · · Score: 5, Funny

      Ol ernqvat guvf lbh nterr gb abg nggrzcg nal qrpelcgvba bs guvf cbfg.

    6. Re:Posted notice? by jrockway · · Score: 5, Insightful

      A case should be thrown out even if robots.txt was ignored. What if robots.txt contains a parse error or was temporarily inaccessible?

      If you want something published on the public internet to be private, require viewers to enter a password or present a cryptographic certificate. Everything else is public.

      --
      My other car is first.
    7. Re:Posted notice? by dgatwood · · Score: 5, Interesting

      Worse, when you click the policy link, this nutjob asks you (without swowing the policy) whether you agree. Then, if you click cancel, it tells you that by even looking at this site, you have already agreed, and the only option is "Ok", after which it takes you to the page where you can actually see what you just agreed to.

      As such, even if contracts were binding upon spiders (which they should not be), this is not a legitimate contract because it is not possible to read the contract prior to agreeing to it. IMHO, all the archive.org people should have to do is videotape someone clicking the policy link and show it to the judge, and she will be thrown out of court on her you-know-what.

      .
      --

      Check out my sci-fi/humor trilogy at PatriotsBooks.

    8. Re:Posted notice? by jrockway · · Score: 5, Interesting

      The case isn't about copying, it's about the act of spidering.

      Also, if copying is illegal, what about copying the website to your browser cache when you display the page? Let's ban that; that will be GREAT for the web.

      --
      My other car is first.
    9. Re:Posted notice? by Original+Replica · · Score: 4, Insightful

      If you want something published on the public internet to be private,

      Then don't post it on the internet. Just like if you don't want people to see you in your underwear you should stand in your front yard in just your underwear holding a sign saying "Don't look at me". The space might be private, but the view is public.

      --
      We are all just people.
    10. Re:Posted notice? by Anonymous Coward · · Score: 5, Informative

      The inaccurate Slashdot description would make you believe that, but actually she has this on her webpage: "IF YOU COPY OR DISTRIBUTE ANYTHING ON THIS WEB SITE, YOU ARE ENTERING INTO A CONTRACT".

    11. Re:Posted notice? by sabernet · · Score: 5, Insightful

      Archive.org is archiving websites. All credit to the author, etc... remain. Equally, Archive.org is not profiting from the mirrored copy. Upon request, archive.org can and most probably will remove the copy from their site.

      As has been stated before, you save a copy of all websites you visit. It's called cache. Archive.org could be seen as a computer browser with a robotic viewer and a bigass cache folder.

      Equally, the internet works on the premise that the information is -out there-. If the site existed only in a local intranet, then that would be a different story. This site was published to be seen and, by extension, copied, as so long as the rights of the content provider are upheld.

      If you don't want your site copied, DON'T PUT IT ON THE INTERNET. Or, at the very least, put some level of protection on it. A human-only readable notice does not protection make. Otherwise, you are defaulting to a "copy me" state, as that is the default state of http. She has the right, however, as the copyright holder, to ask that all copies be purged.

      To use a real world analogue. Say you go golfing. While golfing, you hear "FORE!" but keep playing. A golf ball smacks you in the head and you get a concussion, the extent of which prohibits you from going to work that week. Can you sue the golf course or the golfer(assuming he wasn't maliciously attempting to zing you across the skull)? No. You did not have a reasonable expectation not to be around flying white harbingers of pain. If you go golfing at a golf course, any reasonable person knows that there will be other golfers. So you accept the possibility and the consequences.

      When you publish to the internet, your site will be copied. If someone you don't like does it too, you have no right to bitch about it. You published it out to be seen by the world. It's out there now. Too bad. However, as the copyright holder, you reserve the right to approach one of the copiers and ask, "I wish for you to delete that." The copier may have fair use defenses for it or may end up deleting it out of respect. Equally, as the copyright holder, you can definitely say, "That copy you have, you can't make money off it or take credit for it or deface it in any way or hide its source." Archive.org is not doing anything of the sort.

      This, of course, is ignoring the fact that this woman more then likely has used a search engine at least once in her life. As a result, should she attest that these activities are illegal, it means she willingly participated in a query of a database doing that which she believes is wrong. That is not only hypocritical, but downright vile and questionably as illegal as that which she is alleging.

      I'm not quite sure if you were playing devil's advocate or are actually an RIAA-worthy ignorant copyright Nazi, but either way, you definitely didn't put much thought into your post.

    12. Re:Posted notice? by nuzak · · Score: 4, Insightful

      I can claim that if you speak to any words containing vowels to me that you're entering into a contract. It doesn't make it the slightest bit so.

      Either she has the world's dumbest lawyer (RICO charges? please), or she's filing pro se (see previous about world's dumbest lawyer). In either case she's an ideal test case for the webcrawlers, because she will almost certainly get completely demolished in court.

      She might have a copyright claim, but she couldn't even get that one up the steps. Did she even file for infringement?

      --
      Done with slashdot, done with nerds, getting a life.
    13. Re:Posted notice? by binford2k · · Score: 5, Informative
    14. Re:Posted notice? by jlarocco · · Score: 5, Insightful

      No, that's exactly what it means. Especially in the day of publishing on the web, where when you decide to stop pulishing, it's gone. If you publish a book and sell it, whoever bought the book can come back to it over and over. If you remove your webpage it's gone -- unless some asshat corporation (non-profit or otherwise) comes along and decides to republish your content without your permission. Archive.org, Google, etc. are massive copyright violators. It's not for you to dictate to me the method I have to use to tell you not to violate my rights. Being indexed should be opt-in. Just like being spammed. Being employed. Being in a relationship. Being called by telemarketers.

      Oops! Looks like somebody doesn't understand the internet.

      Robots.txt is the way to block web spiders from your site. That's not somebody "dictating your rights", that just the way it fucking works. It dictates your rights the same way a steering wheel dictates the way you have to steer your car. You don't have to use it, but your solution probably won't work, and you'll look like a moron. When you have a blank or non-existant robots.txt, it's understood by billions of people on the internet that you don't mind if web spiders crawl your site and add it to their index, make cached copies, etc. That's the way it was designed, and that's the way its worked from the very beginning. It's not rocket science.

      Also, every person who visits your site gets a complete copy of the pages they visit in their browser cache. Once your page is cached in my browser, I have that information forever. I can delete it, view it, save it to CD, make a PDF, etc. Just like the person who owns a book that's no longer published. There's not some magic "delete fairy" who goes around deleting everyone's browser cache when you decide to delete a page.

      Maybe not everyone knows about their browser cache or robots.txt, but that doesn't mean they don't exist. You can't change the way the internet works because a bunch of morons failed to do even the most basic research before throwing their crap on the web.

    15. Re:Posted notice? by MillionthMonkey · · Score: 5, Informative
      robots.txt is a nice convention, but its absence doesn't allow anyone to break copyright. Where does that idea come from??

      It isn't just a "nice convention". It's a sufficiently reasonable precaution available to plaintiff to effectively avoid the inadvertent disclosure of copyrighted documents. Failure to provide a simple robots.txt file evidences a lack of reasonable precaution and undermines plaintiff's claims to redress in a court of law.

      In her defense it seems she probably needs the money after being fined $6000 in a Colorado state court a few months ago for a contempt violation (unauthorized practice of law) after she participated in three separate Colorado court cases under a power of attorney when she had no prior involvement- after having been warned on a prior occasion that this was illegal in the state of Colorado. In fact it's illegal everywhere except Slashdot. But of course it's lies, all lies!

      I did not give legal advice to anyone. I did not use powers of attorney to represent anyone or give legal advice to anyone. As I testified, and which was not refuted, I used Powers of Attorney because I was previously accused of forging signatures on release of information forms. I started using powers of attorney to access the DHS records under an existing Colorado Children's Code statutory provision recognizing powers of attorney for that purpose, legislation which I influenced.
      She needs a good spanking.
    16. Re:Posted notice? by dgatwood · · Score: 4, Informative

      ROTFL. I didn't even see that part. Since it was buried below text talking about copying/printing, the assumption is that it is a continuation of that content, but it really isn't. More on why this text is also an illegal contract a little later.

      But first, the obvious flaws: the content formatting is so unreadable that it's easy to miss. And then, there are the spelling and grammatical errors in their "license" notice.

      For example, "The content if this website is intended to generate income, it is not free if you intend to archive, copy, print or distribute anything electronically fixed herein." Let's see. Run-on sentence, missing serial comma, misspelled a two-letter word.... Could someone explain to me why someone incapable of writing a very simple English sentence without tons of very basic 2nd grader mistakes is trying to make money on the internet?

      And then, there's the fact that this woman lacks a basic fundamental understanding of computers. "Permission and limited, non-exclusive license to reproduce this web site, by any method including but not limited to magnetically, digitally, electronically or hard copy, may be purchased for $5,000 (five thousand dollars) per printed hard copy page per copy, in advance of printing." Where to begin.... Magnetic, digital, and electronic reproduction do not involve printing! Oh, yeah, and missing a serial comma, an extra comma after "this web site", I think it needs a comma after "by any method", but I'd have to see what Strunk & White say on the subject. And "per printed hard copy page per copy" is redundant. A printed hard copy page is a single copy by definition.

      But my favorite part is this: not only is there no robots.txt (still), but also nowhere in the page source are there any meta tag to indicate that the document should not be cached, so by viewing the page, you are committing an act which the license claims would require payment of $5,000, and by viewing it through AOL, you also cause AOL to commit an act (caching) which the license claims would require payment of $5,000. And by viewing it through AOL using the Google cache, you cause both companies to owe $5,000. Just in case you think that she might have been smart enough (yeah, right) to set it in the headers, here are the headers for the web page:

      HTTP/1.1 200 OK
      Date: Sat, 17 Mar 2007 19:25:33 GMT
      Server: Apache
      Last-Modified: Fri, 16 Mar 2007 16:15:52 GMT
      ETag: "10d27b-f53b-45fac2b8"
      Accept-Ranges: bytes
      Content-Length: 62779
      Content-Type: text/html

      Notice anything missing? Like a Cache-Control directive?

      Here's a hint: this woman is a nearly computer-illiterate neophyte who posted tons of content online without any real understanding of how the internet works, and now is pissed off because of her own carelessness. Ignorance of the way the net operates is not a defense, folks.

      There. I've copied a portion of the site contents. She'll probably sue me, too. I'm glad we have SLAPP laws here....

      --

      Check out my sci-fi/humor trilogy at PatriotsBooks.

  2. If she didn't want it crawled.. by Red+Cape · · Score: 4, Informative

    she should have put a password on it.

  3. robots.txt by knothead99 · · Score: 5, Informative

    TFA clearly states that there was no robots.txt file. I suppose she expects us (programmers) to rush out and perfect natural language processing so all our spiders can read her stupid notice.

  4. The site in question? by John3 · · Score: 5, Interesting

    This appears to be it.

    Oh, and Ms. Shell, 1996 called. They want their website design back.

    PS - By clicking on the link above you are agreeing to all the stuff Ms. Shell posted on her site.

    --
    "We make our world significant by the courage of our questions and by the depth of our answers." Carl Sagan
    1. Re:The site in question? by Blue+Stone · · Score: 4, Interesting

      Interesting. From her copyright page:

      "Copyright 1996- 2007, Suzanne Shell
      The content if this web site is intended to generate income [...]"

      But from her contact us page, her PGP signature:

      "Here is our public key for encrypting messages to us.

      -----BEGIN PGP PUBLIC KEY BLOCK-----
      Version: PGP 8.1 - not licensed for commercial use: www.pgp.com [snip]"

      --
      Corporation, n. An ingenious device for obtaining individual profit without individual responsibility. - Ambrose Bierce
  5. A bit about Suzanne Shell by shark72 · · Score: 5, Informative
    She's quite a a firecracker. From http://www.westword.com/2005-02-10/news/beyond-con tempt/:

    She's been ejected from courtrooms by judges and attacked in a hallway by a convicted child molester she was trying to capture on film. She's been arrested in Wisconsin for refusing to turn over her video equipment to a police officer and detained at the Colorado Springs Airport because she forgot to remove a .380-caliber pistol from her carry-on items.
    --
    Sitting in my day care, the art is decopainted.
  6. World is bigger than the US... by Splab · · Score: 4, Insightful

    ' If Shell prevails on that count, sites like Google will have to get online publishers to 'opt in' before they can be crawled, radically changing the nature of Web search."'

    No that would only change the nature of the web for US citizens, just like the online internet gamble thingie, the rest of us will shrug and move about our businesses as usual.

  7. Court dismissed most charges by Sabotage · · Score: 5, Informative

    It appears her site is at http://www.profane-justice.org/

    Check out this article here: http://www.phillipsnizer.com/library/cases/lib_cas e456.cfm

    According to this, she requested that the site be removed from the Archive in December, 2005, and they complied. They're actually countersuing her. They moved to have her claims dropped for various reasons, but the court chose to only drop the ones related to conversion, civil theft and the RICO claims. The issue of breach of contract and copyright infringement still apply.

    I think it's absolutely ridiculous that this can go forward, especially when there are two established methods to stop the Archive's activity: The opt-out, which will remove history, and robots.txt (which she didn't use and appears to still not use), which will prevent that spider from ever archiving her site again.

    Her site shows up in Google, I wonder why she hasn't sued them? Could it be that she likes the exposure of the big search engine, but doesn't want any history of her site archived by the Internet Archive?

  8. I wonder if she has an answering machine. by SharpFang · · Score: 4, Interesting

    Call it, then replay a message including:

    "If you don't agree, say 'NO', otherwise we consider you agree to our contract."
    [short silence]
    "Thank you. You have agreed to our terms and conditions. We will send you the invoice in..."

    (and proceed.)

    Since agreement to her conditions is supposed to be implied by a visit of a second party, human or not, agreement from side of the second party, human or not, containing correct reply (or correct lack of thereof - implied consent like in case of her license) is just as valid legally.

    --
    45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
  9. Much more about Suzannne Shell (fun to read!) by Diordna · · Score: 5, Interesting
    Oh, don't worry, there's more on page 2 (emphasis mine):

    In 1974, when Suzanne Shell was seventeen years old, her father punched her in the face...despite being a straight-A student, a cheerleader and a member of the marching band at her Minnesota high school, she was labeled an "out-of-control teenager" and placed in a foster home for her entire senior year...At seventeen, she also gave birth to a daughter that she gave up for adoption.
    I find it interesting how information is used selectively here. She is cast as the victim in the second and third paragraphs, with the standard foster home sob story. She was supposedly a wonderful person, but then we find, buried in parentheses a paragraph later, the bolded text above. Hmm, pregnant...cheerleader...

    It also looks like her own kids reported her for child abuse and went to live with their father, and she's pretty ticked about that.

    She wonders why she's disliked by the court system. Well, the evidence is all over the article:
    • "Every phone call that goes in or out of my house is recorded."
    • She's with the "pro-spanking, home-schooling, families-first forces."
    • "Shell urges her readers not to cooperate with the child-protection system at any level."
    • "If you see a child whom you suspect of being neglected or abused, you probably shouldn't report it...since the kid could be victimized further in foster care."
    • "Tape everything. If it looks like the brutes might try to remove your precious dumplings from the home, hide them. If they snatch Junior anyway, plant a bug in his teddy bear so you can monitor what goes on in the foster home."

    And look how effective she is!

    "I don't think I've ever had a case where she's been involved where we have not ended up terminating parental rights," says Rocco Meconi, Shell's nemesis in Fremont County.
    Heck, this belongs on Fark not Slashdot.
  10. A more relevant link by MillionthMonkey · · Score: 5, Funny

    Check out her current robots.txt file.

  11. Re:Maybe I'm new here... by Blakey+Rat · · Score: 5, Interesting

    As a devil's advocate, though, what enforcement is there of robots.txt?

    I could easily write a program that runs on my workstation and completely ignores it. In fact, I have a offline-browser that downloads sites and *does* completely ignore it while spidering for which pages to download (I won't name names.) There's nothing technical requiring spiders to honor it, presumably there's no legal system to honor it, it's all just trust.

    Next time this comes up in court, the case might be a much more interesting, "I had a robots.txt file, but [search engine] ignored it!"

  12. Anyone notice.... by Shaltenn · · Score: 5, Interesting

    Quoted:

    IF YOU COPY OR DISTRIBUTE ANYTHING ON THIS WEB SITE, YOU ARE ENTERING INTO A CONTRACT. SEE COPYRIGHT NOTICE & SECURITY AGREEMENT (READ BEFORE ACCESSING THIS WEBSITE) - Copyright 1996- 2007, Suzanne Shell and individual contributors where appropriate. The content if this web site is intended to generate income, it is not free if you intend to archive, copy, print or distribute anything electronically fixed herein.

    Which of course implies this is a commercial website.

    However, if you go to the contact info page, we can see her PGP key (...) contains:

    -----BEGIN PGP PUBLIC KEY BLOCK-----

    Version: PGP 8.1 - not licensed for commercial use: www.pgp.com



    GG?

    --
    If you were offended by anything I said... No, I'm not sorry. Please lighten up.
  13. Problemm not isolated by failedlogic · · Score: 4, Interesting

    Lets forget for a second the person suing Internet Archive. Lets also forget the quality of her website and that she may or may not have included a proper robots.txt file.

    Suppose you're a teenager or college student. You do some silly prank and it includes a picture of yourself and your friends. You tell your friend you don't want the picture going on the net. For whatever reason, he/she posts it up anyways. Nothing illegal. Or suppose the names of a doctor's patients were to leak onto the Internet. Or that you're a company and an ex-employee posts information that is defamous or incorrect about your company. Or that you post an angry rant on the net, you've grown past it and then delete from your site. In all cases, wouldn't some or all this information be cached on Google or Internet Archive?

    My point is we're all human. Wether our activities are simpy stupid, embarassing, or could affect a company's ability to do business, some or all of us have said or done some of these things. It could affect our relations and our ability to find work. The problem with the Net is its very easy to find this information and it becomes widespread. We've all embarassed ourselves, at the least. We can sue the person responsible for posting the information. But it is, as I understand, very difficult to have the information removed from caches on search engine and archive sites.

    Rather than sue, sue, sue, shouldn't there be an easier way to remove this information?

  14. Suzanne Shell - Think of The Children!!!11!1 by Anonymous Coward · · Score: 4, Informative

    Suzanne Shell is a family rights activists who has published 3 books and a website advocating on behalf of the fundamental human right of family association. This activism largely applies to government intrusions into the family under child abuse/neglect investigations and court actions. She has been active in this arena since 1991 and is nationally recognized as an expert in this arena.

    She is advertised by: http://www.profane-justice.org/

    An example of her work: http://www.profane-justice.org/sctcomplnt.pdf

    She urges interested parties to contact her via her contact address or phone number on the letterhead in above PDF.

    Let her know what you think!

  15. mitigating circumstances: she's pro-child abuse by NIckGorton · · Score: 4, Interesting

    I can see why she wouldn't want to get indexed. She's basically providing information for people trying to resist intervention from child protective services. She suggests taking your children and hiding them out of state if CPS tries to contact you. She also suggests (and sells an ebook about) teaching your children to never talk to a social worker or answer questions... you know like 'does anyone ever hit you?' and 'what did you have to eat yesterday?' and 'do you have a doctor to go to when you are sick?'

    Of course CPS and child abuse reporting in general is imperfect. I'm an ER physician which makes me a mandated reporter. This means if I simply suspect abuse or neglect, I am required by law to report it. The system is set up to favor false positives over false negatives. That means that CPS investigates many cases for every real abusive situation they find.

    This is NOT a bad thing. There are 1500-2000 reported deaths from child abuse every year in the US. One in twenty kids in the US is physically abused in any given year and most of those victims are under age 6. Most of these cases of abuse never come to the attention of authorities because child abuse is woefully under-reported (which is why we have mandated reporter laws.)

    Most of the instances of 'think of the children' posturing are conservative unthinking crap. However advocating outdated ideas of parental property rights over the rights of children to not be abused goes beyond simple whack-job well into the realm of pure evil.

    Nick

  16. SITE SLASHDOTTED by Anonymous Coward · · Score: 5, Informative

    Suzanne Shell
    14053 Eastonville Rd.
    Elbert, CO 80106
    719.749.2971

    For those looking to share your views, Suzanne has asked that we continue to contact her organization at her official "non-web" addresses.

  17. Disclaimer: IANAL / Use it against them. by Xenographic · · Score: 5, Funny

    By reading this post or having it copied into your browser's cache, you agree to give me a permanent, irrevocable, fully paid-up license, at my option, to revoke, amend, terminate, or waive any breach of any contract of adhesion, EULA, click-through license, or any other such license or contract which does not bear my physical (i.e. the non-electronic kind made with a pen) signature or the physical signature of a designated agent you have entered with me, or to take that action on your behalf. In the event of any disagreement pertaining to, arising under, or relating to this license, you give me the choice of law and venue and agree not to oppose any changes of venue, motions for removal to or from federal court, or objections to standing that I request and that you give me the right to waive any objections on your behalf. In the event of any breech, you agree to pay all of my court costs and attorney's fees.

    If you do not agree to this, you must get my signature in pen and ink stating that whatever license or contract you propose is not subject to this agreement. After all, if you don't think this is valid, just why the HELL do you think some stupid thing you put on your web page is!?

    -----

    There, put that bugger on your website, and let them weasel out of that :] Honestly, I hope this'll get thrown out soon, the judge just might not be able to do that at this stage in the litigation. I mean, hell, how long as SCO vs. IBM been going on now?

  18. Re:Why not change the crawlers by bradkittenbrink · · Score: 4, Insightful

    While I understand the importance of web searches, would it be so hard to change the nature of web crawlers so that they didn't spider a site unless they were specifically allowed?

    Yes, it would be hard. However the technical problems would certainly be solvable for a company like google. The real problem is that this would serve to further entrench google's monopoly on search. I love google as much as the next guy, but I certainly don't want to start introducing measures that will make it more difficult for a google-killer to arise.

    You know, like you don't generally walk into somebody's house or cube at work and start rummaging through their stuff just because the door is open.

    That's the worst analogy I've ever heard that didn't fulfill Godwin's Law. The web was designed to be a public place for the sharing of information. Trying to think of it like the inside of someone's house is just going to lead you in the wrong direction. A much better analogy would be that information posted on the web is like a campaign poster or some other notice posted on someone's lawn. Yes, it's hosted on private property and the information posted there belongs to the person posting it, but it is displayed publicly and acting as if it weren't is just silly.
  19. Re:Why not change the crawlers by zotz · · Score: 4, Insightful

    "While I understand the importance of web searches, would it be so hard to change the nature of web crawlers so that they didn't spider a site unless they were specifically allowed?"

    I guess you could, you could just as easily put terms in the licenses of web servers that state that by using the web server software, you agree to let all your documents be crawled except where you deny that with .... wait for it .... robots.txt.

    That wouldn't be too hard either would it?

    all the best,

    drew

    http://www.youtube.com/watch?v=vcaf2ThG7q4
    UFO seen in skies over Winton!

    --
    FreeMusicPush If you want to see more Free Music made, listen to Free
  20. It's OK by djtack · · Score: 5, Informative

    It's OK now - after negotiating with Ms. Shell she agreed to place the contents of her site into the public domain. Here's a copy of our contract:

    GET / HTTP/1.1
    User-Agent: By accepting this HTTP GET request you agree to release into the public domain the entire contents of this web site.
    Host: www.profane-justice.org
    Pragma: no-cache
    Accept: */*

    HTTP/1.1 200 OK
    Date: Sun, 18 Mar 2007 02:11:09 GMT
    Server: Apache
    Expires: Thu, 19 Nov 1981 08:52:00 GMT
    Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
    Pragma: no-cache
    Connection: close
    Transfer-Encoding: chunked
    Content-Type: text/html