Using P2P To Make Gov't Documents Easy To Find
Trinition writes "Kim Zetter wrote for Wired News that "While legislators in Washington work to outlaw peer-to-peer networks, one website is turning the peer-to-peer technology back on Washington to expose its inner, secretive workings." For once, we have a concrete example to point to when citing the merits of P2P."
Wouldn't "exposing secretive inner workings" make the US government want to shut down p2p even more?
"For once, we have a concrete example to point to when citing the merits of P2P."
Maybe, but this also gives the government one more reason as to why P2P is evil and should be banned, don't you think?
If Bush ever finds this website I'll bet he has the Department of Homeland Security shut it down on the grounds that terrorists could possibly thinking about consider using it for terrorism.
"Curiouser and Curiouser" - Alice
Hmm, there's no bittorrent tracker/seed.
Does anyone have a tracker/.torrent of all the stuff? Or would be willing to host one..
My email addy? should be easy enough.
The site doesn't actually link to anything secret, it is all available to the public. What it does do is make it very easy to find, particulalry compared to getting this stuff of government websites.
The mind boggles...
By the way, isn't this type of thing the raison d'etre for Freenet - how many Freenet nodes are up these days? Any DHS visits to Freenet node operators/sites?
Galileo: "The Earth revolves around the Sun!"
Score: -1 100% Flamebait
Over here in the UK, the government uses the more reliable low-tech approach of real paper documents available from laybys to spread information about its secret inner workings.
++ Say to Elrond "Hello.".
Elrond says "No.". Elrond gives you some lunch.
I'm certain, that as soon as the first secret or confidential documents appear on the network, this will be used as pretext to apply all kind of national security and anti-terrorist laws to the network.
Then we'll see, how anonymous, secure and resilient the P2P-network really is.
As a whole, the concept is interesting, as much as watching mice baiting a cat.
" For once, we have a concrete example to point to when citing the merits of P2P."
Um...What about Bittorrent? Last time I checked it was the best way to download large files like Linux distros. Plus it makes it better to have more people downloading not worse, a big problem for huge servers with popular files. I can remember it taking FOREVER to get my first fresh Linux dostro downloaded
I am guessing this is one site that will have reason to be thankful for being ./ed.
See that long UID - that's what you get for lurking too long
If people download these documents from kazaa or some other p2p network, who is to tell if the information in these documents hasn't been tampered with ? For fun or evil...
You can get weird stories into this world this way.
Lately the government has been reclassifying documents that have been previously declassified.
I don't think they would support this and may even attempt to quash it because it would remove the controls the government has over their own information.
Oh the times we live in.
The preceding message was based on actual events. Only the names, locations and events have been changed.
You're kidding right? How about software distribution? Even though there is lots of software being distributed that shouldn't be, there is a lot of free software out there that is perfectly okay to share that way. Many people get their latest [favorite_linux_distro] ISO images this way. It's very legitimate and has been going on long enough to show it's not an exception to the rule at all.
Maybe the poster didn't think it through when he made the assertion, "For once, we have a concrete example to point to..." P2P is quite legitimate.
...in Japan.
Hmm, there arn't even any of the files being shared, at least on fast track and gnutella (and openft)
At least if there was a BT tracker, you could tell if it was up or not..
I'd start a tracker, or seed the files somewhere, if I could just GET copies of them, but I can't..
Has anyone sucessfully downloaded/mirrored the site?
My email addy? should be easy enough.
The man should try using a network that isnt broken by design. Like eMule. Then he could just post the hashes for the files.
Telling people to download kazaa and linking to kazaa.com is just criminal.
we all love google, however their search technology allows any one to find out anything about the government. one of the special searchs primarily searches US government documents. Not to mention peoples personal information can be found just as easily.
Please don't get me wrong, I love google, and use it, and I especially enjoy these types of searches
Some other comments are saying "But they will just want to ban it all the more!"
In fact, if we use P2P to broadcast all kinds of government dirty laundry, their attempts to ban p2p will look like an attempt to crack down on freedom of information.
It could very well be that free flow of information, anonymous and universally available, is a huge reason why world governments don't like p2p. Of course, the record industry's huge donations to Orrin Hatch don't hurt any either.
I say dump Cryptome onto p2p sites. Dump whatever you can. We have a loophole right now; better try and widen it while we can. We might even give pause to some of the criminals on capitol hill while we're at it.
Intolerance for ambiguity is the mark of the authoritarian personality.
KaZaA users with government documents?
Someone should raise the DEFCON level up a notch.
A computer makes it possible to do, in half an hour, tasks which were completely unnecessary to do before.
I'm downloading AGrikulturalPolicyNOCD+crakz.zip right now.
Micheal moore exposes government 'secrets' and the government will hate him even more, but the point is that although they will want to shut down his operatoion even more so, he got them where it hurts ;)
:)
the government will want to shut down the P2P ever more, but they made a fool of the government and brought on some media attention which is a bad thing for them.
I guess there is more to it, such as a moral debate, but meh i just sctached the surface
I fail to see how or why p2p is important here at all. If he put together a website with these documents on it they would be evan more accessable. Or better, perhaps torrents if he cannot host directly. Not having google to search for stuff usually doesn't speed finding documents. So neat, but I think his energies could have been better spent to get these documents out there in a more accesable way.
With the nature of P2P networks, what safeguards have been taken against "poisoning" the documents? Seems it would be too easy to take a document and modify/censor it then place it back into the network. Neither the article nor the website of the people doing this seem to address this possibility.
=Smidge=
Nearly all game demos and patches are made available through bittorrent. The game publisher saves some bandwidth and gamers don't have to sign their souls over to fileplanet.
Some may argue that Congress wouldn't consider gaming worth of protecting. But just remind Congress that gamers are a billion dollar business, and that'll pique their interest.
If someone says he and his monkey have nothing to hide, they almost certainly do.
Slashdot has/had a duplicate article in the 'Search' results.
When clicked, it said:
Nothing for you to see here. Please move along.
As opposed to officials of the previous administration, who just take the classified documents home with them and "lose" them.
What this tells me is that there aren't enough controls (chain of custody) over documents...
...at The Importance Of... - basically he makes the very sound point that this obfuscated distribution system is entirely unnecessary. All US Government documents are public domain (non-copyrighted) so any web site could put them up for static download without fear of DMCA attacks. It would make them far easier to find just by using Google. Instead "I go to the outragedmoderates.org website, go to the "Government Document Library," look up the documents I want, ignore the fact that I could download them from the website, start a P2P program, enter a search for the document name and/or outragedmoderates.org user name, and then download the documents, remembering that if I don't download the documents from outragedmoderates.org I might be getting inauthentic files."
This is exactly what so many people should be doing in the open-source and free-software communities. We need to prove that many of these tools are only considered "evil" because they take away money from corporations. They are not, by themselves, tools of the devil.
This type of idea can be applied to many more things which can encourage social reform. Not just spreading information and accessing it easily (P2P and the Internet are doing just fine), but with opening tools and software/hardware solutions into the public domain. We need to figure out a way to develop software without fear of piracy (by making it free), and which still compensates those who spend thousands of hours toiling over it.
We should apply this idea at all levels. Move out of the dark realms of piracy and software cracks, and prove that we really DO have better ideas than the current industry.
-Dave
6th Street Radio @ddombrowsky
"It took Anderson about four hours and 2,000 mouseclicks to download more than 13,000 documents related to Vice President Dick Cheney's energy task force" 2,000 clicks for 13,000+ documents?? via an html interface.. now that impresses me.. i map ~ 1 click : 1 document "Pornography, for example, had a role in pushing broadband into more homes." just giving you guys a reason to rtfa
"The more you know, the less sure you are." - Voltaire
you can't tell, same as with the "original" document that the government produced. They could start out pre-tampered. all you can do is find enough of them and compare them to look for inconsistencies. Unless you wrote it and signed it and released, you have no idea that any random government document is accurate,or is in the same form it was originally written in,you have little to no idea if anything in it is accurrate or purposeful disinformation or just busywork or a CYA effort for some reason. None of the above. Look at the way the reasons to invade iraq were presented, as "fact", based on "intel" from "multiple credible sources". Remember the pictures of the "mobile bioweapons labs" the regime was waving around that eventually were proven to be helium weather balloon "mobile labs"? That's just one example, there are probably thousands if not millions more when you think of all the projects government has been into over all these years. Pick any subject, any topic, any government agency, any year, any regime, you can probably find a lot of screwy documents that wouldn't past the honesty criteria.
The system has been broken for a long time. I have yet to meet any civilian or military government employee, willing to talk about matters off the cuff and off the record, who isn't aware of illegal or questionable shenanigans going on, and the system never gets fixed, it just gets more complex and they get better at keeping the bad stuff hidden.
I'm a skeptic, and based on decades of looking and seeing that this vague thing called "government" is just as apt to obfuscate and lie as tell the truth and be open, I am forced to assume anything they say-or release in document form, even so called "leaked" documents-should be treated with a high degree of incredulity. So the best you can do is compare it with some known data, and check multiple and diverse sources.
"free speach,"
"Your all"
"Busch"
Is it because you like Dubya's views on education?
the cloak of secrecy must now be refreshed with the blood of p2p users...
All the torrents you could want.
it's the perfect disinformation channel!
insecurity asks the wrong question irritation gives the wrong answer
"This is EXACTLY why I'm voting for Busch in the Fall."
/. poll:
:-)
Next
Your pick for President of the United States:
o Busch
o Coors
o Blatz
o Guinness
o Cowboy Molson
I'm not saying this isn't without value, but come on . . . I thought that responsible editors were supposed to make sure that such ridiculous exaggeration never make it to press.
* Practical obscurity . . . a term used by courts to indicate that documentation that is a matter of public record, but cumbersome to find (e.g. going to city hall and having to search records to find a specific document). The practical obscurity due to the the effort needed to find the documents affords some level of privacy in and of itself. Putting documents on internet reduces their "practical obscurity."
Using P2P To Make Gov't Documents Easy To Find,
Using Gov't To Make P2P Operators Hard To Find
In the academic community, there are quite a few interesting projects going on. I work on a project called LionShare, which is integrating services like authentication, authorization, and directory in to a federated P2P network.
Ok, reasons to use P2P:
Software downloads - I get all of my Linux ISOs from Gnutella and BitTorrent
Photographs - Yes, 99% of what's shared on Gnuttella in the way of images is porn. That 1% can be DAMN interesting.
Video feeds - Back when the towers fell, the Internet was slow, but usable. Major news sites were effectively dead, though. Gnutella was klunky then compared to now, but was still your best bet for getting video of what was going on.
Rare music - bands that have yet to make a name. Rare recordings from over seas that have never been for sale in the US. There are just so many GOOD things to listen to after you wade through the mainstream garbage.
P2P is a healthy, vibrant community of free speach. That means that a lot of the speach is the sort of thing you'd hear out of the average high school student, true, but that doesn't make the rare, considered speech any less valuable!
>;k
Ah, I sea buy yur speling that you too ar a produkt of the know child left behind.
For once, we have a concrete example to point to when citing the merits of P2P.
Let me offer a few others that have been around for a while:
- Distributing FLOSS. For example, Linux.
- Distributing music with the copyright holder's permission. For example, eTree.
- Distributing internally developed software to employees in a large enterprise. For example, LANDesk and Marimba use peer to peer distribution.
Stop-Prism.org: Opt Out of Surveillance
Just yesterday I downloaded Eclipse3 with bittorrent, I just wish more companies/organisations would over their large downloadables over bittorrent; saves them bandwidth, saves us time.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
For those who can't be bothered to read the article or click the link in the article, there is a website where these documents are collected. In addition to being available on the website, the documents are available via P2P.
http://www.outragedmoderates.org/
Drop me a postcard from Guantanamo, "Thad"... :)
--Rob
Towards the Singularity.
Holy crap, from what I'm reading from their website... I now have strong evidence that the president did not become coo coo, we elected an already coo coo president in.
Wait, I didn't vote, because I was too young...
Trust the old thinker to screw up.
"Hm... George Bush looks old... must make good president..."
Not if he's suffering from dementia.
P.S. Reading down the list, I don't quite mind the spoil of war, in fact, I say those money should go to the soldiers and their family. God knows what they been through over there (I don't know what, but I believe its tough, or worse). That and we won't have to foot the bill.
In US, you can easily buy enough major firearms to wipe out your neighbourhood but a few little fireworks are banned.
Lets make this open source project, and make it great... Take an existing p2p project, and fork it... Change/break compatability with all other nodes(the gov wouldn't want 1,000 terabytes of songs to be among the search results), give it a clean interface, and let it use plugins...
.gov website could use a g2p:// (government-to-people) protocol, and have IE be able to handle the download as if it were a search result in the g2p software... There are atleast 1 p2p program that already does this...
Say, a
I mean really, take the latest CVS snapshot of every major/minor p2p project(including MUTE and Freenet) copy and paste for X hours, to have every feature implemented(and the best code used for duplicate features) and then bugfix...
Want encrypted links? no problem.
Want cryptic(non-direct to peer) routing? Piece of cake.
Heck, lets try and migrate the Gutenberg project's archive to g2p, while we are at it!
LOCKSS-DOCS and even the US GPO Access have already been doing this. But I suppose that given how online government information can go poof or be altered, this project sounds like a good idea, albeit a partisan one.
I post game patches, linux distros, and crap like that on p2p. Makes my ISP work for my money.
There are plenty of legitimate uses for p2p file sharing, even if you don't count music/movies/pirated code.
ad logicam Claiming a proposition is false because it was presented as the conclusion of a fallacious argument.
This is supposed to make the politicians support P2P? We're letting people communicate quicker, more accurately, and using the cost of one FOIA request to get information to thousands or millions. It's in Washington's best interests to smash P2P, now. How about petitioning via P2P? Everyone who gets it will support it, and perhaps the thought of a million voters on P2P networks will turn the tide from banning to strict regulation.
After reading through that report I'm not entirely convinced it's as far fetched as it sounds. Yes, it's clearly a very poorly designed system, but the arguement they are making is that the system wasn't designed to be able to keep running as well as provide the overhead for copying. While they didn't give reasons for it, failed back ups could cause this sort of situation, as well as a system grossly over it's intended maximum capacity.
Think about it, Slashdot crashes systems regularly. Of COURSE they need to fix it, then readdress the requests they receive in the interim. I'll believe it's entirely conspiracy and not just stupidity when they come up with other excuses after the revamp is supposed to be in place.
Never confuse volume with power.
It would be way easy for someone to modify a document and upload it back into the system and then you have two copies of the document. There needs to be some sort of repository of MD5sum's to check against what you download.
Unlike that method of keeping our government transparent the most successful method has been our constitution. The document resides in a meuseum and copies are published in nearly all US History textbooks in our schools. That is one document though and it's far more difficult to maintain say the library of congress and keep things from being edited. Better that the library is archived where people can download copies someday that they could browse through and if anyone was concerned a change was made could consult a network of copies that individuals have maintained.
Jose Padilla?
I wanted to thank everyone for your comments, and address a couple issues.
1) BITTORRENT: Due to a number of emails regarding this, I'm dropping Kazaa and going with Bittorrent. I'll have this set up by the end of the week, possibly earlier.
2) RELIABILITY OF DOCUMENTS: Tonight I will finish synchronizing the names of documents offered via P2P with the names given on the Government Document Library page. Once that is done, if you've downloaded documents online, you'll be able to verify the documents by checking them against the PDF provided by the original source (say, the NRDC or the House Committee on Gov't Reform). The only surefire way I can confirm that you are downloading a reliable document is if you are downloading it directly from my usernames (provided on the Download For Democracy page). Also note that the filenames of all files will include the source. As I mentioned earlier, I'm working all the kinks out of this tonight.
3) ON THIS USE'S EFFECT ON P2P OVERALL: As some people here have pointed out, none of the documents on my site are truly "secret" - I'm not breaking new documents. I consider the site's job to be one of an aggregator (and yes, I use that term because of my obsession with Google News). Anyway, considering that these documents have been made available by other sources - sources that have a degree of credibility that I have not built yet - I don't anticipate that this usage could have a negative effect on P2P. I'm never going to post anything that is not from a major media outlet, a legal or academic source, or the government.
Thanks for your interest, comments, and advice, and keep checking back over the next couple of weeks - the P2P campaign will be improving in terms of the networks used, the number of documents, and the ability to verify documents.
Thad Anderson
outragedmoderates.org
Just give everybody access to Sandy Berger's pants.
Sandy Berger, Clinton's national security adviser and John Kerry advisor, illegally removed classified documents from the National Archives during the 9/11 commission investigation by stuffing them into his jacket, pants pockets, and his socks.
Maybe we need p2p to get this story out, since the mainstream media is doing a good job of burying the story.
I may not agree with how p2p is currently used by the masses but it's a great way to share / find files. If this were applied to the library or congress or even public records it would be a great education tool. That is exactly what government doesn't want.. They don't want the masses to be able to easily find public governemnt documents and educate themselves. Contray to popular belief they would rather have citizens that are as dumb as possible.
Know child's left behind? Isn't that the program used in Catholic schools?
Help save the critically endangered Blue Iguana
This made me have an idea, that someone has probably already tried. Is it possible to link a p2p network directly to a website with something similar to a hypertext link?
This would require a plug-in version of bit-torrent or kazzaa or something that people could download if they wanted to view the content on my site (kinda like the real player, except for the p2p part). The actual link would probably have to have info on the checksum to verify document integrity. Doing this would make one's website scalable for demand. This would be a great way to prove how valuable P2P networks are. Imagine companies using one old slow computer to host their entire website instead of massive servers.
Has this already been implemented somewhere? Can I do it for my site? I am not the greatest programmer, but if anyone would want to try such a thing I would help as best I could.
John
First, it's not just private parties -- the US govt owns websites too, and these are where many important documents lie. Ari Fleischer's transcript was altered to remove the embarrassing comment "Watch what you say." Documents on Iraq on whitehouse.gov were not altered but were intentionally obscured so that search engines couldn't find them easily. And the website of the National Cancer Institute was changed to suggest a connection between abortion and breast cancer, a link that the scientific community has consistently rejected. Worse, many documents are simply not being made available at all -- this has been increasing under the Bush Administration by deliberate design, but it has always been a significant problem. In other cases the documents are being made accessible only through publicly-owned databases that are not on the internet and often require a trip to DC.
Second, there have been instances of private parties doing this to appease government officials. Sometimes they are just ordered to take stuff down, as in the case of Sherman Austin's website. Other times the pressure is less obvious, but it is there, like in the case of the San Jose Mercury News website about drug trafficking started by the Gary Webb series in 1996. The website included photos, audio, video, and scanned court documents and it was really well organized (it was well ahead of it's time, and, if it were still around, I think it would still be ahead of it's time! hard to believe that was 8 years ago). The paper eventually published a rebuttal to its own series, which was also linked on the website (and every page on the website was given a link to the brief rebuttal and a note to the effect of "oops! we were wrong about all this! sorry!") Eventually the whole website was taken down. It reappeared briefly a few years later on a website funded by Webb personally and then disappeared again.
That's just the most compelling and extreme example I can think of; there have certainly been others. Another related problem is the increasing privatization of such information -- do you realize that there are whole segments of federal and state law that are not accessible at all without purchasing privately-held (yet nevertheless public) information from a single party? I'm required to obey the law, but if I want to find out what it actually says I have to pay a private party to use their database? Again, this is a trend that has accelerated dramatically over the last few years.
All of these are good reasons to consider p2p for government documents. Of course, there is risk of corruption of such documents, and such corruption might be harder to trace on p2p (especially since many people will be sharing documents they haven't read or even looked at very closely). But I think it is still a good solution to the problem of making the documents more easily accessible, and when alterations are discovered that can be publicized this way too. I'm not sure it would address the last problem I mentioned (privatization), since in those cases the private companies are legally entitled to control the flow of information (for reasons that make no sense to me). Then again, one might expect p2p to facilitate illegal distribution of such information (and the moral argument would be far more compelling than the one for infringing copyrights on music, movies, etc.)
I think what makes this effort unique is not the sharing of government information but the fact that it's being done systematically with a real index of documents that are available so that you know what the hell you're getting. I've downloaded things that look like they will be interesting docs and turn out to be some photoshopped joke. And a lot of it is hit or miss; I type in "Iraq" and get hundreds of hits, very few of them all that significant. But having a list of what is likely to be available and what the file name would be is really useful. What would be really exciting is a tool to organize many lists of such material that could be easily searched.
I sincerely hope so. It sounds as good a plan as any I have heard. We have an incredible tool with the net for freedom loving honest people around the world to use, not only for mundane personal reasons and profit in all it's various forms, but for throwing off this yoke of dependence on various questionable governments. It might take 10 or more generations, but I am fairly confident one day all humans will be as "free" as they choose to be.
Unfortunately, in between now and then will probably be some serious ugliness, and hopefully, again, we will avoid physicist michio kaku's predictions,his basic one being that most civilisations in the universe (he thinks there are "some", that we aren't alone in the universe), never make it past the discovery of uranium for very long.
the real reason for the above.s hnaziheraldtribunenewscoast.cfm.htm
http://www.btinternet.com/~nlpwessex/Documents/bu
"The president of the Florida Holocaust Museum said Saturday that George W. Bush's grandfather derived a portion of his personal fortune through his affiliation with a Nazi-controlled bank."
ie agent of a foreign gov.
see also "George Bush, the Unauthorized Biography" by Webster G. Tarpley & Anton Chaitkin chapter II
Even my home network could be described as peer to peer as I have no server for 4 client machines.
Its interesting you say that. Client / Server is really only defined at the transport layer or layer 4, and here is why :
What makes a particular device a "client" or a "server" ? Only the applications running on it, and where it matters in the context of TCP/IP is either the UDP or TCP ports the applications are using.
However, even that doesn't really work. What if you are configuring a "web server". To test it out, you fire up a web browser on the same box. Now the box is running both the web server and web client, so is it stil a "server", or is a "client", or is it both a client and a server ?
A point about why I clarified IP as being designed, but not necessarily a peer to peer protocol - NAT. NAT breaks the "equally a sender or receiver" property of IP. This property is one of the ones that have made the Internet what it is today - if you had an IP connection, and an IP address, you used to be able run up a web server, irrespective of any up stream devices. In other words, you were in complete control of the decision to make available a service to the network.
With deployment of NAT, you don't have as much, and depending on your environment, a lot less flexibility in making that "service providing" decision. Groups like RIAA and MPAA are quite happy with this, as they want a "broadcast" only style network, where home users can't deploy their own, possibly competing, services. NAT is the technology that will facilitate that.
There are a lot of other technology limitations that NAT causes, which are fundamentally side effects of violating the "equally a sender or a receiver" property of IP. Here is Keith Moore's list - Things that NATs break
The Internet's nature is peer to peer - 20050301_cs_profs.pdf
"exposing secretive inner workings" [of the US government]
Please, please! Why won't someone think of the children?? Clearly, they should not be exposed to such naughty and immoral things!
>of government dirty laundry, their attempts to
>ban p2p will look like an attempt to crack down
>on freedom of information.
Granted, it will look like that -- to those watching. But how many people, outside of /., are really paying attention to this? I can guarantee if you went out and asked your average Joe about sharing government files/info via P2P networks he'd give you a "huh" look. And if you told him he couldn't trade government files via Kazaa, he'd just ask for a quick assurance that he'd still be able to get Eminem's latest music video on Kazaa and, upon receipt of aforesaid assurance, be quite content.
The sad and simple fact is that most people neither know nor care to know.
--A witty sig proves nothing.--
Color me stupid, but is Microsoft "Windows for Workgroups" not a Peer-To-Peer networking protocol from the "Way Back Machine"?IBM on you, you, you, and UUNET XXXX Me? How hard would you like to XXXX me? Like, duh, MSN Messenger Remote Host IP Address[whois.arin.net]Done#127.0.0.1 Ding! Local Host Direct Inbox#e Yahoo! Perfect SBCIS[0] - Computer Works, The [a definite article][1] WWW The Slashdot Table of Organizer.s? WWW OLE!EXCLAMATION POINT disabled on account of dim wit#us-ascii AOL ART KNOT!!00HTML+>
Regarding point 2, reliability of documents:
You could store MD5 checksums of the file, and also store this list on the web site. Post your PGP/GPG public key to keyservers and on your web site, and you can share a signed version of the MD5 checksums which can be verified by the downloader (the file need only be plain text, and you could even have a $FILENAME.MD5 file containing the MD5 sum of $FILENAME eliminating the need to update a master list which would cause versioning problems).
Summary:
For every shared file, $FILENAME
Create a ASCII text file containing the md5sum of $FILENAME. Call this $FILENAME.MD5
Sign $FILENAME.MD5.