Slashdot Mirror
Online Search Engines Lift Cover Of Privacy
Rican writes "MSNBC has an interesting article about how 'Googledorks' are using the powerful search engine to do searches across the web for sensitive and/or private information. Some of this information includes 'Medical records, bank account numbers, students' grades, and the docking locations of 804 U.S. Navy ships, submarines and destroyers.'"
The worst example I saw was the FBI NCIC 2000 manual [PDF]. It gives you examples of how to look up criminal records and such... which could be very useful to the criminally vested social engineer.
People have used this for years to find things like Bill Gates' social security number and all kinds of things we think should be private. Chances are, if its in a record somewhere, that information will leak onto the internet sooner than most people think.
They don't seem to be, although many could. There's just too many unique ones out there IMHO.
Then again I don't have a WP that'll run those scripts.
it was an AP story- I read the same thing in this morning's washington post.
Gentlemen, you can't fight in here! This is the War Room!
.htaccess anyone?
That, along with an appropriate robots.txt file should be all you would need to prevent a crawl, right?
Overrated / Underrated : Moderation
Here's how it works. Let's say you put a page on your site called
i s/ private_document.html
/
http://yoursite.com/temporary/hidden/dontreadth
And it is not linked to ever.
I realize this is redundant, and you were likely trolling, but Google will leave you right the fuck alone, so long as you put another little file at:
http://yoursite.com/robots.txt
That contains the text:
User-agent: *
Disallow:
I realize this is opt-out rather than opt-in, but there's just one place you have to opt, and there isn't another way that Google could possibly do their job. Everybody else seems to understand that the internet is a publicly accessible network.
So who's to blame? You. You put a sensitive document in a publicly accessible location on the internet, and took no precautions to keep it secure. Not linking to it is not a precaution.
There are no trails. There are no trees out here.
1) This is old. I remember searching for things like '"index +of" vti' and other such things (try it and modify that search if you like, but it was interesting to find out just what sort of interesting tidbits one might find in such a folder).
2) This is an article from MSN. This information was available long before Google, but it is, at the very least, curious to see this sort of article from Microsoft when they have been going to the press lately about how Microsoft intends to develop their own search technology...
Other examples are ".dbx", the file name extension for mail folders in Outlook Express. Or ".pwl", the Windows 9x system password file (supposedly easily crackable with the correct tool).
There are unfortunately clueless users who share their whole hard drive. File sharing programs have however started getting better in discouraging or preventing the users from doing this.
The thing is that most people will literally inadvertantly share their entire hard drive's contents, or at least all "media files".
What I like to do is go on gnutella or kazaa and search for "DSN" or one of a number of similar prefixes. Why? Because most digital cameras save their files in a specific hardwired format, and the kind of people who leave their entire hard drive shared on kazaa are the kind of people who don't rename their digital cameras.
You can find the most random, interesting, occationally personal shit that way.
I'm trying to remember the other common prefixes besides DSN and failing.
-- Super ugly ultraman
"""
one of the central tenets of computer network security: If it is connected to the Internet, it can be accessed
"""
That's not one of the central tenets of computer network security.
If it's not connected to the internet, it cannot be accessed, but that doesn't imply what you've said.
If it's connected to the internet, and there's a daemon which answers requests with the information requested, then it
can be accessed. There's a subtle difference though - namely the daemon which answers the requests. Without that there's no access, and there can never be any access.
YAW.
Your head of state is a corrupt weasel, I hope you're happy.
robots.txt doesn't matter worth a damn, if you're not feeling polite.
The google mediapartners bot which will look at pages for the purposes of advertising such as in Opera is different and seperate from the bot that adds pages to Google's search database. The mediapartners bot does not feed the Google search engine.
Boffoonery - downloadable Comedy Benefit for Bletchley Park
Please webmasters, learn to use the proper code for preventing bots from scanning your page. The Robot meta tag will do that quite effectively. Alternately, you could just /not/ make a webpage with your usernames and passwords, and that would be a lot easier.
*****
Dear Mary,
I yearn for you tragically,
A.T. Tappman, Chaplain, U.S. Army.
Opera doesn't even send such urls to Google.
Boffoonery - downloadable Comedy Benefit for Bletchley Park
Is googledorks a real hacker movement or just some random key word any one with a high ranking web page can abuse?
It appears to be a buzzword that Johnny Long just kinda made up. I used Google to "hack" away and find his website: http://johnny.ihackstuff.com/
It appears his definition of googledorking (?) is not just finding private info, but just anything wacky/weird/different, private is just one of those things.
Do we now call it g00g|3?
> existance of that page get from Opera to Google such that it
> could pin-point (not crawl) that page?
Opera submits URLs browsed to by users, to google, when advert support is turned on.
http://www.opera.com/adsupport/
From that page:
--------
What is the connection between the Web page and the relevant ad displayed by Google?
Opera's interaction with the Google ad system:
The Opera browser sends Google the URL of the web page you are visiting and your IP address (with the exceptions Opera filters out -- see below)
--------
Exceptions are https, forms, passwords, cgi, and non-http URLs.
As an example from my apache log file last night, when I gave a friend a URL to a photo:It's surprising how many Opera users will deny this happens, despite the evidence. That's a 5 minute delay, google is pretty quick with its crawling. Personally, I don't mind. I put things up in my temporary directory and pull them down fairly soon after. I know nothing is secure if it's just an unprotected URL, so I'm not worried like the grandparent poster. However, Opera does send URLs to google, and google does come back and check them out.
You can find out whether personal information about you is available accidentally by searching for your name and a piece of your sensitive information on Google, say, your name and the last four digits of your SSN, the last four digits of a credit card number, parts of your phone number, or your street address. Leaked personal information would have to contain both your name and that other information. Chances are that you will retrieve only a few documents, which you can quickly review.
Keep in mind, however, that Google queries are not encrypted and are not guaranteed to be private or secure, so, for your search, don't use the full SSN or anything else that shouldn't be disclosed.
Boffoonery - downloadable Comedy Benefit for Bletchley Park
Not if the robots.txt file prevents you from accessing that data, which it does.
No, it does not. It provides absolutely NO access control what so ever. It simply tells the a search engine crawler "please do not catalogue these pages".
Finkployd
Just tonight I was Googling for "number personnel U.S. military" and I was surprised to find many links along the lines of "How to find U.S. military personnel." The site with the most links to directories has a Netherlands domain name, which seemed odd. I tried to find some family members and did turn up some information. Some sites were DoD and had recognizable warnings about monitoring. Another was a .com for the military community and required standard registration procedures. I don't know if it's a good idea to have this information online and I wonder what military folks think about it. I reckon there are pros & cons.
a) Mediapartners-google does check robots.txt
b) Opera always has the name "Opera" in it's UA string, even when masquerading as IE.
c) Mediapartners-google doesn't feed the Google search engine. It is only used for Google adverts.
Boffoonery - downloadable Comedy Benefit for Bletchley Park
Google does retain information on search queries in some form. If you go and check the Google Zeitgeist (Weekly Version & the Annual Version) they have statistics on most searched terms, time graphs showing, for example the spike in search queries after the California Quake, and lots of other interesting information.
For the week ending February 2, the top search terms in the US were:
Just another two cents from the Norm...
The MSNBC article fully credits the WP. What's your problem?
Web2.0: I love when people Flickr my cuil and digg my boingboing until my google is reddit and I start to yahoo
User-agent: * /Archives /Archives/bin /Archives/dev /Archives/etc /Archives/ftp /Archives/gopher /Archives/tmp /Archives/usr /cgi-bin /bin /oursite/previews
Disallow:
Disallow:
Disallow:
Disallow:
Disallow:
Disallow:
Disallow:
Disallow:
Disallow:
Disallow:
Disallow:
Omnes arx vestrum sunt adiuncta nobis.
Just wildcard it. Use robots.txt to say that /secretstuff/* should not be indexed, that still won't help the l33t hax0r determine that it's /secretstuff/toodumbtouseapassword/bush-secret-nuk e-codes.lnk.exe.pif.scr which is the hidden file to destroy the world.
Give a man a fish, he'll eat for a day, but teach a man to phish...
Maybe you should try a later version of Mozilla. You know the older ones have bugs that are fixed in later ones.
when everyone gives everything, then everyone everything will get
A lot of the personal data that is publicly accessible was not made publicly accessible by the data subject, but by a third person/party.
People say I'm crazy, I got diamonds on the soles of my shoes...
More specifically, it says "Please do not enter my house and steal my jewelery and banknotes which are in the safe in the bottom-right of the bedroom closet."
/house/closet/bottomright/safe/jewelry /house/closet/bottomright/safe/banknotes
/house/ :)
Sure, you could do
Disallow:
Disallow:
Or, if you want to be simpler, you could just do
Disallow: