Slashdot Mirror
Over 100,000 GitHub Repos Have Leaked API or Cryptographic Keys (zdnet.com)
A scan of billions of files from 13 percent of all GitHub public repositories over a period of six months has revealed that over 100,000 repos have leaked API tokens and cryptographic keys, with thousands of new repositories leaking new secrets on a daily basis. From a report: The scan was the object of academic research carried out by a team from the North Carolina State University (NCSU), and the study's results have been shared with GitHub, which acted on the findings to accelerate its work on a new security feature called Token Scanning, currently in beta. The NCSU study is the most comprehensive and in-depth GitHub scan to date and exceeds any previous research of its kind. NCSU academics scanned GitHub accounts for a period of nearly six months, between October 31, 2017, and April 20, 2018, and looked for text strings formatted like API tokens and cryptographic keys.
I wonder if GitHub could offer a service where if an API key, be it PGP, SSH, or others, it would automatically disable that item on the relevant repository. This wouldn't stop the best of the best, but it would at least be some remedial security... far better than none.
Say a desktop or mobile application distributed as free software in source code form acts as a client for some Internet service. How is the application's developer supposed to distribute the required API key to the user's machine without exposing it in the source code? Or is each user of the application supposed to apply for API keys for his or her own copy of the application?
(See also my previous thoughts on the API key matter)
Gee, if only there was some quick and easy way to migrate from GitHub to Sourceforge... Oh wait!
API key is by design a form of authentication so distributing it to everyone with source code or even binaries makes no sense at all.
API key should be requested by the client after other form of authentication or entered manually on the client after obtaining it in secure way from API provider.
As the author replaces real keys with keyboard mashed keys
It could be the researchers made a mistake in their regular expression that is picking up something that looks like keys but aren't keys.
If this does happen to you it's because you aren't doing code review. If you are solo, then give yourself a quick review by doing a "git add" on each individual file before committing. That gives you a chance to double-check, and you can even do a "git diff" on each file before committing to be extra sure. There are lots of processes you can use to avoid this kind of mistake.
"First they came for the slanderers and i said nothing."
How do they differentiate bogus keys from real keys? In my projects I deliberately include keys that are valid, but won't get you into anything but 'local' applications running with no sensitive data. There are plenty of valid reasons (integration tests, clone-and-run dev applications, etc) to have 'valid' but practically useless keys in github.
that every time microsoft gets ownership of something a few weeks or months later some bad shit like this happens, makes me wonder if a lot of this sort of thing is an inside job,
Politics is Treachery, Religion is Brainwashing
Thanks to webscale your keys will be lost in the crowd, so you can continue with your dog-shit practices, secure in the knowledge that even if they find yours, they probably won't have time to use them.
Keep on developing your Rube Goldberg machines with Twitter, Facebook, IFTTT, Zappier, Slack, Alexa skills...
Fuck security. Ain't nobody got time for that!
From what I gather from the article is that people put the key in their code. It is not that Github did anything unsafe.
I assume what it could do is block sites that put the key in their code somehow. Obviously if I post the key somewhere else (e.g. here) there would not be a lot that could be done. It also means that code from Github can not be called safe to use, as you never know if somebody has not added malicious code to something that was safe before.
If I am somewhere mistaken, please do not hesitate to correct me.
Don't fight for your country, if your country does not fight for you.
So, it looks like they more-or-less did a regex for things that looked like keys.
How did they know they were "real" keys? If I check in some integration tests, they're going to need a key.....and no one should use that key in anything other than a local integration test. Nor would they expect to since it's in "test" folder only used to build and run tests.
Or you might check in a key to provide an "example" mode with all sorts of warnings about "change this key before production", similar to how many web services will out-of-the-box use http instead of https. Not ideal, but not necessarily an issue.
The problem then comes when a service requires that users be 13 to use the service but 18 to register as a developer in order to obtain an API key. It means 13 to 17 year olds are required to either use proprietary software or not use the service.
Whipslash and editor crew: We know you do that for burying posts to game the tree structure of replies here, nothing more. Do you think we don't see and know that?
I guess my question is that are these keys in question in source modules or just configuration files. If they are in configuration files, how do they not know these are just test keys that will then get changed to production values.
If you're talking about something like ssh, you distribute the public half of the key and not the private half. If you're talking about something like https, you get a cert from one of the official places, and don't distribute it at all (you could make your own cert and distribute the public half, but it's more painful). If you're using a key for user authentication, each user is going to need to generate their own key and you aren't distributing anything.
There are valid reasons to check in a private key (integration tests, "dev mode"), and it can be made clear that those keys are not for production use (documentation, it's in the "test" folder and only used by the testing engine, only listen on localhost if it's the checked-in key, etc)
How many of these are actually filler?
Do you care if your public GitHub repository is full of fake SSH private keys, 'pass1234', 'changeme' or 'replace-this-fake-value-in-your-code'? No, you do not.
The article mentions that the basis for guessing that a credential is likely valid is that it is for a single user account instead of a multiple user account.
That is baseless. Attempt to use the credentials. Scan for obvious fakes. There is no reasonable basis to classify these passwords and keys except testing.
If you work with Java then you know that 'changeme' is the default (and often mandatory) password on a Java PKCS12 keystore. Some applications cannot even deal with a different credential - it's hard coded into the software.
But it is sad that almost one in ten of the developers put out something, now permanently in the archives and exports, then tried to redact it. You'd hope that software developers smart enough to use git would know that the more embarrassing it is, the longer it stays on the Internet.
What kind of weirdo edge case are you making up? First you have no clue about basic creeential management, then you go off on some weird tangent about teens? Are you a pedophile?
What kind of weirdo edge case are you making up?
An edge case that has occurred in my own circle of friends. I have relatives who joined Twitter before age 18.
Are you a pedophile?
No.
If you're using a key for user authentication, each user is going to need to generate their own key and you aren't distributing anything.
I'm talking about OAuth, version 1 or 2. The client ID and client secret in OAuth authenticate an application to a service so that the application can receive a session ID representing the user.
Say a million users install an application. Do you think Twitter would appreciate a million requests to register a nearly identical application, differing only by internal timestamps and compiler optimization flags?
And those are going to need to be generated per-app. Otherwise you aren't authenticating anything.
But what's an "app"? Is it the executable program built from a particular repository, or a particular installation thereof?
The featured article is light on details on the patterns used to determine whether a string is "in the format of particular API tokens or cryptographic keys." GitHub's page about "token scanning" likewise doesn't say much. A link deeper in the article to "git secrets" by Amazon gives regular expressions for Amazon API credentials but not those of other well-known providers. The actual regular expressions used are buried in Table III of a PDF linked near the end of the article.
Fortunately, ZDNet is not paywalled.
Take a second to think about this.
If every single install of a program, anywhere on the planet, uses exactly the same identity, how do you know who to let in and who to keep out?
You wouldn't. Which is why you don't give everyone the same identity just because they're running the same executable.
Suppose I write an Android app that uses Google Maps & requires an API key. I dutifully follow Google's instructions, build my app with the key in Strings.xml, compile it, sign it, and publish it to Google Play.
What, exactly, is there to stop someone from obtaining my app through Google Play, ripping it from their phone, deodexing the binary, extracting my API key, then writing THEIR OWN Maps-using app that uses my API key and distributing it to a million users in China (or anywhere else in the world) so the API calls of ITS users end up getting charged against MY account?
I mean, I certainly hope that Google at least does binary-analysis and would reject an app uploaded to Play that attempted to re-use MY API key... but that still does nothing to address API key security in apps NOT distributed via Google Play.
Likewise, Google could theoretically do some form of sanity-checking, and pass along BOTH my API key AND an ID tied to that specific user via Google Play Services, and reject the key's use if the user had never actually installed my app... but that wouldn't help with something like Amazon API keys.
I could see it being secure if API keys offloaded responsibility for registration and payment directly to end users, so that apps would be distributed with dummy keys that were replaced by "real" ones after end users directly authenticated THEMSELVES to Google Maps, Amazon AWS, etc., and used MY app with THEIR key at THEIR OWN expense... but I just don't see how the current way API keys are used with things like Android apps is in any way even REMOTELY secure against key-hijacking.
This is a MAJOR reason why I've always been afraid to publish any Android app that depends upon having an AWS key. At least Google will allow you to distribute a Maps-API-using app with a key that simply gets disabled if you go over the free limit. Amazon won't. They'll give you a certain free allowance, and it might even be fairly high... but Amazon WILL NOT give you the courtesy of setting any kind of absolute circuit breaker and tell them, "if my app exceeds {free-allowance | some-absolute-limit}, just kill it dead and stop any additional charges from accruing". Even worse, Amazon's billing isn't realtime... you can set up all the safeguards you want to notify you if your usage exceeds some amount and kill your account, and even to try and kill it via scripted means... then see the amount you own CONTINUE to skyrocket AFTER you've pulled the plug due to use that happened before you suspended the service, but AFTER the most recent balance update.
Frankly, Amazon's AWS billing policies (above and beyond any concerns about the security of distributing an app with my API key compiled into it) scare me shitless, and apparently I'm far from being the only one who feels that way. I, for one, will NEVER willingly give someone a literal blank check to bill me into bankruptcy.
Hi folks,
Remember the landmark hearing between Google and Oracle?
API's are supposed to be public information for any service that wishes to be accessible. Welcome to the internet.
So, M$ takes over @github and now they start sponsored research to make API publication and keysharing via git repo's look bad? Or is my take too cynical? Their reputation is of the darkest colour, in my three decades experience of the direction of company policies and competitive practices.
And some string magic to google up cryptographic keys? In many cases these are uploaded and shared development key files or included key code to enable services via public key encryption.