Think back to ww2, the Soviet Union in the 1930's, France in the 1950-60's. The US and UK have always enjoyed total access to all other nations data by setting junk standards or allowing international crypto research standards to settle on weak standards for decades.
If a service is secure along the length of transmission, go for the encoding, decoding systems at one end TEMPEST.
If that is secure, ensure a nation picks a junk international standard that is weak by design.
Get to the OS developers, equipment makers over decades. Ensure only tame brands and tame products get international traction and marketing.
Ensure a flood of media and press about standards, the best designers and providers always reaches diplomats and governments. What they buy into from neutral nations or experts is then plain text junk.
Re "Break all ability to make payments or move money online."
Most nations have versions of Australian Transaction Reports and Analysis Centre https://en.wikipedia.org/wiki/...
Real time tracking of all banking. No paperwork needed.
The other option for all other nations is the setting of standard banking interconnect encryption is set at a tame level and the NSA and GCHQ can read it all in real time.
Re "erase cookies using the browser"
Thats really all that can be done to average users by most ad brands legally as the settings and use allow that short or long term access by default. Beyond that and it gets to be equipment interference.
Lots of apps on different OS will find the super cookies, Local SharedObject.SOL shared cross-browser tracking, flash and other deeper tracking options.
"Has this already been done and automated" Different browsers have add ons that can do that based on some level of settings.
The other option was the ISP level deal with brands to alter the users internet experience. Very hard to escape that one as it flows with the basic network.
The final option is the security services or police passing code to detect a user when a visit a site has to resolve the original ip or to classical track a browser of interest for a while.
Classically the option was for ISP backed cookies that only an ad brand could read (2008).
Later users are starting to understand more about Unique Identifier Header (UIDH) and terms like perma-cookie.
A provider using JavaScript to inject packets to show an ad. ie the provider starts altering or initiating data packets for branding, ads.
Other network systems used personalized marketing ie search terms, websites visits, time spent ie all data that a provider can log.
Re "Has this already been done and automated, say, under Linux?"
Search the Firefox add ons some listed are options like: Better Privacy, Self destructing cookies, Cookie time.
Re " Why is it so hard to find the specifics of" Its now been done at the server, isp, web 2.0 provider, social media site level.
Ads have followed the security services thinking, why be in the users machine, just become the network used for all connections.
re "Or have I not really identified the root?"
An average user is now buying generations of hardware and OS software, OS updates from an ad brand... using their ad brand search engine on their OS..
The internet in some countries will be provided by in totally by a social media company or via ad brand hardware. Collect it all.
Every packet in and out is then up for logging, over any browser. ie the classic ISP becomes the advertizing brand not just selling logs to third parties.
The only easy solution might be a new virtual machine with a modern browser and OS on fast dedicated hardware every browsing session. ie a laptop or desktop computer just for the new VM to surf the internat.
Even older movies can get cleaned, 8K scan of the original negative depending on the decade scanned in.
So the huge back catalogue of classics is just another generation of scanning and cleaning away:)
+1 for that. In your own country with the real physical hardware at your own site. Expect and understand every in/out packet will be split at a regional site in your/any country and shared with a growing number of other nations every day, all day. ie Collect it all.
Encrypt and fully understand any default or suggested settings with any hardware of software. When buying any new international hardware under your own brand or company expect it to arrive with extra code, tame/junk encryption settings and hardware trap doors.
German TV had a video in German "Gemeinsam spionieren Deutsch-amerikanische Geheimdienstfreunde" (31.03.2015) http://www.zdf.de/frontal-21/w... video.
An animation at 2:22 has packet path options for regional splitting.
Re "Because it seems the US likes technology plenty"
Playing computer games and slowly upgrading to faster broadband is not really the funded thinking production, educational side.
Re: "US R&D centers, and many of them have a lot of US production."
Mostly for branding, per state and federal tax breaks, historical, top level US security clearances, past unique gov funded educational excellence.
Re "mainstay in virtually every segment of software"
Who is sitting next to the emerging generation of US students soaking up the same educational decades? Getting US post graduate positions and learning even more per university, every year? Foreign paying or paid for students are soaking up what made the US unique for free and then returning home as smart or smarter than the very top few % of US science, math, computer graduates..
Why? The US like the propaganda aspect when they return home, some might invest or be a gateway for US products and services back home after years of exclusive, intensive US educational experiences.
The payments per seat is a nice win for the educational institution too.
What was 1950-90's inward looking US academic exceptionalism is now a world wide production line of random students equal to the best emerging US academic experts. Advanced math, science, crypto, physics, design are now just gifted away to anyone, any nation with the cash, every generation.
Other nations are repatriating that US academic excellence, merging it with their own regional low costs and winning.
Merit based US education is also a long term question given the costs of seats per year per class and pure academic top percentile over the nation and decade.
Is US standardize testing still ensuring only the very best academically get the limited places with the very best educators?
The US can bring in a lot of experts from around the world to keep cost down and have union free staff but at a point its going to need a lot of security cleared trusted locals of a very high standard.
A vast pool of US taskforces, ONDI, OICI, Dept of Energy, NSB, treasury, NRO, NSA.. will all need next gen staff from a shrinking pool of top, trusted cleared graduates.
So will the US private sector that services the above mil and gov sectors. How can the US stock its "strategic partnerships" with staff? Try NZ, UK, Australia, Canada with staff that work on US sites? Try Germany again? Virtual desk from some distant land for the US mil?
Re "Also as an aside, what's wrong with being #2 or #3 in something?"
Look back to emerging jet, nuclear, computer and rocket design and production lines in the 1930-70's. Lots of nations had their own projects but had to drop out/got forced out and all the expert work just drops away a over a generation of workers. Once a local work force from the workers, scientists, technicians, engineers drop behind the costs to catch up or even import what is needed is expensive or not an option.
A lot of complex projects demanding equal access to the same number of trusted graduates.
Thats why South Africa, Brazil, India, China, Russia worked so hard to create and keep their own national jet, rocket, nuclear, computer production lines at any cost.
The UK has its Skynet project from the 1970's - huge imports of US hardware as the UK just did not have the ability to produce or create the needed mil standard sat parts for own military communications networks and terminals. In the end it had to buy in US export grade sat systems with the US having some say in what got offered and how the UK would be allowed to use the final product. Never good to be #2 or #3 in something a nation needs to have total control over.
Yes in the long term US intelligence service aspect seems to have been pulled into very simple domestic court issues.
The GCHQ always seemed to have a deeper understanding of never going near courts or short term political requests.
Early in the 1920-30's the UK found a fast way into Soviet embassy codes, one about links deep in the UK, staff, unions, cash. It was too good to be true but the UK had to spread the results of the code work as it was just such a perfect document as a domestic political win. Codes where changed and Russia understood the needs of fast crypto vs secure crypto. Code breaking and results much more hidden from that point on.
The US seemed to have demand very public results from its very secret clandestine gathering systems vs just collect all and sort. No courts, news or press at any cost, no legal staff, drop convictions in open courts if needed. The use of the RUC Special Branch and CID are also telling in how the UK offered its most on time and vital gathered intelligence. The UK had a very clear understanding of who was doing criminal investigations and who was an intelligence service.
If the UK needed crypto help in public, police like units thats could be seen in public could be presented to the secure or public legal system. Government Technical Assistance Centre (~ a Home Office unit), National Technical Assistance Centre. That would be very court friendly for early emails and computer hard drives that where decrypted quickly and legal teams, the press could see. No unsafe convictions or later questions about the speed of decryption of advanced trusted computer encryption software:)
The US seems to have hoped that that its courts, lawyers, the press would always be unable to trace back basic domestic signals intelligence use for a few more decades?
Thats why weak junk encryption per US backed standard networks/software over decades is so problematic.
If the mil has keys, so do federal taskforces, all nations staff that worked with the US, ex staff, former staff... nations, groups that can gain insights into the methods.
eg SISMI-Telecom scandal, Italy 2006 https://en.wikipedia.org/wiki/...
re "Encryption can still prevent becoming a suspect in the first place."
The US has a way of 'updating' your standard cells telco network day to day function over the network.
"Did a Rogue NSA Operation Cause the Death of a Greek Telecom Employee?" (2015-09-29) https://theintercept.com/2015/...
"...the report concluded that someone had loaded unauthorized “corrections,” i.e. malware implants"... " secretly activate the lawful intercept’s tapping function while at the same time hiding the fact that it had been turned on"
So the nation/mil or state backed entity can just keep tracking without anyone/telco/courts ever having a log or understanding of what/who is working telco network wide.
Under "Collect it all" all users on any cell network is suspect and has been for years as designed. The keys and who gets to activate the functions is the only question.
Beyond the logs left to be found full of ip's and the 9 to 5 time zone day shift in that part of the world? Traces in the code that was used was found to be what is expected?
Any advanced nation can fake that trail as the public now understands from the recent whistleblowing news over the past few years.
Ex staff, former staff, nations that allowed access to advanced 5 eye nation sites and had smart local support staff working on site. Some other nation built their own more interesting "collect a network" system while understanding just how "collect it all" was hidden.
Methods, mid network junk encryption standards and advance hardware soon get into the hands of a lot of other nations after they have seen the 5eye network magic.
The only trick is to really make it look like another nation with all the expected methods, time, ip's, a trail back thats logged and all gov, mil, private sector contractors can agree on.
The other question is why did the US place its lists on an open server, in english without encryption for some time.
After all the years of expert digital security per mil/gov taskforce, agency, councils, executives, offices why the rush/need/demand/request to list on in plain text on an open server.
The other option is the list found was created with US disinformation with junk tracked names, SS numbers, projects, letters with project names.. mixed in with contractors and gov workers. Who would be allowed even create such a list thats human readable?
Bait, a trap the US had as a vast readable honeypot or some other nation tested their own methods.
Re: " don't realise is that the Chinese Intelligence is so secretive it doesn't even have a name"
China produces a lot of internal documents. What any one person can walk out with to a US/UK embassy is never really that useful.
The UK was reduced to rebuilding container ships with signals equipment and having cargo ships move far up rivers in China. Did the UK get much? No. Hong Kong, Little Sai Wan, Chum Hom Kok, the large Demos 1-4 NSA designed dish upgrades did not really get much going into the 1980's for the US and UK.
Most of that collection is now done from Australia after 1997. Lots of the usual, expected military grade traffic any nation pumps out everyday is collected in bulk.
Re "Someone has been lifting the personal identities of government employees"
The US and its helper nations had secured, all its projects and accounts safe from the Soviet Union and other nations well into the 1990's on vast digital databases.
Every department, agency, mil and gov digital site seemed have been kept secure or was a honeypot as bait, junk busywork projects to be tracked when found or traps.
ie at some point and for some reason the US gov selected to place its databases in an easy to read plain text format on an open network facing file structure.
The "dog and pony" is now for the securing, clean up and private sector experts to rent cyber solutions over the issue of why databases online that are not encrypted is not a great idea.
Re "what is left to protect"? Most of the US gov basic, entry level systems may start as parts from China, parts of Asia, Spain, Germany, France, the UK that are then cleaned up during a knock-down kit supply line thats sold on as Made in the USA as far as regulations go. Great for contractors and lawyers to have the paper work for.
A vast network of parts plans and designs floating around long global just in time supply chains.
The mil space/sat work is bespoke, 100% hand crafted in the USA but is limited run spy sat production lines.
The only way for the US to be secure is to totally revert to a US only supply chain again. That would be blocked by the powerful contractors and lawyers lobby who are happy to support their lucrative import fronts.
Military espionage is very easy to protect against, every nation can do it, but its more tempting for the US to buy in cheap, buy from allies ensuring their locked into US/NATO standards and the price difference between a rebranded import and US domestic mil price is a great win.
The UK thinking on "would be far less far reaching than they are now and intelligence gathering could be much more targeted and effective." has evolved from the Ireland issues of small groups of people meeting without cell or set locations in the 1970-80's.
The solution was to track small groups of people down the the cell level and then offer each member a personal option: work for the UK intelligence services or not.
Over the years people where successfully advance up the cell structure exposing all existing and new cells they had command and control over.
Thats why the UK likes to watch everything and connect everything as it cannot be sure of how, where and when people of interest will meet and if they can even be observed without sat or other very costly and advanced platforms per person.
The "without further authorization" shows not much has changed from the 1920's or the classic Home Office Warrant (HOW) days. The pipes are fast, wide, always expanded as domestic and international data grows and collecting it all over every generation of optical upgrade.
Re: simply is no sound way to keep secrets safe.
The US gov and mil and all the Western mil's did a good job over many years. Encrypted, per site, no public net access.
No great issues going back decades given the US had a great early start in advanced digital databases.
At some point all the US data was placed on a network facing the 'internet' and the data was not encrypted.
That gov/contractor need for a massive easy to read and use database was 'worth' more than a lot of secure encrypted files.
Some mil or gov group wanted to find skills fast and did not want to ask a lot of different networks or be logged asking for keys to vast different databases.
The solution seems to have been a huge plain text effort kept online. The other option would have been in the creation of many different account per person per project per contractor. A simple to use database in English open to the net would have been useful to insert entire histories as bait/traps or to totally hide complex work histories/projects.
The language issue might have been that need. Put everything in from every contractor, mil, gov and see if any have much needed languages or could be trained given past testing or education... it would not be the first time the US needed language skills quickly.
Contractors, gov, mil liked the open, easy to use system so much it just stayed open, online and readable.
Or it was all just bait, one huge trap with a percentage of names and projects total fiction for other nations to sort and wonder about.
Re Cheap or Easy?
The GCHQ showed the easy way with its Temproa system https://en.wikipedia.org/wiki/...
The UK also put a lot of funding into signals intelligence modernisation programme (SIGMOD) over the past years too.
That allows the UK to reconcile all types of data created in the UK and sent with in the UK and other interesting communications magic.
Onion routing is then just another data set to match origin to destination on a gov or mil database every day.
Does Russia face the same network issues? Russia faces constant pressure from NATO backed, created and funded NGO and many well funded Western backed color revolution groups. Rather easy to find using classical police methods given huge funding and low public support numbers.
US and UK spy networks may test any Russian walk in offer with "internet" contact then if viable offer very advanced sat or low power wireless technology that is not internet based. Safe houses, sites for low power re transmission of data.
Russia fully understands what it faces politically and from a Western spy perspective. How safe is Onion routing from any well funded gov? As safe as it is from US federal police efforts as seen in open US courts. Onion routing is now within the normal budgets of any well funded federal police support unit globally.
Pick a decade, any decade, the two-party duopoly has always been interested in collect it all.
Main Core https://en.wikipedia.org/wiki/... from the 1980's
Salon has uncovered new evidence of post-9/11 spying on Americans. Obtained documents point to a potential investigation of the White House that could rival Watergate. (Jul 23, 2008) http://www.salon.com/2008/07/2...
1960's Project MINARET https://en.wikipedia.org/wiki/...
Re 'How does that justify the huge government project?"..
AC most of the cash would have gone into mapping and working out what was a "no coverage" area and would stay as "no coverage" long term.
Was a remote area in need of coverage, having no other traditional services and lacking in the ability to attract traditional private shared telco interest.
Height, location, costs, floods, numbers of users, power options, existing telco connections in the area that could be used or extended, what providers in the area will share or have to build out from very limited local support.
All that complexity would have to be found, created as raw data and considered. Where a service is really needed and the private sector is cautious due to location or existing non cell services...
No duplication of existing or planned private sector builds, no systems with too few users. How much for power and a network connection to a new site?
Who pays for the new connection? How many providers would even be interested per site long term? A government gifted tower that looks after a few lost people and a few locals at a huge loss over many years? Or a tower that could be projected to make some profit after gov support up front?
The limitations and media event thats a for profit "public private partnership" Tax payers take all the risk, private sector gets all the profit even with fewer than expected users. Support costs given a more remote site and faults over the years of the project for a very small set of users? A huge distance to cover or some strange local hills, mountains that offer unexpected coverage over distance. Still worth the build? An overlay of mil and sensitive sites that have stingray like cell perfection but dont show as having much private sector coverage on databases. No need to create issues with a new telco cell tower.
AC re "As if money isn't an incentive. Even relatively small amounts can buy information."
If a person is going to sell information, they will want or need to spend that extra cash is strange ways.
That character or lifestyle change is one of the most easy traits for the US gov to track in todays digital world.
Facial recognition via public-private partnerships, collecting all internet use, averaging out payments and cash flow over all gov/mil staff watching for slight extra spending changes.
Colleagues out of spite, jealousy, past issues take note of the changes and start to gossip. What worked "years ago" is now just a trap. Even the classic approach by "foreigners" with accents is usually a trap based on personality profiling by domestic security teams. If not reported fully the trap worked.
Thats why most nations working with humans go for staff they can place at a low level and have work their way up for ideological or patriotic reasons. No sloppy cash trail and the level of trust is perfect over many decades. Too many good networks where lost when cash was spent in the 1950-80's.
Thats why a big long gov/mil list in plain text left to be "found" on the internet is not really much use. Too much could be bait, junk, double and triple profiles all been watched for any contact. But its great for the press and a fun read.
Re: "Ok, so they've got this super secret info from a government employee database and some medical records, maybe? And?"
Any contact attempt will be well tracked by domestic efforts like the NSB ie all fake SS numbers, work, projects, letters all set, waiting for contact.
What was really readable on open, unencrypted, networked computers by the US gov as plain text lists to be found in the wild?
So the super secret part is really in doubt or is a created mix to test networks.
How much would the OICI nuclear and energy sector labs, ODNI, NRO, Treasury really allow to float around different unencrypted databases outside their very own internal secure databases?
The data seems more like a honey pot, domestic bait and a great talking point about selling/renting more 'security' products back to the US mil and gov.
Re "embarrassing personal information is already known by the government, because you already told them."
Its not the classic negative human traits that all skilled foreign intelligence operations and espionage hope to uncover.
The US and UK now offer their staff better wages and living conditions to stop just that kind of easy temptation by all foreign intelligence operations.
The negative worked well in the 1920-60's when US/UK mil/gov wages where low and any extra cash would help a lot on average as a gift.
With digital networks any extra spending by US gov staff or 5 eye helpers is tracked, all new lifestyle changes reported and looked into. That new car, holiday, spending does not go undetected.
Foreign intelligence services never want the "embarrassing" side of staff who are just staying in gov or the mil and at a set wage or position thanks to past security work or their private sector brand as part of a larger project. People like that are stuck doing work given to them for decades. The entire project might be busy work. Not much use to any skilled foreign operation deep within the USA of they never advance and can not be trusted. The real magic is getting deep into policy formation over decades. That needs access to the best of the best from entry, up the ranks to the top. Russia has that method perfected and the US has no understanding of who to trust based on testing or long term productivity. Passive collection over decades is very hard to detect as Russia does no act classically on the information collected as it did in the past.
China just floods the West with university students and learns next to the US and UK's best university students. What they learn China understands for the low price of years of collage tuition. As smart as any western crypto, math, computer, engineer entering the US or UK mil or gov service with the same skills.
Other nations spies have moved way beyond even caring about "embarrassing personal information" from gov or mil staff who may or may not have anything useful to sell or just be bait and traps.
An easy to find US database thats in plain text, on the net... or... a complex NSB honey pot just waiting for any domestic approaches as all the names are bait?
Its not "staggering" given the access demands needed to find and upgrade contractors and skill sets needed for US global mil operations.
"Top secret " details placed in a simple, readable, network facing database in plain text.... ?
Great for finding needed no bid contractors via a cloud..
Any other US mil or gov sector would have kept its own data secure, encrypted and not on the public facing 'net'
Why was this done, when was it done and what was allowed to drift out in plain text?
Plain text letters of commendation listing secure project names in English on an open network?
What has the US really lost? Lists from staff from what wars and over what decade of contractors?
The US has entire new services with total domestic support to stop foreign intelligence operations and espionage.
Too many new services that got created over the past decade to totally replace the decades of experts that had kept all this data totally secure in the past?
Slashdot Mirror
Think back to ww2, the Soviet Union in the 1930's, France in the 1950-60's. The US and UK have always enjoyed total access to all other nations data by setting junk standards or allowing international crypto research standards to settle on weak standards for decades.
If a service is secure along the length of transmission, go for the encoding, decoding systems at one end TEMPEST.
If that is secure, ensure a nation picks a junk international standard that is weak by design.
Get to the OS developers, equipment makers over decades. Ensure only tame brands and tame products get international traction and marketing.
Ensure a flood of media and press about standards, the best designers and providers always reaches diplomats and governments. What they buy into from neutral nations or experts is then plain text junk.
Re "Break all ability to make payments or move money online."
Most nations have versions of Australian Transaction Reports and Analysis Centre
https://en.wikipedia.org/wiki/...
Real time tracking of all banking. No paperwork needed.
The other option for all other nations is the setting of standard banking interconnect encryption is set at a tame level and the NSA and GCHQ can read it all in real time.
Re "erase cookies using the browser" .SOL shared cross-browser tracking, flash and other deeper tracking options. .
Thats really all that can be done to average users by most ad brands legally as the settings and use allow that short or long term access by default. Beyond that and it gets to be equipment interference.
Lots of apps on different OS will find the super cookies, Local SharedObject
"Has this already been done and automated" Different browsers have add ons that can do that based on some level of settings.
The other option was the ISP level deal with brands to alter the users internet experience. Very hard to escape that one as it flows with the basic network.
The final option is the security services or police passing code to detect a user when a visit a site has to resolve the original ip or to classical track a browser of interest for a while.
Classically the option was for ISP backed cookies that only an ad brand could read (2008).
Later users are starting to understand more about Unique Identifier Header (UIDH) and terms like perma-cookie.
A provider using JavaScript to inject packets to show an ad. ie the provider starts altering or initiating data packets for branding, ads.
Other network systems used personalized marketing ie search terms, websites visits, time spent ie all data that a provider can log.
Re "Has this already been done and automated, say, under Linux?"
Search the Firefox add ons some listed are options like: Better Privacy, Self destructing cookies, Cookie time.
Re " Why is it so hard to find the specifics of" Its now been done at the server, isp, web 2.0 provider, social media site level.
Ads have followed the security services thinking, why be in the users machine, just become the network used for all connections.
re "Or have I not really identified the root?"
An average user is now buying generations of hardware and OS software, OS updates from an ad brand... using their ad brand search engine on their OS..
The internet in some countries will be provided by in totally by a social media company or via ad brand hardware. Collect it all.
Every packet in and out is then up for logging, over any browser. ie the classic ISP becomes the advertizing brand not just selling logs to third parties
The only easy solution might be a new virtual machine with a modern browser and OS on fast dedicated hardware every browsing session. ie a laptop or desktop computer just for the new VM to surf the internat.
Even older movies can get cleaned, 8K scan of the original negative depending on the decade scanned in. :)
So the huge back catalogue of classics is just another generation of scanning and cleaning away
+1 for that. In your own country with the real physical hardware at your own site. Expect and understand every in/out packet will be split at a regional site in your/any country and shared with a growing number of other nations every day, all day.
ie Collect it all.
Encrypt and fully understand any default or suggested settings with any hardware of software.
When buying any new international hardware under your own brand or company expect it to arrive with extra code, tame/junk encryption settings and hardware trap doors.
German TV had a video in German "Gemeinsam spionieren Deutsch-amerikanische Geheimdienstfreunde" (31.03.2015) http://www.zdf.de/frontal-21/w... video.
An animation at 2:22 has packet path options for regional splitting.
The UK had regional listening stations over generations.
Yes the US had trained and looked after generations of their top military staff. If your interesting your about as encrypted as in the US.
Re "Because it seems the US likes technology plenty"
Playing computer games and slowly upgrading to faster broadband is not really the funded thinking production, educational side.
Re: "US R&D centers, and many of them have a lot of US production."
Mostly for branding, per state and federal tax breaks, historical, top level US security clearances, past unique gov funded educational excellence.
Re "mainstay in virtually every segment of software"
Who is sitting next to the emerging generation of US students soaking up the same educational decades? Getting US post graduate positions and learning even more per university, every year? Foreign paying or paid for students are soaking up what made the US unique for free and then returning home as smart or smarter than the very top few % of US science, math, computer graduates..
Why? The US like the propaganda aspect when they return home, some might invest or be a gateway for US products and services back home after years of exclusive, intensive US educational experiences.
The payments per seat is a nice win for the educational institution too.
What was 1950-90's inward looking US academic exceptionalism is now a world wide production line of random students equal to the best emerging US academic experts. Advanced math, science, crypto, physics, design are now just gifted away to anyone, any nation with the cash, every generation.
Other nations are repatriating that US academic excellence, merging it with their own regional low costs and winning.
Merit based US education is also a long term question given the costs of seats per year per class and pure academic top percentile over the nation and decade.
Is US standardize testing still ensuring only the very best academically get the limited places with the very best educators?
The US can bring in a lot of experts from around the world to keep cost down and have union free staff but at a point its going to need a lot of security cleared trusted locals of a very high standard.
A vast pool of US taskforces, ONDI, OICI, Dept of Energy, NSB, treasury, NRO, NSA.. will all need next gen staff from a shrinking pool of top, trusted cleared graduates.
So will the US private sector that services the above mil and gov sectors. How can the US stock its "strategic partnerships" with staff? Try NZ, UK, Australia, Canada with staff that work on US sites? Try Germany again? Virtual desk from some distant land for the US mil?
Re "Also as an aside, what's wrong with being #2 or #3 in something?"
Look back to emerging jet, nuclear, computer and rocket design and production lines in the 1930-70's. Lots of nations had their own projects but had to drop out/got forced out and all the expert work just drops away a over a generation of workers. Once a local work force from the workers, scientists, technicians, engineers drop behind the costs to catch up or even import what is needed is expensive or not an option.
A lot of complex projects demanding equal access to the same number of trusted graduates.
Thats why South Africa, Brazil, India, China, Russia worked so hard to create and keep their own national jet, rocket, nuclear, computer production lines at any cost.
The UK has its Skynet project from the 1970's - huge imports of US hardware as the UK just did not have the ability to produce or create the needed mil standard sat parts for own military communications networks and terminals. In the end it had to buy in US export grade sat systems with the US having some say in what got offered and how the UK would be allowed to use the final product. Never good to be #2 or #3 in something a nation needs to have total control over.
Re "There was more separation between intelligence and law enforcement in the US prior to" 2000? 1990? 1980?
Pick a decade, any decade.
Main Core https://en.wikipedia.org/wiki/... "The data which is believed to come from the NSA, FBI, CIA, and other sources,[1] is collected and stored without warrants or court orders"
MAINWAY https://en.wikipedia.org/wiki/...
Operation CHAOS https://en.wikipedia.org/wiki/...
Project MINARET https://en.wikipedia.org/wiki/...
Project SHAMROCK https://en.wikipedia.org/wiki/...
Yes in the long term US intelligence service aspect seems to have been pulled into very simple domestic court issues. :)
The GCHQ always seemed to have a deeper understanding of never going near courts or short term political requests.
Early in the 1920-30's the UK found a fast way into Soviet embassy codes, one about links deep in the UK, staff, unions, cash. It was too good to be true but the UK had to spread the results of the code work as it was just such a perfect document as a domestic political win. Codes where changed and Russia understood the needs of fast crypto vs secure crypto. Code breaking and results much more hidden from that point on.
The US seemed to have demand very public results from its very secret clandestine gathering systems vs just collect all and sort. No courts, news or press at any cost, no legal staff, drop convictions in open courts if needed. The use of the RUC Special Branch and CID are also telling in how the UK offered its most on time and vital gathered intelligence. The UK had a very clear understanding of who was doing criminal investigations and who was an intelligence service.
If the UK needed crypto help in public, police like units thats could be seen in public could be presented to the secure or public legal system. Government Technical Assistance Centre (~ a Home Office unit), National Technical Assistance Centre. That would be very court friendly for early emails and computer hard drives that where decrypted quickly and legal teams, the press could see. No unsafe convictions or later questions about the speed of decryption of advanced trusted computer encryption software
The US seems to have hoped that that its courts, lawyers, the press would always be unable to trace back basic domestic signals intelligence use for a few more decades?
Thats why weak junk encryption per US backed standard networks/software over decades is so problematic.
If the mil has keys, so do federal taskforces, all nations staff that worked with the US, ex staff, former staff... nations, groups that can gain insights into the methods.
eg SISMI-Telecom scandal, Italy 2006
https://en.wikipedia.org/wiki/...
re "Encryption can still prevent becoming a suspect in the first place." ... " secretly activate the lawful intercept’s tapping function while at the same time hiding the fact that it had been turned on"
The US has a way of 'updating' your standard cells telco network day to day function over the network.
"Did a Rogue NSA Operation Cause the Death of a Greek Telecom Employee?" (2015-09-29)
https://theintercept.com/2015/...
"...the report concluded that someone had loaded unauthorized “corrections,” i.e. malware implants"
So the nation/mil or state backed entity can just keep tracking without anyone/telco/courts ever having a log or understanding of what/who is working telco network wide.
Under "Collect it all" all users on any cell network is suspect and has been for years as designed. The keys and who gets to activate the functions is the only question.
Beyond the logs left to be found full of ip's and the 9 to 5 time zone day shift in that part of the world? Traces in the code that was used was found to be what is expected?
Any advanced nation can fake that trail as the public now understands from the recent whistleblowing news over the past few years.
Ex staff, former staff, nations that allowed access to advanced 5 eye nation sites and had smart local support staff working on site. Some other nation built their own more interesting "collect a network" system while understanding just how "collect it all" was hidden.
Methods, mid network junk encryption standards and advance hardware soon get into the hands of a lot of other nations after they have seen the 5eye network magic.
The only trick is to really make it look like another nation with all the expected methods, time, ip's, a trail back thats logged and all gov, mil, private sector contractors can agree on.
The other question is why did the US place its lists on an open server, in english without encryption for some time.
After all the years of expert digital security per mil/gov taskforce, agency, councils, executives, offices why the rush/need/demand/request to list on in plain text on an open server.
The other option is the list found was created with US disinformation with junk tracked names, SS numbers, projects, letters with project names.. mixed in with contractors and gov workers. Who would be allowed even create such a list thats human readable?
Bait, a trap the US had as a vast readable honeypot or some other nation tested their own methods.
Re: " don't realise is that the Chinese Intelligence is so secretive it doesn't even have a name"
China produces a lot of internal documents. What any one person can walk out with to a US/UK embassy is never really that useful.
The UK was reduced to rebuilding container ships with signals equipment and having cargo ships move far up rivers in China. Did the UK get much? No. Hong Kong, Little Sai Wan, Chum Hom Kok, the large Demos 1-4 NSA designed dish upgrades did not really get much going into the 1980's for the US and UK.
Most of that collection is now done from Australia after 1997. Lots of the usual, expected military grade traffic any nation pumps out everyday is collected in bulk.
Re "Someone has been lifting the personal identities of government employees"
The US and its helper nations had secured, all its projects and accounts safe from the Soviet Union and other nations well into the 1990's on vast digital databases.
Every department, agency, mil and gov digital site seemed have been kept secure or was a honeypot as bait, junk busywork projects to be tracked when found or traps.
ie at some point and for some reason the US gov selected to place its databases in an easy to read plain text format on an open network facing file structure.
The "dog and pony" is now for the securing, clean up and private sector experts to rent cyber solutions over the issue of why databases online that are not encrypted is not a great idea.
Re "what is left to protect"? Most of the US gov basic, entry level systems may start as parts from China, parts of Asia, Spain, Germany, France, the UK that are then cleaned up during a knock-down kit supply line thats sold on as Made in the USA as far as regulations go. Great for contractors and lawyers to have the paper work for.
A vast network of parts plans and designs floating around long global just in time supply chains.
The mil space/sat work is bespoke, 100% hand crafted in the USA but is limited run spy sat production lines.
The only way for the US to be secure is to totally revert to a US only supply chain again. That would be blocked by the powerful contractors and lawyers lobby who are happy to support their lucrative import fronts.
Military espionage is very easy to protect against, every nation can do it, but its more tempting for the US to buy in cheap, buy from allies ensuring their locked into US/NATO standards and the price difference between a rebranded import and US domestic mil price is a great win.
The UK thinking on "would be far less far reaching than they are now and intelligence gathering could be much more targeted and effective." has evolved from the Ireland issues of small groups of people meeting without cell or set locations in the 1970-80's.
The solution was to track small groups of people down the the cell level and then offer each member a personal option: work for the UK intelligence services or not.
Over the years people where successfully advance up the cell structure exposing all existing and new cells they had command and control over.
Thats why the UK likes to watch everything and connect everything as it cannot be sure of how, where and when people of interest will meet and if they can even be observed without sat or other very costly and advanced platforms per person.
The "without further authorization" shows not much has changed from the 1920's or the classic Home Office Warrant (HOW) days. The pipes are fast, wide, always expanded as domestic and international data grows and collecting it all over every generation of optical upgrade.
Re: simply is no sound way to keep secrets safe.
The US gov and mil and all the Western mil's did a good job over many years. Encrypted, per site, no public net access.
No great issues going back decades given the US had a great early start in advanced digital databases.
At some point all the US data was placed on a network facing the 'internet' and the data was not encrypted.
That gov/contractor need for a massive easy to read and use database was 'worth' more than a lot of secure encrypted files.
Some mil or gov group wanted to find skills fast and did not want to ask a lot of different networks or be logged asking for keys to vast different databases.
The solution seems to have been a huge plain text effort kept online. The other option would have been in the creation of many different account per person per project per contractor. A simple to use database in English open to the net would have been useful to insert entire histories as bait/traps or to totally hide complex work histories/projects.
The language issue might have been that need. Put everything in from every contractor, mil, gov and see if any have much needed languages or could be trained given past testing or education... it would not be the first time the US needed language skills quickly.
Contractors, gov, mil liked the open, easy to use system so much it just stayed open, online and readable.
Or it was all just bait, one huge trap with a percentage of names and projects total fiction for other nations to sort and wonder about.
Very good point, just drop connections, suspected users accounts are logged and see if the traffic stops per user :)
Re Cheap or Easy?
The GCHQ showed the easy way with its Temproa system https://en.wikipedia.org/wiki/...
The UK also put a lot of funding into signals intelligence modernisation programme (SIGMOD) over the past years too.
That allows the UK to reconcile all types of data created in the UK and sent with in the UK and other interesting communications magic.
Onion routing is then just another data set to match origin to destination on a gov or mil database every day.
Does Russia face the same network issues? Russia faces constant pressure from NATO backed, created and funded NGO and many well funded Western backed color revolution groups. Rather easy to find using classical police methods given huge funding and low public support numbers.
US and UK spy networks may test any Russian walk in offer with "internet" contact then if viable offer very advanced sat or low power wireless technology that is not internet based. Safe houses, sites for low power re transmission of data.
Russia fully understands what it faces politically and from a Western spy perspective. How safe is Onion routing from any well funded gov? As safe as it is from US federal police efforts as seen in open US courts. Onion routing is now within the normal budgets of any well funded federal police support unit globally.
Pick a decade, any decade, the two-party duopoly has always been interested in collect it all.
Main Core https://en.wikipedia.org/wiki/... from the 1980's
Salon has uncovered new evidence of post-9/11 spying on Americans. Obtained documents point to a potential investigation of the White House that could rival Watergate. (Jul 23, 2008)
http://www.salon.com/2008/07/2...
1960's Project MINARET https://en.wikipedia.org/wiki/...
Re 'How does that justify the huge government project?"..
AC most of the cash would have gone into mapping and working out what was a "no coverage" area and would stay as "no coverage" long term.
Was a remote area in need of coverage, having no other traditional services and lacking in the ability to attract traditional private shared telco interest.
Height, location, costs, floods, numbers of users, power options, existing telco connections in the area that could be used or extended, what providers in the area will share or have to build out from very limited local support.
All that complexity would have to be found, created as raw data and considered. Where a service is really needed and the private sector is cautious due to location or existing non cell services...
No duplication of existing or planned private sector builds, no systems with too few users. How much for power and a network connection to a new site?
Who pays for the new connection? How many providers would even be interested per site long term? A government gifted tower that looks after a few lost people and a few locals at a huge loss over many years?
Or a tower that could be projected to make some profit after gov support up front?
The limitations and media event thats a for profit "public private partnership" Tax payers take all the risk, private sector gets all the profit even with fewer than expected users. Support costs given a more remote site and faults over the years of the project for a very small set of users? A huge distance to cover or some strange local hills, mountains that offer unexpected coverage over distance. Still worth the build? An overlay of mil and sensitive sites that have stingray like cell perfection but dont show as having much private sector coverage on databases. No need to create issues with a new telco cell tower.
AC re "As if money isn't an incentive. Even relatively small amounts can buy information."
If a person is going to sell information, they will want or need to spend that extra cash is strange ways.
That character or lifestyle change is one of the most easy traits for the US gov to track in todays digital world.
Facial recognition via public-private partnerships, collecting all internet use, averaging out payments and cash flow over all gov/mil staff watching for slight extra spending changes.
Colleagues out of spite, jealousy, past issues take note of the changes and start to gossip. What worked "years ago" is now just a trap. Even the classic approach by "foreigners" with accents is usually a trap based on personality profiling by domestic security teams. If not reported fully the trap worked.
Thats why most nations working with humans go for staff they can place at a low level and have work their way up for ideological or patriotic reasons. No sloppy cash trail and the level of trust is perfect over many decades. Too many good networks where lost when cash was spent in the 1950-80's.
Thats why a big long gov/mil list in plain text left to be "found" on the internet is not really much use. Too much could be bait, junk, double and triple profiles all been watched for any contact. But its great for the press and a fun read.
Re: "Ok, so they've got this super secret info from a government employee database and some medical records, maybe? And?"
Any contact attempt will be well tracked by domestic efforts like the NSB ie all fake SS numbers, work, projects, letters all set, waiting for contact.
What was really readable on open, unencrypted, networked computers by the US gov as plain text lists to be found in the wild?
So the super secret part is really in doubt or is a created mix to test networks.
How much would the OICI nuclear and energy sector labs, ODNI, NRO, Treasury really allow to float around different unencrypted databases outside their very own internal secure databases?
The data seems more like a honey pot, domestic bait and a great talking point about selling/renting more 'security' products back to the US mil and gov.
Re "embarrassing personal information is already known by the government, because you already told them." ... a complex NSB honey pot just waiting for any domestic approaches as all the names are bait?
Its not the classic negative human traits that all skilled foreign intelligence operations and espionage hope to uncover.
The US and UK now offer their staff better wages and living conditions to stop just that kind of easy temptation by all foreign intelligence operations.
The negative worked well in the 1920-60's when US/UK mil/gov wages where low and any extra cash would help a lot on average as a gift.
With digital networks any extra spending by US gov staff or 5 eye helpers is tracked, all new lifestyle changes reported and looked into. That new car, holiday, spending does not go undetected.
Foreign intelligence services never want the "embarrassing" side of staff who are just staying in gov or the mil and at a set wage or position thanks to past security work or their private sector brand as part of a larger project. People like that are stuck doing work given to them for decades. The entire project might be busy work. Not much use to any skilled foreign operation deep within the USA of they never advance and can not be trusted. The real magic is getting deep into policy formation over decades. That needs access to the best of the best from entry, up the ranks to the top. Russia has that method perfected and the US has no understanding of who to trust based on testing or long term productivity. Passive collection over decades is very hard to detect as Russia does no act classically on the information collected as it did in the past.
China just floods the West with university students and learns next to the US and UK's best university students. What they learn China understands for the low price of years of collage tuition. As smart as any western crypto, math, computer, engineer entering the US or UK mil or gov service with the same skills.
Other nations spies have moved way beyond even caring about "embarrassing personal information" from gov or mil staff who may or may not have anything useful to sell or just be bait and traps.
An easy to find US database thats in plain text, on the net... or
Its not "staggering" given the access demands needed to find and upgrade contractors and skill sets needed for US global mil operations.
"Top secret " details placed in a simple, readable, network facing database in plain text.... ?
Great for finding needed no bid contractors via a cloud..
Any other US mil or gov sector would have kept its own data secure, encrypted and not on the public facing 'net'
Why was this done, when was it done and what was allowed to drift out in plain text?
Plain text letters of commendation listing secure project names in English on an open network?
What has the US really lost? Lists from staff from what wars and over what decade of contractors?
The US has entire new services with total domestic support to stop foreign intelligence operations and espionage.
Too many new services that got created over the past decade to totally replace the decades of experts that had kept all this data totally secure in the past?