Slashdot Mirror
Edward Snowden Kills Team Trump's Conspiracy Theory By Explaining How The FBI Can Quickly Comb Through Email (geekwire.com)
FBI director James Comey told Congress Sunday that the further investigation of emails related to Hillary Clinton didn't turn up anything that would cause the bureau to recommend charges against her. The FBI had reviewed over 650,000 emails under nine days. Upon hearing this, GOP presidential candidate Donald Trump and his supported started to question whether the FBI could go through all those emails in such a short period of time. We will never know for sure until the FBI explains its process to us all (which is unlikely to happen), so people turned to Edward Snowden over the weekend for answers. And Mr. Snowden didn't disappoint. From a report on GeekWire: How easy would it be to cull out the duplicate emails? Outspoken journalist Jeff Jarvis posed that question to Snowden in a tweet, and got a quick response: "Drop non-responsive To:/CC:/BCC:, hash both sets, then subtract those that match. Old laptops could do it in minutes-to-hours."
Seriously, anyone who's ever had to do de-duplication or pattern matching or anything like that could have told you how easy this is to do. It's almost like computers are good for this kind of stuff!
Rawr
I work for an organization that is heavily involved in electronic discovery processing for large corporations, law firms and the United States government.
Email threading, and duplication detection / dedupe are standard tasks that are performed on a daily basis on huge datasets. (As part of the Processing phase of the EDRM model.)
It is not at all unfeasible that the FBI could have used standard, off the shelf software to identify duplicates and generate an exception report for all 'new' emails that were not in the previously collected datasets.
https://en.wikipedia.org/wiki/...
There were no "non-dups". There were no new emails. Not too hard to grasp.
In this case we have a trove of emails . Also note what Comey said: he said that this doesn't change their decision with regards to recommending to indict Clinton or not, so that means once they hit this point all they have to do is figure out if Clinton had sent any of the remainder of the emails, which is easily accomplished with a simple search.
Badda bing, easy work.
Rawr
The point is that Trump's supporters have no solid evidence that there was not enough time to review the emails.
Their extreme view of it is that all 650,000 emails were relevant, and that therefore it should have taken 18 months * 650,000 emails / 80,000 emails = 146.5 months to review them.
The other extreme of possibilities is that the FBI filtered the emails by "To/From 'Hillary Clinton', date within period of being secretary of state, not a duplicate of any of the already reviewed emails" and the output of the filter was 0 emails.
The truth is likely to be somewhere between the two, it's also likely to be towards the very low end of the range.
This seems to be the case. Demographic polling shows him losing the vote with anyone who has more than a High School Diploma.
Rawr
the point is, if they can do it this quickly, why did take some many months the first time they investigated this?
Because the first time, they had tens of thousands of emails, none of which were duplicates of ones they already had, and all of which were sent to or from Hillary Clinton, and all of which were sent during her time in office as Secretary of State. Further, they had to investigate several different avenues for finding more emails.
This time, they have hundreds of thousands of emails, only a small percentage of which were sent to or from hillary clinton, only a small percentage of the remaining were sent while she was in office as secretary of state, only a small percentage of the remaining were not duplicates of existing emails that they had already reviewed. The result is that even though the original number was larger than the original cache they had to search, it's likely that they only had to look through a couple of hundred in the end this time.
I don't get why people are having such a hard time grasping this.
Well, considering that the Trump crowd, including a few posters here who should have the ability to actually write the code to de-dupe a bunch of fucking text files, claiming this was some impossible task that could not be completed in a few days, I think it was useful to have story reminding those poor suffering Trump-support /.ers who seemed to have a major brain fart about some pretty trivial algorithms.
The world's burning. Moped Jesus spotted on I50. Details at 11.
Presumably part of the difficulty with solving any problem the first time is figuring out a good method to use, implementing it, and testing to make sure it works properly. Once you've solved that problem, it becomes much easier to do it a second time because you know what needs to be done and people have experience doing it so it goes considerably faster. That isn't proof that they were actually capable of going through everything, but if we want to think about it logically, the outcome was always going to be the same.
First of all, if someone actually had something really damning they would have released it months ago if they had any intention of going public at all. Anything that's immediately and undeniably legally actionable gives you perfect blackmail material that can be used to control the president of the United States. No one in a position to collect that kind of information (blackmail) is going to waste that kind of opportunity. If you want to argue that someone who might have said information wants to release it to cause disarray, there's a more compelling argument that disarray is maximized if you only release the data some time after a Clinton victory.
It's therefore safe to assume that there's no silver bullet in the new data dump to start with and that it only contains more of the same, which the FBI have already said isn't going to get anyone to indict Clinton, even though they've essentially stated she's been pretty duplicitous about the whole thing. She's hardly the only corrupt person in D.C. and it's more likely than not if she were to go down, she could take a lot of other people with her on both sides of the isle. As much as the Republicans love talking about how corrupt she is, exposing it probably slits many of their own throats in the process. Elections are basically a trial in the court of public opinion anyways, so making swing voters think Hillary is guilty is effectively just as good as legally proving it.
Not really. I thought Occupy was an absurd waste of time, and I thought the whole "1%er" nonsense was simply contrived. Not that I don't want to see the wealthy made more accountable, and large corporations brought more firmly under the rule of law, but to imagine a guy like Trump, whose business history has been one of screwing over investors, using every trick in the book to evade taxes, and who is, by definition, one of the Elite, was going to bring the "1%ers" to bear was so ludicrous and laughable that I just have to imagine that most of his supporters are either complete morons or were more likely hoping he'd be so fucking awful that he'd bring the system down (which is absurd, the Founding Fathers built the system to deal with even the most terrible Presidents).
The world's burning. Moped Jesus spotted on I50. Details at 11.
Which once again begs the question why Comey broke the FBI guidelines to not insert himself in the middle of the political process, especially so close to the election. His ass should be canned for throwing all that red meat to the Trump campaign 11 days before the election as Clinton was pulling away in the polls. How do you defend yourself against innuendo from the FBI?
FOX news seemed to be getting daily updates on how Clinton was going to jail immediately after the election from an unnamed FBI source. They would report them in primetime with great fanfare and then retract them Friday morning where the viewership is much smaller with no fanfare.
The FBI is a mess right now.
That you elected a lying bitch instead of a lying asshole?
As one Republican consultant said in a Politico article, "Given a choice between crooked and crazy, the American people will always vote for crooked."
I don't think anyone can actually understand what he's saying. His supporters pick out the good bits of his word salads and declare him a genius, his opponents pick out the bad bits (not exactly a hard job), and declare him a dangerous idiot. There never really was a serious effort to put out a message or a coherent set of policies. It was just sound bites wrapped up in some sort of bizarre alpha male charisma schtick. Trump was the product of a whole lot of peoples' imaginations. Honestly, up until the last week or so, he hasn't even acted like someone who had the vaguest hope that he'd ever be president, and to wait until the last week of an election before you decide you're going to behave with some self control and dignity indicates to me that you're either a complete idiot or you never seriously wanted the job to begin with.
Trump has wasted a vast number of the GOP's resources, probably harmed a number of downticket races, enough that it's likely the Senate will either be deadlocked or at least marginally in the Democrats' hands, not to mention the damage done to the GOP's efforts in states like Florida and Arizona to reach out to minority voters. And for what? To be a hit with a demographic that the GOP has recognized for eight years now will fade in importance?
Any Republican angry at what will transpire tomorrow shouldn't blame Clinton, they should look at the fools in their own party that put one of the most unsuitable presidential candidates in modern US history in the place he's in right now.
The world's burning. Moped Jesus spotted on I50. Details at 11.
That still doesn't explain why the FBI boasted that they had 400+ agents working for many months for 33,000 emails and yet magically can go through 650k in just a few days. Either they were lying before, or they are lying now.
There's a third explanation. You don't understand how computers and basic problem solving works.
The 400+ agent review involved someone personally reading and evaluating every email.
The 650k email review involved extracting the small subset of email to/from Clinton, extracting the even smaller subset of emails not in their previous already-reviewed sample, and then reviewing those.
That may have been as simple as going through a few hundred personal emails that weren't part of the initial dump.
I stole this Sig