Slashdot Mirror
Is The DOJ Using Obsolete Software To Subvert FOIA Requests? (theguardian.com)
"A new lawsuit alleges that the U.S. Department of Justice intentionally conducts inadequate searches of its records using a decades-old computer system when queried by citizens looking for records that should be available to the public," reports The Guardian. Slashdot reader Bruce66423 writes:
An MIT PhD student has filed a suit in Federal court alleging that the use of a 21-year-old, IBM green screen controlled search software to search the Department of Justice databases...constitutes a deliberate failure to provide the data that should be being produced.
Ryan Shapiro's lawsuit alleges "failure by design," saying that the Justice Department records are inadequately indexed -- and that they fail to search the full text of their records when responding to requests "When few or no records are returned, Shapiro said, the FBI effectively responds 'sorry, we tried' without making use of the much more sophisticated search tools at the disposal of internal requestors." The FBI has a $425 million software system to handle FOIA requests, but refuses to use it, saying that would be "needlessly duplicative...and wasteful of Bureau resources."
Ryan Shapiro's lawsuit alleges "failure by design," saying that the Justice Department records are inadequately indexed -- and that they fail to search the full text of their records when responding to requests "When few or no records are returned, Shapiro said, the FBI effectively responds 'sorry, we tried' without making use of the much more sophisticated search tools at the disposal of internal requestors." The FBI has a $425 million software system to handle FOIA requests, but refuses to use it, saying that would be "needlessly duplicative...and wasteful of Bureau resources."
When the Federal government is invloved, don't blame on intentional malice that which can be explained by consequences of Republican budget cuts.
No. This is wrong. They already have the tools to do better. They simply refuse to use them. If they had to buy some new software, you'd have a point. They don't, so you don't.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Millennials.......old age.
Political flamebait works both ways. The other side of the coin is that Democrats set up overly complicated systems that can't work without an ever-increasing price tag, then complain (loudly) that they just aren't getting the support they need.
Let's move on.
When the Federal government is [involved], don't blame on intentional malice that which can be explained by...
...anything else.
Bureaucracy in general is a breeding ground for unintentional malice. There are literally thousands of people in the federal government with the ability to influence programs like this, and they often have conflicting priorities. Some are mostly concerned about the economic cost, thinking that a strong economy is the clearest path to "general welfare". Others want social support services, being of the opinion that minimizing hardship makes everyone's lives better. Some think that government should do as little as possible, allowing individuals to decide for themselves how to pursue happiness, while still others believe that a life led according to religious principles leads to a better eternity.
Those are only a few examples, and not terribly nuanced, either. People can have multiple opinions, conflicting opinions, and even different opinions for different subjects derived from the same principles. The representative government reflects the opinions of the people, and in a country of over 300 million people, it is perfectly reasonable to have a very complicated set of opinions in government.
The most that we as individuals can hope for is that occasionally, enough people agree on an issue that they'll do something matching one of our strong opinions.
You do not have a moral or legal right to do absolutely anything you want.
More recently, the FBI’s own investigation into the September 11 attacks found that “[o]n September 11, 2001, the Bureau’s information technology was inadequate to support its counterterrorism mission”, noting further that “[t]he FBI’s legacy investigative information system, the Automated Case Support (ACS), was not very effective in identifying information or supporting investigations”.
kinda bad when even you are on record as saying the system sucks.
Anons need not reply. Questions end with a question mark.
Having previously worked for a large corporation, I found that the costs are always limited to the ticketed price. There are others including employees to run and manage the system; process change; new audits; infrastructure; politics; expertise; training; software limitations leading to longer execution time.
Sometimes saving money leads to spending more money and creating more headaches. Not always, but you do need to understand the nature of the environment you are dealing with.
In our particular case we had a lab where we believed we would be doing good for the company with the $200/month network connection vs the $1000/month connection. Boy were we wrong. The $1000/month connection would have saved so much trouble with internal resources, even if the $200/month connection would have worked just as well.
Jumpstart the tartan drive.
Especially if the web interface was essentially a vt100 interface wrapped in a REST service.
Jumpstart the tartan drive.
The DoJ is one of many departments these days that are run for the benefit of the of the administration, not to serve justice or even the American people.
Alcohol, Tobacco and Firearms should be the name of a store, not a government agency.
You're kidding right??? They have plenty of money to conduct their obscenely detailed internal searches... This has nothing to do with the two fully inbred political parties. It has everything to do with the protection of the hegemony of the State.
Dear Microlimp: I give you 2 valid product keys for win7 and you reject both of them. Piss off you wankers!!!
By requesting a search that should have netted his own earlier FOIA requests, and didn't, Ryan Shapiro was able to demonstrate the inadequacy of the index-term search the FBI is using. Clever.
Political flamebait works both ways. The other side of the coin is that Democrats set up overly complicated systems that can't work without an ever-increasing price tag, then complain (loudly) that they just aren't getting the support they need.
The problem is that that assertion doesn't line up with reality. Go down to your DMV some time, and observe the kinds of systems that they're using. They're using databases built in the 80s and 90s on top of DOS, running on ancient computers with CRT monitors (at least around here).
What reasonable business do you know of that hasn't upgraded their systems since that time to allow for more efficiency savings, faster processing, reduced staff costs etc?
There's a lack of investment in this kind of system, plain and simple, being disguised as "government efficiency" by the republicans.
Frequent posts from The Guardian and BBC cover important events in the US that local media fail to report. This should make us wonder why American news media aren't on top of these stories.
Of course tight budgets constrain many traditional news outlets and restrict the ability to really investigate anything. The de-funding of Public Broadcasting was a disaster in American history, forcing a dependence on advertising and fund raising.
But those of us who entertain conspiracy theories may suspect that the 'free press' in the US is heavily influenced by various pressures from government and advertisers. For instance, many media are now forbidden at Trump rallies because they have offended The Donald by asking serious questions. Some media are unwelcome at White House briefings. Your local city/state politicians also have preferred, cooperative, outlets for their announcements. Cooperation with big advertisers is also important for American media to survive financially. Evil Monsanto stories go on page 3 or nowhere at all.
Most US publishers share with their readers the political posturing of government officials and the promotional 'news' of advertisers but fail to investigate anything. The remainder of US news is crime, weather, celebrities, a smattering of drama about terrorist activity, and no mention of large parts of the world like Latin America.
So, thanks Guardian and BBC, for a fresh look at the world and my own country.
...omphaloskepsis often...
The allegation is that only searches of an incomplete index are ever performed for FOIA purposes, and such searches are (1) archaic and unusual nowadays, (2) rarely find the requested material. That's more likely misfeasance than innocent.
The DOJ is an agency that ought to find compliance with law of primary interest, and arguments of 'needlessly duplicative' ring false. The plaintiff's test was amusing: he asked DOJ to find his previous FOIA requests, and was told there was no record...
It's a nice simple argument to make to a judge as well:
"Your Honor, if you approved a subpoena for records and the response was 'We searched and found nothing responsive,' would you accept that response if you knew that the search consisted of nothing but looking at a list of filenames? After all, that's a search - a very poor one, but a search nonetheless."
fencepost
just a little off
The problem is that that assertion doesn't line up with reality. Go down to your DMV some time, and observe the kinds of systems that they're using. They're using databases built in the 80s and 90s on top of DOS, running on ancient computers with CRT monitors (at least around here).
...And is that a problem? Does the thickness of the monitor really impact how legibly they can print your drivers' license?
What reasonable business do you know of that hasn't upgraded their systems since that time to allow for more efficiency savings, faster processing, reduced staff costs etc?
As one example, in the mid-2000s, I worked at a company whose main computer was built in 1988, with only minor upgrades (disk capacity, and a modem that was occasionally plugged in so it could be maintained remotely) since its construction. It had survived the obsolescence of its product line, the rise of DOS and Windows, and had only a minor stumble for Y2K. For a system whose primary purpose was tracking orders moving through departments, and tracking employees' time cards, it did the job perfectly well. That particular company was in the top 10% of the industry by order volume and profits, so it seems to have done just fine by most standards of "reasonable".
There's a lack of investment in this kind of system, plain and simple, being disguised as "government efficiency" by the republicans.
Again, to show the other perspective, there is grossly excessive spending in other kinds of systems, being disguised as "upgrades" by the Democrats.
I'm not promoting any particular political party here. Rather, my point is to illustrate that every partisan criticism in this thread has an equally-valid counterpoint that is too-often glossed over. When the Republicans shout about "spending", the Democrats shout "obsolescence". Nobody ever seems to want "get what's useful and nothing more", or "review the cost/benefit analysis for every component in the system".
I've worked for the federal government before, notably on one particular system whose lifespan was about 20 years. The system was designed and built to be state-of-the-art, using top-of-the-line COTS hardware available at the time (as a cost-saving measure, naturally). Ten years into the system life, those original components were obsolete, and being replaced with new top-of-the-line hardware, with the promises you mentioned: efficiency savings, faster processing, reduced costs, et cetera.
However, the basic workflow hadn't changed at all, and the software hadn't been rewritten (as that'd be prohibitively expensive), but only ported up to newer technologies. Even though each part of the process was indeed faster, the system as a whole hadn't changed significantly. It could run perfectly fine on modern (for the day) mid-grade or even low-end hardware, but because "upgrades" were seen as desirable, the system continued to be built with top-of-the-line parts, for about triple the cost.
Towards the end of the project lifespan, there was an effort to re-engineer it using minimal hardware, but by that point the idea had grown into something of a legend. The managers (and bureaucrats) who had seen the system's early versions and knew its original cost couldn't believe the system could actually run on such a low hardware budget. Every actual test was successful, but the mantra that "you get what you pay for" had become such an integral part of common sense that actually getting approval for a cost-efficient system was impossible. Eventually, my team ended up inflating our quoted costs to get approval, then delivering a working system under budget and getting extra praise.
That tale doesn't meet my idea of "reasonable", but it was definitely the reality that I saw.
You do not have a moral or legal right to do absolutely anything you want.