Slashdot Mirror
The US Government and Open Standards: a Tale of Personal Woe (thevarguy.com)
An anonymous reader writes: This article details a Linux user's struggles to submit a grant application when the process requires finicky, proprietary software. It also covers familiar ground made timely by the upcoming elections: the U.S. should prefer open source software and open standards over proprietary alternatives. The grant application required a PDF created by Adobe Acrobat — software Adobe no longer supports for Linux. Once the document was created, attempting to submit it while using Ubuntu fails silently. (On Windows 7, it worked immediately.) The reader argues, "By requiring Acrobat the government gives preference to a particular software vendor, assuring that thousands of people who otherwise would not choose to use Adobe software are forced to install it. Worse, endorsing a proprietary, narrowly supported technology for government data poses the risk that public information could become inaccessible if the vendor decides to stop supporting the software. Last but not least, there are privacy and fairness issues at stake. Acrobat is a totally closed-source program, which means we have to take Adobe's word for it that nothing sketchy is going on in its code. ... It would seem to be in the interest of the public for the government to prefer an open source solution, since it is much harder to hide nefarious features inside code that can be publicly inspected."
since it is much harder to hide nefarious features inside code that can be publicly inspected
Not THAT crap again.
Heartbleed should put that right to bed.
I don't understand your point here. It was found and then fixed in a few days, and the patches were widely released to anyone willing to update. The system worked exactly like it was supposed to: the fact that a single critical bug garned that much attention should give you an idea of how uncommon it is.
In contrast, Adobe Reader has had not one, not two, but 26 different cripplingly severe vulnerabilities in the last six months alone, and that's only because I got tired of counting after #26. How many people patch Adobe Reader? Would you like to compare Libreoffice to Microsoft Word, FreeBSD to Windows, or Internet Explorer to Firefox? Maybe Apache to IIS, or perhaps OpenJDK to Sun java? Amarok to Itunes? Our very own Adobe Reader to Okular or Evince?
Open source software does indeed have a demonstrably better security record than closed source software, that is undeniable. Further more, even if it didn't, it wouldn't matter because the statement was that it was easier to discover vulnerabilities in open wource software. And he's right. What do you rather do: read source code, or dissassemble a binary?
"Set a man a fire, he'll be warm for the rest of the night. Set a man afire, he'll be warm for the rest of his life."
... Is the worst program to use to create PDFs. Just use one of the free applications.
PDF is the open standard for sharing documents. Adobe does not offer any open source or free creation tools, but there are half a dozen great PDF creation tools available some of them open source, many of them free.
Troll is not a replacement for I disagree.
To add to this: why the hell does it even matter if one particular software solution contained a serious security issue? The whole point of having open standards is the ability to have multiple software solutions all capable of interoperably working on the same data formats. This is one area where HTML shines, though HTML isn't quite well suited for physical paper print material though.
When opened in Acrobat Reader it had a form with a button at the bottom to submit the information. He tried to process it using the most recent version of acrobat for each of the following operating systems:
The takeaway is this: a government process used a supposedly open format but ruined it by using a proprietary extension that only worked on a recent version of proprietary software running on a recent version of a proprietary operating system.
Hmm. I use Firefox / Fedora to access both the NIH and NSF web sites without any problems.
I also use Adobe Reader / Windows to fill out the SF424 forms because, well, if it screws up because you've got your panties in a twist about not using one company's software versus another's, and you don't get the grant because the form was unreadable or inconsistent, you have no one to blame but yourself.
Indeed, I was just submitting the JIT (Just In Time) information for a DARPA award and the PDF form wasn't working correctly despite having been recently downloaded. Whom do you think gets the sharp end of the stick if I were to submit a wonky form? You go ahead and be pendantic and self-righteous and blame the government; I want to keep doing science. So the old copy was deleted, and a new copy re-downloaded. Fortunately, it wasn't some hidden, Fed-sponsored pro-Adobe conspiracy, but likely a simple TCP error during the first download, as the newer copy worked just fine.
Moreover, when it comes down to it, grant applications to the US government are likely accessible by the public through FOIA requests, so it's not like the information is really private or protected in any deep sense. What sort of nefarious activity does the OP suspect Adobe will commit with the data in the application anyway?
The current use of a PDF-based application is phenomenally better than it was before when the applications required a specific program to be downloaded in order to fill them out. That was frustrating to say the least, highly non-portable, and full of bugs. The present PDF-based mechanisms are great, simply great, in comparison. They also work very, very reliably.
There are battles that are worth fighting, and those that aren't. I'm always pleased when the US government allows me to use my Linux box (and I do that preferentially), but as a realist, I also have a dedicated Windows box on my desk for exactly the times when the assumption has been made that Windows is the computational substrate. That the government no longer requires .DOC files in its grant applications (at least the ones I see), and takes PDFs instead is a huge, huge win.
Put my fist through my alarm clock with its ding-dong death inside my ear. - The Blackjacks.
No, in this case I disagree. I'm not usually a fan of Hanlon's razor, but I think it applies here. I recently had an experience with the US Govt in this regard submitting an application to the NIH.
It was a second stage grant so a chunk of the proposal was how you did on the first stage. And they let you submit a video. So far so good!
What about the formats, well, not only did they allow wmv and mov, they also allowed the industry standard, open (if not unencumbered) and widely supported h.264 in an mp4 file.
Woah! That's amazing. Open standards are great, that should work anywhere, easy to make, etc etc nice happy flowers and bunnies and rainbows and unicorns yay!
Oh and the file has to be embedded in a PDF.
er, what? I mean, u wot m8? I mean WHAT THE EVER LIVING WHAT WHY WHY WOULD YOU DO THAT WHAT DO YOU EVEN MEAN???
I am not kidding that was a requirement. So this comes with about a billion problems. First, "embedded" is ill defined: some versions of PDF support video playing in the PDF, but they can also hold files you can simply download. In the former case, Adobe (tm) decided to do it two different ways in two different versions. The first (older) way is to embed the video file and use the system's video player to play the video.
That's moderately sane. Was PITA before every platform supported MPEG4, but even back then I had a PDF which would play on Windows, OSX and FreeBSD (probably Linux too---sis not check). These days it should be easy---just use MP4.
Except it doesn't work that way any more. No, the newest version which not everyone has will only play stuff using flash. So, you have to find a flash player for the video and convert the video to flv and embed it that way. So far, so bad. Flash player is getting somewhat rare now, at least the standalone flash plugin not bundled with a browser (chrome?). And it ain't bundled with acroread.
Well that's all pretty obnoxious. Firstly the methods are mutually incompatible, of course. Naturally because one is for older acroread, one for newer. The file size is strict so you can't embed it both ways and hope for the best. Actually we couldn't get the flash version to work on anyone's (windows) machine. Well, fuck you very much Adobe.
So what I did was the third method which is to have it as an attached file. Double clicking on it invites you to save or open it.
Naturally of course NONE of these things work in anything other than acroread. None of the other PDF readers---the sort everyone seems to have now, like the firefox and chrome ones, the mobile ones or the one embedded in newer versions of windows---work with these methods.
And thankfully someone figured out how to do this in LaTeX. Scott Pakin of course---anything sufficiently obscure in LaTeX always ends up there. Anyone else noticed that?
So there it was, I had the nice, standard works anywhere video file embedded in a uh... PDF where you had to piss around to open it. It was still accessible to submit for anyone using open tools, but WTF?
Oh and of course I tried including a youtube link for when it didn't work and the PDF got bounced with a snippy message pointing out angrily that of COURSE links weren't allowed (heaven forfend!) because then someone might CHEAT by linking to a longer video than is allowed!
This is one of the cases where I think only incredible incopmetence and not malice describes the situation.
SJW n. One who posts facts.
Except that the "open" PDF standard you're talking about is only a small subset of the oldest, most primitive image/text drawing features of said file format
That's not even remotely true. Read the PDF 1.7 specification (chapter 8, specifically) and you'll see all of that stuff documented. JavaScript has been part of the spec since PDF 1.3. The fact that some viewers don't implement features that have been part of the spec for over 10 years is not the fault of the spec.
You might be thinking of the PDF/A family of standards. These are ISO standards for long-term document archiving and specify an intentionally restricted subset of PDF features to ensure that it will always be easy to implement readers for them.
I am TheRaven on Soylent News
At work, my clients use PDFs to submit to us regularly. We immediately convert this to a TIF for our use...
This is not so good as it seems.
We have one client that uses a custom font. Yeah, really. Being not just custom but copyrighted, and they do not include it in the PDFs, when they submit, and our converter makes the best choice it can to make this into something we can use internally. Sadly, the mapping is off by one character code. The original word 'carrot', for instance ends up being 'dbsspu'. Really. They could not change this. We could not change this. They submit using PNGs now. 'Solved'.
Another client uses some third-party PDF software to send those to us. Their solution results in perfectly readable files that our converter refuses to recognize as a PDF. I looked at the data, and it looks ok to me with an unuusal qualifier in the header. Seems their software creates PDFs with version numbers that can't really exist... Solution? Open the PDF before they send ti to us, save it, and magically somehow it changes things.
Another client sends us PDFs that often convert perfect images, hidden behind what can be described as zebra stripes. Except for when it looks like black & white leopard stripes. Solution? Send us JPGs.
PDFs are a lot more complex and difficult than people think. So many third-party apps that generate almost-compatible PDFs, Adobe probably trying to kill these by modifying the file format, adding features that just don;t come out so well, it's not bliss with PDFs.
But to the OP, what open document format would we want the government to use? It should be first, read-only when needed, for instance for applications and submissions, though read/write as an option, of course. Signable. Able to secure, probably via certificate. Forms capability of course. Does this readily exist?
deleting the extra space after periods so i can stay relevant, yeah.
I am an Administrative Official for a large organization. Uploading grants is literally a major part of my job. (As a research scientist, I also write my own grants - so I understand this from several angles.)
The argument that open standards should be used is a fair one, but it is missing the bigger picture here. The vast majority of grants (NIH, NSF, Veterans Affairs, DoD, etc.) are SF-424 NIH standard packages obtained through Grants.gov and submitted by an AO such as myself, not by the applicant. Very few grants require the person authoring them to be the signing official who agrees on behalf of the organization to administer funds if the grant is successful. The vast majority of the applicants therefore route grants through a corporate or University network, where Windows (and to a lesser degree OS X - I'm a Mac user myself) predominate. In all of these cases, the organization will be providing the tools necessary - Acrobat is handed out like candy in my organization. It's part of the corporate image for all computers. Using Acrobat forms streamlines and simplifies submission for 99% of the applicants. The government is not going to change this to address a few edge cases.
The suggested alternative - web forms - is laughable. It might be good for one person, but in an average submission cycle I am sending 10-15 grants with widely varying requirements including esoteric formatting issues, hard-coded naming conventions, and etc. - not to mention that the typical grant includes dozens of required components and attachments, each with set formatting restrictions. It is hard enough to comb through an assembly SF-424 package to check for errors prior to submission as it is. If I had to manually upload each of these grants, one at a time, one piece at a time, into a web forms system, I would not be able to do my job. Period.
Post-submission, forms are processed by a clunky system in eRA Commons, then get referred to Grants.gov for eventual routing to the reviewing agency. The system has a series of automated checks built in to verify that the package is complete before it is assembled. This requires the various bits and pieces to be separate documents, as they are in an Acrobat package (and it is a package, with embedded attachments, not a flat PDF). This process is flaky and fragile enough as it is. Web forms are not going to improve the process, but they certainly would increase the workload for the AO by about 1000% and would definitely increase the error rate. This is also ignoring the fact that the forms are modular, in that some sections (like the budget) are only inserted as needed, and the necessity of being able to assemble and pre-check these things offline precludes any kind of web form system. The article writer is being intentionally obtuse and a bit naive here to make a shallow argument in favor of open standards. Heart is in the right place but reality is being ignored here.
Tl;dr version: it's hard. We do the best we have with the tools provided. Just be glad Grants.gov didn't decide to use InfoPath instead of Acrobat.