The US Government and Open Standards: a Tale of Personal Woe (thevarguy.com)

An anonymous reader writes: This article details a Linux user's struggles to submit a grant application when the process requires finicky, proprietary software. It also covers familiar ground made timely by the upcoming elections: the U.S. should prefer open source software and open standards over proprietary alternatives. The grant application required a PDF created by Adobe Acrobat — software Adobe no longer supports for Linux. Once the document was created, attempting to submit it while using Ubuntu fails silently. (On Windows 7, it worked immediately.) The reader argues, "By requiring Acrobat the government gives preference to a particular software vendor, assuring that thousands of people who otherwise would not choose to use Adobe software are forced to install it. Worse, endorsing a proprietary, narrowly supported technology for government data poses the risk that public information could become inaccessible if the vendor decides to stop supporting the software. Last but not least, there are privacy and fairness issues at stake. Acrobat is a totally closed-source program, which means we have to take Adobe's word for it that nothing sketchy is going on in its code. ... It would seem to be in the interest of the public for the government to prefer an open source solution, since it is much harder to hide nefarious features inside code that can be publicly inspected."

Comments

who here can fix that?

[Bite The Pillow]

"On Windows 7, it worked immediately."

Oh, you fixed it. I don't have time to be outraged about this. Get a citizens united corporate backing and fight, otherwise fuck off Bennett hassleton.

I didn't ask why I should care, I know that. I just don't have time to do more than ask if anonymous helpless cares more than just preach to the choir.

Re:who here can fix that?

[pz]

Hmm. I use Firefox / Fedora to access both the NIH and NSF web sites without any problems.

I also use Adobe Reader / Windows to fill out the SF424 forms because, well, if it screws up because you've got your panties in a twist about not using one company's software versus another's, and you don't get the grant because the form was unreadable or inconsistent, you have no one to blame but yourself.

Indeed, I was just submitting the JIT (Just In Time) information for a DARPA award and the PDF form wasn't working correctly despite having been recently downloaded. Whom do you think gets the sharp end of the stick if I were to submit a wonky form? You go ahead and be pendantic and self-righteous and blame the government; I want to keep doing science. So the old copy was deleted, and a new copy re-downloaded. Fortunately, it wasn't some hidden, Fed-sponsored pro-Adobe conspiracy, but likely a simple TCP error during the first download, as the newer copy worked just fine.

Moreover, when it comes down to it, grant applications to the US government are likely accessible by the public through FOIA requests, so it's not like the information is really private or protected in any deep sense. What sort of nefarious activity does the OP suspect Adobe will commit with the data in the application anyway?

The current use of a PDF-based application is phenomenally better than it was before when the applications required a specific program to be downloaded in order to fill them out. That was frustrating to say the least, highly non-portable, and full of bugs. The present PDF-based mechanisms are great, simply great, in comparison. They also work very, very reliably.

There are battles that are worth fighting, and those that aren't. I'm always pleased when the US government allows me to use my Linux box (and I do that preferentially), but as a realist, I also have a dedicated Windows box on my desk for exactly the times when the assumption has been made that Windows is the computational substrate. That the government no longer requires .DOC files in its grant applications (at least the ones I see), and takes PDFs instead is a huge, huge win.

Re: who here can fix that?

[sumdumass]

Pen and paper should be enough but standards and not software programs should be the acceptable forms.

And yes, it has everything to do with rights and culture. Poorer people cannot always afford specific programs and people may for whatever ideological reasons not want to use specific programs and that should be their right because of arbitrary costs associated. If the government is to require specific software instead of acceptable standards, they need to ensure it is free from the bottom up. This means an operating system, software, and perhaps even computers and internet. There should be no barriers to government services artificially put in place. Standards allow options -

Re:who here can fix that?

[tburkhol]

I also use Adobe Reader / Windows to fill out the SF424 forms because, well, if it screws up because you've got your panties in a twist about not using one company's software versus another's, and you don't get the grant because the form was unreadable or inconsistent, you have no one to blame but yourself.

This is true: if you want the grant, you must comply with whatever rule the funding agency requires. If they ask for a photograph of the PI with a herring on his head, then you better fish-up.

The question is whether those bureaucratic rules are necessary or appropriate. Is your DARPA grant allowed to include a windows computer for the sole purpose of filing grants? Is there some technical superiority to Acrobat forms over html or javascript forms?

I'm always pleased when the US government allows me to use my Linux box (and I do that preferentially), but as a realist, I also have a dedicated Windows box on my desk for exactly the times when the assumption has been made that Windows is the computational substrate.

Seems pretty wasteful if you, and dozens (? hundreds?) of other people, have to go buy a second computer, OS, and other software from a specific vendor just for communicating with the government. I don't think anyone's asking you to pick up the banner and fight the power, but don't denigrate the people who are trying to push the government to move to better systems.

The form progression has, so far, followed a minimum-change model. First the paper: typewritten onto pre-printed forms. Then Word template documents to print and mail. Then the dedicated program where you load pdfs generated from the old Word templates into fixed fields (I think I still have a copy of that monstrosity, which was definitely worse for the applicant). Now the Acrobat fillable-forms with pdf-attachments. It gets a little better every time, so let's not stop improving it.

Re:who here can fix that?

[vtcodger]

"The current use of a PDF-based application is phenomenally better than it was before when the applications required a specific program to be downloaded in order to fill them out. That was frustrating to say the least, highly non-portable, and full of bugs."

No kidding. After I retired from real 40-50 hour a week IT work, I worked part time for a school in Vermont. Every year we got two or three new/altered forms from various state agencies trying to collect data on all sorts of stuff. Some of the stuff was probably pure bureaucracy, but a lot of it looked meaningful.

Not one of their data collection tools/techniques actually worked out of the box. Not one. Never. I was able to tweak some to usability. The state fixed others. Much of the time, everyone gave up and we just sent in the data as a text file or email or sheet of paper and they presumably reentered it by hand.

Re:Not that crap again

[EmeraldBot]

since it is much harder to hide nefarious features inside code that can be publicly inspected

Not THAT crap again.

Heartbleed should put that right to bed.

I don't understand your point here. It was found and then fixed in a few days, and the patches were widely released to anyone willing to update. The system worked exactly like it was supposed to: the fact that a single critical bug garned that much attention should give you an idea of how uncommon it is.

In contrast, Adobe Reader has had not one, not two, but 26 different cripplingly severe vulnerabilities in the last six months alone, and that's only because I got tired of counting after #26. How many people patch Adobe Reader? Would you like to compare Libreoffice to Microsoft Word, FreeBSD to Windows, or Internet Explorer to Firefox? Maybe Apache to IIS, or perhaps OpenJDK to Sun java? Amarok to Itunes? Our very own Adobe Reader to Okular or Evince?

Open source software does indeed have a demonstrably better security record than closed source software, that is undeniable. Further more, even if it didn't, it wouldn't matter because the statement was that it was easier to discover vulnerabilities in open wource software. And he's right. What do you rather do: read source code, or dissassemble a binary?

Adobe Acrobat

[wisnoskij]

... Is the worst program to use to create PDFs. Just use one of the free applications.

PDF is the open standard for sharing documents. Adobe does not offer any open source or free creation tools, but there are half a dozen great PDF creation tools available some of them open source, many of them free.

Re:Adobe Acrobat

[wisnoskij]

PDF-XChange is a good viewer/editor, and the free version has most of the features of the pro version. I use it for filling in and signing forms.

Re:Not that crap again

[darkain]

To add to this: why the hell does it even matter if one particular software solution contained a serious security issue? The whole point of having open standards is the ability to have multiple software solutions all capable of interoperably working on the same data formats. This is one area where HTML shines, though HTML isn't quite well suited for physical paper print material though.

Re:Not that crap again

[sims 2]

Just fyi OpenSSL was vulnerable to heartbleed for over 2 years before it was discovered then it was fixed at the same time it was announced.

Otherwise yeah adobe reader is large security risk for Any windows computer...actually now that I think of it wasn't jailbreakme.com based on a PDF exploit?

So even just PDF in general seems to have security problems in implementation for some reason.

Horrible Summary: Some clarifications

[Duckman5]

The application process required opening a PDF in Adobe Acrobat READER. It used some proprietary extension and if opened in any other application just had a note that said to use Acrobat Reader.

When opened in Acrobat Reader it had a form with a button at the bottom to submit the information. He tried to process it using the most recent version of acrobat for each of the following operating systems:

• On Linux, the button did nothing.
• On Windows XP in a virtual machine, the button half worked (asked for login info)
• On native boot Windows 7, it worked all the way

The takeaway is this: a government process used a supposedly open format but ruined it by using a proprietary extension that only worked on a recent version of proprietary software running on a recent version of a proprietary operating system.

Re:Horrible Summary: Some clarifications

[Anonymous Coward]

It was probably written using the Adobe LiveCycle program and uses the XFA form technology. XFA PDF's are different in that there are no actual postscript commands in the PDF and they do not use the AcroForms technology. The layout and form inputs are defined in an XML document embedded into a PDF container. Adobe Reader then dynamically generates the postscript to render the document on the fly when the PDF is opened. If the PDF reader being used doesn't understand XFA (for example. pdf.js), then they get the generic "Please open in Adobe Reader." The benefit to XFA is that since the drawing content is dynamic, it allows for growing the length of the PDF when you have forms that take a variable number of inputs, like a purchase order with a variable number of line items. One downside is that Adobe Reader does some crazy undocumented proprietary shit when the form also has digital signature blocks.

Re:Horrible Summary: Some clarifications

[nmb3000]

XFA PDF's are different in that there are no actual postscript commands in the PDF and they do not use the AcroForms technology. The layout and form inputs are defined in an XML document embedded into a PDF container. Adobe Reader then dynamically generates the postscript to render the document on the fly when the PDF is opened. If the PDF reader being used doesn't understand XFA (for example. pdf.js), then they get the generic "Please open in Adobe Reader."

Good god. Just when I started thinking Acrobat couldn't get any worse... What ever happened to the Portable part of the Portable Document Format? :(

Adobe needs to stop riding on the coattails of the PDF standard and just create their own damned document format completely separate from PDF. They've been shoving more and more of this kind of stupid shit into PDF files for years, all under the guise of PDFs being a "standard" -- just to encourage the spread their bug-ridden malware by making the files unusable in other programs. It's gotten worse than the ActiveX webpages from the early 2000's.

Re:Horrible Summary: Some clarifications

[temcat]

What ever happened to the Portable part of the Portable Document Format?

Now it's Proprietary Document Format.

Re:Horrible Summary: Some clarifications

[vtcodger]

"Now it's Proprietary Document Format."

PDF = Preposterous Document Format ?

Re:"attempting to submit it while using Ubuntu fai

[larkost]

My guess is that this was one of Adobe's form systems. Those produce overly-complex PDF's that then submit the form content back to specially crafted servers in a non-documented way. Creating these "workflow systems" are how Adobe has been making money on the Acrobat "platform" for some time now. So none of this corresponds to any standards, so nothing works except Adobe Acrobat (usually only on Windows, even MacOS need not apply).

Five years ago I might have thought that using these systems was an ok idea, but web forms have long since surpassed what is possible on these systems, and since mostly they just produce XML it should be cheap and easy to replace them. My guess is that this is an old system, and it just works, so it is hard to justify the money of replacing it. If someone really wanted to do some good, they could organize a hackathon to replace this.

Linux supporters again denying real problems.

[Anonymous Coward]

While Linux hasn't always been known for having the most supportive community, things have gotten particularly bad lately.

Like your comment shows, it's getting quite routine for a user to describe some problem they're having with open source software, and instead of getting anything resembling help we instead see Linux and open source supporters just flat out deny that the problem exists. This isn't a case of giving snooty answers, or even just ignoring the questions. It's outright denial we're seeing now, typically without any sort of evidence to support this denial.

It's an extremely disrespectful attitude to have, and when you direct it towards somebody asking for help then you'll most likely just drive them away to proprietary software.

We see this attitude from the GNOME 3 community, which now consists of a small number of people trying to force their awful software on a much larger community. These GNOME 3 supporters just deny that the UI is now unusable.

We see this attitude from the systemd community. Again, this is a relatively small number of people trying to force their awful software on a much larger community. These systemd supporters just deny that their init system is bloated, full of architectural flaws (binary logging and doing everything are to examples), and has caused a lot of people a lot of problems.

We see this attitude from the Firefox community. Once more, this is a relatively small number of people trying to force their awful software on a much larger community. These Firefox supporters just deny that the UI is now awful, that there are performance issues, and that there are years-old bugs that haven't been fixed.

When users come forward with problems with GNOME 3, systemd or Firefox we just see open source supporters like you treat these people like they're total shit.

It isn't Microsoft, or SCO, or Apple, or Adobe, or any other company that truly harms the adoption of Linux and open source software. It's the Linux and open source communities themselves who cause this harm, all thanks to how poorly they treat so many of the users of this open source software.

You're learning just now

[RightwingNutjob]

that the US government is Bad At Computers? Where have you been this whole time? And are you interested in buying a bridge? I've got in Brooklyn that just happens to be for sale.

Of course the government prefers Adobe...

[OEasygoDiodoB]

between Acrobat and Flash, Adobe provides the bulk of the vulnerabilities the NSA needs to operate. Quid pro quo.

Re:Not that crap again

[nmb3000]

since it is much harder to hide nefarious features inside code that can be publicly inspected

Not THAT crap again.

Heartbleed should put that right to bed.

Heartbleed had nothing to do with the potential for "nefarious" hidden functionality in closed-source systems. If anything, Heartbleed might be a counterpoint to Eric Raymond's proposal that "given enough eyeballs, all bugs are shallow" -- but the truth is that complex problems usually have complex solutions, and the more complex the solution the smaller the percentage of those eyeballs that has expertise.

As far as the article's argument goes, I'm torn. I can see immense value in requiring the software that government uses be open source. It levels the playing field in terms of accessibility while promoting the transparency required for a successful democracy. It also falls in line with other existing aspects of the government, such as the requirement that any works created by the Federal government (generally) are put into the public domain.

However I can see the argument that computers and software are just tools to get stuff done. Interoperability can be simplified when you target specific application versions (say, "MS Office 2010 and newer") rather than a more squishy target such as "Open Document Format". While there's no reason you couldn't do that, it seems like when it comes to software today, sadly, sticking to an implementation of a (possibly proprietary) standard rather than a generic standard itself ends up causing fewer problems.

That said, I'd definitely agree that it should not be required to use Adobe Acrobat to submit a grant application.

Re: Not that crap again

[TheReaperD]

The PDF security problems stem from its early days when they were trying to get their adoption rates up. In order to try and get every business to adopt it, they asked people at the companies what features they would like to see in Acrobat. And they got mostly marketing managers replying with every bell and whistle they could think up: scripting support with system and drve access, embedded binaries, ability to connect and send commands to Outlook, etc. I'm not sure anyone at Adobe cared if they were a good idea, much less secure. All they cared about was adoption. They've been trying to clean up the mess ever since. Many of these features have had to be removed or severely restricted just to try and put their fingers in the gaping holes that is Acrobat security.

Re:Not that crap again

[ShanghaiBill]

This is one area where HTML shines

It is also an area where PDF shines. PDF is a license free open standard, and there are open source tools that can generate and manipulate the format. It isn't as easy to work with as HTML, but it isn't that hard either. TFA is just uninformed whining. PDF is a perfectly acceptable open format for the government to use, and it is a big improvement over requiring something that is actually proprietary, such as MS-Word.

Re:The "Trust us" aspect is intentional

[serviscope_minor]

No, in this case I disagree. I'm not usually a fan of Hanlon's razor, but I think it applies here. I recently had an experience with the US Govt in this regard submitting an application to the NIH.

It was a second stage grant so a chunk of the proposal was how you did on the first stage. And they let you submit a video. So far so good!

What about the formats, well, not only did they allow wmv and mov, they also allowed the industry standard, open (if not unencumbered) and widely supported h.264 in an mp4 file.

Woah! That's amazing. Open standards are great, that should work anywhere, easy to make, etc etc nice happy flowers and bunnies and rainbows and unicorns yay!

Oh and the file has to be embedded in a PDF.

er, what? I mean, u wot m8? I mean WHAT THE EVER LIVING WHAT WHY WHY WOULD YOU DO THAT WHAT DO YOU EVEN MEAN???

I am not kidding that was a requirement. So this comes with about a billion problems. First, "embedded" is ill defined: some versions of PDF support video playing in the PDF, but they can also hold files you can simply download. In the former case, Adobe (tm) decided to do it two different ways in two different versions. The first (older) way is to embed the video file and use the system's video player to play the video.

That's moderately sane. Was PITA before every platform supported MPEG4, but even back then I had a PDF which would play on Windows, OSX and FreeBSD (probably Linux too---sis not check). These days it should be easy---just use MP4.

Except it doesn't work that way any more. No, the newest version which not everyone has will only play stuff using flash. So, you have to find a flash player for the video and convert the video to flv and embed it that way. So far, so bad. Flash player is getting somewhat rare now, at least the standalone flash plugin not bundled with a browser (chrome?). And it ain't bundled with acroread.

Well that's all pretty obnoxious. Firstly the methods are mutually incompatible, of course. Naturally because one is for older acroread, one for newer. The file size is strict so you can't embed it both ways and hope for the best. Actually we couldn't get the flash version to work on anyone's (windows) machine. Well, fuck you very much Adobe.

So what I did was the third method which is to have it as an attached file. Double clicking on it invites you to save or open it.

Naturally of course NONE of these things work in anything other than acroread. None of the other PDF readers---the sort everyone seems to have now, like the firefox and chrome ones, the mobile ones or the one embedded in newer versions of windows---work with these methods.

And thankfully someone figured out how to do this in LaTeX. Scott Pakin of course---anything sufficiently obscure in LaTeX always ends up there. Anyone else noticed that?

So there it was, I had the nice, standard works anywhere video file embedded in a uh... PDF where you had to piss around to open it. It was still accessible to submit for anyone using open tools, but WTF?

Oh and of course I tried including a youtube link for when it didn't work and the PDF got bounced with a snippy message pointing out angrily that of COURSE links weren't allowed (heaven forfend!) because then someone might CHEAT by linking to a longer video than is allowed!

This is one of the cases where I think only incredible incopmetence and not malice describes the situation.

Re:Not that crap again

[Narcocide]

Except that the "open" PDF standard you're talking about is only a small subset of the oldest, most primitive image/text drawing features of said file format, and the aforementioned government website is not only requiring use of a PDF document that used some of the newer (massively insecure) JavaScript-enabled interactive form input/validation features not included in said "open" PDF standard or implemented outside of Acrobat, but apparently they even then used said features to code the document such that it blocks you from even trying to read the document without Acrobat.

Go ahead. Go download it and try to open it with Xpdf, let us know how that works out.

Re:Not that crap again

[serviscope_minor]

PDF is NOT an open standard. Oh sure little bits of it are (the document part) but the bulk is not. Adobe's PDF has loads of weird and messed up features that are 100% proprietary and that for some reason government IT wonks absolutely love for no discernible reason.

This story is about one of these bits.

Re:Not that crap again

[EmeraldBot]

I am sorry, but if it was fixed in few days, it was not found in few days. This bug existed for many versions of OpenSSL before being finally discovered. That's not quite true to say it was discovered in days.

Microsoft had a flaw in Windows that lasted for almost 20 years before being fixed, and they also had one that took 17 years to fix, and another one that took 15 years to fix. There are many, many more with shorter lifespans but are just as severe in terms of how much they compromise. Heartbleed was in use for 2, being introduced in March 2012 and fixed April 2014.

My point here is that open source software has a better track record for security, and you don't seem to be really disputing that.

Adobe 9.5.5 and Linux

[Zombie Ryushu]

In order to get this working under Linux, you have to install the (Ancient) Adobe 9.5.5 Reader and its associated npppf module. Then it will work. I have alot of experience with this. While Okular, Evince, and XPDF can fill out forms, there is no support for submitting an XFA Form under anything other than the real Acrobat Reader.

Re:Not that crap again

[TheRaven64]

Except that the "open" PDF standard you're talking about is only a small subset of the oldest, most primitive image/text drawing features of said file format

That's not even remotely true. Read the PDF 1.7 specification (chapter 8, specifically) and you'll see all of that stuff documented. JavaScript has been part of the spec since PDF 1.3. The fact that some viewers don't implement features that have been part of the spec for over 10 years is not the fault of the spec.

You might be thinking of the PDF/A family of standards. These are ISO standards for long-term document archiving and specify an intentionally restricted subset of PDF features to ensure that it will always be easy to implement readers for them.

If you're using Win98 you will be 0wn3d

[tepples]

Acrobat reader is free, dumbass, and if you can afford a computer it will come with a Windows license.

There is no acroreader for Windows 98/me

Support for Windows 98 and Windows Millennium Edition ended in July 2006. If you are still using one of these two operating systems on a PC connected to the Internet, you are using software with exploitable security vulnerabilities that will not be fixed this century.

and yes, all sorts of new computers are sold without Windows or a Windows license.

I know. Many are servers, which are not intended to display GUI apps in the first place. Many are made by Apple Inc., and they come with an OS X license that can run Acrobat for OS X. Many are ARM-based devices, and you have a valid point that Adobe refuses to port the features at issue to Acrobat for mobile operating systems.

Re:Not that crap again

[vtcodger]

Where PDF shines is its ability to accurately render a document pretty much EXACTLY the way its author intended. HTML usually can't do that. Nor was it intended to. The M stands for MARKUP -- which is not the some thing as LAYOUT.

Other than that, I can't say much nice about PDF. When confronted with a purportedly editable pdf form, my experience has been that trying to edit the bloody thing without paying for Acrobat is a waste of time in both Linux and Windows. (foxit purportedly can edit pdfs, but I found the user interface to be beyond my limited comprehension). Anyway I just convert editable pdfs to Jpeg and use an image editor like kolourpaint. Probably not what the agencies distributing the stuff have in mind, but it satisfies MY obligations.

In fairness, government folk face a major problem when trying to gather data in a usable format other than unadorned ascii text. There really doesn't seem to be any such format. Those folks have a day job and that job surely is not dealing with the IT industry's near total lack of meaningful standards.

PDFs are not PDFs

[rickb928]

At work, my clients use PDFs to submit to us regularly. We immediately convert this to a TIF for our use...

This is not so good as it seems.

We have one client that uses a custom font. Yeah, really. Being not just custom but copyrighted, and they do not include it in the PDFs, when they submit, and our converter makes the best choice it can to make this into something we can use internally. Sadly, the mapping is off by one character code. The original word 'carrot', for instance ends up being 'dbsspu'. Really. They could not change this. We could not change this. They submit using PNGs now. 'Solved'.

Another client uses some third-party PDF software to send those to us. Their solution results in perfectly readable files that our converter refuses to recognize as a PDF. I looked at the data, and it looks ok to me with an unuusal qualifier in the header. Seems their software creates PDFs with version numbers that can't really exist... Solution? Open the PDF before they send ti to us, save it, and magically somehow it changes things.

Another client sends us PDFs that often convert perfect images, hidden behind what can be described as zebra stripes. Except for when it looks like black & white leopard stripes. Solution? Send us JPGs.

PDFs are a lot more complex and difficult than people think. So many third-party apps that generate almost-compatible PDFs, Adobe probably trying to kill these by modifying the file format, adding features that just don;t come out so well, it's not bliss with PDFs.

But to the OP, what open document format would we want the government to use? It should be first, read-only when needed, for instance for applications and submissions, though read/write as an option, of course. Signable. Able to secure, probably via certificate. Forms capability of course. Does this readily exist?

Uploading grants is literally my job.

[caution live frogs]

I am an Administrative Official for a large organization. Uploading grants is literally a major part of my job. (As a research scientist, I also write my own grants - so I understand this from several angles.)

The argument that open standards should be used is a fair one, but it is missing the bigger picture here. The vast majority of grants (NIH, NSF, Veterans Affairs, DoD, etc.) are SF-424 NIH standard packages obtained through Grants.gov and submitted by an AO such as myself, not by the applicant. Very few grants require the person authoring them to be the signing official who agrees on behalf of the organization to administer funds if the grant is successful. The vast majority of the applicants therefore route grants through a corporate or University network, where Windows (and to a lesser degree OS X - I'm a Mac user myself) predominate. In all of these cases, the organization will be providing the tools necessary - Acrobat is handed out like candy in my organization. It's part of the corporate image for all computers. Using Acrobat forms streamlines and simplifies submission for 99% of the applicants. The government is not going to change this to address a few edge cases.

The suggested alternative - web forms - is laughable. It might be good for one person, but in an average submission cycle I am sending 10-15 grants with widely varying requirements including esoteric formatting issues, hard-coded naming conventions, and etc. - not to mention that the typical grant includes dozens of required components and attachments, each with set formatting restrictions. It is hard enough to comb through an assembly SF-424 package to check for errors prior to submission as it is. If I had to manually upload each of these grants, one at a time, one piece at a time, into a web forms system, I would not be able to do my job. Period.

Post-submission, forms are processed by a clunky system in eRA Commons, then get referred to Grants.gov for eventual routing to the reviewing agency. The system has a series of automated checks built in to verify that the package is complete before it is assembled. This requires the various bits and pieces to be separate documents, as they are in an Acrobat package (and it is a package, with embedded attachments, not a flat PDF). This process is flaky and fragile enough as it is. Web forms are not going to improve the process, but they certainly would increase the workload for the AO by about 1000% and would definitely increase the error rate. This is also ignoring the fact that the forms are modular, in that some sections (like the budget) are only inserted as needed, and the necessity of being able to assemble and pre-check these things offline precludes any kind of web form system. The article writer is being intentionally obtuse and a bit naive here to make a shallow argument in favor of open standards. Heart is in the right place but reality is being ignored here.

Tl;dr version: it's hard. We do the best we have with the tools provided. Just be glad Grants.gov didn't decide to use InfoPath instead of Acrobat.

Re:SWF spec is available

[TheRaven64]

The Flash spec has been published since at least the '90s, though the click-through license agreement prohibited writing tools for playing back flash until about 10 years ago. There were numerous third-party tools for producing Flash, just as there were for PDF and PostScript, because that's always been Adobe's explicit policy for getting adoption for their formats.

Re:Not that crap again

[BarbaraHudson]

Look, anyone making a grant application who can't beg or borrow someone else's computer for a bit and thinks that the solution is to whine about it has bigger problems. When you're asking for money, better to just go with the flow. Maybe include the purchase of a windows computer in the request. Or get one of those cheap windows tablets.

Re:Not that crap again

[bigman2003]

HTML forms are a bad idea for proposal submission.

I've written quite a few grant submission systems (I have a grant cycle running right now, with a deadline of this Friday...yay...). It's a pretty standard deal- web based system that allows for a fair amount of meta data (PIs, co-operators, institutions, name of grant, funding request, etc.). These of course are all part of the HTML forms.

BUT- the proposals themselves- the 2-20 page document where they explain the project- is always a complete mish-mash of stuff that could never go into an HTML form. Formulas, images, etc. Tons of formatting. And typically it is a document that has been shared/edited with other researchers. I ran one system about 15 years ago that was HTML only, and the number of projects that had 8 different PIs, who all wanted edit rights at the same time was way too high. This was pre-Google Wave, and the idea of 8 people simultaneously editing the same text on the web was insane then...as it is now.

Plus, the way that researchers/PIs handle these submissions is to turn everything in at the last possible minute. Any complication on the receiving system will just cause you to get your ass chewed out in the hallway at the next big conference.

I absolutely, 100% never ever want to hear someone say, "I tried to submit my proposal, I typed everything in, then there was an error." Because really, these people will open the page, then sit on it for 3 days as they dink around. When they finally hit 'submit' they're surprised that there was an error. Yes, there are technical ways to mitigate this problem...and the very best way is to have the applicants submit documents.

But, in the case of this article...I usually provide support for these systems. I've been doing this for about 20 years, so I'm fairly good at it. And the absolute quickest way to provide support to someone having problems is to say, "Just email me the document, and I'll submit it for you." 90% of the time I get an email that says, "I figured it out...thanks for your help." 8% of the time people say, "I tried to email the document, but it failed...my file was corrupt, so I re-saved it and then submitted...thanks for your help." The last 2% send me the file, I convert it if necessary, and we move on. (that's 2% of the problems, not 2% of the submissions)

There is no reason for me to make a 100% bullet-proof, all-inclusive system that will handle every single different scenario perfectly. It would take too much time. For the very small number of people with a problem, I just do it the old fashioned way. So if somebody told me, "I'm on Linux, and I can't convert my file to PDF, and I don't want to use one of the billion on-line PDF conversion tools, why is the government supporting Adobe and Microsoft!!!, blah blah blah" I just tell them to send me the file. In about 3 minutes I'm done and they are happy. Once upon a time I even hired temps to do this work- but these cases are really about .5% of submissions, and it just isn't worth it.

The article wasn't about the practical aspects of using PDF, it was about the (crap, can't think of the word...) aspect, where someone got their panties in a bunch because the government doesn't facilitate their worst-case-scenario approach to proposal submission.

Source: Been doing this for 20 years for the gub'ment. Yes, there is a guy like me behind most of those systems. See the part of the submission site that says, "For technical assistance...". Yeah, call me or send me an email and I'll take care of it for you. That's why they pay me, and good service is how I make the system look good.

***On the other hand, when you send an email to me, my boss, the funding organization and the overarching agency describing how the system does not function properly, and you were not able to submit your proposal...yes, I will send back a very detailed screenshot laden email pointing out step by step how you failed, and probably send the logs showing that you logged on one time 3 hours before the submission deadline. Goddam I hate it when people blame their failings on the system.