Slashdot Mirror
Fulfilling the Promise of XML-based Office Suites?
brentlaminack asks: "Almost a year ago Tim Bray of XML fame
said 'when the huge universe of MS Office documents becomes available for processing by any programmer with a Perl script and a bit of intelligence, all sorts of wonderful new things can be invented that you and I can't imagine.' Now that
MS has dropped the ball on the XML Office front, and
StarOffice has fulfilled its XML promise, where are all those 'wonderful new things?' Is anybody out there writing Perl/Java/whatever programs to take advantage of StarOffice XML? Could this be an opportunity for Free/Open/Libre software to leapfrog MS Office in real productivity as XML proponents have promised all along?" What kinds of new and wonderful things can you come up with?
This is just a return to part of what made Unix so powerful in the first place: text formats that can be manipulated by the whole suite of command line tools. "Those who don't understand Unix are doomed to re-invent it, poorly" (Henry Spencer).
Back in the 70s we used nroff/troff for document formatting, producing in some cases professional-quality camera-ready books...but the source code was easily fed to spell checkers, formatting-command-strippers, sort, wc, etc etc etc.
XML is ok...not bad as a meta-format...but it's not some kind of new magic; it's just more of the same as what we always used to do.
The great step forward is moving away from the crud that happened in the middle: proprietary underdocumented binary formats that couldn't be fed to filter pipelines.
In this case, moving backwards is progress. But expecting something amazing to be invented is a bit much; it was already invented a long time ago.
P.S. pet peeve...people credit Knuth (admittedly an amazing guy for the Art of Computer Programming) for reinventing typesetting with TeX. Now, TeX is nicer than nroff/troff in multiple ways, but it's worse in some others (TeX is not set up for command line filters!), and in any case is only an incremental improvement, not a revolution over the older Unix tools. Credit is not properly being given.
Professional Wild-Eyed Visionary
I sure would like a apache module that can CSS and display native open and star office documents.
Got Code?
I created a PHP script a few months ago that allowed a client to upload StarOffice templates for company documents. Then the the script automatically generate documents by pulling data from a database and inserting it into the StarOffice document.
Was really easy, StarOffice documents are zipped files that contain the XML files. I just unzip'ed the file, inserted the appropriate data into the content.xml file and zipped it back up.
I was absolutely amazed by how easy the StarOffice files were to work with. I'm really excited about the possibilities that are in store for us, especially ones that are better than my little hack.
Brandon Petersen
The biggest dream that the financial world has ever had with an XML concept has been the concept of standardised financial reports.
Imagine a world where any finacial (excel based or otherwise) report from any public company can be compared with any other company report and we can all be sure of how the figures were calculated and what they mean.
AND they are fully comparable. And fully importable into any financial package. No longer is any one company dependant on one financial package. Come to think of it there is no way the vendors of such products will ever allow this to happen!!!
http://www.xbrl.org/
jech
If there was a way to render out the open office/star office documents on the command line it would explode in the reporting area. Being able to have the end user making a really nice template and have a perl script fill it then pass it off to a pdf or printer is key.
My team & I just got done building some billing software for one of our customers, and OpenOffice.org's XML based documents turned out to be perfect for generating reports. Our customer is able to open up the document and change the formatting of any report at will, and then we have some Ruby code on the backend that parses the XML document, fills in all the real data from the database and then uses the CLI interface to OpenOffice to render the document as postscript. It was a quick easy way to get powerful report generation with a format that non-technical people could edit that required just a little bit of glue code on the backend, and it's the XML format that made it all possible.
I did take some time and decompress a StarOffice document -- I was attempting to write a couple of modules for manipulating StarDraw images to create dynamic flowcharts.
It took some time to get up to speed, as the compressed XML is split across four different files (content, meta data, settings and styles). Mostly, I was concerned with modifying the content document.
Each of the documents is written with space in mind, and for the document I was dealing with, the content was 20K on a single line. I had to process the XML just so I could understand the physical structure. Once that was done, it really wasn't that difficult to manipulate the doc by hand, re-zip the content and open in StarOffice.
(Unfortunately, I didn't have the time to even start, much less complete, the modules. Damn day job).
XML developers and Web designers are now able to work on some XML-to-HTML transformer that matches closely what the average office user is spending his time creating with the WYISWYG Writer program. This could be a nice alternative to Frontpage, for example.
Of course, OpenOffice 1.1 already comes with a nice HTML tool, but that doesn't stop anyone from trying to do better.
I bring it up because my organization paid Crystal reports $10,000 to be able to do this. If I could have written a little perl script that connects to the database and emits an OpenOffice doc, then I could have saved the organization ten thousand dollars, and saved myself a world of pain. (The only thing more evil than Crystal Reports is crystal meth.)
You might be wondering why I wouldn't just use HTML and some library that automatically creates chart PNG images -- the reason is we have to email the report to our board members because they're demanding like that. So we use Crystal to generate pretty PDFs with all the charts. We also let the board members log into our system to generate their own reports via the web, which they can then email to the group.
So having an XML-based document format for this would be wonderful, especially if OpenOffice would provide a command-line utility for converting from OO format to PDF.
Wow, a lucrative publishing contract! I don't have to be evil anymore. --Meteor
At my company, once a failed startup with new life under the wings of a huge corporate parent, we have been using a homebrewed Web publishing system that takes Word 2000 or XP documents, saves them in RTF format, then uses a utility created by Majix to transform the document to XML. From there we use perl, and some XSL to get the document into XHTML combined with some JSP to produce documents that we deploy on our production env. The good part: the system was entirely free of license fees (other than office and Windows of course). The bad: it was a pain in the behind to get all the parts together.
The steps to produce valid XML from Word are the biggest hack I have ever been a part of as an engineer. We had to write a custom VB DLL we run inside (what else) an IIS server which takes the documents uploaded by authors, then saves the documents as RTF. Control is then handed over to Tomcat, which takes the RTF and uses some custom classes that make Majix a server to transform the documents into XML. All in all we had to use VB, VBA, Java, JSP; two separate server configurations (IIS and Tomcat) and a bunch of really ugly glue to stich all the parts together.
I for one, and I am sure I speak for my entire team, would love a solution which saves us this ugly cludge.
First Off
Microsoft did not drop the ball with XML. Microsoft disappointed the slashdot crowd by not going completely open... geee...... big shock there. Microsoft maintains dominance to their office suite by controlling the file formats behind it. Opening that up, without reason would be absolutely stupid from a business point of view. Granted, its an un-popular stance, but that doesnt make it any less true. MS played along with the XML game to be able to use XML as a buzz word... and in some ways, they truly have embraced XML... just not in their holy cash cow called Office. Take a look at Visual Studio (dot) Net, and you will see how strongly MS has infact embraced XML.
Secondly...
XML is perhaps one of the most over hyped technologies ever. Self describing datatypes are nothing new. The only really remarkable thing about XML is how embraced by the industry it was. In all honesty... the difference between XML and CSV files really isnt that signifigant. Granted... XML is far beyond anything a CSV ever did, but they all present the same result. In the current work environment I am in, all our enterprise systesm support input/output now via CSV. In addition, im in the auto industry, so the whole hype of Webservices+XML really isnt that special either. RIght now, they have ANX and EDI... granted... XML + Web Services would be much more straight forward... but in 20+ years of evolution... has it really come that far?
Sorry for the anti-status-quo opinion, but I cant help but believe that XML is way overhyped. Useful... sure... but definatly overhyped!
XML does make it extremely easy to create documents on the fly, whether a plain old document or a slideshow presentation, all it needs is some template XML, original text, and some programming language to put it together.
I wrote a song lyric storage system using PHP and MySQL, and I had the idea to have it be able to be put onto a slideshow to teach it to a group of people (or whatever). With the XML format provided by OpenOffice.org, I was able to quickly put it together and show it off, impressing quite a few people in the process. Of course, those people think Word/PowerPoint run the world, and the file format is all but a mystery to them. Hence having something generated on the fly via a webpage has its cool factor, and not to mention it was a good chance to introduce this free word processing suite to them. Also a good chance to tell them that if I were to rely on ASP/PowerPoint it would have costed much, much more.
Open document format is the way to go in the future, because it definitely allows interoperability.
Please direct all bug reports to
I helped spec out a document management metadata database 18 months ago for an engineering firm that wanted to catalog its files. They started out wanting just to categorize their CAD drawings, then decided to include all types of project files.
Our solution was a tcl front end that forced the entry of a minimal amount of metadata *during file creation,* to be picked from preset categories and subcategories. We also provided for free text entry but that was to be used only after the other fields.
The points are
a) The general metadata categories were known; the engineering tasks weren't new.
b) No one is going to go back after the fact and enter the metadata. You have to integrate its entry into the new file work procedure.
c) It's got to be as easy as file/new in a GUI.
d) Its utility has got to be very very apparent when juxtaposed with a subdirectory / filename scheme.
Not me but I am writing C# apps that make use of Excel's XML format. I wrote about using XSLT on the Excel XMLSS format in my blog a few months ago when I had to update date values in certain columns. I also posted the XSLT stylesheet.
Disclaimer: I work on the XML team at Microsoft but not directly with Microsoft Office.
I've been using these XSLT OOo <-> Docbook-XML filters for a little while.
They work pretty well (if you can manage to get them installed with the broken install instructions) but only for a limited subset of Docbook. There's no support for the programlisting tag, and lists are currently broken.
If anyone out there has superior XSLT kung fu, getting those two things working would be most appreciated : )
(I know the basics, but I don't yet have time at work to justify it. Maybe if this project gets done on time...)
Well I don't know about Free/Open/Libre or XML development for Office... but I do know about the proprietary APIs Microsoft distributes for Office.
If you wanna give them a try sometime, assuming you got Windows, VB5+, and Office installed... just add Office to your references (try Microsoft Office in the Project References menu) and give it a whorl. It's fairly easy to program in if you've used Office... most of the concepts that make for a good Office user translate directly into programming concepts for the Office object model.
And yet Office Automation programmers are in scarce supply.
Microsoft even offers a cert specifically for Office Automation programmers!
But I haven't seen too many well written Office applications. My speculation is that its not for lack of tools, but that its for lack of concepts. Other than the obvious reporting needs that any large organization has, are there any compelling reasons to spend an afternoon coding an office application?
I think it is this lack of compelling reasons, and not a lack of easy-to-use programming tools that causes the lack of good free open add-ins...
I am disrespectful to dirt! Can you see that I am serious?!
...Of course, not very well--but it's pretty easy to compile, say, the Docbook 4.1 DTD in Wordperfect and edit moderately complicated documents. Or import... The limitations are that it uses its own formating system, rather than XSLT; and it uses DTDs instead of schemas, because the technology derives from SGML (which wordperfect also supports). Arguably, WordPerfect has better support than any of the alternatives within the word processing space (i.e. discounting pure editors such as EMACS).
Author of Permanence and Ventus, co-author of The Claus Effect and The Complete Idiot's Guide to Publishing SF.
I still don't get this thing about MS dropping the ball. I've played with Office 2003, and the XML features in particular (mostly Word & Infopath, not the other programs) and I think they are quite well done.
/. goes on about mostly. Yes, it's pretty ugly XML, but you are trying to represent non-structured data in a structured format - of course it's going to be ugly. But it is documented & there is a publicly available XSLT from Microsoft to work with it. The other mode is to import and XSD and tag up the document as you like. You can save this in "rich" mode (with all the office formatting - unstructured again) or "clean" mode in which the XML is as pure as your XSD is.
Word has two different modes. One is where you can save an ordinary word document in an XML format. This is the one
InfoPath simply rocks. Where else can you create a end user friendly UI that outputs clean XML (with XHTML islands if you choose) and will submit directly into a web service & make the whole thing start to end in a few minutes (for a simple form, of course).
I just don't get it. Seems like mindless MS bashing to me.
Read reviews of shopping cart software
That probably sounds icky and scary, but should not be all that hard.
I don't know what the formats are, but there's a whole pile of flexibility in XSL and FOP so building a very accurate version could take some fiddling. But producing a close approximation is probably very straightforward.
I don't want an "Office Suite" shoved down my throat. I want to use the graphing tool I think is best, I want my favorite email app, I want to use the word processor I like, and the spreadsheet I like, etc. I want to be free to try the newest software without converting everything I might need in the future. If the "office productivity programs" all used xml file formats, I could interchange files for one app to the next easily. I would NOT be locked into a single vendor's "suite" or programming HELL.
If the apps were using XML, easy migration would be a given, and programmers could spend time "enhancing" the user interface.
. there used to be a sig here.....
I agree with you SOOO much. Often times, it seems applications are written by programmers/computer geeks FOR computer geeks. I work on a workflow-based web application (It uses oracle workflow). We recently completely redid the app to do away with the Oracle-generated web pages for "notifications" (stages in the workflow) to do our own and send messages to the engine via API. Why? Our users just didn't "get" the workflow concepts and we had to design vastly more complicated UI that had pictures, etc.
and yet we met with massive resistance from the other IT groups... "Why are you doing that, workflow does that" "that's a training issue (code phrase for 'the users are stupid') and "don't you know how to say no?" and (getting to your central point) "you've dumbed it down. Your application doesn't any of the powerful search, etc, features the workflow web interface has" (never mind NO ONE used these things).
I think it was a piece from Douglas Adams who told a story of someone he knew using word who wanted all the junk removed from Word's menus that he didn't use. He showed him how to remove menu items thru customization and he ended up with just Open, Save, Bold, Italic, Print and Spell check.
DO NOT DISTURB THE SE
Having an XML representation of a Word (MS, Open, whatever) document as a stream is really no more useful to me than RTF: I can parse them both.
The better part is when you can structure your document. Not just a heading surrounding a bunch o' paragraphs, but a (to use the stuff I have to work with) Research Report contains a Title Page, a Synopsis, an Introduction, Materials Section, etc. You can't put tables and figures on the title page or Introduction, you can in the Synopsis and Materials Section. TOCs and things like that are created as part of rendition, between the Synopsis and Introduction, without the user messing with it.
Now even more than storing those sections (which would, in the HTML world, be DIVs and SPANs), I want control over the UI: disable that table button in the title page, even down to where bold and italics can be used.
Office 2003 has some facility to implement this, but it's kind of awkward -- it's an extension of how their SmartTags work. Generally pretty ugly, to control everything.
I don't want to use an XML editor, my users know Word, are used to Word Processors, and they cost 1/5 of XML editors, less in bulk licenses.
I'd be implementing this now, if it weren't for two things: a) I work for a big corporation that never buys into new releases for a couple of years, and 2) they're laying me off -- closing all the facilities in Chicago (sigh).
Design for Use, not Construction!
True! But it is widely under-appreciated that this can and was done even with troff, and still is today in an important way: the "apropos" command that scans for relevant man pages works by looking at a DB built by searching for semantic tags in the man pages.
This very handy feature would not be possible if troff just did presentation style.
It's true that this is not the main emphasis of troff, and that one is at the mercy of whoever wrote the macro package, etc, but that's true of XML sublanguages too.
I'm playing devil's advocate. I realize (and posted elsewhere here) that there's a difference in that XML, when used as intended, is supposed to be primarily about semantics, with style as a secondary transformation, whereas it's the other way around with troff...it's intended for presentation, but people have nonetheless done handy things with it at a higher level of abstraction.
But still, the point is that nothing XML does is brand new. It just represents new industry awareness of some old good ideas.
Similar to how Java has popularized the 40 year old notion of doing garbage collection, so now people say GC is "new technology". Not at all. Just being more widely used.
Professional Wild-Eyed Visionary
The fact that the format is XML doesn't say anything about the format being "open". That's why Microsoft was proposing XML to standard bodies, and trademarking DTDs and Schemas...
What other people in the thread is for Microsoft to give us 100% of the schema, and so far Microsoft has shown zero will to do so witout legislation compelling it to. 100% of the schema would allow Corel and/or IBM to feature-copy 100% of Microsoft's Office features, and they certainly will of course say that legistlation to force them to give away their competitive advantage would be anti-american.
Someone with a different agenda would probably say such a thing would have provided a better, more balanced punishment to Microsoft's monopoly than the minimal slap on the wrist they had.
I personally think they should have been made to refund 50% of the purchase price of all Windows licenses, as half of the value was created by "Everyone else is using it" and that advantage was gained through illegal monopoly, and very creative enforcement of copyright laws. But that's neither here nor there.
You can configure most office suites to display the document properties dialog on save. I'm sure you could also build templates with macros that would check and update these. Yes, it's a real problem and most businesses do not have strategies to address it. It's a document management issue very few address.
It's a similar problem with web publishing; there is little or no metadata to identify documents. I've always thought that the Dublin Core set would serve as a very good repository for a kind of CVS on the status of documents. Have wanted to build a back end to something like Apache/Cocoon using this model, which would also serve as the data repository for populating both the metadata in the web documents and also all the other data for semantics and accessibility, all done on the fly out of a DC metadata repository.
One thing the Open Source office suites don't (yet) have much of an answer for is an XML data collection/management system along the lines of Microsoft Office InfoPath. A natural standard for such applications is W3C XForms.
Read all about it--fullly GFDL and online now--from the O'Reilly book at my site.
.micah
--- Learn XForms today: http://xformsinstitute.com
I've been thinking about a document management system that has an integrated word processor.
/. archive of this post as prior art (although I'm sure this kind of system is already in use in some organisation somewhere).
To create a new document users would first be presented with a DE screen asking for some meta-data (perhaps with some manditory fields) before being dropped into the more familiar wordprocessor gui.
Someone with admin rights to the document management system might define the fields that go into the initial DE screen.
Users might have to choose beforehand whether the document will be emailed, faxed or printed (eg for snail-mail), and the document would be "attached" to a client record, along with any replies (eg by email).
The "save" feature would be replaced by "save draft" and "save final", because once the document is sent to an external party you need to "freeze" the document as a record of what's been sent.
Maybe some kind of versioning & rollback would be useful too (something more powerful than undo/redo).
I'd do it myself, of course, but I don't have the time or the skills.
If I ever see a patent application for this idea, I'll point to the
You make the mistake of thinking you can educate the fundamental stupidity out of people. You can't.
"and a bit of intelligence"
Using a MS Word template, ActiveState Perl, and a number of modules including Win32::OLE I created a documentation generation system that pulled information from a database and created a Word document with dynamic headers, footers, formating, content, etc. I used it to created 1000+ password protected, pre-formatted Word documents that we provided to the client. Anytime the format needed updating or any data needed to be changed all I had to do was rerun the Perl script rather than update all of those docs.
I'm not going to say that this was easy by any means, it took quite a bit of research and tweaking to finally get right. XML would, no doubt, make this task easier but I don't necessarily think it is the panacea that will FINALLY permit us to automate docs and reports that need to be generated and shared. My point is that with "a Perl script and a bit of intelligence" document automation is something that can be done now.
On the other hand, OO.o's XML format + schema will be available even to competitors and theoretically beyond the life span of OO.o. One way for OO.o to encourage users to think in a structured is through style sheets. Style sheets and document templates can save a lot of wasted time and effort. But again, what would people do with the spare productivity if formatting were done in 5 minutes, instead of spending 2 days formatting manually and re-formating manually various reports and presentations?
Beta is broken and the link to classic doesn't work. Stop wasting our time or there won't be anybody left here.
> Is anybody out there writing Perl/Java/whatever programs to
> take advantage of StarOffice XML?
Yes, actually I started doing that yesterday: I'm using Perl and XSLT to build documents in StarOffice XML (or actually OpenOffice.org XML), converting some 500 XHTML pages into one huge OpenOffice.org document. It's amazingly easy!
The gigantic propaganda campaign about the "wonderful new things" that semantic markup would make possible was always just a masturbatory fantasy by people who'd never implemented anything, encouraged by SGML contractors who saw an opportunity to broaden their target market.
At the root of this delusion is what I call "Goldfarb's conjecture"-- the claim that document styles are superficial representations of underlying semantics. If Goldfarb were right, then tagging document semantics would be no harder than tagging styles, so this sort-of-works for titles and highlighting.
But hardly any other semantics have associated styles, so tagging them becomes sheer drudgework for almost no payoff. It's absurd to have to tag every name as a name, every place as a place, etc. This metadata belongs in headers, not as embedded tags.
So the real outcome of the XML-scam is that the effort to add metadata to webpages has been set back at least five years. What should have been emphasized was META headers for: Yahoo topic-category, DMoz topic-category, list of persons, list of places, list of companies, list of things, dates discussed, document type (eg timeline, image gallery, biography, etc).
Maybe because its not a closed format, hence all the open-source pdf generation programs.
Frankly, I'd rather see more PDF generation than XML. If I sit down and spend hours designing a book or report it's more important to know that it will appear as designed than that it can be converted into a mass of raw data and presented in any half-arsed way by someone so primative that they still think PowerPoint is a pretty good idea.
TWW
"Encyclopedia" is to "Wikipedia" what "Library" is to "Some people at a bus stop"