Extended TeX: Past, Present, and Future

First time accepted submitter Hamburg writes "Frank Mittelbach, member of the LaTeX Project and LaTeX3 developer, reviews significant issues of TeX raised already 20 years ago. Today he evaluates which issues are solved, and which still remain open and why. Examples of issues are managing consecutive hyphens, rivers of vertical spaces and identical words across lines, grid-based design, weighed hyphenation points, and overcoming the the mouth/stomach separation. Modern engines such as pdfTeX, XeTeX and LuaTeX are considered with regard to solutions of important problems in typesetting." Note: When TeX was first released, Jimmy Carter was president.

Comments

TeX for Math

[rrhal]

When TeX was new people were not accustomed to seeing well type set documents unless they came from a legitimate publisher. I wrote several college papers in TeX and I think the presentation let me get a few mistakes past my teachers. I've not seen anything better for formulas - even today TeX documents have a more polished feel to them.

Re:TeX for Math

[ModernGeek]

HTML 5.1 needs a TeX tag... I'd do anything to see it. What's stopping it from happening? Someone should fork WebKit and do it.

Re:TeX for Math

[fuzzyfuzzyfungus]

Presumably imply the existence of a feature-complete TeX implementation in the browser, that would render and display whatever TeX snippet was included within the tag...

I'm pretty sure that there are some server-side convenience plugins for at least a few of the major OSS CMS packages that will let you use TeX or LaTeX and then digest the results into images that get plunked into the actual HTML that gets shoveled out to clients; but the odds of coaxing browser makers to include a completely separate, extremely powerful, and highly mature(if baroque) rendering engine alongside the one they already have, just to support a TeX tag seem slim...

Re:TeX for Math

[PopeRatzo]

Way back when my wife was getting her PhD math, I had to learn Tex to help her. She let me bounce my head against Tex while she was busy with her fluid dynamics and then I'd get to explain it to her. I could never understand how there are mathematicians who can easily write in Tex the way I'd write in a word processor. It all just seemed so opaque. De-bugging errors was among the most frustrating things I've ever done on a computer. But the results are impeccable. I still don't think there is a better program for typesetting equations. Or I should say, I don't know of a better one. But if there is one, can someone kindly tell me?

Re:TeX for Math

[K. S. Kyosuke]

There's always TeXLive.js, if you actually need full (La)TeX environment in your browser.

Re:TeX for Math

[jd]

Whereas now, people are still not accustomed to seeing correctly typeset documents and are now completely used to vast numbers of typos, malformed web pages, poor indexing via the semantic web, gratuitous XML, excessively long style sheets, browser incompatibilities, Javascript...

Re:TeX for Math

[semi-extrinsic]

Modern LaTeX is quite a bit better at giving good error messages, unless you try to do very complex things. Combining lots of packages for heavy customization is the prime example; I once spent half a day setting up custom chapter titles with a side-by-side miniature table-of-contents and epigraph below a title where the chapter number extruded into the margin, and drop caps at the start of the first paragraph. But the end result was beautiful.

Re:TeX for Math

[jd]

Well, with WebKit up the proverbial creek these days, a new rendering engine would make sense.

The question would be whether you could create a TeX-alike engine that supports the additional functions required in HTML and can convert any well-formed SGML document into a TeX-alike document. If you could, you can have one rendering engine and subsume HTML and XML entirely within it.

The benefits of doing this? The big drawback of style sheets is that no two browsers agree on units. TeX has very well-defined units that are already widely used. These also happen to be the units industry likes using. Eliminating browser-specific style sheets would be an incredible benefit.

The big drawback of the Semantic Web is that everyone, their brother, cat and goldfish have designed their own ontologies, none of which interoperate and few of which are any good for searching with SPARQL. LaTeX has a good collection of very standard, very clean methods for binding information together. Because it's standard, you can have a pre-existing ontology libraries which can be auto-populated. And because LaTeX is mostly maintained by uber-minds, rather than Facebook interns during their coffee break, those ontologies are likely to be very, very good. Also, microformats will DIE!!!! BWAHAHAHAHAHAHAHAHA!

The big drawback with HTML 5 is that the W3C can't even decide if the standard is fixed, rolling or a pink pony. TeX is a very solid standard that actually exists.

Ok, what's the downside of TeX? There's no real namespace support, so conflicts between libraries are commonplace. I'm also not keen on having a mixture of tag logic, where some tags have content embedded and others have the content enclosed with an end tag. It's messy. Cleanliness is next to Linuxliness.

Parsing client-side is a mild irritant, but let's face it. AJAX is also parsing client-side, as is Flash, as are cascading style sheets, etc, etc. The client is already doing a lot (one reason nobody has a fast browser any more), so changing from one set of massive overheads to another really wouldn't be that much of a pain.

Ok, so if we consider TeX the underlying system, do we need a TeX tag? No. We would rather assume all parts of a document not enclosed by an SGML tag are TeX. This would be a transitory state, since you could then write SGML-to-TeX modules for Apache, IIS and other popularish web servers. The world would then become wholly TeXified, as it should be.

Re:TeX for Math

[gweihir]

Indeed. We just decided to move to LaTeX again for all documents that customers do not have the right to edit (most of them). The alternative was Word 2010.

Reasons are far better look, far better to edit, no distractions while edition (MS GUIs suck), can be edited on any OS, .eps capability, svn compatible, easy separation of documents into separate files, etc. Took me 3 days of LaTeX hacking to make the style file and templates match the Word Template, but well worth it, as now it is done and will not surprise us all the time like the toy-level MS Word does.

For stuff that customers do edit, we are stuck with MS trash, unfortunately. But even there we are thinking of writing it in LaTeX first and then move it over with latex2rtf for the final version. Far more efficient.

Re:TeX for Math

[Daniel Dvorkin]

I could never understand how there are mathematicians who can easily write in Tex the way I'd write in a word processor.

I can't really explain it other than to say "you get used to it." After a while, the markup becomes transparent; if you're typesetting an equation, for example, you see the layout in your head while you're typing the markup. Which makes it much easier than using an equation editor in a word processor, really--compounded by the fact that equation editors are universally awful.

Re:TeX for Math

[gtall]

I just gave a talk for management. One fellow remarked on the quality of my slides and didn't think they were done using PP. Yep, I said, they are Latex (Beamer), and I can cut and paste from my papers. Using PP for math will make you go blind.

Re:TeX for Math

[michael_cain]

...no distractions while edition (MS GUIs suck)...

At some point there was an internal study at Bell Labs after WYSIWYG word processors were beginning to be available that found most people spent 20% of their time futzing with how the document looked instead of writing. Most of that time was wasted because subsequent changes were going to wipe out whatever the little tweaks had been intended to accomplish.

Interesting that today you can buy programs whose primary purpose is to blank all of your display except for a green-on-black mono-spaced text window. Sold as an aid for professional writers who need to pound out umpteen pages of text per day, so need to avoid interruptions and distractions while composing.

Re:TeX for Math

[Anonymous Coward]

The question would be whether you could create a TeX-alike engine that supports the additional functions required in HTML and can convert any well-formed SGML document into a TeX-alike document. If you could, you can have one rendering engine and subsume HTML and XML entirely within it.

TeX is a document typesetting language. HTML, regardless of its flavor, is a markup language that describes the document's contents but doesn't tell the browser how to lay it out.

There's no hope for TeX-as-HTML ever working because they're built on fundamentally incompatible document models.

-JS

P.S. If you want to see what math for HTML looks like, go look at MathML. Next, try actually writing a non-trivial equation in MathML, something like

$\int_{-\infty}^{\infty} e^{-x^2} dx = \sqrt{\pi}$

Then you'll understand why those of use who do this for a living still write our papers in TeX. Even if you don't know TeX, you can probably guess what my equation should look like; you wouldn't say that about the MathML.

Re:TeX for Math

[sFurbo]

When all your writing is text, the whole point of Tex is lost, and you might as well use word.

No it isn't. Apart from the ease of writing equations, one major point for LaTeX is that you can just write the damn text and you don't have to worry about how it looks. The final result is going to be beautiful. In Word, you can choose the font and size of all levels of headings, the line spacing and the margins, and even when you have spent time doing that, it still doesn't look as good as LaTeX does out of the box.

Re:TeX for Math

[jgrahn]

We just decided to move to LaTeX again for all documents that customers do not have the right to edit (most of them). The alternative was Word 2010. Reasons are [...] svn compatible, [...]

That's a rather laconic way of putting it. Real revision control gives you a whole array of essential things, like collaborative editing, an audit trail, the ability to work on version 2 while version 1 is still being finalized ... That's why *all* binary document formats (not just MS Office) fail my personal test.

Re:Jimmy Carter was president of TeX?!

[fyngyrz]

Yes, but there was that helicopter crash; the typesetting community always blamed TeX for that, and ever since then, TeX has been relegated to doing font work in the third world, charity undertakings and the like. And really, who wanted to be limited to 55 fonts per document? Some of us type in the fast lane, buddy.

Wrong Font For Program Name

[Freshly Exhumed]

Oh dear oh dear, I hope Donald Knuth doesn't see that Slashdot doesn't seem to allow the correct METAFONT for displaying the program name! Pissing off Donald Knuth would be like kicking the Dalai Lama.

Is it possible to represent it in it's proper format via this version of Slashcode?

Re:Wrong Font For Program Name

[mark-t]

Knuth doesn't seem to be too bothered by the fact that his program is rendered in standard ascii characters as 'TeX', rather than how the name is typeset in his books, since his very own home page seems to do likewise.

Re:Wrong Font For Program Name

[martin-boundary]

True, but he uses a special Tektronix terminal with a custom ROM that computes on the fly the optimal TeX rendering of any HTML document while he's browsing it. Please nobody tell him what the rest of us see...

Old tech, and limited

[Okian Warrior]

I just recently reviewed the landscape of document writing systems for a client.

TeX (and LaTeX, and such) are a fine choice for specific purposes. There's a lot of functionality, it's robust and widely used. If you're writing a journal submission paper, it's a good choice.

The publishing landscape has changed. There are now many more types of document (help files, web pages, books, articles, owner's manuals, laws, contracts) that people want to write, and the TeX family is inconvenient for many of them.

XML is a more comprehensive document content specification. It easily covers all of the common document types (including those for which the TeX family is useful) and is extensible in a straightforward manner.

As a specific example, DocBook (a specific XML scheme) covers all cases where TeX is useful, and many more. An XML processing system can convert to any presentation format (HTML, XHTML, PDF, Microsoft Help, Text), and it's straightforward to build converters for new formats.

(There are also other XML schemas.)

The drawback of DocBook and XML in general is that installation is a nightmare. So far, there's no "one package install" that gets the author up and running. XML processing is a series of steps, with each step served by one of several open source packages. The author must choose and install software for each step, usually without any indication which is best for his purposes. This only needs to be done once, though. (For open source - paid software packages have this sorted out.)

(For example, see how long it takes you to install DocBook 5.x on a windows system.)

The TeX family is a good choice, but if you're not already using it consider learning a more recent solution.

Re:Old tech, and limited

[retchdog]

structure is one thing, output is another. show me some docbook-rendered math that doesn't suck.

Re:Old tech, and limited

[jd]

Never had any problem writing books in LaTeX. The main difficulty has been in deciding whether I want a modern or medieval structure.

Docbook, on the other hand, I hated. I helped with the writing of a few chapters of the Linux Advanced Traffic Control book, which was abandoned in part because Docbook was such a disgusting system.

XML is useless for typesetting. It's not really that useful for organizing anything - you'll have used XML-driven databases, but you'll have never used an XML-driven database that had any performance or serious functionality. (LaTeX doesn't do databases, either, but it doesn't pretend to. It has external engines for databases, which are actually quite nice.)

Web pages? Never had any problem embedding HTML in LaTeX. In fact, I have very very rarely found ANY document style to be LaTeX-incompatible. Load up the correct document type, load up the appropriate stylesheets and you're good. Yes, spiral text is hard. Yes, embedding HDR images can be a pain. Yes, alpha blending isn't that hot. But how often do you use any of these for owner's manuals or contracts?

There are more table classes than I'd really like, and some of the style coding is scruffy, but I challenge anyone to find a genuine, common document type that LaTeX* cannot do as well as or better than any non-TeX wordprocessor, DTP solution or XML-based system. (Non-TeX means you can't compare TeX with Scientific Word, TeXmacs or any other engine that uses TeX behind the scenes.)

(To make it absolutely clear, "as well as or better than" can refer to any one or more parameters. So if I get better-quality output, that's better than. If I can achieve comparable results with cleaner, easier-to-maintain syntax, that's also better than. To win, your solution has to not merely equal but actually exceed what I can do on EVERY parameter, or you have failed to demonstrate something that supercedes.)

A bitcoin to anyone who can do this.

*I am including all dialects of LaTeX here, so LuaLaTeX, PDFTeX, etc, are all things I can consider on my side, as are all WYSIWYG and WYSIWYM editors, Metapost, supplemental services, style sheets, etc. Since this is versus a specific alternative, anything comparable for that specific alternative is fair game for you to use, but you can't mix in other alternatives. It has to be one versus the complete TeX family if you want to prove your point.

Re:Old tech, and limited

[jd]

TeX has control elements for describing structure, since structure is a key part of typesetting. Since these elements are macros, they're programmable, although not truly abstract as in XML. About the only thing I can think of that XML can do for document structure that TeX cannot is out-of-order elements, and I'd argue that out-of-order is incompatible with structure.

In database terminology, XML is a key-data pair system. The data can be anywhere in the XML file and you need some sort of key to know where it is and/or when you've found it. (Since XML is not organized, you can't do random access to get at the key. You have to load it in and organize it, in which case it isn't XML, or you have to sequentially search it.)

TeX is a semi-sequential structure, with relationship links between specialized data tables. Again in database terms, it's a set of batch sequential files with crude but useful support for concrete data association. Because it's batch sequential, real-time usage gets hairy. Big deal. Those in the middle of writing should be concerned with the writing. It would be nice if editors had better error-detection, but it's not usually that critical.

so old it must be replaced...

[smoothnorman]

nothing that old can possibly be relevant anymore (cf "trust no one over 30"). it should be replaced with something more responsive to a one-hand touch interface abbrev friendly imho. math, a central theme in TeX, no longer has any relevance to the modern world (just ask any millionaire agile scrum extreme php programmer). any remaining bits of math are done entirely by app; the vestigial remnants of the usefulness of "math" can only be found in the interjections of animated characters. only a tiny ancient dying breed of tenured academics (and i suspect *europeans*) would ever seek typography beyond the standards of MSWord. page layout was forever perfected by expensive per seat layout software around 1996 and requires no more changes. markup languages are too hard to learn. anything that requires a compiling phase has gone the way of C++. the world is better now as everything old and outmoded quickly recedes. sine-die.

What about pictures?

[TechyImmigrant]

TFA doesn't address the extreme crapitude of embedding pictures in Tex. I could drive myself to drink converting everything to EPS, or poorly scalable bitmaps before embedding them, but I don't want to.

The issues of imperfect typesetting are not the barrier to entry for potential TeX users. Picture embedding is.

Re:What about pictures?

[archshade]

I know you are trying to be funny, but searching for ``latex images'' on google the first page is all tutorials on how to insert an image into a Latex document. The third link was a link to a google image search wich did have the kind of thing you are implying. still 9/10 relevant results is not bad.

Re:TeX for Otherstuff

[xclr8r]

Couple of good reads regarding LaTeX. 50 Shades of LaTeX: The Pain the Pleasure http://airminded.org/2005/11/18/latex-the-pain-the-pleasure/ http://crookedtimber.org/2005/04/27/fetishizing-the-text/

Re:The last command-line word processor

[PhamNguyen]

I agree, the need to compile is a big time sink. Hunting for a missing brace or dollar is just horrible. I and many people I know (all long time users of LaTeX) switched to using LyX and only exporting to LaTeX for the final formatting (e.g. using a journal's style guide). Unfortunately there is no quick fix for LaTeX: the power of the language means that gui's like LyX can only deal with a subset of the language, and yet this power is necessary in order to allow for all the packages that LaTeX supports (and especially to support existing packages).

mixed feelings

[ssam]

while i have written a thesis in latex, and could not imagine using anything else for papers, i still get frustrated by it.
* any problem is solvable with enough searching online, but the solutions are often like magic. for example i often have figure filenames like "x2.3_y3.4.pdf" latex gives a weird message, search around and eventually you find a forum thread that tells you to put some extra arguments in the includegraphics call, or if you are lucky you might find a mention of the grffile package. in all the years of using it I have never built up an intuition for solving these issue (by comparison programming and linux pretty much make sense to me).
* multiple ways of doing things. should i use \begin{center} or {\centering text text tex}. probably they both work fine, but each of them breaks something else in some obscure case.
* why are some things \command{text text text}, some {\command text text text} and some \begin{command}. compare with XML/SGML where everything is achieved with nesting tags.
* can the output be cleaned up? when i run pdflatex i get several screen-fulls of messages. really it should be showing me errors and optionally warnings.
* the interactive mode when it hits an error. i am sure there is nothing productive i can do in that shell. why is it so hard to get out of. why is -halt-on-error not default?
* why do i have to run pdflatex twice? why can't it figure out if a reference has changed? latexmk (or a good makefile) helps, but it took me years to find it.

Re:The last command-line word processor

[dfghjk]

"TeX still had a compile-run-debug workflow, and without a graphic display, you had to run a hard copy on something like an electrostatic printer or a daisy wheel printer to check the results."

When TeX was new, "a graphic display" was common and there were many previewers available pre-1.0. The idea that anyone previewed work on a daisy wheel printer is absolutely ludicrous. Never happened and would be useless.

"Then you could go to the phototypesetting machine."

No, you would go to screen previewer, then a laser printer, and then only to a phototypesetter if you were publishing. You sound like someone who didn't use TeX in those days.

"Once everybody got an interactive display good enough to view the output of TeX..."

You mean like a PC in 1985? Seriously, you pretend to be a historian but you aren't one. I, on the other hand, cowrote one of the first PC TeX previewers, in...1985. I am actually familiar with how these tools were used then, and it's clear you weren't a TeX user. As an Interleaf user, it seems you were the type privileged by limitless company money. Not many even had access to a machine capable of running Interleaf in those days. In my next job I worked with someone who was an Interleaf fan and who had the clout to get the company to buy him, and only him, a seat. He liked it, no one else used it or really even got to see its output, and TeX worked well at zero cost.

Many consider "compile-run-debug" to be an advantage but perhaps not since it became trendy to call that a "workflow". What-You-See-Is-All-You-Get.

Re:hah

[Genda]

Pretty much just incidental. Reagan's interest in "High Tech" was mostly limited to military applications (like Star Wars) which turned out to be huge financial boondoggles. He killed financial support for solar and other alternative energy technologies and put James Watt in charge of the environment (a man who believed we should use up the environment as soon as possible to hasten the second coming.)

By comparison, Eisenhower set the economic stage for Bell Labs, IBM, Dupont and DOW Chemicals and a brand new government space program named NASA. The JFK expanded all of these things dramatically including the mandate for a man on the moon. Clinton was the seminal power behind America's global advance in internet technologies and in 2000 we were leading the world. With Bush's cutting of support of the internet and redirecting economic focus on fossil fuel, war and housing, America has fallen behind Asia and even Europe. So Ronny may have presided over the 1980s expansion of high tech it would be very hard to claim he was a friend to advancing technology.