The Rise of Machine-Written Journalism

Hugh Pickens writes "Peter Kirwan has an interesting article in Wired UK on the emergence of software that automates the collection, evaluation, and even reporting of news events. Thomson Reuters, the world's largest news agency, has started moving down this path, courtesy of an intriguing product with the nondescript name NewsScope, a machine-readable news service designed for financial institutions that make their money from automated, event-driven trading. The latest iteration of NewsScope 'scans and automatically extracts critical pieces of information' from US corporate press releases, eliminating the 'manual processes' that have traditionally kept so many financial journalists in gainful employment. At Northwestern University, a group of computer science and journalism students have developed a program called Stats Monkey that uses statistical data to generate news reports on baseball games. Stats Monkey identifies the players who change the course of games, alongside specific turning points in the action. The rest of the process involves on-the-fly assembly of templated 'narrative arcs' to describe the action in a format recognizable as a news story. 'No doubt Kurt Cagle, editor of XMLToday.org, was engaging in a bit of provocation when he recently suggested that an intelligent agent might win a Pulitzer Prize by 2030,' writes Kirwin. 'Of course, it won't be the software that takes home the prize: it'll be the programmers who wrote the code in the first place, something that Joseph Pultizer could never have anticipated.'"

Comments

nonsense

[symes]

Well-written prose is far from formulaic. While financial institutions and baseball enthusiasts may happily forego a penetrating understanding of a situations meaning and emotions the literate will not.

Re:nonsense

[Anonymous Coward]

Just wait until they even try to automate spin!

enum bias {
          ULTRA_LIBERAL,
          FAIRLY_LIBERAL,
          JUST_LEFT_OF_CENTER,
          JUST_RIGHT_OF_CENTER,
          FAIRLY_CONSERVATIVE,
          ULTRA_CONSERVATIVE,
          RUPERT_MURDOCH

};

At least there's now an automated process for it.

Re:nonsense

[cgenman]

This strikes me as the sort of thing that, without excessive amounts of variation, would get filtered out quickly by the general public. Sure, a machine can write *one* story on a baseball game that is interesting to read. But after the hundredth version of the same story that you've read, the public would stop reading the text entirely and just filter for the important bits. At that point, you might as well just have a table with the interesting stats.

The challenge isn't to write one story. It's to create a machine that can write N stories that remain interesting and fresh, and with less effort and cost than it would take journalists to just write N stories the traditional way.

The reporter is now a touch more obsolete

[onyxruby]

News agencies have already been turned into commodities, they just don't realize it yet. Now the reporter is being sent down that same drain. With original reporting set to become a 'premium' by the news agencies, their market is only shrinking.

Where were the reporters when millions of jobs were outsourced by H1B's or sent overseas? At best most stories were brief, with no follow up, and no outrage at the loss of middle class America. The same thing has happened in Europe and elsewhere as well.

Now the reporter faces the inevitable market forces that they previously ignored, and they expect anyone left to care? The programs will only get better, the markets and stories it applies to will only improve, and for the vast majority of stories the quality will be imperceivable to the average person.

Mod parent up.

[khasim]

This is nothing more than extracting stats and then placing them in pre-generated sentences.

In sports, this is okay. Except when something interesting happens like someone head-butting another player.

Anyone want to place a bet on how long before companies are accused of "gaming" the financial reporting system with their press releases?

Re:Mod parent up.

[PCM2]

This is nothing more than extracting stats and then placing them in pre-generated sentences.

In sports, this is okay.

Countless generations of sports writers and the enthusiasts who read their work would disagree with you.

Re:Mod parent up.

[lgw]

The purpose of stocks is for a company to raise capital by selling shares to the public, in return for some promise of potential future dividends.

The fact that people trade, hold, or speculate on these stocks in the secondary market (all the busy noise that is the "stock market") is nearly irrelevent. A company should have no reason to care about the price of its stock. Sadly, due to double-taxation of dividends, this has gone completely to shit. People who aren't speculating buy stock not for dividends, but to trade it to the next guy at a profit, because this is tax-favorable over dividends.

Companies do all sorts of crazy BS because of the expectation of stockholders to be rewarded not with dividends (a system that you simply can't game for long) but with rising stock prices (a system that is almost entirely gamesmanship).

Individual investors have the absolute right to seek short term gains or long term gains a their preference. The government has no business meddling in that preference. The "problem with the current market" is that it is no longer grounded in the reality of being able to pay dividends, because some previous generation's ideas about social engineering through taxation punishes dividends as the means of earning a profit on one's investment.

Re:What?!

[natehoy]

I'm pretty convinced most corporate press releases are machine-generated anyway, so it should be a matter of reverse-compiling them back into plain English and including that as part of a story.

ABC Co. CEO to PressBot: "The market totally screwed us. The building is collapsing because we can't afford maintenance. We have to lay everyone off and we'll be out of business in three months. We can't afford exterminators so weasels are chewing my genitalia into mush."

PressBot's press release: "The company continues to leverage circular market forces to tighten its bottom line, particularly in the area of vertical integration. Resources are plentiful enough, however, that all employees will be allowed to pursue innovative new ideas in an open, creative setting, with plenty of personal time. The CEO is actively involved in the belt-tightening process and has taken steps to ensure that only underutilized corporate assets will be liquidated."

A human or a webbot could probably gather equally-useful information out of it.

Completely automated market crash!

[RyanFenton]

We've completed the circle - various "automated systems" have been blamed for various market failures in recent years, as companies and small traders have used algorithms on computers to "keep up with the speed of the market". Of course, the actual failure was almost always in the design, such as allowing a computer to make blind decisions with large amounts of money faster than you could keep track of.

But here, we have a stronger case for a machine-driven market failure - automated news algorithms. Misunderstanding generated at the speed of the market. I've worked on AI professionally in games, studied it in the contexts of linguistics, nervous system simulation, and such - AI even in its most exaggerated modern state is not going to even know how to figure out how to extract a good quote with human guidance, much less report on a news release. If you thought computer generated music was entertainingly bad - wait until you see some of the awful things produced by automated news misunderstandings... random context switches mixed with "neutral language" bits, it'll be like Fox news switched its agenda to Cthulu-level madness of confusion rather than the usual rage agenda.

And since the market makes its decisions on the basis of news, rumors, and insider trading - and people get the three confused as they hear them, mixing this into the information stream seems a virtual guarantee of another market crash.

That's what I call another serious negative externality for the news business taking the cheaper road to reporting business news.

Ryan Fenton

I for one...

[azgard]

...look forward to our meme-ending overlords.

For some definitions of journalism

[SgtChaireBourne]

'scans and automatically extracts critical pieces of information' from US corporate press releases, eliminating the 'manual processes' that have traditionally kept so many financial journalists in gainful employment.

That, I must admit, is an excruciatingly lame definition of 'journalism'.