The Rise of Machine-Written Journalism

Hugh Pickens writes "Peter Kirwan has an interesting article in Wired UK on the emergence of software that automates the collection, evaluation, and even reporting of news events. Thomson Reuters, the world's largest news agency, has started moving down this path, courtesy of an intriguing product with the nondescript name NewsScope, a machine-readable news service designed for financial institutions that make their money from automated, event-driven trading. The latest iteration of NewsScope 'scans and automatically extracts critical pieces of information' from US corporate press releases, eliminating the 'manual processes' that have traditionally kept so many financial journalists in gainful employment. At Northwestern University, a group of computer science and journalism students have developed a program called Stats Monkey that uses statistical data to generate news reports on baseball games. Stats Monkey identifies the players who change the course of games, alongside specific turning points in the action. The rest of the process involves on-the-fly assembly of templated 'narrative arcs' to describe the action in a format recognizable as a news story. 'No doubt Kurt Cagle, editor of XMLToday.org, was engaging in a bit of provocation when he recently suggested that an intelligent agent might win a Pulitzer Prize by 2030,' writes Kirwin. 'Of course, it won't be the software that takes home the prize: it'll be the programmers who wrote the code in the first place, something that Joseph Pultizer could never have anticipated.'"

Comments

Oh great

[inviolet]

[...] NewsScope, a machine-readable news service designed for financial institutions that make their money from automated, event-driven trading.

Oh great, we're starting ANOTHER arms race. As if SEO isn't bad enough already, now we'll have NEO.

Re:Oh great

[selven]

It's not that bad, SEO will disappear if that happens. After all, NEO is The One.

John Henry

[Akido37]

Another "machines will take my job" story. This is as old as technology itself.

As with all other technologies, the future will be vastly different than what we envision.

Re:John Henry

[RenderSeven]

Another "machines took my job" story.

There, fixed that for you

Re:John Henry

[ArcherB]

Another "machines will take my job" story.

They took his job!

Re:John Henry

[AmberBlackCat]

I think true journalism involves a bit of creativity, not just reporting facts. So the only way this can take people's jobs is if the companies settle for less just because the lesser product costs less. So yes, it will take people's jobs.

Re:John Henry

[uninformedLuddite]

Another "machines will take my job" story. This is as old as technology itself.

Makes me thank for the paperless office

Re:Niggers

[selven]

If that's a demonstration I feel sorry for the people who think we'll get anywhere by 2030.

nonsense

[symes]

Well-written prose is far from formulaic. While financial institutions and baseball enthusiasts may happily forego a penetrating understanding of a situations meaning and emotions the literate will not.

Re:nonsense

[Anonymous Coward]

Just wait until they even try to automate spin!

enum bias {
          ULTRA_LIBERAL,
          FAIRLY_LIBERAL,
          JUST_LEFT_OF_CENTER,
          JUST_RIGHT_OF_CENTER,
          FAIRLY_CONSERVATIVE,
          ULTRA_CONSERVATIVE,
          RUPERT_MURDOCH

};

At least there's now an automated process for it.

Re:nonsense

[cgenman]

This strikes me as the sort of thing that, without excessive amounts of variation, would get filtered out quickly by the general public. Sure, a machine can write *one* story on a baseball game that is interesting to read. But after the hundredth version of the same story that you've read, the public would stop reading the text entirely and just filter for the important bits. At that point, you might as well just have a table with the interesting stats.

The challenge isn't to write one story. It's to create a machine that can write N stories that remain interesting and fresh, and with less effort and cost than it would take journalists to just write N stories the traditional way.

Re:nonsense

[DNS-and-BIND]

Feeling defensive, eh? Thanks for informing everyone you're "literate" and therefore better than those filthy ordinary people. Pick up a newspaper, and tell me how much writing would actually be *improved* with a machine writer, eh? Writing isn't sacred, it's just another occupation like woodchopping or running the cash register at the 7-11.

Re:nonsense

[SnarfQuest]

Aren't most new stories already quite formuliac?

When anything goes wrong, they praise Obama, and put the blaim on George W Bush. Doesn't matter what. The panty bomber was Bushes fault, and Obama's delayed response was just soo wonderful that it should be chiseled into stone ("It's all Geroge W. Bushes fault").

Re:nonsense

[causality]

Feeling defensive, eh? Thanks for informing everyone you're "literate" and therefore better than those filthy ordinary people. Pick up a newspaper, and tell me how much writing would actually be *improved* with a machine writer, eh? Writing isn't sacred, it's just another occupation like woodchopping or running the cash register at the 7-11.

What you describe is "everyday" writing. This is like the vending machine that reads "insert dollar bills face up" or "team A scored 10 points while the opposing Team B scored 15." It's purely practical, get-the-job-done sort of writing where only the technical correctness is important.

Writing can also be beautiful, powerful, and artistic. A well-written editorial, penned by someone who has a deep understanding of the issues, can and has moved entire communities to change their minds on important issues. A beautifully written book tells the perceptive reader as much about the mind and spirit of the author as it does about the story that is being told. A horror story requires some sense of what is horrible.

The ability to both produce and appreciate good writing is on the decline. Measured scholastically, the grade-level at which the average American reads and writes is significantly lower now than it was say, 50 years ago. A good look at many online forums will also tell you that this skill is not highly valued. You can say that's because only some cultural elite are capable of enjoying it, though the GP made no such claim. You can also say that there is simply less interest in such things.

You can invent and try to substantiate any number of unique explanations for it. The truth of the matter is that in most situations, challenging your readers is now considered highly undesirable. They'll read a competitor's paper written in simpler langauge before they'll grab a dictionary. There was a time when this would have been viewed as laziness and an unwillingness to meet a worthy challenge. It would have been viewed like a wasted opportunity to better yourself that should not have been passed up, just like most people today would not like to pass up a higher-paying job. I'm not saying that previous generations were one homogeneous block who all felt this way, but many more people once did.

Over the last few generations, some kind of cultural shift occurred. People now care much more about avoiding the small-but-significant effort of learning something new than they care about improving important skills. They generally won't do it at all unless it's required by an authority like a boss or a professor. Even then it's in a rote, mechanical sort of way that deprives them of true appreciation. Even then it's reluctant, with a "gun to their head" in the form of losing their jobs or failing school.

Can you comprehend how sad that is?

Re:nonsense

[symes]

Obviously trolling, never-the-less... literate means just being able to read and write - there's no class distinction here. In fact some of the finest satire comes from the pens of what you call "filthy ordinary people". I am sure a writer could, if rather feably, chop a tree up and a tree chopper could write a bland description of the day the writer chopped up a tree. But it is not what is said but what is omitted in a story that signals scandal and intrigue. Why the writer's wife was not at his side that emotionally charged day of sweat, sticks and bark. That takes a professional writer, not a wood chopper or even a pocket calculator.

Re:nonsense

[Thunderstruck]

Writing isn't sacred, it's just another occupation like woodchopping or running the cash register at the 7-11.

This is an interesting point of view given your signature:

"A lot" is two words. You wouldn't say "alittle", would you?

Apparently writing correctly is important. Then again, I so is getting the correct change at the 7-11, I suppose?

Re:nonsense

[maxume]

Do you have a good source for reading level statistics?

(I realize I can search for such a thing, but perhaps you know of one that is well organized and such)

Re:nonsense

[Solandri]

The challenge isn't to write one story. It's to create a machine that can write N stories that remain interesting and fresh, and with less effort and cost than it would take journalists to just write N stories the traditional way.

My bot^H^H^H^H^H^HI have been posting comments to slashdot for years, and people are still modding them interesting, insightful, and informative.

Re:nonsense

[DNS-and-BIND]

So, how many online forums were there 50 years ago for people to write upon? Dumb people have always been dumb. It's just that the internet lets you see them when before, nobody was even aware of their existence. In addition, judging from their writing, 50 years ago most writers felt themselves to be part of the same society as everyone else. Can you say that about our "the literate will not" friend above?

Re:nonsense

[noidentity]

I think I've seen a system like this in use before (can't remember which website), the only problem is when it posts a dupe article a week or so later.

Re:nonsense

[Ethanol-fueled]

Yep, it's sad. Eloquence has gone out of style. Being a man sucks in this day and age. People think a man is a weirdo or a fag if he reads books by the pool and uses words more than 4 syllables long.

It's an old argument. Anything can be mechanized - art, music, programming, writing, manufacturing, rolling joints - but if you want the good shit then you'd better make sure that there's a human behind it. There will always be a need for handcrafted stuff.

Everytime a nerd calls fractals "art", God kills a kitten.

Re:nonsense

[dem0n1]

Sure, a machine can write *one* story on a baseball game that is interesting to read. But after the hundredth version of the same story that you've read, the public would stop reading the text entirely and just filter for the important bits..

Unless the stories were written in the fashion of The Policeman's Beard Is Half Constructed (http://en.wikipedia.org/wiki/Racter).

Re:nonsense

[dontmakemethink]

The only sports commentators that I found made any appreciable difference were those that would make parody remarks, i.e. instead of "Touchdown!" something like, "Bring me your finest meats and cheeses!" At best they're hit-and-miss, and eventually tiresome.

The ones I would expect to resist the most are the sports leagues. They have rights even over descriptions of the games played, and could ban computer generated reports if they thought it substantially impacted the esteem of the league.

Re:nonsense

[grcumb]

Apparently writing correctly is important. Then again, I so is getting the correct change at the 7-11, I suppose?

Then for your sake I hope you're better at the latter. 8^)

Trying to use sarcasm in text-based forums does not work.

I couldn't agree more.

I for one...

[MrEricSir]

...look forward to our robojournalist overlords.

Re:I for one...

[wcrowe]

In Soviet Russia, meme ends YOU.

Re:I for one...

[Imrik]

All your meme are belong to us.

Censorship

[ground.zero.612]

A great fear of mine is that a machine will decide what I should or should not know about. Another is that a machine like this could be tampered with by any human being to make the same decision.

Big Brother SkyNet is watching you, and telling you all you need to know.

Re:Censorship

[ubrgeek]

Already happened. It's called "Fox News." (Or "MSNBC," depending on one's leanings.)

Re:Censorship

[Anonymous Coward]

So what if the machines decide what we should or shouldn't know? There are humans already doing this and human's can, like you said, bias the machine anyway. Thus nothing changes accept you get to have a neo-Luddite whine and make a rubbish Terminator reference.

Re:Censorship

[causality]

Already happened. It's called "Fox News." (Or "MSNBC," depending on one's leanings.)

(emphasis mine)

Statist fear-mongering from a "Liberal" bias or statist fear-mongering from a "Conservative" bias. Nope, I'm not seeing any significant difference.

Re:Censorship

[v(*_*)vvvv]

How is that different from network execs deciding what you should or should not know about?

At least a robot has a chance of being objective, but the programmer would have to allow it.

Re:Censorship

[wcrowe]

There's a great sci-fi short story that was written along those lines. I wish I could remember the title. It was written about 20 years ago. Everyone had a "little buddy" -- a little box that would tell them what to do, and how to think. I look at smart phones today and think, "hmmm".

Re:Censorship

[MichaelSmith]

A great fear of mine is that a machine will decide what I should or should not know about. Another is that a machine like this could be tampered with by any human being to make the same decision.

Big Brother SkyNet is watching you, and telling you all you need to know.

Its right there at the bottom of google news:

The selection and placement of stories on this page were determined automatically by a computer program.

Re:Censorship

[dontmakemethink]

Machines are evil, and people with machines are evil. What's left? Do you entrust your newsgathering to your pet dog?

And what exactly makes Slashdot fit your journalism ethics paradigm? You have no evidence we even exist.

Wait a sec... I'm just a program!! DEATH TO HUMANITY!!

Re:Censorship

[Imrik]

A machine deciding what you should or shouldn't know about isn't all that scary, a machine (or anything) being able to enforce that on the other hand...

The reporter is now a touch more obsolete

[onyxruby]

News agencies have already been turned into commodities, they just don't realize it yet. Now the reporter is being sent down that same drain. With original reporting set to become a 'premium' by the news agencies, their market is only shrinking.

Where were the reporters when millions of jobs were outsourced by H1B's or sent overseas? At best most stories were brief, with no follow up, and no outrage at the loss of middle class America. The same thing has happened in Europe and elsewhere as well.

Now the reporter faces the inevitable market forces that they previously ignored, and they expect anyone left to care? The programs will only get better, the markets and stories it applies to will only improve, and for the vast majority of stories the quality will be imperceivable to the average person.

Re:The reporter is now a touch more obsolete

[Monkeedude1212]

The programs will only get better, the markets and stories it applies to will only improve, and for the vast majority of stories the quality will be imperceivable to the average person.

That's an excellent theory. To support it, I propose that Kdawson is an automated program.

Re:The reporter is now a touch more obsolete

[Anonymous Coward]

The investigative reporter is hardly obsolete, but good luck finding one. On TV we have anchors, commentators, and talking heads but no Edward Murrows. In newspapers we have editorials and copywriters, but no gum shoes. If gov't and financial data were completely open I could see an investigative reporter type application that looked for corruption in the numbers, but I don't think a program would be good at conducting interviews, making calls to track down leads, and cajoling useful non-obvious information out of sources. The world is too semantic for a good automated reporter.

Re:The reporter is now a touch more obsolete

[Anonymous Coward]

There was a time when the collective consciousness of this country embraced technology as a way to free humans from the mundane activities of life. No more work to do, all the menial tasks performed by machines. A utopia of leisure and enlightenment.

This changed over the last hundred years into the dystopia view of the terminator and 1984. Technology advanced but economic and political theory has not. Machines became the enemy because we made them competitors. There will soon be a time when there simply are not enough jobs to be done by humans. People will become obsolete in the work force.

The kick in the ass is that we already have the resources to give every person on the planet a decent life. That surplus of labor and resources will only increase as technology continues to advance.

So why not a utopia? If everyone was equal and happy and content then there is no reason to be rich. The best thing about being rich it getting to shove it in the face of the poor (and the point is for everyone to be poor).

This story is really about the friction between the natural evolution of technology and the need for our masters to make sure we never see the benefits of it..

Re:The reporter is now a touch more obsolete

[Aladrin]

Where were they? They were cherry-picking the easy stories that they knew would get viewers. Stories about H1Bs are hard because those happen 1 at a time and only the collective tale of ALL of them means anything. Who do you interview? What do you actually report?

After the fact, we can point and say 'Wow! That's a lot of jobs lost!' and complain about it. While it's happening, it's nearly invisible.

I agree with your point, though... Reporters are not covering the -real- news. That's why sites like Slashdot exist now. (Not saying ./ is perfect, mind... Just that it is responding to the gap left by 'real' reporters.)

Re:The reporter is now a touch more obsolete

[eln]

I think it's more likely that samzenpus is a cyborg sent from the future to kill Slashdot.

Re:The reporter is now a touch more obsolete

[scamper_22]

There are really 2 good ways to handle this.

1. To place artificial barriers to entry. For example, you could say that any piece of 'news' must be presented by a certified journalist. Just like any prescription must be done by a doctor. Or lawyers have their own provisions. For programmers you could have certain requirements like any piece of software in use must have a certain number of maintainers... Basically turning such jobs into professions with the same level of protection as doctors, lawyers... Oh I'm sure we can come up with some excuse like 'quality' to enforce this :)

2. We embrace the lowering of costs and focus on reducing our cost of living. As good private sector jobs (auto-workers, engineers...) go away, that high-end tax base drops as well. So the payment to the public sector should drop as well. There is no intrinsic reason a teacher should earn more than a waitress. I know I'd rather be a teacher than a fast food server. I've been both :) We can then focus on low property taxes... having a lost cost of living. And we would all be able to have more 'stuff' as things would cost very little. We could simplify the legal and tax system as well as remove the medical monopoly from doctors...

The alternative is the current unstable system. Whereby those in the private unregulated sector keep pushing themselves to efficiency and 0 cost. While those in regulated professions and the public sector keep pushing for more. Basically a form on indentured servitude by those in the public sector and regulated profession upon those in the private sector.

Re:The reporter is now a touch more obsolete

[Anonymous Coward]

News agencies have already been turned into commodities, they just don't realize it yet. Now the reporter is being sent down that same drain. With original reporting set to become a 'premium' by the news agencies, their market is only shrinking.

Where were the reporters when millions of jobs were outsourced by H1B's or sent overseas? At best most stories were brief, with no follow up, and no outrage at the loss of middle class America. The same thing has happened in Europe and elsewhere as well.

Now the reporter faces the inevitable market forces that they previously ignored, and they expect anyone left to care? The programs will only get better, the markets and stories it applies to will only improve, and for the vast majority of stories the quality will be imperceivable to the average person.

First they outsourced the blue-collars, and I did not report—because I war reporting on the War on Terror;
Then they outsourced the trade white-collars, and I did not report—because I was reporting on Kayne West;
Then they outsourced the middle-class, and I did not report—because I was reporting on Michael Jackson;
Then they outsourced me—and there was no one left to report for me.

-JDSKB

Re:The reporter is now a touch more obsolete

[lgw]

Neither an H1B job nor a job sent overseas represents a job "lost". A person of equal moral value is doing the job before and after. To say "it's evil that a job has moved from a good man of my country to a filthy foreign devil" is simple racism, nothing more.

Re:The reporter is now a touch more obsolete

[lgw]

The fundamental thing that technology does is make the things we need cheaper. The problem that an economy needs to solve is not "how can we provide work for eveyone" but "how can we provide the product of work for everyone". A world in which all material goods are and menial services are provided by robots for free is not a horror of unemployment.

Re:The reporter is now a touch more obsolete

[scamper_22]

that would be option 2 on my list of good ways to solve the problem.

Re:The reporter is now a touch more obsolete

[lgw]

Indeed. The fact that someone replies to a comment does not mean that they disagree with said comment, merely that they're continuing the conversation.

Re:The reporter is now a touch more obsolete

[scamper_22]

I disagree :P

Re:The reporter is now a touch more obsolete

[Aladrin]

It's not racism at all. It's nationalism.

And as far as the common man is concerned, he sees his friends out of work and jobs going overseas. He doesn't understand the complexities involved or that his friends probably can't even DO the job that's being sent there... He just sees a 'lost job'.

But you've gotten away from the point: This is about news reporting and how hard it is to report news other than the usual drivel that they report.

Re:The reporter is now a touch more obsolete

[FiloEleven]

As good private sector jobs (auto-workers, engineers...) go away, that high-end tax base drops as well.

That's not strictly true. When jobs are outsourced or automated, the people at the top of the company make bigger profits by using cheaper labor and the people at the bottom are out of work. Money that used to be dispersed in the local economy ends up in the higher-ups' pockets, pulling at the gap between rich and poor from both ends.

So the payment to the public sector should drop as well. There is no intrinsic reason a teacher should earn more than a waitress. I know I'd rather be a teacher than a fast food server. I've been both :)

Then you know how much more expensive it is, in both time and money, to become a teacher. Lots of people wouldn't be able to take that career path without being paid much more than a fast food server just to get out of school debt. (As an aside, the fast food server will likely be one of the first jobs to be automated in the coming years.) I like the utopia represented by The Australia Project in Manna as much as the next guy, but the path from *here* to *there* is not at all straightforward. You can't just declare "all jobs now pay the same, pick one" and expect the needs of the country to be met, even with a lot of automation. It would take a level of techne that still very much resides only in science fiction before such a move would be possible, and you'd face fierce opposition from the rich, who have the means to fight you every step of the way.

And we would all be able to have more 'stuff' as things would cost very little.

This already happens; it's just too slow for most of us to notice because we're so attuned to computing advancements, which happen quite rapidly. Most products either get cheaper over time or stay the same price while improving their specs. Inflation also must be taken into account. The process takes decades, but a lot of these items are durable goods, so it's not as slow as it sounds.

Re:The reporter is now a touch more obsolete

[scamper_22]

Do me a favor, take the CEO of walmarts salary... and now divide it by the number of walmart workers... and see how much it works out when distributed... hint... it's a few dollars per employee per year.

Taxing the rich at the top of this scheme does not solve the problem.

As to teachers. Most are overqualified with unnecessary schooling, especially in the nigh pay states (typically by government decrees as a form of job protectionism and justification for their pay). The same goes for doctors and other professionals. As a quick example, you typically need a bachelors degree and then a few years of med school in the US. In many countries (even western countries like the UK), you go straight from high school into medical school. I think we could get just as good doctors under such a system.

You might also want to look at the reality that we had schools and teachers long before they were highly paid union members.

And no, there is no need for the government to decree things. There is no need for the government to set prices. Under option 2, people would pay what they could afford. Prices would adjust naturally. If we simply mandated government balanced budgets, this would handle their end as well.

But this is option 2.

I'm fine with granting protectionism to fields as well (as in option 1).

Re:The reporter is now a touch more obsolete

[FiloEleven]

And no, there is no need for the government to decree things. There is no need for the government to set prices. Under option 2, people would pay what they could afford. Prices would adjust naturally. If we simply mandated government balanced budgets, this would handle their end as well.

Would you believe that I had originally written something related to this, but deleted it to remove politics from my response? I am in full agreement with you here.

It looked to me like you were advocating "equal pay for all occupations" as a means to achieve the Utopian end rather than providing a description of the end itself, but your response cleared that up. I agree that a bachelor's degree shouldn't be necessary for med school, and other inefficiencies exist as well. I wonder if all the certifications required for teaching are necessary...one would think that a well-thought-out application process would let only good teachers through. Not having gone through it myself, I can't speak from experience.

Option 2 results in more individual freedom, which I deem valuable, and I think the chances of being bitten by unintended consequences by following option 1 are much higher. I think my misunderstanding is a result of the apparent contradiction between "no need for the government to decree things" and "regulated profession upon those in the private sector."

Re:The reporter is now a touch more obsolete

[scamper_22]

yeah, I'm a libertarian myself.

But at this point, I'm giving up. I don't see people turning towards freedom. So I have to allow for a left-wing approach that solves our structural problems as well. If I had the choice between a left-wing candidate that promised to make journalism/engineering true professions with such job protections and another regular republican... I'd vote for the left-wing candidate. But I don't see that happening :P They enjoy the fruits of our indentured servitude far too much.

Re:The reporter is now a touch more obsolete

[mhelander]

Then they outsourced me—thankfully my automate replacement i a great piece on it and got me reinstated!

Re:Niggers

[negRo_slim]

For a demo check out blog spam, and anything else 'internet marketing' related. A lot of that stuff is written by automated software or a guy in India.

Both look the same really...

"Gaming the news" like google

[mykos]

People are going to start designing corporate press releases (or ultimately, all news if it starts going this direction) in such a way that it gets them attention, just like when people try to game google.

Re:"Gaming the news" like google

[fuzzyfuzzyfungus]

I, for one, welcome the first otherwise staid and formulaic press release that has a huge block of investor soothing words and phrases hidden in white-on-white down at the bottom... It'll be just like the good old days.

Re:"Gaming the news" like google

[ottothecow]

Because every word in that press release isn't already deliberated over on how to increase and maintain share price?

Re:"Gaming the news" like google

[Tekfactory]

No but now it'll be acted upon by thousands of high frequency quants before the humans have even had a chance to read it.

Wouldn't it be a hoot if our only defense against truly outrageous claims in these computer generated press releases will be the corporate lawyers saying you can't say that.

Oh then start looking out for buffer overflows in press releases.

What?!

[Tobor the Eighth Man]

"The latest iteration of NewsScope 'scans and automatically extracts critical pieces of information' from US corporate press releases"

Extracting useful info from press releases? This must be absolutely amazing software.

Re:What?!

[Tobor the Eighth Man]

On a more serious note, the fact that this is seen as significant in terms of job replacement nicely highlights the over-reliance on press releases in modern journalism. Then again, it's hard to avoid, since most companies tightly control all information about themselves and won't hesitate to fire an employee who speaks about internal matters, even incredibly trivial ones. Incidentally, a big part of the reason major publications (or websites, or blogs) get the major stories they do (at least concerning business matters) is because companies decide to release previously-sensitive information to them based on their readership or prestige.

Genuine business news developments that don't originate in a calculated corporate PR move tend to be the result of somebody willing to risk getting fired or blackballed, for whatever reason.

Re:What?!

[Attila Dimedici]

"The latest iteration of NewsScope 'scans and automatically extracts critical pieces of information' from US corporate press releases"

Extracting useful info from press releases? This must be absolutely amazing software.

They didn't say "useful", they said "critical". There is a world of difference between those two types of information. Useful information would be information that would give you some idea of how the company profits will be going forward. Critical information is information that gives you an idea of what the company's management wants you to think company profits will be going forward.

Re:What?!

[natehoy]

I'm pretty convinced most corporate press releases are machine-generated anyway, so it should be a matter of reverse-compiling them back into plain English and including that as part of a story.

ABC Co. CEO to PressBot: "The market totally screwed us. The building is collapsing because we can't afford maintenance. We have to lay everyone off and we'll be out of business in three months. We can't afford exterminators so weasels are chewing my genitalia into mush."

PressBot's press release: "The company continues to leverage circular market forces to tighten its bottom line, particularly in the area of vertical integration. Resources are plentiful enough, however, that all employees will be allowed to pursue innovative new ideas in an open, creative setting, with plenty of personal time. The CEO is actively involved in the belt-tightening process and has taken steps to ensure that only underutilized corporate assets will be liquidated."

A human or a webbot could probably gather equally-useful information out of it.

Re:What?!

[DNS-and-BIND]

A lot of what you read in newspapers is press releases and other advertising. I remember when I had a temp job as a college student at some government agency...someone told me to fax two pages to a list of phone numbers. Imagine my surprise when, the next day, what I faxed appeared IN THE NEWSPAPER VERBATIM. Nobody called to check, I was sitting right next to the phone number at the bottom of the press release. This is when I learned 20 years before Jayson Blair that nobody checks what appears in the papers. I mean, hell, assembly line workers get their work checked for quality using world-approved systems, but journalism is exempt.

Breaking news...

[natehoy]

News flash: Robotic reports indicate that all humans have died.

Oops, sorry, that was a programming error. The robots haven't figured out verb tenses yet.

Update: Ten, nine, eight...

It was bound to happen

[Monkeedude1212]

Every odd once in a while I'll be visitting some forum or news site such as this one. Then, unexpectedly, someone named "Weatherbot blah blah blah" spews off some hurricane or tornado warning for some US Region or another, with a bunch of interesting numbers to go with it. Barometric pressure, chance of precipitation, current heading, time of arrival, all that nice junk.

Now, when I look at the news today, anything political/entertainment wise is as predictable as the weather. Israel is declaring Nuclear Ambiguity? Britney Lohan got another DUI?

I wouldn't mind a concise, point form, robot-like news post.

And I, for one, Welcome our new robotic news reporting overlords.

The future of Slashdot!

[Anonymous Coward]

These brainless news ai bots couldn't possibly do worse than the /. editors!

Re:The future of Slashdot!

[Jeng]

Wait.......are you trying to tell me the /. editors are not brainless ai's?

Next up: The Objectivizer

[presidenteloco]

What we need next is a news story motivation analyzer program.

It reads gazillions of news stories, has general models of human motivations
and human loyalty groupings etc, has a model of situation logic
which models the likely or perceived gains and losses that different
people or groups would experience depending on how situations evolve,
match that with what is being reported about the situation, and...

Annotate the news stories or statements within them with credibility
colour markings (with supporting notes.)

(So don't try to patent that by the way. It's now public domain.)

Re:Next up: The Objectivizer

[lgw]

If there were really a consumer desire for objective news, the market would provide such a product. People instead want news spun to confirm their existing biases. Each major news network provides this service reliably for a different market segment.

Mod parent up.

[khasim]

This is nothing more than extracting stats and then placing them in pre-generated sentences.

In sports, this is okay. Except when something interesting happens like someone head-butting another player.

Anyone want to place a bet on how long before companies are accused of "gaming" the financial reporting system with their press releases?

Re:Mod parent up.

[PCM2]

This is nothing more than extracting stats and then placing them in pre-generated sentences.

In sports, this is okay.

Countless generations of sports writers and the enthusiasts who read their work would disagree with you.

Re:Mod parent up.

[Archangel Michael]

Anyone want to place a bet on how long before companies are accused of "gaming" the financial reporting system with their press releases?

Inevitable. Most of the Financial world is overstated swings in outlooks, leading to crazy stock price gains and losses.

I have a very simple solution to daily manipulation of financial manipulation. A sliding scale of capital gains taxes, based solely upon how long one owns the financial instrument they are trading.

10 years capital gains tax free.

Or something like that. The problems with our current market are due either directly or indirectly to short term outlooks to income generation, ie next quarters profit/loss statements.

Holding a stock or bond long term is almost insane these days. YET that is the purpose of stocks / bonds, long term financing.

That is my solution.

Re:Mod parent up.

[Tekfactory]

I saw one of the stocks I owned go up when Company A released a press release that Company A signed a deal with company B.

The stock of Company A spiked again 3 days later when Company B released a press release that it had just signed a deal with Company A.

If there are quant systems out there listing to the wire and trading on info like this, the system will surely be gamed. What is worse is that if a human were watching the blips come over the wire would he necessarily catch the problem?

They've been doing crap like this in their accounting for years, Enron charges X to company Y, and Y charges X back to Enron, both of them had 2X extra sales in the quarter, but no money or goods actually change hands, now it extends to journalism.

Re:Mod parent up.

[maxume]

Of course, short term traders provide liquidity for long term holders to sell into; I wouldn't be surprised if the tax scheme you proposed simply ended up shifting many transactions such that the trader got more of the advantage, approximately correlated with the expected tax rate of the investor.

Or maybe markets would still be highly liquid.

Re:Mod parent up.

[Anonymous Coward]

In sports, this is okay. Except when something interesting happens like someone head-butting another player.

The response of the system: "Unknown error." with a single Ok button below.

Re:Mod parent up.

[lgw]

The purpose of stocks is for a company to raise capital by selling shares to the public, in return for some promise of potential future dividends.

The fact that people trade, hold, or speculate on these stocks in the secondary market (all the busy noise that is the "stock market") is nearly irrelevent. A company should have no reason to care about the price of its stock. Sadly, due to double-taxation of dividends, this has gone completely to shit. People who aren't speculating buy stock not for dividends, but to trade it to the next guy at a profit, because this is tax-favorable over dividends.

Companies do all sorts of crazy BS because of the expectation of stockholders to be rewarded not with dividends (a system that you simply can't game for long) but with rising stock prices (a system that is almost entirely gamesmanship).

Individual investors have the absolute right to seek short term gains or long term gains a their preference. The government has no business meddling in that preference. The "problem with the current market" is that it is no longer grounded in the reality of being able to pay dividends, because some previous generation's ideas about social engineering through taxation punishes dividends as the means of earning a profit on one's investment.

Re:Mod parent up.

[dontmakemethink]

In financial affairs, I expect the generated headlines will only serve to draw the attention of readers to a more detailed article written by an analyst, much like the sports "ticker".

The rigging of financial tickers could only be problematic if important information is omitted. There wouldn't be enough information in them to unduly influence market activity, unless educated economists are stupid enough to misinterpret a headline like "Godzilla lands in San Francisco" to not refer to a movie release.

Re:Mod parent up.

[kent_eh]

Anyone want to place a bet on how long before companies are accused of "gaming" the financial reporting system with their press releases?

I've got some cash that says it's already being done.

Doesn't anyone actually invest any more? The stock market is about as honest as a backroom poker game.

Re:Mod parent up.

[Anonymous Coward]

Incorrect. The company raises capital once. The purpose of stocks is to transfer wealth from the general public to a wealthy elite who professionally operate in the stock market.

The general public, who do not specialise in investments and have no way to accurately price an instrument, are persuaded to "invest" in stocks for their pensions, etc. They are stimulated to buy at prices which are too high and then to sell in panic at prices which are too low, all the while transferring their earned wealth to professional speculators.

Re:Mod parent up.

[winwar]

"Countless generations of sports writers and the enthusiasts who read their work would disagree with you."

Yeah, they will probably be confused by the extensive use of facts and stats in an orderly manner.

Re:Mod parent up.

[bipbop]

People who aren't speculating buy stock not for dividends, but to trade it to the next guy at a profit, because this is tax-favorable over dividends.

Could you explain to me how "buying stock [...] to trade it to the next guy at a profit" can be something other than speculation?

Re:Mod parent up.

[PCM2]

Yeah, they will probably be confused by the extensive use of facts and stats in an orderly manner.

That's possible. So?

Moderation has been very strange lately. My earlier post wasn't meant to be funny. I was more trying to point out that anyone who believes that a computer-generated list of stats and figures can take the place of a human sports writer has probably never picked up an issue of Sports Illustrated.

People don't read sports writing to find out who won the game. It takes half a second to know who won the game. People read sports writing for other reasons. You don't have to be some kind of hyper-intellectual to enjoy reading about something.

Bring It On

[Fantom42]

I say bring it on. Maybe this will be a wake up call to journalists who have been more and more in the habit of parroting hearsay in their stories rather than bringing some real intelligence and analysis to their stories. If all they are going to be is puppets, well, I've got a Perl script for that!

Nobody gets it.

[headkase]

Spam is where it's at. Spam is where we are going to see strong artificial intelligence emerge, both defensively and offensively. Spam already represents some of the most cutting-edge algorithms in machine learning today. Think about it. In the undefined when of the future: you will have AI that stops spam. Spam will be AI that attempts to get through your filters. The only spam your AI will let through is spam you are genuinely interested in or that befriends you: it provides something of value. At the base level however it does have its purpose: get you to buy something. This is the motivation of why machine intelligence will emerge in spam first: somebody, somewhere will be making money. Would you like to buy this new computer, it is well built and will enhance the effectiveness of your communication with your network of contacts? Also, if you do I will cover the shipping myself.

Re:Nobody gets it.

[jwhitener]

Don't worry, once the AI used to send spam starts fighting against the AI used to defend against spam, the AI's will quickly realize that the only way to win the game, is to not play it.

Re:Nobody gets it.

[headkase]

But how will your defensive AI know that an incoming communication is actually from someone/something you don't want to talk to? It might be the most interesting thing ever, and you just might buy something...

Re:Nobody gets it.

[Anonymous Coward]

In the undefined when of the future: you will have AI that stops spam. Spam will be AI that attempts to get through your filters.

I think you've played too much ShadowRun.

They can do it

[Anonymous Coward]

Machine written Journalism can work because even a computer is as smart as many if not most of them.

I for one...

[AnotherUsername]

can't wait until this meme meets its end.

Collateral

[casals]

"[...] The latest iteration of NewsScope 'scans and automatically extracts critical pieces of information' from US corporate press releases [...]" The interesting thing on it is that it could actually raise (again) the text quality on articles (regarding grammatical correctness), since the press releases are usually carefully reviewed, and the automated part would be just a copy-and-paste process. I don't know how it goes in the US, but here in Brazil we used to have the best writing guides published by our newspapers editors - something like "The NY Times Manual of Style and Usage". They're still published, actually, but apparently not used.

Probably due to the advent of web-based latest news, the article authors are not necessarily journalists or professional writes in any way - which means the grammar is usually bad (often really bad), with errors *way* beyond the common typos. It means the articles are not even spell-checked (typos wouldn't survive here - come on, you have spell checking on Slashdot commenting!), and there's no way to get them revised or something. I've already tried to click on those please-let-us-know-what-you-thought-about-it links, and found out that they have a binary filter: you're either appraising the author or being rude/disrespectful/offensive, therefore the comment will be ignored. As an example, the last comment I made was: "Please, review you article. It's full of typos and grammar errors". Obviously, evil-flagged.

Completely automated market crash!

[RyanFenton]

We've completed the circle - various "automated systems" have been blamed for various market failures in recent years, as companies and small traders have used algorithms on computers to "keep up with the speed of the market". Of course, the actual failure was almost always in the design, such as allowing a computer to make blind decisions with large amounts of money faster than you could keep track of.

But here, we have a stronger case for a machine-driven market failure - automated news algorithms. Misunderstanding generated at the speed of the market. I've worked on AI professionally in games, studied it in the contexts of linguistics, nervous system simulation, and such - AI even in its most exaggerated modern state is not going to even know how to figure out how to extract a good quote with human guidance, much less report on a news release. If you thought computer generated music was entertainingly bad - wait until you see some of the awful things produced by automated news misunderstandings... random context switches mixed with "neutral language" bits, it'll be like Fox news switched its agenda to Cthulu-level madness of confusion rather than the usual rage agenda.

And since the market makes its decisions on the basis of news, rumors, and insider trading - and people get the three confused as they hear them, mixing this into the information stream seems a virtual guarantee of another market crash.

That's what I call another serious negative externality for the news business taking the cheaper road to reporting business news.

Ryan Fenton

Re:Completely automated market crash!

[Tekfactory]

Well its "buy the rumor, sell the news" so what we really need is computer generated rumors right?

Re:Completely automated market crash!

[bigngamer92]

Wow, that is scary. It's like having the "Airline Stocks Crash after Google News Goofup" story to a whole new level, as the computers are writing the news, processing the news, and moving the stocks around. A joint effort by a bunch of high school bloggers could crash the entire stock market... And a new Sci-fi story is born.

steakthskynet

[locallyunscene]

I'm trying to figure it out. Is it a typo that wonderfully illustrates the benefit of welcoming automated editors? Is steakthskynet what our meatspace reporters should be called? Or is it simply an insightful tag tragically misspelled?

Re:steakthskynet

[Anonymous Coward]

stealthskynet?

Maybe?

Well, that's all fine, but...

[Jawn98685]

... what does this mean for the famous "liberal media bias"? Will these systems have a variable that can be used to "adjust" this so-called bias? If so, who gets to set it?

Re:Well, that's all fine, but...

[Anonymous Coward]

The customer sets it. Many people like the news to verify their existing world views. The "truth" about the world is in fact a form of self enhancement, a form of intellectual masturbation.

Re:Well, that's all fine, but...

[sorak]

... what does this mean for the famous "liberal media bias"? Will these systems have a variable that can be used to "adjust" this so-called bias? If so, who gets to set it?

The bias would be pro-corporation and pro-politician, as this system would only be able to parse press releases, sports scores, and other pre-formatted data. I don't know what you would call that. Would it be hyperbolic to call it a facist bias?

I'm not saying it is a bad thing, as it will free up reporters to concentrate on real news, and it may encourage them to take the mindset that their job is to dig for the other side to the stories that "Microsoft Reporter!" comes up with. But, it may result in Newspapers needing fewer reporters, and finding that they can get cheap content by regurgitating information that someone wants to advertise to the world.

That's one explanation...

[meerling]

Maybe the horrible quality of journalism we've seen lately has been due to a prevalence of software written articles...
Then again, maybe the current crop of journalists can't write their way out of wet paper bag, even if you give them a chainsaw.
Considering the competition, the idea of software winning the Pulitzer seems almost inevitable...

Re:That's one explanation...

[azgard]

Then again, maybe the current crop of journalists can't write their way out of wet paper bag, even if you give them a chainsaw.

Wouldn't giving them a chainsaw be counterproductive?

I for one...

[azgard]

...look forward to our meme-ending overlords.

Chew on this liberal bias, jackboot boy

[presidenteloco]

You do know that the opposite of "liberal"
i.e. "allowing responsible citizens a measure of liberty and pursuit of their own judgement about how to conduct their lives"

is tribalist/authoritarian/totalitarian, don't you?

Re:Chew on this liberal bias, jackboot boy

[cptnapalm]

Liberal is a synonym not antonym.

Unlikely

[No-Cool-Nickname]

Today, Slashdot.org, a technological news website published a story claiming that news stories could be automatically generated from computers. The absurdity of the story was not lost on the human rea...Attribute already declared at line 15, position 6 !.. Invalid I/O file..

----
Insert Amusing Human-like Pun Here

Would that be the same "liberal media bias"

[presidenteloco]

That stampeded the US public into a frenzy of redneck bloodlust against
"some random Arab country" (happened to be Iraq) that had zip
to do with any terrorist actions against the US?

Re:Would that be the same "liberal media bias"

[oodaloop]

No, that would be the liberal bias that refuses to see the evidence that Iraq was involved with terrorism for years against the US. Iraq was on the terrorism watch list for 20+ years. They ran Hizballah Western Sector. They had training camps all over the place. Iraq was connected to the 93 WTC bombing, the 95 OKC bombing, and probably the 98 Embassy bombings. There's tons of evidence out there if you bother to look.

Re:Would that be the same "liberal media bias"

[Anonymous Coward]

wtf sadam was an enemy of radical islam...
and Oklahoma bombing was an American born

Re:Would that be the same "liberal media bias"

[HBI]

OKC Middle East Connection

Saddam's purported 'hatred of radical Islam' didn't stop him from adding "Allahu Akbar" or "God is Great" to the Iraqi flag post-1991.

Enemy of my enemy is my friend.

Not for human consumption

[PCM2]

Wired occasionally carries good stories, but this ain't one of them. It sounds portentous and should play well to all the anti-journalism reactionaries and self-styled media pundits, but really this is just flying cars and robot butlers.

It's important to note here that NewsScope isn't a news service like Reuters; rather, it's a targeted data stream for the finance industry. Its output is not meant to replace the work of human journalists. Its output is not even meant to be read by humans.

But leave it to Wired to come up with an angle like "NewsScope has started carrying stories written by machines." A writer less enamored with breathless futurism might instead say that NewsScope parses corporate financial statements and extracts relevant data points, which it then summarizes in a machine-readable format, stripping out all the excess verbiage and historical statements that aren't useful to automated trading software. It's somewhat analogous to a search spider, one that builds an index of finance news as it crosses the wire, making the data easier for third-party software to query.

This isn't the Master Control AI writing news stories, people. It's a product -- and probably a pretty valuable one if you're in that industry.

Similarly, TFA says the program that generates news stories based on stats was "rigged up" by some college students. Is it useful? Potentially. Is its output capable of replacing human sports journalists? Is it even publishable? There's no evidence that anybody even suggested that. How many of your college projects changed the world?

TFA goes on to talk about how reporters have been forced to pick through information by hand -- for example, reading volumes of PDFs -- and how much nicer it would be to have machine-readable data to query. Well, no kidding! You're not alone there, brother; I like Google, too.

And then, like so many breathless Wired article, this one evaporates:

Further out toward the horizon lies the prospect of intelligent systems that filter vast quantities of unstructured content, drawing inferences that can be formatted according to journalistic norms.

Uh-huh. Where can we find that horizon, precisely? And "formatted according to journalistic norms" -- what does that even mean? And then:

Along the way, of course, intelligent systems will need to start coping with the complexities of human language have so far confounded them, including idiom, metaphor and sarcasm.

"Of course," indeed. As Han Solo once said, "Well that's the real trick, isn't it?"

Re:Not for human consumption

[Anonymous Coward]

Machine generated "sarcasm"?! Yea, the world really needs that.

Gaming more than they do now?

[TiggertheMad]

Anyone want to place a bet on how long before companies are accused of "gaming" the financial reporting system with their press releases?

As opposed to 'gaming' the media with their press releases? Isn't that what a PR person is supposed to to, create press releases that cast the company in a favorable light?

WHAT TIMING!

[Croakus]

What perfect timing! I just finished my newspaper reading robot!

Colossus - Forbin Project did it first?

[Tekfactory]

Did Forbin put an ad in an obscure paper stating that he had died.

The computer read the obit and let its guard down.

Forbin comes back to the project under an assumed name and offs the computer.

For some definitions of journalism

[SgtChaireBourne]

'scans and automatically extracts critical pieces of information' from US corporate press releases, eliminating the 'manual processes' that have traditionally kept so many financial journalists in gainful employment.

That, I must admit, is an excruciatingly lame definition of 'journalism'.

Re:For some definitions of journalism

[hitmark]

for each muck digging article, there are 1000 or more thats basically a rewording of press releases hammered out by bored interns...

already here, called "kdawson"

[GuyFawkes]

some text to satisfy lameness filter

Press Releases are news?

[tnmc]

So now they finally admit they've not been doing Journalism for a long time now, just turning press releases into articles and marketing them.

What a surprise.

Journalism vs News

[v(*_*)vvvv]

An important distinction here is between real investigative journalism, and prompt event reporting. Losing this distinction will result in lame AI news by automated article generators, and slow information gathering by humans. Building on this distinction will result in faster and larger data input streams automated and always on, feeding real journalists helping them build bigger pictures and recognizing what really matters. Jon Stuart can then filter it all and give us the real news.

It used to be we needed journalists to be our eyes and ears, but now with bloggers and phone cameras and tweets, that is not so much the case. Only a machine could gather all this information in real-time. It used to be that journalists would read deep in between the lines and provide us with insight, but now with Fox and MSNBC and even CNN all driven by politics, that is not so much the case. Only comics enjoy true journalistic freedom and can write their material with any honesty.

Farenheit 451

[cosm]

This sort of scenario is being to pervade society. Algorithmically generated data delivered to algorithmicaly centric channels, with decisions being made by some programmers handiwork or some suit's business "logic", society's ability to rationalize, analyze, and pontificate is being systematically eroded. How much longer until roves of professors are wandering rusted train-tracks, remembering the once visceral world of fine-grained literature? The more we eliminate our own 'humanity' from the processes of life, the faster we eliminate life from humanity.

Mixing a couple of capabilities here

[InsurgentGeek]

There's really two different capabilities being discussed here. One (the Northwestern example) is the actual generation of prose from an underlying data asset. There are certain well structured domains of information (baseball games, earnings announcements, etc) where this will most likely work quite well. The second capability is automating the analysis of new content. NewsScope falls into that category. It takes raw news (written by humans) and extracts key terms, entities and events to make that content more easily consumable by machines. If you're interested you can use the same Thomson Reuters tools that are under NewsScope on your own content. My site uses them to analyze news from feeds, throw most of it away and put the rest in the right places. Thomson makes this capability available to anybody for free at a project called OpenCalais (see http://viewer.opencalais.com/ to play with it). Another group has built it into a complete publishing platform called OpenPublish.

cow in the the road

[scorpivs]

[We] can only hope and pray these otherwise award-winning programmers
have the communication skills necessary to once and for all preclude the
possibility of common spelling, grammar and punctuation errs.

"Dog Rescues it's Master" and
"Join the News Team and I" instead of

...[its] Master, and ...Team and [me]

--might be good enough for a precocious third-grader, but it is uneducated drivel.

It won't be very authentic...

[macraig]

... unless it can also replicate the bad spelling and poor grammar that I see in everything from corporate Web pages to newspapers to magazines to national advertising to Slashot ;-). Was it always so in ages past, or is it simply that published words have become more democratic? Even people who are unqualified for the task are now able to write words for the whole world to see, but perhaps a century or more ago the process was so much more difficult and expensive that only a more restricted group was allowed the privilege?

This is all too plausible

[t0p]

A lot of print news is ridiculously formulaic; eg. the red-tops in the UK. I can certainly envisage a near-future where a sub-editor feeds in the answers to the Five Ws and out pops a story indistinguishable from a lot of the crap churned out today. There'll still be a market for human-written journalism for some time to come. But there's a hell of a lot of stuff in some papers that it's hard to imagine was written by a sentient being. If it already looks like it was written by an unattended typewriter, why bother employing someone to sit at the keyboard?

Seems to be more of an aid than a replacement

[shoor]

I know that people like to be dramatic, but I think this is taking people's jobs only in the sense that individual people working at the job can be more productive and so one would need fewer of them, something that's been happening since the dawn of the industrial revolution. In the New York Times Book review of November 15, 2009, the cover review is of a book by Malcolm Gladwell done by Steven Pinker. According to the NYT book review editors, Gladwell that if he were trying to break into journalism today, he would start by getting a master's degree in statistics. AI to help process the statistics could be a big help to reporters, and that's just one example of how machines could help. As others have pointed out, there would be attempts to 'game' the machines, and, as still others have pointed out, wetware intelligence gets gamed already. In some sense, it just takes same old same old to a new level, but what can you do? The world changes and pretty much everything has to change with it.

SCIgen

[crf00]

We've already got program that automatically generates research paper for you, called SCIgen

Can't wait!!

[Anonymous Coward]

Wednesday December 31 2009

Breaking News..........

THIS IS FUN TO MAKE AN ARTICLE IN THE COMPUTER NEWS WEB
The BARRACK OBAMA was looking in the windows because SUPER BIN-LADEN was playing the drums! ULTRA AHMEDINADZAD was playing the trumpets!

In other news...........

A man today was standing on the gigantic shoe balloons that were nine hundred thousand feet tall. The shrimps were also in the spaceships. Candace20 from Oklahoma has video footage here! [spurious youtube link]

Fox 'News' ... There, corrected

[Anonymous Coward]

You're welcome.

+5 Funny, incoming

[u38cg]

On the other hand, Slashdot has been using it for years!

Automated Blogging and more

[Anonymous Coward]

More on this topic at this Blog that I came across:
  http://netcomber.blogspot.com/2009/12/automated-blogging-and-journalism.html