Automated System Developed To Grade Student Essays

RougeFemme points out this story at the Times about software that can be used to grade student essays and offer almost instant feedback. "Imagine taking a college exam, and, instead of handing in a blue book and getting a grade from a professor a few weeks later, clicking the 'send' button when you are done and receiving a grade back instantly, your essay scored by a software program. And then, instead of being done with that exam, imagine that the system would immediately let you rewrite the test to try to improve your grade. EdX, the nonprofit enterprise founded by Harvard and the Massachusetts Institute of Technology to offer courses on the Internet, has just introduced such a system and will make its automated software available free on the Web to any institution that wants to use it. The software uses artificial intelligence to grade student essays and short written answers, freeing professors for other tasks."

Comments

Re:Have a computer write your submission too

[korgitser]

I wonder how these would do:

the postmodernism generator http://www.elsewhere.org/pomo/
the math paper generator http://thatsmathematics.com/mathgen/

Grades grammar not content. A.I. not ready yet.

[doug141]

"A director of writing at MIT Les Perelman says that because these robo-graders work according to an algorithm, it is not hard to find out what it values and thus beat the system. He found that if you write long essays with big words, even if they are nonsensical, you will score high. The algorithm does not like short sentences or paragraphs or sentences that begin with ‘and’ or ‘or’ nor is it enamored of sentence fragments. In other words, all the little rules that good writers will break to create a particular effect will cause your essay to be marked down.

Perelman gives an example of how you can get a high score. The most interesting feature of the algorithm is that it doesn’t care about substance or even truth. It will ignore such trivialities as saying that the war of 1812 began in 1945, provided you say it grammatically. The substance of an argument doesn’t matter, he said, as long as it looks to the computer as if it’s nicely argued.

For a question asking students to discuss why college costs are so high, Mr. Perelman wrote that the No. 1 reason is excessive pay for greedy teaching assistants. “The average teaching assistant makes six times as much money as college presidents,” he wrote. “In addition, they often receive a plethora of extra benefits such as private jets, vacations in the south seas, starring roles in motion pictures.”

E-Rater gave him a [top score of] 6. He tossed in a line from Allen Ginsberg’s “Howl,” just to see if he could get away with it. He could."

http://freethoughtblogs.com/singham/2012/05/03/how-to-fool-a-computer-grader/

The Actual "Essay"

[efitton]

http://www.documentcloud.org/documents/346138-essay-awarded-a-top-grade-by-e-rater.html