Automated System Developed To Grade Student Essays

RougeFemme points out this story at the Times about software that can be used to grade student essays and offer almost instant feedback. "Imagine taking a college exam, and, instead of handing in a blue book and getting a grade from a professor a few weeks later, clicking the 'send' button when you are done and receiving a grade back instantly, your essay scored by a software program. And then, instead of being done with that exam, imagine that the system would immediately let you rewrite the test to try to improve your grade. EdX, the nonprofit enterprise founded by Harvard and the Massachusetts Institute of Technology to offer courses on the Internet, has just introduced such a system and will make its automated software available free on the Web to any institution that wants to use it. The software uses artificial intelligence to grade student essays and short written answers, freeing professors for other tasks."

Comments

Have a computer write your submission too

[hawguy]

Seems like it's a small step from this to having computer algorithms that automatically write your paper for you too - then you can let it go through thousands of submit-edit-submit cycles until the scoring computer gives you a perfect score.

Kind of like the guys that came up with software to generate nonsense scientific papers and actually had a few accepted at conferences and journals.

They had these back in 1991 too

[GoodNewsJimDotCom]

My friend wrote a story about his cat that was grammatically correct,and used big words, but made little to no sense. The auto-grader program told him he was approaching PHD level English. So he took his paper into school and showed it to the English teachers who reviled at it. He was like,"Show's what you know, the computer told me I'm university level."

Re:AI has not come far enough for this

[dgatwood]

Computers suck at even the most basic grammar checking. I once decided to try a bunch of online grammar checkers to see if they would be useful at providing a sanity check for my novels. I concluded that they report so many bogus mistakes that it simply wasn't practical to use their output at all. To test them, I fed them a block of content, some with intentional errors that the grammar checker should have caught, others with deliberately (or accidentally) tricky bits that should not have produced any errors.

• Upon seeing that, Joseph resolved to stop. Several grammar checkers thought "seeing that" was used idiomatically, and suggested replacing it with because. Upon because, Joseph resolved to stop. Yes. Much better.... Oh, and some others suggested that "Upon" is archaic.
• “Time to impact: seventy-six hours, fifteen minutes, twelve seconds,” the computer intoned. Oddly, several checkers suggested that "twelve seconds" was a fraction and should be hyphenated. Ugh.
• It's simple, really. There must be some mistake. Several spell checkers suggested "their". Others said that "must be" is passive voice. Uh, no, not every use of "to be" is passive construction.
• This isn’t your class anymore. Some checkers reported an agreement problem with "class". Huh?
• The room was dark, its plant-covered landscape shimmering green in the light of their headlamps. At least one checker suggested replacing "in the light of" with "considering". Eek!
• Joseph climbed up first. Several spell checkers suggested that "climbed up" is redundant. Apparently, their editors have never climbed down something.
• One checker even called "chided" archaic, but did not comment on the highly offensive swear word that I placed elsewhere in the sentence.

And so on. Heck, my phone doesn't even know the difference between "its" and "it's" and tries to auto-correct me into looking like I failed first grade English. And these folks expect me to believe that computers can feasibly help students learn to write better papers? Give me a break. Maybe in thirty to fifty years (*) we'll get there, but....

* Which many grammar checkers would probably suggest changing to "thirty-two fifty".

Re:This is horrid

[Anonymous Coward]

I went through the same system and it taught me all sorts of useful things unrelated to my actual physics curriculum, like
1/2 != 2/4
0.5 != 1/2
x != x+1-1
x^2 != x*x

Re:This is horrid

[reve_etrange]

I had the same experience in university calculus and physics. Even for problems with one right answer, there are typically many (even infinite) ways of expressing that answer. Even something as advanced as Mathematica or Maple can be fooled, and the websites in question are no Mathematica.

Re:This is horrid

[Jstlook]

I went through that type of system for a Chemistry class. After class, the entire class would wander down to the computer lab and do the homework together. We'd get a question, find the [book] answer, then have each person try to obtain the correct [computer-identified] solution on the first try, trying various syntax adjustments each time. Of the three chances we got, someone usually got the right syntax before everyone had failed the question the second time.

Best benefit? Getting a group of people in the same place to research, debate, and agree on a single answer, then be open-minded and organized enough to shape the solution to fit the constraints given.

Essay grading machines have been in use for years

[milkasing]

All your GRE essays are evaluated by a machine and have been for years -- the e-rater. http://www.ets.org/research/topics/as_nlp/writing_quality/
The rating is also done by humans. It works well in practice and ensures that essays are graded fairly. If there is a significant discrepancy between the two ratings for a essay, that essay is examined further by another specialist. It prevents students from being victims of someone having a bad day at the office, and also does not encourage writing an essay to beat a machine.
The significance of the EDX news is not the concept of automated grading, it is that that such software is now free and opensource.