Slashdot Mirror
Automated System Developed To Grade Student Essays
RougeFemme points out this story at the Times about software that can be used to grade student essays and offer almost instant feedback. "Imagine taking a college exam, and, instead of handing in a blue book and getting a grade from a professor a few weeks later, clicking the 'send' button when you are done and receiving a grade back instantly, your essay scored by a software program. And then, instead of being done with that exam, imagine that the system would immediately let you rewrite the test to try to improve your grade. EdX, the nonprofit enterprise founded by Harvard and the Massachusetts Institute of Technology to offer courses on the Internet, has just introduced such a system and will make its automated software available free on the Web to any institution that wants to use it. The software uses artificial intelligence to grade student essays and short written answers, freeing professors for other tasks."
One of my kids had something like this: not for English, but for physics.
The teacher couldn't be bothered to assign and grade proper homework.
Instead, he fobbed the kids off onto a web app.
- go to the site
- get a problem
- solve the problem
- type in the numerical answer
- right answer? go on to the next problem
- wrong answer? try again
The web app allowed maybe 0.5% margin for rounding error, and you got 5 tries before it failed you on that problem.
It sounds reasonable in the abstract, but in practice it was utterly wretched.
All learning is, at some level, an interaction--a conversation--between student and teacher.
Even if it is nothing more than a red check mark or a red X on a homework paper,
you have communicated some thing to some person and gotten some response.
You don't realize how important this is until it is gone.
With nothing but a machine to talk to, it stops being about learning.
It is just about satisfying the machine by whatever means necessary.
In his rage and frustration my son told me that the easiest way to solve the problems was to copy and paste the problem text in to google.
This would reliably return the general formula for solving that problem;
plugging in the numbers that the web app had generated for your instance of the problem would then yield the correct answer.
By the end of the school year, I was telling him that if he didn't want to deal with the web app, he should use google to get his grade,
and if he wanted to learn physics, I would teach it to him.
Automated essay grading is going to be even worse.
There is no point writing prose unless a human is going to read it.
When I want to talk to machines, I write code.
Writing songs, that voices never shared...
-- Paul Simon
``Your grade is C. To improve your grade in the future, you need to do the following:
use 25-30 words per sentence; include more words from the wordnet entry for the topic of your essay; avoid simplistic or run-on sentences as measured by number of noun and verb phrases detected by our proprietary NLP tokenizer.
As a helpful reminder, our preparatory guides are available as a subscription service and include 100 practice submissions per week; only $29.95 per month."
"They were pure niggers." – Noam Chomsky
Grading is not, or should not be, about the grade, it should be about the feedback that the lecturer gives to the student. Even if the computer can grade an essay well (which I remain to be convinced of, although I am sure I will soon have the chance to test it for myself), there is no claim made about the computer giving useful advice to the student. Can a computer explain how to refine a research question or structure an argument? Sadly, many lecturers don't in fact give good feedback, but we should be looking for ways to enable lecturers to give better feedback, not accepting poor feedback as the norm.
Every era has its snake-oil salesmen and their marks. Sadly, in this case it will not be the customers who suffer, but their hapless students.
Those who would give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety.
That's interesting in its own way, but a much more interesting comparison would be between the essays' lengths and the respective SAT Verbal scores of their writers. I would bet that they are also correlated quite closely.
News flash: when presented with an essay topic, smart people spend a few minutes planning and then proceed to write voluminously about the subject, because they are fluent writers. Dumb people start muddling along, lose track of where they are, and stop when they've stated (though not proved) their main point, because they're not. Fun game: ask a room full of people to write nonstop for five minutes on any topic(s) of their choosing, then compare word counts vs IQ/class grades/whatever.
If you're a HS student reading this (and I imagine there are a lot of you who are): practice writing. Practice writing. Practice writing. It's important. It's probably the most valuable skill you will ever acquire for dealing with people you don't meet with face-to-face. Bad writing is universally considered a sign of low intelligence. It takes a lot to overcome the negative impression that bad writing gives, and you often will not have the opportunity to try - when given a stack of 100 resumes for two positions, guess how the initial winnowing occurs? Toss anything on colored paper, anything written in a funny typeface, and anything with grammatical or spelling errors. I cringe today when I read some of the stuff that I wrote in HS, but it's grammatical and correctly spelled, even if the verbiage is ponderous (and occasionally verges on purple prose).
IME, smart people write concisely and to the point of the prompt, while dumb people write voluminous, rambling, redundant, and unfocussed walls of text.
"Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away." --Antoine de Saint-Exupery