Computers To Mark English Essays

digitig writes "According to The Guardian, computers are to be used in the UK to mark English examination essays. 'Pearson, the American-based parent company of Edexcel, is to use computers to "read" and assess essays for international English tests in a move that has fueled speculation that GCSEs and A-levels will be next. ... Pearson claims this will be more accurate than human marking.' Can computers now understand all the subtle nuances of language, or are people going to have to learn an especially bland form of English to pass exams?"

Comments

Don't they already do this?

[darkshot117]

I seem to remember back in school my English teachers would grade as if they were a computer, failing to actually read into the meaning of things and simply complain about obscure grammar errors (which no one in the real world even knows about) and simple typos. From the sound of this, nothing is going to change.

Re:Don't they already do this?

[Anonymous Coward]

As a writing instructor, let me put it this way: I very, very seldom see a paper with misspellings and grammar mistakes that is nonetheless a well-written paper. It happens, but not often. Grammar and spelling mistakes are a symptom of sloppiness, as are poor reasoning, lack of organization, and lack of adequate support. If you can't be bothered to remember primary-school English, it is not likely that you are willing to master rhetorical structure.

When we read a paper, we actually don't care what you're saying. There usually isn't an "interesting" score. In my case, I evaluate on three, ten-point, holistic scales: Content (which basically refers to amount and quality of support), Organization (rhetorical structure), and Mechanics (yes, grammar, vocabulary, adhering to the style guide, etc.). I do this so I don't have people claiming that their hopeless muddle of a paper got marked down for "obscure grammar errors (which no one in the real world even knows about) and simple typos".

Guess what? Writing is not speaking. Those "obscure rules" are, indeed, usually only applied in writing. I ramble, swear, and disregard the conventions of "proper" English when speaking. But that is because those rules do not really apply in that sociocultural setting. In formal writing--you know, what you're being taught in writing class--they matter a great deal. If you don't follow them, you sound like an idiot, and no one will listen to you.

Why are these "obscure" rules used as a "canary test" of your intelligence and noteworthiness?

Because of what I wrote in my first paragraph. Intelligent, methodical, and rational people care enough to follow them.

I'm sorry, but that's how it works in the "real world".

Re:Don't they already do this?

[DirePickle]

What?

Re:Don't they already do this?

[SanityInAnarchy]

Just because I write a book of philosophy that is grammatically incorrect but possibly deeply insightful doesn't not make it any less important.

If you're capable of writing a book of philosophy that is deeply insightful, you should also be capable of writing one that's grammatically correct. Doing so would set you apart from someone who is capable of neither, and it'd set you apart at a glance.

It's also common courtesy to the reader. Generally, people have no trouble reading something that's grammatically correct, no matter how poor their own grammar is. However, it's at least annoying, and sometimes frustrating and difficult to understand something that's incorrect. Depending how incorrect you are, I might decide that deep insight you have isn't worth the effort of reading your book.

In other words, if you want your philosophy book to actually be read, you'll proofread it, spellcheck it, and clean it up -- just as, if you want to actually be hired, you'll shower, shave, and put on a tie for the interview.

how many authors have had no editors?

An editor is helpful for two reasons: To catch the mistakes you don't, and to ensure that the publisher's name doesn't get tarnished by subpar writing.

It shouldn't be the editor's job to remind you to capitalize the first word in a sentence. Meet them halfway.

What's more, we're rapidly moving towards mediums that don't need a "publisher", per se -- anyone can start up a blog, or ramble on Slashdot, without any editor at all. If you think it's worth having an editor correct your grammar in a dead-tree book, surely it's worth having correct grammar in what you write online -- but do you really want to hire an editor for your blog? At that point, it just makes economic sense to learn some "basic common English" skills yourself.

Re:Don't they already do this?

[BryanL]

I would add to your list the third-person plural pronoun in place of the third-person singular that many people use to be gender neutral. I know many English teachers hate it, but it is commonly used, hence a de facto part of our grammar. Also, it beats the hell out of constantly switching back and forth between she and he, using s/he or he/she (she/he) or using "one".

Re:Don't they already do this?

[Anonymous Coward]

Of course formal written language is different from spoken colloquial language, but as for the rest of your post, you're 100% wrong. By any objective standard, none of those things are actually mistakes. Were you to compare such rules to a corpus compiled from the most learned, eloquent and respected published writing in the English language, you would find all of the above supposed errors to be not only normal, but often preferred. It's an unfortunate fact that our writing education is needlessly bogged down with voodoo rules that have nothing at all to do with actual informed usage, which often leave students in a state of nervous anxiety about their grammar for no good reason at all.

Re:Don't they already do this?

[digitig]

My point is that it develops through a lot of mechanisms. It's naive to think that it develops completely organically, as the post I was replying to seemed to suggest. But it would be just as naive to think it's entirely down to definite conscious decisions (except in the case of artificially constructed languages). Hence: "Partly but not entirely..."

Sure...

[Greyfox]

That'll work great when the software can write a nasty response to your assertion that Herman Melville was a loud-mouthed pratt who only wrote those books because he liked to hear himself talk. Of course, given the quality of most student English essays, it would probably be fine if the software just verified that the student wasn't just plagiarizing from the wikipedia entry on the subject and then randomly assigned a passing grade.

Probabilitistic grammer

[genericpoweruser]

Colorless green ideas sleep furiously. A computer would read this sentence and see nothing wrong. Any human can tell that it lacks any meaning at all. Just because the sentence has the proper subject/verb structure doesn't mean it is a good one.

In my opinion, you can't practically replace an old-fashioned human for such things, with the possible exception of strong AI.

The beginnings of Newspeak

[Amigori]

eh hem...put on tin-foil conspiracy hat... Could this be the beginning of a real-world "Newspeak?" With everything else the UK has done in recent years, it is merely one more step toward 1984. For those unfamiliar with Orwellian Newspeak:

• 1984
• Newspeak
• Newspeak Dictionary

This is stupid.

[icannotthinkofaname]

Computers can't even grade source code. How are they supposed to understand English?

Or is my professor's grading script simply stupid when it comes to source code?

kairos

[epine]

All you have to do is detect how many lolcat/txting words are in their essay and mark accordingly. Anybody who can put two sentences together without using any is "advanced".

Allow me to pee on your fantasy world with actual knowledge.

Clive Thompson on the New Literacy
"I think we're in the midst of a literacy revolution the likes of which we haven't seen since Greek civilization," she says. For Lunsford, technology isn't killing our ability to write. It's reviving it--and pushing our literacy in bold new directions.
...
The Stanford students were almost always less enthusiastic about their in-class writing because it had no audience but the professor: It didn't serve any purpose other than to get them a grade. As for those texting short-forms and smileys defiling serious academic writing? Another myth. When Lunsford examined the work of first-year students, she didn't find a single example of texting speak in an academic paper.

Re:kairos

[Anonymous Coward]

Although I take no part in this debate, I would ask you not to mistake an appeal to authority as factual knowledge.

Re:kairos

[rastilin]

Although I take no part in this debate, I would ask you not to mistake an appeal to authority as factual knowledge.

I begin to suspect that quoting "logical errors" is a new form of karma whoring. The appeal to authority only means that a person isn't automatically correct simply because they are in a position of power. What you failed to note in your flurry of smugness is that we have a person who actually has first-hand information on the subject. Thus making his perspective, while not automatically right, far more relevant to the subject than that of a thousand slashdotters.

Re:Depressing

[psnyder]

I had seen a student who knew very little about biology do her homework by scanning in her book for specific phrases mentioned in the questions and looking for some semblance of an answer once she's found the phrases. By the time she was done, she hasn't even read the chapter but her answers would probably get her a "C"

This is the way I always did it, and it got me A's. In fact I was taught to do this in a 6th grade "Study Skills" class. Ironically, it's a very good skill to have in the "real world" as it's a way of quickly obtaining the information you need. You could even draw a parallel between this and Googling something or any kind of computer "find" or "search".

The ability to skim for an answer is not a problem. It's one of the solutions that children employ to deal with a school system that puts more emphasis on grades rather than inspiring them to actually learn a subject. The "inspiration" to get good grades works for some (especially with parental support), but with "average" being a 'C' (often a very shallow understanding), it can be argued that it's not working for most.

As you said, "It took a college education and many years of reading to undo these "lessons" and really discover the joy of writing essays."

Skimming is a skill. Learning a system, and figuring out to survive in it is also a skill. The emphasis on that 'joy' is what's usually lacking. Get a student inspired and the rest usually takes care of itself.

Indeed not new..

[wanax]

What you just described is what started happening on wall street at least 20 years ago. Once an algorithm err.. VAR is part of measuring score.. err risk, the people involved settle into two camps: Since there is money to be made, the traders.. err students quickly learn the weaknesses of the algorithm and start to write essays that make a farce of the assumed Gaussian distribution. The Execs raking in options.. er.. I mean the test administrators and the Board Members er.. I mean trusted graders who are paid a fixed sum + part of the throughput quickly learn that their compensation er.. filthy lucre is all based on getting a check mark from the computer, since 'computers are objective.'

And in the end, a test much like the current SAT, GRE, etc etc emerges: Unless you're a very top or bottom scorer, connections not performance are the heart of the matter.

Re:Depressing

[jonadab]

> more often than not such programs flag
> perfectly acceptable usage as erroneous

If that were the worst of it, they might actually be useful.

But in fact automated grammar correction software frequently *introduces* error into otherwise correct material. If the starting text is of even mediocre quality, the software actively makes it worse.

Re:Graduate Record Exam

[alba7]

In the German speaking countries many variations of this pastry are known. And they go by a lot of different names. For example in Austria we call it "Krapfen", and the people of Berlin call it "Pfannkuchen".

Only recently, through the cultural influence of the US (e.g. McDonalds and Starbucks) the name Berliner was introduced to a wider audience and is now known as an American pastry.

Re:Cheatcode

[Eudial]

Any sensible essay should include the sentence "This statement is a lie." That way, somewhere, the grading computer spins it's sparking head and flails it arms as it screams "DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE!"

It's a nice prospect.