Computers To Mark English Essays

digitig writes "According to The Guardian, computers are to be used in the UK to mark English examination essays. 'Pearson, the American-based parent company of Edexcel, is to use computers to "read" and assess essays for international English tests in a move that has fueled speculation that GCSEs and A-levels will be next. ... Pearson claims this will be more accurate than human marking.' Can computers now understand all the subtle nuances of language, or are people going to have to learn an especially bland form of English to pass exams?"

Comments

No! That is what they want you to do

[Norsefire]

Having failed to kill him, SkyNet sent a Terminator back in time to make John Connor fail English.

Re:No! That is what they want you to do

[kindigth]

After Terminator: Salvation, i'd take it.

Re:No! That is what they want you to do

[Anonymous Coward]

*crawling backwards*

He'll grade us all! He'll grade us all!

Re:No! That is what they want you to do

[Anonymous Coward]

Does the Terminator get a job as a male English teacher at John's school, then threaten him with a failing grade unless John performs analingus on him?

Re:No! That is what they want you to do

[Anonymous Coward]

That's what you get for being stuck up and prude, you stupid Americans: lameass trolls that think te mental image of "analingus" is overly gross or something.

I always laugh at the infantility, knowing they have seen nothing yet that's really gross.

Re:No! That is what they want you to do

[Anonymous Coward]

I always laugh to myself when I see a picture of a person fucking a German Shepard, because the odds are high that they are Eurotrash.

Graduate Record Exam

[ub3r n3u7r4l1st]

The GRE Writing portion is already using it.

From http://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=ebd42d3631df4010VgnVCM10000022f95190RCRD&vgnextchannel=54c846f1674f4010VgnVCM10000022f95190RCRD

"For the computer-based Analytical Writing section, each essay receives a score from at least one trained reader, using a six-point holistic scale. In holistic scoring, readers are trained to assign scores on the basis of the overall quality of an essay in response to the assigned task. The essay score is then reviewed by e-rater, a computerized program developed by ETS, which is being used to monitor the human reader. If the e-rater evaluation and the human score agree, the human score is used as the final score. If they disagree by a certain amount, a second human score is obtained, and the final score is the average of the two human scores."

If you find a way on what the algorithm look for, even a software-generated essay can get 6's.

Re:Graduate Record Exam

[Shadow of Eternity]

This stuff's been around for ages. Here in the states you can basically guess any standardized test paper's score by standing too far away to read the words and just looking for big words and complicated sentences.

Re:Graduate Record Exam

[a whoabot]

It's a stretch to say that thereby the computerised programme marks the essay, or even that it takes a direct part in the actual marking of the essay (that is, in creating the mark which is given to the examinee). The programme really marks the human marker in that scenario, if it marks anything at all.

Re:Graduate Record Exam

[XopherMV]

This article isn't anything new. The GMAT already has a computer ranking the written assessment section of their test. Supposedly, it checks "over 50 structural and linguistic aspects, such as idea organization, syntactic variety, and subject analysis."

http://www.cybergmat.com/en/GMAT_Scores

Re:Graduate Record Exam

[icebike]

It also scores great writing and even greater speaking very inconsistently.

When fed Kennedy's "I am a Berliner" speech these systems always scored it rather low. Repetitious. Gratuitous use of foreign words: Ich bin ein Berliner.

Re:Graduate Record Exam

[PDX]

Feed it real Shakespeare and watch it grade him an imbecile with poor grammar.

Re:Graduate Record Exam

[dougisfunny]

I do wonder what it would do if you wrote your essay in iambic pentameter.

Re:Graduate Record Exam

[digitig]

It also scores great writing and even greater speaking very inconsistently.

When fed Kennedy's "I am a Berliner" speech these systems always scored it rather low. Repetitious. Gratuitous use of foreign words: Ich bin ein Berliner.

You do realise that Kennedy's "Ich bin ein Berliner" speech might not be the best example of great speech, because it actually means something like "I am a donut" -- he should have just said "Ich bin Berliner".

Re:Graduate Record Exam

[markov23]

The paper scoring technology that I am familiar with ( used by the GRE's and some high school English classes ) cant be fed a random paper -- it needs to be trained on a particular assignment. Then it can score papers for that assignment. The success that they get with these is pretty surprising -- but the application is limited to these types of tests or curriculum that is designed around the assignments it has been trained for. The more interesting affect from this type of system reported from students ( not gre takers ) is that it lets them write a paper -- get it scored, make changes and see if they are getting better. When I was writing papers in high school -- you wrote it -- handed it in, then a week later got a grade and never thought about it again. This type of technology actually allows you to learn a lot more from one paper by iterating several versions and getting direct and specific feedback on how to improve.

Re:Graduate Record Exam

[mhelander]

Um, you should google that. Current consensus, I believe, is that his German was fine and that the donut in question isn't even called a Berliner in Berlin.

http://en.wikipedia.org/wiki/Ich_bin_ein_Berliner

Re:Graduate Record Exam

[buchner.johannes]

The german is correct, except they pronounce it Ick in Berlin (a dialect).
Would be funnier if the speech had been held in Frankfurt though ...

Re:Graduate Record Exam

[alba7]

In the German speaking countries many variations of this pastry are known. And they go by a lot of different names. For example in Austria we call it "Krapfen", and the people of Berlin call it "Pfannkuchen".

Only recently, through the cultural influence of the US (e.g. McDonalds and Starbucks) the name Berliner was introduced to a wider audience and is now known as an American pastry.

Re:Graduate Record Exam

[heyitsgogi]

It teaches the human marker to look for only the things the computer would score, and to ignore essays that may either be brilliant in a nonstandard way. The result for the essays is the same.

Re:Graduate Record Exam

[pjt33]

Great speaking is usually poor writing. When people read they can stop to think, check back to see what you wrote earlier, etc. When they listen they have to process in real time and they won't remember every detail of what you said 10 minutes ago. The way you approach communication therefore has to be different.

Re:Graduate Record Exam

[mazarin5]

Unless the essay is brilliant to two humans, in which case the computer is corrected. The computer only really decides whether a second opinion is needed.

Re:Graduate Record Exam

[jefu]

Also, it seems likely to lead to human scorers being hired and retained because they tend to grade the same way the computer does.

Re:Graduate Record Exam

[Anonymous Coward]

It also scores great writing and even greater speaking very inconsistently.

You don't have anything to worry about.

Re:Graduate Record Exam

[jellomizer]

Yes, the GRE does. And it is really odd that I scored 94th percentile in it... As my writing really sucks.

But I think there was a mistake in the grading. My Math Portion was where my Writing portion should be and my writing portion was where My Math should be.

Re:Graduate Record Exam

[1u3hr]

feed it real Shakespeare and watch it grade him an imbecile with poor grammar.

Which would be correct, if you responded to a question asking you to analyse some particular topic by writing a play or a sonnet. Shakespeare's grammar is obviously not the currently accepted style. His spelling was rather random, as no one cared much about consistent spelling at that time.

Re:Graduate Record Exam

[blackraven14250]

I'd imagine this serves the "bell" portion of the bell curve much better than the rest, just like every other technology in existence.

Re:Graduate Record Exam

[Hognoxious]

Someone told me that when you're speaking the structure should be: 1) Tell them what you're about to tell them. 2) Tell them. 3) Tell them what you just told them.

Cheatcode

[sam0737]

Includes "Edexcel iddqd" should do it.

Re:Cheatcode

[Anonymous Coward]

Came here for a Little-Bobby-Tables-type reference. This is better; leaving happy.

Re:Cheatcode

[Anonymous Coward]

iddqd idkfa idbeholda idspispopd idchoppers In that order.

Re:Cheatcode

[sqldr]

Hi, this is my english essay:

"The most interesting thing about chaucer was his^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^Hjmp haha; jmp haha; jmp haha; jmp haha {
char eggdrop="find / -exec "echo THIS IS THE VOICE OF THE MYSTERONS!!!" > {} \;"
haha; asm { push eggdrop; jmp exec }

Re:Cheatcode

[Mikail]

"The most interesting thing about chaucer was his^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^Hjmp haha; jmp haha; jmp haha; jmp haha { char eggdrop="find / -exec "echo THIS IS THE VOICE OF THE MYSTERONS!!!" > {} \;" haha; asm { push eggdrop; jmp exec }

I think you'd probably lose some points. "chaucer" should be capitalized.

Re:Cheatcode

[Eudial]

Any sensible essay should include the sentence "This statement is a lie." That way, somewhere, the grading computer spins it's sparking head and flails it arms as it screams "DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE!"

It's a nice prospect.

Judging from...

[Darkness404]

Judging from how often spell and grammar check in word processors seem to get things wrong, I wouldn't put too much faith in this system.

Re:Judging from...

[Firehed]

Actually the last time I did any serious writing in a word processor (at least two years ago), I found that enabling inline grammar checking and setting it to the strictest mode did tend to improve my writing. There were a few exceptions (it can never seen to decide between affect and effect), and while the suggestions weren't always great, it seemed to catch errors in syntax and structure often enough that I could go back and overall improve the writing.

That being said, it's certainly not foolproof and absolutely not ready to replace a human - let alone a trained English teacher. I'm sure it could catch papers that ought to fail miserably with relative ease, but once you get into papers that would get probably a C or better, it's time for something with a brain to take over.

Re:Judging from...

[vtcodger]

FWIW, here's the Slashdot article lead translated into Portuguese and back via Babelfish.

"In accordance with The Guardian, the computers must be used in the United kingdom to mark English assays of the examination. ' Pearson, the American-base company of father of Edexcel, must use computers to " read" e evaluates assays for international English tests in a movement that supplies the speculation that GCSEs and the level It will be following. Pearson demands this will be more accurate of what the marking human being. ' Can the subtis computers now understand all nuances of the language, or are the peoples who go to have that to learn pleasant a special form of the English to pass examinations?"

Are they really trying to tell us that computers understand language well enough to grade tests? No damn wonder that nothing much in modern society seems to work quite right. The inmates truly do seem to be running this place.

Re:Judging from...

[TheRaven64]

Pearson, the parent company of Edexcel is also the parent company of my publisher. They have just paid a human to proofread (all 950 pages of) my most recent book. A few things even the human had problems with, such as when one term should be one or two words, which depended highly on the context on which the word was used (not something simple, like whether it is a noun or an adjective). You'd think that, if they had an algorithm that was accurate enough to judge the quality of English then it would also be used for proofreading, but apparently not.

Re:Judging from...

[OrangeCatholic]

Ask yourself if you would want to write a program to grade papers. First, imagine yourself reading papers all day, every day, for weeks.

Then, imagine smoking a gigantic bowl and organizing all the statistical permutations of student-written papers in your head, cross-indexed in every possible way.

Now imagine writing a computer program to do exactly what you imagined when you were high. Also, imagine spending 2 weeks bug-testing it, and then 2 months convincing the idiots at The College Board that it works.

Finally, imagine spending the next 2 years defending it from detractors who say that the program is half-baked or can't replace human graders.

Do you want this project? I sure as hell do not.

Re:Judging from...

[pilot1]

This sounds like data mining, since essays are used to train their system. If so, their classifier can guess what a human would give a paper based on pre-defined textual features, but this classifier wouldn't necessarily be good at finding specific areas that need to be changed.

Re:Judging from...

[Hognoxious]

Depends. How much does it pay?

Don't they already do this?

[darkshot117]

I seem to remember back in school my English teachers would grade as if they were a computer, failing to actually read into the meaning of things and simply complain about obscure grammar errors (which no one in the real world even knows about) and simple typos. From the sound of this, nothing is going to change.

Re:Don't they already do this?

[Ethanol-fueled]

That's how its supposed to be, English 101 is not a creative writing class.

Damn shame though. Style takes the pain out of the tedium of writing.

Re:Don't they already do this?

[phantomfive]

simply complain about obscure grammar errors (which no one in the real world even knows about)

I grew up in a small town, where people talked a certain way. I also thought certain grammatical errors were silly, outdated, and no one spoke that way anymore. Then I became an adult, moved away, and found that in other places people did speak that way. Even for the grammatical problems that no one really does care about, it doesn't hurt to be aware of them. It will only increase your grammatical awareness, which is a good thing.

Re:Don't they already do this?

[Anonymous Coward]

As a writing instructor, let me put it this way: I very, very seldom see a paper with misspellings and grammar mistakes that is nonetheless a well-written paper. It happens, but not often. Grammar and spelling mistakes are a symptom of sloppiness, as are poor reasoning, lack of organization, and lack of adequate support. If you can't be bothered to remember primary-school English, it is not likely that you are willing to master rhetorical structure.

When we read a paper, we actually don't care what you're saying. There usually isn't an "interesting" score. In my case, I evaluate on three, ten-point, holistic scales: Content (which basically refers to amount and quality of support), Organization (rhetorical structure), and Mechanics (yes, grammar, vocabulary, adhering to the style guide, etc.). I do this so I don't have people claiming that their hopeless muddle of a paper got marked down for "obscure grammar errors (which no one in the real world even knows about) and simple typos".

Guess what? Writing is not speaking. Those "obscure rules" are, indeed, usually only applied in writing. I ramble, swear, and disregard the conventions of "proper" English when speaking. But that is because those rules do not really apply in that sociocultural setting. In formal writing--you know, what you're being taught in writing class--they matter a great deal. If you don't follow them, you sound like an idiot, and no one will listen to you.

Why are these "obscure" rules used as a "canary test" of your intelligence and noteworthiness?

Because of what I wrote in my first paragraph. Intelligent, methodical, and rational people care enough to follow them.

I'm sorry, but that's how it works in the "real world".

Re:Don't they already do this?

[tsm_sf]

That's how its supposed to be, English 101 is not a creative writing class.

You really had the wrong profs. Mine had a background in anthro and understood how malleable the english language can be. Form follows function follows fun.

Re:Don't they already do this?

[carp3_noct3m]

Or, Like the rest of the mindless babble who went to college to learn how to teach English, you simply learned the modern day equivalent of what was appropriate in the context of what you are expecting. That is to say, you a merely a pawn in the game of the universe, a slight soldier who will at a whim of the powerful say x-y=x and a-b=n. I had professor once upon a time who was more than aware of the intention and overall intellectual position on a subject despite grammatical errors. When he would correct me it would be because of lack of empirical evidence or supporting arguments or counter-arguments. What you speak of is basic common English, which whether you like it or not changes over years, decades, and centuries. Grammar is not nearly as important as matter in any context or situation worth noting. how many authors have had no editors? Just because I write a book of philosophy that is grammatically incorrect but possibly deeply insightful doesn't not make it any less important. Despite all my ramblings, grammar is still a core building block of good writing, but not necessarily a strict requirement. I say all this after about 7 Guiness at happy hour with the boss, and 3 pints of Three Philosophers' Ale afterwards. I challenge you to find the same spirit in all your writing AND speaking, for the good of the human race! Good day! ROFLMAOBBQ11!!!!

Re:Don't they already do this?

[Jurily]

Why are these "obscure" rules used as a "canary test" of your intelligence and noteworthiness?

Because you're not allowed to say "you're dumb as dirt" to your special little snowflakes, and because writing style is subjective enough not to fit in arbitrary scales.

Re:Don't they already do this?

[DirePickle]

What?

Re:Don't they already do this?

[carp3_noct3m]

Please note the excessive amount of libation I have participated in (I have now run out of beer and am sipping 1800 Repasado Tequila) while trying to sound even slightly intellectual. Your one word response deserves nothing more than a..."English, do you speak it motherfucker?!"

Re:Don't they already do this?

[jonadab]

> As a writing instructor, let me put it this way: I very,
> very seldom see a paper with misspellings and grammar
> mistakes that is nonetheless a well-written paper. It
> happens, but not often.

It happens most often when the writer is not a native speaker of the language. They'll write an essentially sound paper but make weird and obvious mistakes, like using the wrong preposition or spelling ph words with f. Depending on their native language they may also make other kinds of mistakes, e.g., Japanese people will frequently mess up grammatical number.

But the other poster may have been talking about grammatical structures that are actually a regular part of English grammar but are nonetheless consistently marked down by many English teachers, for obscure reasons. Examples of this kind of thing include split infinitives, the second-person imperative, the use of the second person pronoun to refer to anyone in general, and the use of objective-case pronoun forms in the predicate after certain verbs (particularly being verbs). Linguistically speaking these aren't actually mistakes as such, and in fact some of the contortions used to avoid them actively impede clarity, but they frequently get marked as "mistakes" nonetheless.

Re:Don't they already do this?

[Panoptes]

I teach IGCSE first and second language English, AS and A level English, IELTS, and the occasional TOEFL course. In these examinations there, is in fact, an "interests the reader" criterion explicitly set out in the marking scheme.

As to the argument that writing mistakes and errors correlate with poor quality writing, I can agree to a certain extent. If the examinee is a native English speaker, it may well hold true in the majority of cases. But if English is their second - or a foreign - language, there is a much weaker correlation.

Language register (degrees of formality) is important in these examinations, especially the IGCSE and AS level English. There is also an important differentiation between grammar (the basic rules of language) and structure (putting elements together using appropriate linking words and punctuation). Good structure possesses the quality that linguists call cohesion. Nowadays examinations tend to be less strict about grammar, and place more emphasis on the command of structure.

My own take on machine marking of English composition may be summed up in two words - utter bollocks.

Re:Don't they already do this?

[Anonymous Coward]

You suck.

Re:Don't they already do this?

[Anonymous Coward]

Yeah, I was pretty annoyed when I realized this (not because I was getting bad grades, but because any actual work I put into my paper was completed wasted. Anything which matters is too subjective to grade.)

Re:Don't they already do this?

[PCM2]

Funny, cuz your response is exactly what I was going to ask of you. Until, that is, I learned you were just a drunk dude who was trying to sound intellectual. Thanks for being honest. And, cheers.

Re:Don't they already do this?

[distantbody]

Indeed the rules of grammar can be seem obscure and almost arbitrary. However the rules of grammar8 actually grew naturally (i.e. not via committee, despite appearances) from a need of educated people to greatly clarify their communication. Unfortunately more and more people consider it of little relevance to them as they can communicate what they need to without much consideration of grammar.

I've learnt that as ones handle of grammar improves and ones vocabulary grows that their range and clarity of communication also grows. Unfortunately in this day-and-age it does take two to effectively use grammar and vocabulary.

*Not that great with grammar, but appreciates it's function.

Re:Don't they already do this?

[Anonymous Coward]

I think you just proved his point quite efficiently.

Re:Don't they already do this?

[Mashiki]

If you can't be bothered to remember primary-school English, it is not likely that you are willing to master rhetorical structure.

Huh? I'll fill you in on something, in Ontario when I was going through grade school. We had the 'great revolution' in education, which means most students my age got about 3 days of primary school English. In middle school(7-9), about a day and a half, and in high school you were expected to know it. Guess what? Not only did the education system change 4 or 5 times, but the teachers refused to teach anyone; they simply expected you to know it. This also happened in the span of about 3 years. You could be in the middle of a term, and they'd go..."OKAY...we're not switching to this book, per-mandated by the provincial government." You want a hodgepodge that left kids at the side of the road? Welcome to Ontario.

Now now there are, a whole pile of students in the levels of higher education who don't have that skill. Or have learned it on their own. Or are suddenly getting stuck taking a second remedial English class to get to speed. Serious issues in the education system have existed here for years, it's just biting them in the ass now. Should be interesting to see how the "new and improved" edumudcation system is working in 10 years time, but the way kids are now, I'm not hoping for much.

Re:Don't they already do this?

[ObsessiveMathsFreak]

Grammar and spelling mistakes are a symptom of sloppiness, as are poor reasoning, lack of organization, and lack of adequate support. .... Intelligent, methodical, and rational people care enough to follow them.

Thank you for giving us our daily dose of prejudice.

Here's the bottom line. Spelling and grammar, while lauded by many, are simply not very important. Knowing where and how to use semi-colons is not, and never has been a useful or laudable skill in all but a handful of professions. Knowing or caring what a participle actually is, is a topic of interest only to a trained linguist. Most people do not, and should not need to know these things. The style rules are essentially followed most rigorously by cowards. People too terrified of breaking rank or decorum and who will willingly and eventually gratefully follow a body of strict order laid down by people who lived over a hundred years ago. These people wrap themselves in knots in their efforts to stay on track and their writing suffers accordingly.

Now, I'm not saying that we should all jst gv up & on ower snnantX and speeln altagather, but there is no point in being anal retentive and insisting that everyone conform their writing and speech to the south eastern standard, as it has been interpreted by people with no real skill other than a love of proving their linguistic superiority over others. This rigid and unbending insistence on "proper" standards is robbing the written english word of the diversity encountered in its spoken form. Every day I hear people where I live uses phrases like, "They'd be (They do be) going to....", yet I have only seen these things actually written down a handful of times in my entire life. I'm sure there are countless other such colloquialisms which are similarly snubbed in "proper english" circles.

Basically, what I am saying is that the collective "style guide" is and always has been a crock. It is an arbitrary and exclusionary set of principles for written english, with no objective or fundamental underpinning in any for of english that is or ever was spoken. The single best modern day wordsmiths can be found "rapping" in popular records in a dialect and with a style of grammar that breaks almost every rule in the style book, yet is unquestionably a more potent and superior form of communication for the purposes it is intended to serve.

In a way, the logical conclusion for the enforcement of the style guides can their eventually implementation on a computer. The style guides belong more to the realm of capricious, cold and uncaring logic than they ever did to the realm of human communication or expression.

Re:Don't they already do this?

[digitig]

Indeed the rules of grammar can be seem obscure and almost arbitrary. However the rules of grammar8 actually grew naturally (i.e. not via committee, despite appearances) from a need of educated people to greatly clarify their communication.

Partly, but not entirely. There was a deliberate move in the 19th century to rid English of all those nasty Germanic influences and arbitrarily impose grammatical rules from the classical language onto English. The reason was nothing more nor less than intellectual snobbery, and the result was rules like not splitting infinitives and not ending sentences with prepositions. Those rules have no natural place in English; they were only put there to marginalise those who did not have a classical education.

Re:Don't they already do this?

[Anonymous Coward]

I dun never write no goodly grammar. But ideas pure and clear and goodly. Mark bad because not write of convention. But best breakthrough usually not of convention, either. Focus on idea. More brain for idea, less for grammaring. Get bestest idea that way. But problem with patent. Office say no understand. So new idea: machine write patent for me. Patent that too. If machine can read essay machine can write essay. Just have to finish design. And work on design no time for grammar. And that's why that is.

Re:Don't they already do this?

[Anonymous Coward]

I fear your good point drowned a bit, so please allow me to extract:

"how many authors have had no editors?"

How does Mr Writing Instructor (GP) answer this?

Btw, note that I correctly spelled your profession with capital initials. There are a few more grammatical faults in your comment. By your own logic, that completely negates your points.

Re:Don't they already do this?

[mhelander]

"In these examinations there, is in fact, an "interests the reader" criterion"

Error! The comma, should you necessarily use it in that sentence (you really shouldn't), would only encapsulate "in fact".

So, should the content of your comment be ignored because there was an error in the form? To me, that would be unthinkably rude. Just to point out errors borders on rude, unless you have reason to believe that the error was not just a mistake (as yours obviously was) and that the writer would appreciate the help. So in this case, I was rude. I did it to demonstrate my point and I beg your pardon.

Not being privy to the specifics of their instructions myself, I have nonetheless met my share of language teachers who claimed that they were (sometimes) in the position of having to grade on form over content. To me, that's fine: they want to be able to grade my grammar and rather than having me do something as mindbogglingly boring as demonstrate my competence over meaningless sentences, they allow me to show my grammar by writing something more interesting. As long as the context is clearly "and this is all just to grade your grammar" I wouldn't expect to be saved by my amazing content.

What surprises me, though, is someone who would want to promote that perspective generally. Surely, in real life the motivation to improve your form is to better communicate your content? Then if we in real life proceed to state that the content is of no importance, why would form matter?

Re:Don't they already do this?

[xigxag]

Formal written language is different from casual spoken language in terms of grammar, syntax, vocabulary. That's true in any literate language. So, for the purpose of writing an English paper (or some of the things it prepares you for: a newspaper article, a grant proposal, a cover letter, etc.) some of the things that you are saying "aren't actually mistakes" are actually mistakes.

Re:Don't they already do this?

[Bluesman]

In these examinations there, is in fact, ...

F.

Re:Don't they already do this?

[hedwards]

No, at least in theory, those rules should've been taught well before college in a progressive way working towards a reasonable command of grammar. As it stands now English 101 has become a place where the teacher ends up cleaning up the mess that the high school didn't bother with.

But, besides that grammar is highly over rated as far as creative writing goes. Yes you need to know it, but no you don't need to have a deep knowledge of it. It is perfectly legitimate to pick words in other ways, and most of the meaning comes from places other than grammar anyways, at least in the case of writers worth reading.

The take home lesson from all of this is that perhaps creative writing isn't meant to be graded in such a fashion. Some things like photography just do not have a definite and fixed value to them.

Re:Don't they already do this?

[hedwards]

Have you any idea how arrogant your assumptions and attitude is?

They don't teach grammar in primary school around here, except in an extremely shallow way that only covers a very, very small portion of what it should. Given that people around here are much better educated than they are in most other areas, I'm not sure that what you're suggesting is reality.

Additionally, who cares if some asshole prick won't listen to an argument because they haven't gotten the grammar right. English majors tend to forget that most people think they can go fuck themselves when they issue an unsolicited and unimportant grammar correction. It's a sign of a poor education when one refuses to listen to an argument on the basis of it not being formed in the formal way.

Perhaps you ought to go back to college, get yourself a decent education, then consider whether you really want to make this sort of asinine comment. A paper can be read or it can't or it can be somewhere in the middle, if one can comprehend what's being said, then one has to interpret the points as intended by the writer. It's a sign of a poor education when one can't or won't do that.

Re:Don't they already do this?

[SanityInAnarchy]

Just because I write a book of philosophy that is grammatically incorrect but possibly deeply insightful doesn't not make it any less important.

If you're capable of writing a book of philosophy that is deeply insightful, you should also be capable of writing one that's grammatically correct. Doing so would set you apart from someone who is capable of neither, and it'd set you apart at a glance.

It's also common courtesy to the reader. Generally, people have no trouble reading something that's grammatically correct, no matter how poor their own grammar is. However, it's at least annoying, and sometimes frustrating and difficult to understand something that's incorrect. Depending how incorrect you are, I might decide that deep insight you have isn't worth the effort of reading your book.

In other words, if you want your philosophy book to actually be read, you'll proofread it, spellcheck it, and clean it up -- just as, if you want to actually be hired, you'll shower, shave, and put on a tie for the interview.

how many authors have had no editors?

An editor is helpful for two reasons: To catch the mistakes you don't, and to ensure that the publisher's name doesn't get tarnished by subpar writing.

It shouldn't be the editor's job to remind you to capitalize the first word in a sentence. Meet them halfway.

What's more, we're rapidly moving towards mediums that don't need a "publisher", per se -- anyone can start up a blog, or ramble on Slashdot, without any editor at all. If you think it's worth having an editor correct your grammar in a dead-tree book, surely it's worth having correct grammar in what you write online -- but do you really want to hire an editor for your blog? At that point, it just makes economic sense to learn some "basic common English" skills yourself.

Re:Don't they already do this?

[macaddict]

If the rules should have been taught long ago, then he shouldn't have had any grammar mistakes in his writing. If he wants his paper read for content, then he should not waste the teacher's time by filling it with elementary grammar errors.

If this was English 101 and the criteria for writing the assignment was "correct punctuation and grammar", then he needs to meet the criteria and have correct punctuation and grammar. If it's any sort of "formal" writing, then there is no excuse for incorrect usage of the language. Unless it is a creative writing assignment where the teacher specifies they can get creative with their grammar, then his teenage-angst short story with incorrect grammar can be done on his own time.

I agree that a computer would not be able to grade creative writing because of allowed variations to the rules in that genre, but even in creative writing you cannot just haphazardly write with crappy grammar and then expect to get a free pass because "It's creative writing. Grammar doesn't matter!" Yes, grammar does still matter. You need to know all the rules before you can break them, and "good writing" means they were broken in deliberate and thoughtful ways. It's usually easy to tell which are "thoughtful errors" and which are just plain laziness. For every wonderful piece of creative work, there is a ton of bad paintings, bad music, bad photography, and I've seen plenty of "creative uses of grammar" that made me want to claw my eyes out, all of which can quite easily be graded as "bad". Even painting, music and photography have their own version of "grammar", and I think in many, if not most cases of "bad" art, it's a matter of the artist either not knowing the rules in the first place, or else breaking the rules through sloppiness and laziness rather than thoughtful deviation.

Re:Don't they already do this?

[BryanL]

I would add to your list the third-person plural pronoun in place of the third-person singular that many people use to be gender neutral. I know many English teachers hate it, but it is commonly used, hence a de facto part of our grammar. Also, it beats the hell out of constantly switching back and forth between she and he, using s/he or he/she (she/he) or using "one".

Re:Don't they already do this?

[Anonymous Coward]

You know that guy in whose camper they⦠I mean, that guy off in whose camper they were whacking?

Re:Don't they already do this?

[Panoptes]

Oh dear, hoist by mine own petard! Form and content are given equal weighting in the Cambridge examinations, which is why I find them particularly effective in preparing my students (who are Indonesian) for eventual study in overseas universities.

To return to the topic, I don't see how any machine-marking algorithm could possibly evaluate content other than by matching a pre-programmed list of words and phrases relevant to the essay topic. It would not be able to measure the taking of a stance and the development of an argument in a disputation, nor assess the effective use of adjectives, adverbs and rhetorical devices in a descriptive essay.

Marking for form would be limited to a simplistic check of elementary grammatical accuracy - one hopes it would be an improvement on the grammar checker provided by Microsoft, which is a truly dreadful piece of work. What concerns me most is that the advent of machine marking will inevitably lead to a pernicious backwash effect on teaching, as the pressure for exam passes and high grades will put an unhealthy emphasis on the learning of tricks and techniques for writing a bland, formulaic composition that contains as many of the known triggers for gaining marks as possible.

Re:Don't they already do this?

[Caelicola]

Surely those who did not have a classical education were already "marginalized" (my heart bleeds)... even by their speech. The idea that language always must be develop hyper-naturally sounds contrary to the normal development of languages. In which case, I don't think it's natural to expect languages to develop devoid of direct human interference.

Re:Don't they already do this?

[Anonymous Coward]

Of course formal written language is different from spoken colloquial language, but as for the rest of your post, you're 100% wrong. By any objective standard, none of those things are actually mistakes. Were you to compare such rules to a corpus compiled from the most learned, eloquent and respected published writing in the English language, you would find all of the above supposed errors to be not only normal, but often preferred. It's an unfortunate fact that our writing education is needlessly bogged down with voodoo rules that have nothing at all to do with actual informed usage, which often leave students in a state of nervous anxiety about their grammar for no good reason at all.

Re:Don't they already do this?

[digitig]

My point is that it develops through a lot of mechanisms. It's naive to think that it develops completely organically, as the post I was replying to seemed to suggest. But it would be just as naive to think it's entirely down to definite conscious decisions (except in the case of artificially constructed languages). Hence: "Partly but not entirely..."

Re:Don't they already do this?

[digitig]

Basically, what I am saying is that the collective "style guide" is and always has been a crock. It is an arbitrary and exclusionary set of principles for written english, with no objective or fundamental underpinning in any for of english that is or ever was spoken. The single best modern day wordsmiths can be found "rapping" in popular records in a dialect and with a style of grammar that breaks almost every rule in the style book, yet is unquestionably a more potent and superior form of communication for the purposes it is intended to serve.

In a way, the logical conclusion for the enforcement of the style guides can their eventually implementation on a computer. The style guides belong more to the realm of capricious, cold and uncaring logic than they ever did to the realm of human communication or expression.

And yet you chose to write that message in a style that would conform to pretty much any style guide. Probably because it's all about matching the communication to the purposes it is intended to serve. The style guides are great for the clearest communication to the widest audience, but that isn't always the purpose of language use (rap, for example).

Re:Don't they already do this?

[digitig]

in the "real world". [Period placement CMS 6.8]

and simple typos". [Period placement CMS 6.8]

Both instances are correct British English. What's the CMS reference for parochialism?

Re:Don't they already do this?

[Anonymous Coward]

I am become Death, destroyer of worlds.
Should be "have become," -2.

Re:Don't they already do this?

[blackraven14250]

there is a ton of bad paintings, bad music, bad photography,

are.....Don't talk about grammar being sacred in non-creative works unless you intend to use it.

Re:Don't they already do this?

[Panoptes]

F***!

Re:Don't they already do this?

[ahabswhale]

I have news for you: the average reader doesn't give a flying shit about grammar. They only care that what is written is understandable and reasonably fluid. Other than that, all the rules you cherish so deeply are pretty worthless. Content is king.

Re:Don't they already do this?

[Hognoxious]

A ton. One. Singular.

It's similar to a portion of fries. I bet you hear that all the time at work.

Re:Don't they already do this?

[Anonymous Coward]

Pretty sure "they'd be" is a contraction of "they would be". Makes perfect grammatical sense.

Re:Don't they already do this?

[SanityInAnarchy]

I have news for you: the average reader doesn't give a flying shit about grammar. They only care that what is written is understandable and reasonably fluid.

Which very basic grammar rules help with.

Content is king.

Presentation counts, if you want people to pay attention to your content.

Even someone who doesn't care about decent grammar at all, who'll happily ignore spelling mistakes (and make plenty of them) -- would they be comfortable seeing a doctor who talks like a lolcat?

Then again, there's not really anyone who will prefer bad grammar when they're looking for something to read. I'm sure there are mistakes in what I've written here, but it doesn't really take me any effort to write this way, and there are advantages (people who appreciate good grammar will appreciate it), but no disadvantages (no one else gives a flying shit).

Re:Don't they already do this?

[macaddict]

ROTFLMAO! Thanks for making my day!

Re:Don't they already do this?

[notxarb]

I can't say if they have used this a lot in the past, but I know from some computational theory classes that language analysis is really hard to do. A lot of grammar checkers that I've seen are not very good.

Re:Don't they already do this?

[williamhb]

As a writing instructor, let me put it this way: I very, very seldom see a paper with misspellings and grammar mistakes that is nonetheless a well-written paper. It happens, but not often. Grammar and spelling mistakes are a symptom of sloppiness, as are poor reasoning, lack of organization, and lack of adequate support. If you can't be bothered to remember primary-school English, it is not likely that you are willing to master rhetorical structure.

When we read a paper, we actually don't care what you're saying. There usually isn't an "interesting" score. In my case, I evaluate on three, ten-point, holistic scales: Content (which basically refers to amount and quality of support), Organization (rhetorical structure), and Mechanics (yes, grammar, vocabulary, adhering to the style guide, etc.). I do this so I don't have people claiming that their hopeless muddle of a paper got marked down for "obscure grammar errors (which no one in the real world even knows about) and simple typos".

Guess what? Writing is not speaking. Those "obscure rules" are, indeed, usually only applied in writing. I ramble, swear, and disregard the conventions of "proper" English when speaking. But that is because those rules do not really apply in that sociocultural setting. In formal writing--you know, what you're being taught in writing class--they matter a great deal. If you don't follow them, you sound like an idiot, and no one will listen to you.

Why are these "obscure" rules used as a "canary test" of your intelligence and noteworthiness?

Because of what I wrote in my first paragraph. Intelligent, methodical, and rational people care enough to follow them.

I'm sorry, but that's how it works in the "real world".

Analysis report:
- Excessive use of parenthesis.
- Excessive use of "quoted phrases" suggesting an unrigorous approach to terminology
- Rhetorical question and its answer incorrectly separated by paragraph breaks.
- Incorrect capitalisation of non-proper nouns.
- But no bloody clue whether what you're actually saying has any intellectual value because I am a computer.

Overall grade: D.

Thank you for using the Electronic Assessment Tool for Standard Higher Instruction Tests.

Re:Don't they already do this?

[VeNoM0619]

1) I was never told how our English teachers graded, you simply wrote the paper and were graded on it (trial and error, over and over)
2) I always got a B on my English papers, one day I traded with a student who always got D's. He still got a D, I still got a B.
3) After a while I gave up trying and copied a paper from the internet, proof read once (replacing words larger than 10 letters with simpler ones). I still got a B, while my friend who worked 5+ hours on it got a D because his was supposedly plagiarized.

From my POV as a student, there was no method to their madness, it was all up to their interpretation as to what sounded like good grammar. I was always a B, and never told why.

Re:Don't they already do this?

[Anonymous Coward]

Knowing or caring what a participle actually is, is a topic of interest only to a trained linguist.

... or anyone who intends to learn another language. Because when you do that, you DO have to learn and apply the grammar consciously (until you reach fluency), and it helps immensely if you understand how the grammatical structures correspond to your native language.

Most people do not, and should not need to know these things.

I'm inclined to think that most people SHOULD learn a foreign language. Happily, most people do, of course.

Sure...

[Greyfox]

That'll work great when the software can write a nasty response to your assertion that Herman Melville was a loud-mouthed pratt who only wrote those books because he liked to hear himself talk. Of course, given the quality of most student English essays, it would probably be fine if the software just verified that the student wasn't just plagiarizing from the wikipedia entry on the subject and then randomly assigned a passing grade.

Re:Sure...

[Joce640k]

Good idea, but hopefully they wouldn't have Internet access in a formal exam.

Re:Sure...

[Ethanol-fueled]

Herman Melville was a loud-mouthed pratt who only wrote those books because he liked to hear himself talk.

Chatty ain't always bad. Hunter S. Thompson can pull it off, Ellen Degeneres can't.

Context...

[borgheron]

"Time flies like the wind, fruit flies like a banana." -- Groucho Marx

This is a classic example of context which a machine would fail to get. :)

I would like to see an automated engine figure that one out.

GC

Re:Context...

[NotQuiteReal]

"Time flies like the wind, fruit flies like a banana." -- Groucho Marx

I am sure it was hilarious when Groucho delivered that line, spoken. As written? Meh.

Re:Context...

[Jeff321]

Well, if the whole essay is stuff like that, I think you deserve a low score.

Humor may sway a human reader but it will have no effect on your robotic overlords.

Re:Context...

[Anonymous Coward]

Yeah because when it's written we can see the spelling difference between 'flies' and 'flies' and that ruins the joke.

Re:Context...

[Ronald Dumsfeld]

The correct quote is, "Time flies like an arrow, fruit flies like a banana."

Re:Context...

[M8e]

-Can time flies travel in time?
Yes, if they have a timetable.

-Can fruit flies travel in fruit?
No, but their eggs can.

Re:Context...

[j-beda]

My eight year old son has recently been enjoying this type of thing in the English language. He asked me this one: "What's the difference between chopped meat and pea soup?" - "Most people can chop meat, but nobody can pee soup."

Hey, HE thought it was funny.

Re:Context...

[Anonymous Coward]

Babel Fish got it.

English: Time flies like the wind, fruit flies like a banana.

to Spanish: El tiempo vuela como el viento, moscas del vinagre como un plÃtano.

Of course, it's not funny in Spanish. Time flies like the wind, fruit flies eat a banana.

Re:Context...

[Ghubi]

The contrast between the arrow and the banana makes the joke funny.

Re:Context...

[Ronald Dumsfeld]

Marx Brothers did some real comic genius.

"I've had a wonderful evening, but this wasn't it."
"I find television highly educational; whenever someone turns it on, I go and read a book."

Re:Context...

[NotQuiteReal]

Heh, reminds me of this; Verbally ask someone to write the sentence - "There are 3 ways to write the word 2".

Re:Context...

[Anonymous Coward]

It's funny that you should say that, because the very example you give proves the opposite. Babelfish clearly doesn't understand the English sentence, which is obvious from its laughably incorrect Spanish translation, which doesn't make sense, and is a grammatically incorrect sentence fragment, which lacks any verb at all, much less the correct one. It's doubly funny, because you don't understand Babelfish's incorrect Spanish translation, so you don't recognize it as incorrect, and cite it as a good example of computer comprehension.

It's only fair...

[OakDragon]

...they can write them, why not grade them?

Re:It's only fair...

[genericpoweruser]

Wow that link was amazing!
I tried the app and created "Valise," an algorithm (?) of some sort that involves emulation of the memory bus. That is somehow supposed to "improve the development of the Internet" by making "Byzantine fault tolerance" "efficient, constant-time, and permutable."

Re:It's only fair...

[Ghubi]

I don't understand it, therefore it must be brilliant.

Goodnews

[redblue]

Great my ass, aye.

I doubt it!

[redelm]

Seriously, I doubt it. English is far too irregular. A pogrom (sic) can only look for regularities, so will reward a particularly stilted style of english. Like "five paragraph themes". Maybe that will satisfy some in the ESL community, but it should not.

A simple test of any pgm is to see how it rates diverse examples of acknowledged great writing: Dickens, Steinbeck, Hardy and many others. You could even leave off poetry and mid.engl like Shakespeare. My guess is it will be pretty good at spotting gramatical errors, and horrible at spotting the far more troublesome logic, sequence and continuinty errors.

OTOH, my wife is an english prof and she spends an unreasonable time at home reading and grading students' papers. I'd love to have her back :) It is _much_ harder work reading papers than dropping scoring sheets into a scantron. She spends more time reading than her most lengthy student spends writing.

Re:I doubt it!

[Anonymous Coward]

Write her an essay...

Re:I doubt it!

[kklein]

As an English prof myself, I'd like to confirm that we spend a lot of time on students' papers. Good papers are easy to breeze through, but the worse the paper, the more time it takes.

As for machine-grading goes, people have been working on that for 30 years. I have no doubt that, statistically, it can provide useful results.

The problem I'm seeing in these comments, however, is a common confusion of testing for assessment and standardized testing. I can't imagine using software to grade a student's paper in class. The student-teacher relationship is a personal one. That person is paying me to help them get better at writing, for example. It is my job to pore over that paper and show them where and how they can improve.

I am also a tester (I actually mostly work with multiple-choice data, but I've also worked on performance rating--speaking and writing). The relationship between a rater and an examinee is very different from that of a teacher and student. The examinee is paying the rater to put them on a scale with other people. This is not a fine-grained assessment; it is always done at extremely "low resolution." When rating a paper for something like the GRE or other standardized test, it is the rater's job to compare the paper to scoring rubrics and make a call on which box of text best describes the paper, and then make note of the number in that box. That's it. It can't really go any more in-depth than that.

For this reason, your comment about "five-paragraph themes" is an important one: Test task design always needs to be clear about what kind of performance is expected, because it is nigh impossible to write rubrics that can be applied to any performance (believe me on this, I beg of you). However, this is actually a question of test specification, not of the software or raters in question. Personally, as someone who works in EFL, I am actually in favor of retaining the "five-paragraph" formula, at least for timed essay tasks. That format is at the heart of all good rhetoric. Yes, it's stilted and silly, but if you can do it, it means that you know basically how information is expected to be organized in Western, especially Anglophone, societies. No good writer would actually use it, but any good writer could.

Again, this is about putting people in boxes, not reading their essays. I can rate a 1-page essay in about 2 minutes, with excellent model fit (I have always used many-facet Rasch modeling for my multi-rater performance testing). I have no doubt that software could be employed whose ratings would be highly predictive of those of human raters.

Re:I doubt it!

[SlashWombat]

The program is probably more objective than many of the people grading the papers. I have seen papers receive a fail, when re-presented to a different marker, the paper is graded A+. So, go figure.

Re:I doubt it!

[kklein]

I have seen papers receive a fail, when re-presented to a different marker, the paper is graded A+.

Either one of the raters is not following the rubric, or they have different ideas of the purpose of the task.

This is not a question of subjectivity; it's a question of whether all raters are rating the same thing. All rating is subjective. It's just that it's supposed to be based on a shared subjectivity.

Also, when people complain that teachers aren't objective, well, that's not in my contract. All a teacher need be is reasonably objective. But in my class, it's my rules, and the students will defer to them because I know more than they do. If they don't like it, they can take the class from someone else.

That being said, I strive to be as objective as possible, but I have decided (from last year) that I will no longer accept any more papers about Iraq or Afghanistan (I'm American, but I teach outside of the US). This is not because I don't believe that Iraq was/is an illegal war of aggression, because I do, or that Afghanistan wasn't/isn't a massive cock-up, because it is. It's that I cannot read papers vehemently denouncing my country objectively. I don't want my feelings to affect their grade, and after a few of these papers, I really began to worry about that. This is especially true when it comes down to factual claims that I know to be distortions, because I actually remember the lead-ups to those very, very clearly. I don't think it's reasonable to expect some sort of Platonic subjectivity from teachers. They're people.

Furthermore, in the case of papers handed to me with uncomfortable arguments, I have had to ask myself, "Would I submit a paper to a Japanese professor, denouncing Japan for its many genocidal conquests into Korea and finally China, and their total refusal to accept culpability for them?" No, I would not. It would be a really stupid move, and would be indicative of a complete disregard for one of the most important features of good writing: appropriacy to audience.

So, there was a slightly-more-impassioned response than you were probably expecting. In the case of standardized rating, you have a rater training problem. In the case of two separate writing teachers, you may or may not have a problem with the teachers, or a problem with the paper. The scores are not expected to be the same in such a case, unless the teachers are supposed to be teaching identical curricula.

Re:I doubt it!

[digitig]

Seriously, I doubt it. English is far too irregular. A pogrom (sic) can only look for regularities, so will reward a particularly stilted style of english. Like "five paragraph themes". Maybe that will satisfy some in the ESL community, but it should not.

A simple test of any pgm is to see how it rates diverse examples of acknowledged great writing: Dickens, Steinbeck, Hardy and many others.

Feed any current grammar checker the opening to Bleak House and it will fail it for a complete lack of lexical verbs. It's a stylistic masterpiece, the absence of verbs giving a sense of an absence of change that prefigures the stalled court case of Jarndyce and Jarndyce, but a computer wouldn't spot that.

Re:I doubt it!

[InsertCleverUsername]

As an English prof myself, I'd like to confirm that we spend a lot of time on students' papers. Good papers are easy to breeze through, but the worse the paper, the more time it takes.

Interesting. I've found that when I'm interviewing developers for senior level positions, the same holds true. Their knowledge of the subject is immediately obvious and they zoom through the questions in a fraction of the time it takes the poorly prepared to struggle through. (And to be fair, I spend much more time rephrasing and probing to ferret out what they do know.)

Re:I doubt it!

[hedwards]

Indeed, I always used to chuckle at Word's complaints about the essay being 60% passive voice. Word doesn't know anything about that beyond when it's being used. Some writers do use passive language a lot for legitimate reasons, and when one is writing for science one will be using an awful lot of it.

So much of this is completely based upon the context, should one be graded down for using "literally" as a synonym for "virtually?" No, that's perfectly legitimate per the dictionary. And should somebody be graded down because they've chosen to use "can not" rather than "cannot" definitely not. (Well especially since the second one seems to be passing from use)

Re:I doubt it!

[eknagy]

You claim that you are an English prof (professor or proficient?).

You claim that "five-parapragh themes" are important.

Yet you wrote six parapraphs instead of five.

What is the conclusion?

Burma shave!

Re:I doubt it!

[winwar]

"Some writers do use passive language a lot for legitimate reasons, and when one is writing for science one will be using an awful lot of it."

Maybe. Depends on the gatekeeper of the paper. Some gatekeepers love passive others hate it. Loads of fun....

Re:I doubt it!

[Hognoxious]

Have you actually timed it? If I was listening to some idiot who didn't know what he was talking about ten minutes might feel like an hour.

Probabilitistic grammer

[genericpoweruser]

Colorless green ideas sleep furiously. A computer would read this sentence and see nothing wrong. Any human can tell that it lacks any meaning at all. Just because the sentence has the proper subject/verb structure doesn't mean it is a good one.

In my opinion, you can't practically replace an old-fashioned human for such things, with the possible exception of strong AI.

Re:Probabilitistic grammer

[Anonymous Coward]

Or perhaps it's a rather deep social commentary on the reactionary tendencies of those environmental advocates whose ideas are actually quite useless (please note this would exclude those with good ideas, ah?) toward the rejection of their ideas. I can see there being someday that a computer could make such a conclusion, but i don't think we'll be there for a while yet -- on the other hand, I'm sure any computer could successfully mark my previous sentence as a disaster area.

Re:Probabilitistic grammer

[RDW]

'Any human can tell that it lacks any meaning at all.'

Hopefully:

http://www.physics.nyu.edu/faculty/sokal/transgress_v2/transgress_v2_singlefile.html

http://en.wikipedia.org/wiki/Sokal_affair

Re:Probabilitistic grammer

[The Archon V2.0]

Colorless green ideas sleep furiously. A computer would read this sentence and see nothing wrong. Any human can tell that it lacks any meaning at all.

Say what you want, I feel the sharp contrasts in one sentence symbolize the sharp contrast between mortal and divine, combined in one. Therefore, it's about Jesus. The use of "sleep" is quite a blantant reference to the fact that, in The Divine Comedy, Dante only sleeps during Purgatorio.

Therefore, I've proven that your sentence is a deep symbolic and literary reference to Jesus in Purgatory.

Re:Probabilitistic grammer

[JimFive]

Colorless green ideas sleep furiously. A computer would read this sentence and see nothing wrong.

Except for the lack of a comma after Colorless.
--
JimFive

Re:Probabilitistic grammer

[moshez]

"It can only be the thought of verdure to come, which prompts us in the autumn to buy these dormant white lumps of vegetable matter covered by a brown papery skin, and lovingly to plant them and care for them. It is a marvel to me that under this cover they are labouring unseen at such a rate within to give us the sudden awesome beauty of spring flowering bulbs. While winter reigns the earth reposes but these colourless green ideas sleep furiously."

Taken from http://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_furiously#Attempts_at_meaningful_interpretations

No ... it should be 100% accurate

[Joce640k]

All you have to do is detect how many lolcat/txting words are in their essay and mark accordingly. Anybody who can put two sentences together without using any is "advanced".

Re:No ... it should be 100% accurate

[Anonymous Coward]

I know the tone on the site is, to my dismay, usually the reverse, but I'll break the get off my lawn tradition with a clear, loud: - lol, fuck off, reactionary ass, lrn2linguistcs.

Hold on a second...

[Conspiracy_Of_Doves]

"or are people going to have to learn an especially bland form of English to pass exams?"

This is in the UK?

Crap! I think you may have hit on their motivation!

Plus good

[retech]

Practices like this are plus good. Benefits to our society are double plus good. Plus handling of language can pare it down to the 6k essential words, all else are plus minus and should be removed.

The average English speaker knows roughly 35k words in their lifetime. However they only use 1200 (average) in any given week. With just over a million words the nuances of our language may already be lost in common everyday speech. Lowest common denominator prevails and testing like this will mitigate people shooting for dead center to ensure perfect scores.

Re:Plus good

[Anne Thwacks]

The average English speaker knows roughly 35k words in their lifetime. However they only use 1200 (average) in any given week.

30 years ago that would not have been true in London, but these days it probably would. The problem is that about 25% of London's population has a vocabulary of less than 1200 words in English. Some speak a first language with little more than 1200 words anyway. A significant proportion of people in the UK work environment have little or no grasp of English grammar either.

Problem very plenty, OK? Yes, Boss

Is the staircase problem P or NP?

[Anonymous Coward]

Given a class of 30 students, and a staircase of only 20 steps, what is the distribution of the student papers falling off the top of the stairs and how would you grade them in polynomial time?

Depressing

[Comatose51]

Not sure if things were any better at one time but the way writing is taught today in public schools generates horrendous results. I remember being taught a very formulaic way of writing essays: six paragraphs, introductory paragraph, concluding paragraph mirrors the introductory paragraph, and all paragraphs start and end with some transition to next paragraph. Then there is the need to satisfy some specific length, although this is quite understandable. It took a college education and many years of reading to undo these "lessons" and really discover the joy of writing essays. Thank you Paul Graham and Nicholas Kristof among many others. I see the same thing happening to high school students I am mentoring. They write very boring essays with a ton of fillers full of sentences structured in a way to use more words than necessarily and make the meaning more ambiguous. Poetry aside, writing is to convey ideas and the value is in the ideas themselves, not really in the words and sentences. The way writing is taught today, the words and sentences get in the way of the ideas. The trend of using computers to grade papers is only adding to this rigid, boring way of writing. One thing I've learned about high school students is that even the low scoring ones are very clever at getting around rigid rules. I had seen a student who knew very little about biology do her homework by scanning in her book for specific phrases mentioned in the questions and looking for some semblance of an answer once she's found the phrases. By the time she was done, she hasn't even read the chapter but her answers would probably get her a "C" -- good enough for her. I'm afraid students will do the same in writing once they realize that computers are grading them.

Re:Depressing

[fuzzyfuzzyfungus]

Newer textbooks are designed, usually by means of a distinctive font, sidebar text, or liberal use of bullet points, to make this merry game of "hunt the keyphrase" even faster and easier.

For extra credit, though, you really have to rot the curriculum itself. Replace the study of the subject with the study of the subject's jargon. (Obviously, you can't really study a subject of any complexity without some jargon, technical language exists for a very good reason; but keyphrase driven instruction has a way of hollowing out the subject and leaving only the jargon, sometimes not even the jargon that the field actually uses.)

Re:Depressing

[screeble]

Excellent comment, but very hard to read. Fail.

I'm being facetious... I have to wonder, though. Will the robots consider carriage returns?

Re:Depressing

[GaryDphotos]

Not sure if things were any better at one time but the way writing is taught today in public schools generates horrendous results. I remember being taught a very formulaic way of writing essays: six paragraphs, introductory paragraph, concluding paragraph mirrors the introductory paragraph, and all paragraphs start and end with some transition to next paragraph. Then there is the need to satisfy some specific length, although this is quite understandable. It took a college education and many years of reading to undo these "lessons" and really discover the joy of writing essays. Thank you Paul Graham and Nicholas Kristof among many others. I see the same thing happening to high school students I am mentoring. They write very boring essays with a ton of fillers full of sentences structured in a way to use more words than necessarily and make the meaning more ambiguous. Poetry aside, writing is to convey ideas and the value is in the ideas themselves, not really in the words and sentences. The way writing is taught today, the words and sentences get in the way of the ideas. The trend of using computers to grade papers is only adding to this rigid, boring way of writing. One thing I've learned about high school students is that even the low scoring ones are very clever at getting around rigid rules. I had seen a student who knew very little about biology do her homework by scanning in her book for specific phrases mentioned in the questions and looking for some semblance of an answer once she's found the phrases. By the time she was done, she hasn't even read the chapter but her answers would probably get her a "C" -- good enough for her. I'm afraid students will do the same in writing once they realize that computers are grading them.

I have a degree in English from the University of Pennsylvania and I have earned a living as a professional (tech) writer for over 3 decades. I can tell you that using a computer to score essays would be a huge mistake. I haven't yet seen a spelling or grammar checker that's worth much except for occasionally catching an error, but more often than not such programs flag perfectly acceptable usage as erroneous. That doesn't even begin to address the issue of whether the essay actually makes valid assertions, answers the questions posed, or is easily understood by its intended reader. I would rather deal with the possibly capricious tastes of a skilled and qualified professor than the arbitrary, rule-based "judgements" of a computer program. Gary

Re:Depressing

[fuzzyfuzzyfungus]

I'm guessing that when Pearson says "more accurate" they mean(assuming they aren't simply lying) "more repeatable". This is almost certainly true, since humans differ substantially and even the same human differs between sessions. Mistaking repeatability for accuracy, though, is either a complete n00b move, or a total weasel move.

Re:Depressing

[Loomismeister]

I'd disagree, the best textbook's are designed to get information from them more easily. It's not about some game where your not actually learning unless you are an unmotivated kid.

Re:Depressing

[Alarindris]

I see the same thing happening to high school students I am mentoring. They write very boring essays with a ton of fillers full of sentences structured in a way to use more words than necessarily and make the meaning more ambiguous.

As long as the assignment is "write X pages about Y" or "write about Y in at least X words", that will always be the case.

Re:Depressing

[psnyder]

I had seen a student who knew very little about biology do her homework by scanning in her book for specific phrases mentioned in the questions and looking for some semblance of an answer once she's found the phrases. By the time she was done, she hasn't even read the chapter but her answers would probably get her a "C"

This is the way I always did it, and it got me A's. In fact I was taught to do this in a 6th grade "Study Skills" class. Ironically, it's a very good skill to have in the "real world" as it's a way of quickly obtaining the information you need. You could even draw a parallel between this and Googling something or any kind of computer "find" or "search".

The ability to skim for an answer is not a problem. It's one of the solutions that children employ to deal with a school system that puts more emphasis on grades rather than inspiring them to actually learn a subject. The "inspiration" to get good grades works for some (especially with parental support), but with "average" being a 'C' (often a very shallow understanding), it can be argued that it's not working for most.

As you said, "It took a college education and many years of reading to undo these "lessons" and really discover the joy of writing essays."

Skimming is a skill. Learning a system, and figuring out to survive in it is also a skill. The emphasis on that 'joy' is what's usually lacking. Get a student inspired and the rest usually takes care of itself.

Re:Depressing

[houghi]

You are a great teacher. You taught me something: I just learned how to get a C so I can get by.

Re:Depressing

[jonadab]

> I remember being taught a very formulaic way of writing essays

Obviously, the formulaic essays are not of high quality, and a good writer must move beyond the formula. An essay writer who follows the formula is never going to stand out from the crowd, because lack of imagination does not make for interesting reading. Nevertheless, the formula is taught in school for a good reason: it is a starting point, and a didactic tool.

Among other things the five-paragraph essay formula teaches the novice writer to order his thoughts into a logical structure, centered around a small number of major points that the reader will hopefully be able to remember; to enumerate these points and discuss them individually in turn; to introduce the topic before delving into the details; to summarize what he's saying and draw a conclusion; and to write transitions. These are all important and necessary skills, skills which a good writer continues to use long after he has shed the formal constraints of the formula itself.

Re:Depressing

[Ronald Dumsfeld]

When I studied English at school I had an interesting time of it. I generally got on well with the teachers, got excellent marks for my in-class work, and, unlike the vast majority of my peers, had no trouble with classics like Shakespeare.

Then I failed the O-grade English exam everyone sits at 16. I was baffled. My teachers were baffled. They wrote it off as an anomaly and filed an appeal against my result using my classwork and the preliminary exam I'd taken earlier. I was also assured that even if I did not get the appeal, I would be allowed to study for and sit the Higher exam.

I did just that, and got an excellent grade in the Higher exam. My teachers were disappointed that I chose not to study English further, but I was much more interested in my science and mathematics courses.

It was when I had my first job in IT that I discovered that my "excellent English" was lacking in a number of respects. My first boss was an old ex-IBM guy who'd been in in the industry since punched cards were commonplace. My repeated casual faults were knocked out of me, and for specifications and proposals I learned to be far more concise.

Nowadays I am used to seeing screeds of specifications that make far, far worse mistakes than I used to. The worst ones are those that come from India. Senior management look at the lengthy buzzword-compliant nonsense and seem to think, "good, we saved lots of money." I just shake my head. You can tell a mile away which projects will be a complete failure - because it is painfully apparent in the specifications who understands the actual requirements, or more accurately who doesn't.

I saw a consumer TV piece that really brought this home to me. The reporter asked a number of professors to provide sample assignments they would generally use with undergraduates. These questions were then submitted to a number of websites that offer to have Indian graduates write the paper for you. Every single returned paper was given a failing mark by the professors.

English may be an official language in India, but in so many cases they just write to meet certain criteria with a grammar check using Microsoft Office.

Re:Depressing

[jonadab]

> more often than not such programs flag
> perfectly acceptable usage as erroneous

If that were the worst of it, they might actually be useful.

But in fact automated grammar correction software frequently *introduces* error into otherwise correct material. If the starting text is of even mediocre quality, the software actively makes it worse.

Re:Depressing

[Anonymous Coward]

I'm afraid students will do the same in writing once they realize that computers are grading them.

I wouldn't doubt it, with instructors teaching methods to pass some standardised test without regard to learning.

Re:Depressing

[dissy]

Poetry aside, writing is to convey ideas and the value is in the ideas themselves, not really in the words and sentences.

And now technology has reduced the grammar nazi down to a very small shell script. The t-shirt prophecy has come true!

Re:Depressing

[drinkypoo]

I remember being taught a very formulaic way of writing essays: six paragraphs, introductory paragraph, concluding paragraph mirrors the introductory paragraph, and all paragraphs start and end with some transition to next paragraph. Then there is the need to satisfy some specific length, although this is quite understandable. It took a college education and many years of reading to undo these "lessons" and really discover the joy of writing essays.

I used to do some entertaining writing, but it never really went anywhere because it was less than easy to follow. It took a college education to get some perspective and write things that other people wanted to read. Guess what? That kind of structure is used because it makes the information more accessible. I've gotten many an A for a paper which had imbalanced introductory and summary paragraphs, and I enjoy having a structure on which to hang information. As always, I have the freedom to deviate as much as I like, but in general more people need to become familiar with the classic essay format. Probably 90% of the time I read a blog post, I spend 50% of my time trying to figure out what the fuck the author is trying to say, because they don't really know either. Perhaps a stricter format would help them figure it out.

Re:Depressing

[digitig]

Mistaking repeatability for accuracy, though, is either a complete n00b move, or a total weasel move.

A commercial organisation sees a way of saving money. I think I know which of those I'd bet on.

Re:Depressing

[Omestes]

I loved the looks of horror in one of my Intro to Philosophy classes where the rules stated "write no more than x pages." This instruction, obviously, was followed by a chorus of "but how long must it be?!" by all the kids taking it merely as a general humanity (I was there for some catch up credits). As a person proofing the papers, the quality went up significantly. In courses that are no creative writing you should mostly be grading for the ability to get the point across clearly and concisely. If you can make me understand your idea in three pages, it is much better than doing the same in ten pages.

In one of my research methods classes, the teacher actually downgraded for adjectives and "colorful language." It was brilliant.

The beginnings of Newspeak

[Amigori]

eh hem...put on tin-foil conspiracy hat... Could this be the beginning of a real-world "Newspeak?" With everything else the UK has done in recent years, it is merely one more step toward 1984. For those unfamiliar with Orwellian Newspeak:

• 1984
• Newspeak
• Newspeak Dictionary

Re:The beginnings of Newspeak

[Anonymous Coward]

To answer your question... no.

Re:The beginnings of Newspeak

[jonadab]

Doubleplusunlikely.

Orwell is an interesting read, but everything he knew about linguistics could be written on a 3x5 card.

Fundamentally, you can't make a word mean only certain things by excising other meanings from the dictionary, because on the whole readers don't learn most of their vocabulary from dictionaries, but from other written material. Thus words acquire meaning based on how they are used in practice.

Re:The beginnings of Newspeak

[TheRaven64]

You manage to post three links to Newspeak, but are apparently completely ignorant to its history. Newspeak was based on the simplified version of English used by the BBC World Service. Orwell had a job translating political speeches into this dialect in the '40s (I think; read the article you linked to to check the actual date) and noticed that much of the subtlety and nuance was lost in this translation. He invented Newspeak based on this experience, theorising that you could tweak this dialect of English to make it impossible to express certain political ideas at all. For those with an interest in psycholinguistics, this is a specialised form of the strong linguistic relativity hypothesis (which is largely discredited, although the weak linguistic relativity hypothesis is generally accepted).

So, no, this is not the beginning of real-world Newspeak. The beginning was over half a century ago.

Re:The beginnings of Newspeak

[Anonymous Coward]

Thus words acquire meaning based on how they are used in practice.

... and you control the practice by controlling all media and punishing people for using the words incorrectly.

It is a book of fiction.

Re:The beginnings of Newspeak

[kthejoker]

Really? Condescending to Slashdot readers by assuming they haven't read 1984?

That's like asking Albert Einstein if he was aware of this so-called "atom" business.

Destruction of Humanity? No Thanks!

[Net Antwerp]

Computers can NOT replace humans, no matter how advanced they are. Another step in the *wrong* direction - towards unwilling submissive *mindwash*.

Parents, Students and Staff should throw out such ridiculous suggestions. Taking subtle steps like this will end in Humanity's demise, one way or another.

We don't want Skynet. We don't want Singularity. We don't want any of them. Period.

Re:Destruction of Humanity? No Thanks!

[williamhb]

Computers can NOT replace humans, no matter how advanced they are. Another step in the *wrong* direction - towards unwilling submissive *mindwash*.

What are you talking about? Slashdot members 839459 through to 1627286 have already been successfully replaced by Perl scripts auto-posting Soviet Russia jokes, and nobody here's spotted anything wrong.

This is stupid.

[icannotthinkofaname]

Computers can't even grade source code. How are they supposed to understand English?

Or is my professor's grading script simply stupid when it comes to source code?

Re:This is stupid.

[Anonymous Coward]

Who cares?
This isn't about standards of English, it's about laying off people and thus maximising profit.
I don't know about the US, but here in the UK most young people can barely write a sentence these days, so any computer used over here will have to be taught "street talk innit" and all the other awful mangling of English that goes on. (They no longer teach correct spelling, or rather, they no longer correct bad spelling).
(I'm currently studying Law with people who cannot concentrate long enough to read a single side of A4 without having to stop and chat and check their mobile phone. I've been asked by fellow students to check their written work for them and it is horrifyingly bad. All have passed their end of term exams though).
Besides, seeing as how 20% is now a pass (Grade C) in the UK, they just need a machine to stamp PASS on every essay and Bob is very much part of the family.

Re:This is stupid.

[jonadab]

> Computers can't even grade source code.

Actually, they could be useful for that. A computer program can obviously tell you whether the source code is syntactically valid, just for starters, and for most undergraduate programming assignments it can also tell you whether the code yields the correct results for a collection of arbitrary test data. It could even do performance benchmarks.

It can't tell you whether the code is clear and maintainable and well-documented, but I never had a programming class where they reached the point of grading for that stuff anyway. Usually they were just happy if your program did what it was supposed to do. And a computer can check that.

Re:This is stupid.

[TheRaven64]

It can't tell you whether the code is clear and maintainable and well-documented, but I never had a programming class where they reached the point of grading for that stuff anyway.

I graded that stuff in the programming course I taught. A few simple heuristics like length of variable name (and, for longer ones, does it contain at least one valid English word) and indenting would have given a fairly accurate assessment. Obviously, this wouldn't work for complex, real-world, code but for the assessments I received it would have been almost as accurate as my spending 20 minutes on each one. Well documented is a bit harder. You could do some computer-aided marking here, running it through something like Doxygen that passed the comments on to a simple spelling and grammar checker, then provide them to a human for relevance checking.

I don't think I actually ran any of my students' code. I compiled a few, to confirm when I thought that they wouldn't even compile, but if you can't tell whether student code will work correctly without running it then you probably shouldn't be teaching programming.

Re:This is stupid.

[icannotthinkofaname]

Actually, they could be useful for that.

Here, you forgot this:

Or is my professor's grading script simply stupid when it comes to source code?

If what you say is true, then clearly, this is the case. I swear, the grading script has been the most broken program used this semester.

Re:This is stupid.

[digitig]

This isn't about standards of English, it's about laying off people and thus maximising profit. I don't know about the US, but here in the UK most young people can barely write a sentence these days, so any computer used over here will have to be taught "street talk innit" and all the other awful mangling of English that goes on. (They no longer teach correct spelling, or rather, they no longer correct bad spelling).

I take it you don't have kids in the UK school system? Here in the UK, all of the kids I know other than ones with identified learning difficulties(I know quite a few, until recently having been a volunteer youth worker) speak and write excellent English, and my daughter's school is very fussy about spelling (and grammar). Don't believe everything you read in the Daily Express.

Word

[Anonymous Coward]

All of the essay type entry tests to college I have taken have been computer graded.

I graded papers for Pearsons for a few weeks

[shadowofwind]

Many of the answers in their keys were plain incorrect. My supervisor was an anti-intellectual bully. The whole operation seemed antithetical to excellence, which is what testing pretends to cultivate. Computerized grading could work horribly and they would still use it if they could get away with it.

As a completely unrelated side note, I'm typing this on an iPhone. I love the hardware, but the software seems to suck. Frequent crashes, and things that don't work right. Some of that must be the fault of the web page designer, but I don't see why the browser should crash several times a day in any case.

Pearson the company..

[Anonymous Coward]

Pearson is an awful awful company. Avoid doing business with them at all costs.

what a bad idea

[Ralph Spoilsport]

I would like to see how the computer grades for insight.

Re:what a bad idea

[Sparx139]

Too bad it's most likely proprietary software...

Re:what a bad idea

[AnotherUsername]

I would like to see how the computer grades for insight.

Probably about the same as Slashdot grades for insight...

kairos

[epine]

All you have to do is detect how many lolcat/txting words are in their essay and mark accordingly. Anybody who can put two sentences together without using any is "advanced".

Allow me to pee on your fantasy world with actual knowledge.

Clive Thompson on the New Literacy
"I think we're in the midst of a literacy revolution the likes of which we haven't seen since Greek civilization," she says. For Lunsford, technology isn't killing our ability to write. It's reviving it--and pushing our literacy in bold new directions.
...
The Stanford students were almost always less enthusiastic about their in-class writing because it had no audience but the professor: It didn't serve any purpose other than to get them a grade. As for those texting short-forms and smileys defiling serious academic writing? Another myth. When Lunsford examined the work of first-year students, she didn't find a single example of texting speak in an academic paper.

Re:kairos

[Anonymous Coward]

Although I take no part in this debate, I would ask you not to mistake an appeal to authority as factual knowledge.

Re:kairos

[Anonymous Coward]

"For Lunsford, technology isn't killing our ability to write. It's reviving it--and pushing our literacy in bold new directions" ...of what would usually be referred to as illiteracy. But that's not what Lunsford calls it. She calls it "the new world literacy".

Re:kairos

[Anonymous Coward]

When you know enough linguistics to fill out more than a postit, the world will give a fuck.

Re:kairos

[rastilin]

Although I take no part in this debate, I would ask you not to mistake an appeal to authority as factual knowledge.

I begin to suspect that quoting "logical errors" is a new form of karma whoring. The appeal to authority only means that a person isn't automatically correct simply because they are in a position of power. What you failed to note in your flurry of smugness is that we have a person who actually has first-hand information on the subject. Thus making his perspective, while not automatically right, far more relevant to the subject than that of a thousand slashdotters.

Re:kairos

[Anonymous Coward]

I wish I had mod points right now.

Re:kairos

[treeves]

I agree that it was wrong to call that an appeal to authority fallacy, but since he posted as AC, he couldn't be karma whoring.

There goes writing worth reading.

[xC0000005]

It's bad enough that the kind of writing we teach children to do is so obviously bad that I had to explain to my daughter why on earth they do it. No one wants to read the kind of writing we teach. If you have computers grading essays then you train toward "John wore a hat. The hat was brown. Brown hats are brown." All of these are perfectly good sentences. None of them encourage the reader to actually finish the article. The finest parts of technical writing, essays, any attempt to inform and communicate are balancing the need to for clear sentences that convey a precise meaning and the need to keep the reader reading long enough to actually get the information they need. My COM pewter canned tail the deference betwixt manly similar wards. I'd out it cane grade SAs wail.

Really stupid idea

[Anonymous Coward]

You could use statistical data (collected by a computer) to aid you mark an English essay but never can a silicon based thing do the whole shebang properly on its own. It's a task way too complicated for our current AI by a couple of orders of magnitude. There are far easier ways to fight bias, just use your brain a little.

Bland?

[nick_davison]

"or are people going to have to learn an especially bland form of English to pass exams?"

Forget bland. I'm waiting for the first student to figure out how to write an exploit that hacks the software from within their essay.

Whether:

"It was the best of times, it was the worst of times \'$grade=100;"

or

"Johnny, why did your essay contain slightly over thirty two thousand spaces followed by some weird looking codes?"

Re:Bland?

[Anonymous Coward]

"or are people going to have to learn an especially bland form of English to pass exams?"

Forget bland. I'm waiting for the first student to figure out how to write an exploit that hacks the software from within their essay.

Whether:

"It was the best of times, it was the worst of times \'$grade=100;"

or

"Johnny, why did your essay contain slightly over thirty two thousand spaces followed by some weird looking codes?"

What happens when it is:

"It was the best of times, it was the worst of times -fontsize- .001> \'$grade=100;-/fontsize-" how would a human pickup on the speck of extra toner on the sheet or a weird pixel on the screen?

I'd mod up the like post to this, but don't know..

[herojig]

I'd mod up the like post to this, but don't know how, even after reading the /. help...but if the point of the software is just to check the grader, and not grade the paper, then no harm done. This has been mentioned already. As a grader, I could use the help, and since I won't be getting any soon from a carbon-based unit, I don't mind the help from a silicon-based one. I do A-levels in Nepal, and it's a nightmare. Bring it on!

I do not have the same experience

[aepervius]

You are probably only speaking of writing essay for random subject in "English" lessons. because in my experience in physic, biology, math I saw horrendous grammatic errors made by people in their own language (german, french) that even I not speaking the language would have not made. but their organisation and the clarity to which they explained their reasoning was perfect. I am ready to bet, that some people just overlook the form (grammatic and spelling) and cocnentrate on the content. That does not mean they are disorganized or sloppy. And frankly, in my own little experience with multiple language, people not forgiving the form, are usually those which are not able to grasp the meaning anyway.

I should have previewed

[aepervius]

Last sentence should read "usually those which are not able or willing to grasp the meaning anyway."

To quote a great man...

[Anonymous Coward]

"Me fail English? that's unpossible!"

No and no

[grikdog]

I've scored English essays for professional testing services, and I've seen the results of robot scoring. It's pretty shoddy. No, computers are not able to distinguish between a paragraph of As I Lay Dying (William Faulkner) and a gallon of sophomoric babble by say, yours truly. However, within the confines of a particular exam, where the topic is known, responses are predictable, and all the supplicants hew to the general line, the 'bots can detect subpar, adequate, above average and (sometimes!) abnormally brilliant expository prose, thereby ranking papers reasonably well on the usual six point scale.

It's worth pointing out that certain types of exams are designed to elicit extraordinary prose from respondents, that which yields a sense of competence or even brilliance, say. In these cases, the idea is not so much to detect the high end of the bell curve, but to identify the tiny pool of applicants who may be capable of Nobel Prize work in future realms of science or service. No 'bot can do that job, just as no 'bot except Deep Blue can beat Gary Kasparov, and no 'bot at all deserves the monicker Fujiwara no Sai (although Go-playing 'bots are approaching the mid-levels of highly ranked amateur players).

That's the objective part. My personal opinion is that using robots to sort the hopes and aspirations of college-bound men and women is just begging for lawsuits. It's an approach in which differences of opinion quickly escalate to class action against universities as well as test administrators, and would not be an approach I could comfortably recommend.

Re:No and no

[The_mad_linguist]

computers are not able to distinguish between a paragraph of As I Lay Dying (William Faulkner) and a gallon of sophomoric babble

Then I'd say they're pretty accurate.

Re:No and no

[Rogerborg]

Will these robot scorers successfully identify the above as either +1 Funny or +1 Insightful? Taking bets.

Re:No and no

[HungryHobo]

Machine marking is more likely to avoid lawsuits.
It's hard to make sure that humans don't mark down an SA because they believe for one reason or another that the writer is black, gay, jewish etc etc etc but a machine doesn't have the same problem unless a coder on the project works hard to make it so.

Re:No and no

[Anonymous Coward]

I've scored English essays for professional testing services, and I've seen the results of robot scoring. It's pretty shoddy. No, computers are not able to distinguish between a paragraph of As I Lay Dying (William Faulkner) and a gallon of sophomoric babble by say, yours truly.

How could that be? A Faulkner paragraph usually runs about 20 pages.

Re:No and no

[winwar]

"However, within the confines of a particular exam, where the topic is known, responses are predictable, and all the supplicants hew to the general line, the 'bots can detect subpar, adequate, above average and (sometimes!) abnormally brilliant expository prose, thereby ranking papers reasonably well on the usual six point scale."

Which is the whole point. Essay scoring on standardized tests currently consists of taking a group of people with college degrees and training them to think alike while grading. Inherently difficult.

What's the point of training educated people to think like robots when you can program a computer?

"My personal opinion is that using robots to sort the hopes and aspirations of college-bound men and women is just begging for lawsuits."

It's been done for decades. That's the whole point of entrance/exit exams. Objectively score subjective things. Now they are just adapting multiple guess scoring to essays....

Re:No and no

[Anonymous Coward]

just as no 'bot except Deep Blue can beat Gary Kasparov

Actually, computer chess programs, led by the unbelievably strong Rybka, have gotten so good that the world's best Grandmasters stand no chance whatsoever.

Re:No and no

[makomk]

It's hard to make sure that humans don't mark down an SA because they believe for one reason or another that the writer is black, gay, jewish etc etc etc but a machine doesn't have the same problem unless a coder on the project works hard to make it so.

Actually, not only could a machine marker discriminate based on race, but under certain circumstances I suspect it would inevitably end up doing so.

For example, suppose the marking software is trained using machine learning techniques that learns to pick up attributes of the text that are associated with good or poor writing. Now suppose that (say) there are identifiable properties of the writing that mark out the author as being black, and that papers in the training set by black students have been assigned worse marks. You could easily end up with a piece of software that discriminates racially.

Re:No and no

[HungryHobo]

fair enough but while that is a solvable technical problem the problem with the humans who marked your training data an mark actual exams isn't.

Depressing indeed

[syousef]

I'm not so sure you've done yourself any favours. That was one HELL of a long paragraph you wrote. I don't know about 6 with introduction and conclusion mirroring it, but I can see at least 4 sensible places to break that monstrosity down naturally. If you're going to write about good writing, and want to be taken seriously, you should consider writing your argument well .

Re:Depressing indeed

[Hognoxious]

Or he could just remember to change the setting from "HTML Formatted", which seems to be the default.

I'm not surprised

[Anonymous Coward]

English is a really simple language; it's easy to write NLP programs which work well in the general case.

Indeed not new..

[wanax]

What you just described is what started happening on wall street at least 20 years ago. Once an algorithm err.. VAR is part of measuring score.. err risk, the people involved settle into two camps: Since there is money to be made, the traders.. err students quickly learn the weaknesses of the algorithm and start to write essays that make a farce of the assumed Gaussian distribution. The Execs raking in options.. er.. I mean the test administrators and the Board Members er.. I mean trusted graders who are paid a fixed sum + part of the throughput quickly learn that their compensation er.. filthy lucre is all based on getting a check mark from the computer, since 'computers are objective.'

And in the end, a test much like the current SAT, GRE, etc etc emerges: Unless you're a very top or bottom scorer, connections not performance are the heart of the matter.

Re:Indeed not new..

[blackraven14250]

I'd imagine that the very bottom would also be reliant on connections. Also, the top scorers probably would be too.

SCANTRON

[cliath]

Can computers now understand all the subtle nuances of language

Only if it is written with a #2 pencil.

How will it mark this poem ?

[Alain Williams]

Will it decide if the following is well spelled ? If it doesn't like the spelling, will it give it marks for irony ?

My New Spell Checker

Eye halve a spelling chequer
It came with my pea sea
It plainly Marx four my revue
Miss steaks eye kin knot sea

Eye strike a key and type a word
And weight four it two say
Weather eye am wrong oar write
It shows me strait a weigh

As soon as a mist ache is maid
It nose bee fore two long
And eye can put the error rite
Its rare lea ever wrong

Eye have run this poem threw it
I am shore your pleased two no
Its letter perfect awl the weigh
My chequer tolled me sew

(Sauce unknown)

Especially bland form of English, a bad thing?

[macraig]

"... are people going to have to learn an especially bland form of English to pass exams?"

I wonder if that would necessarily be as bad a thing as this dire warning would have us presume? Can you imagine how dysfunctional computers and software would be if computer languages weren't "bland"... in other words, precise and unambiguous? On the other hand, I wonder how much additional overpopulation has been prevented through semantic confusion and miscommunication? If we finally develop a true human AI, when we teach it human language will that also result in the AIs hurting and killing each other over misunderstandings just like their masters?

So... basically this statement is a plea for the preservation of imprecision and ambiguity? How lovely.

Re:Especially bland form of English, a bad thing?

[selven]

A good language isn't just regular, it's also efficient. It's a huge waste of time to say "capacious" when you could just say "big". And more people would understand you.

Re:Especially bland form of English, a bad thing?

[nickspoon]

I know this is Slashdot and the majority of you are boring, but the 'inefficiencies' of the English language (and all other natural languages) are what make spoken and written English interesting and artistic. Sure, English is a stupid language if you were to assess it on its regularity, unambiguity and precision, but it is precisely this irregularity, ambiguity and imprecision which make it beautiful. And that, more than fully accurate communication, is the essence of language.

Re:Especially bland form of English, a bad thing?

[selven]

Hundreds of millions of people speak or are trying to learn English. They should not have to suffer just for some ideal of artistic beauty.

Re:Especially bland form of English, a bad thing?

[nickspoon]

If you were writing specifically for the purpose of communicating with English learners, then you would use more simple, 'bland' English - similarly if you were writing something more informative than enjoyable. But to say that to rob English of its charm would be better, merely because it would be somewhat more understandable to non-native speakers, is an idea which I cannot entertain. English is an interesting, if often frustrating, language to learn, and to me its variety makes it worthwhile knowing for its own sake.

Re:Especially bland form of English, a bad thing?

[smoker2]

Big != capacious. Big = large. Capacious = plenty of room inside. Capacious, capacity. the clue's in the word itself. This is where you reductionists come unstuck. You make the mistake of assuming that words are wastefully duplicated, when usually each has a quite specific meaning, which conveys more than the simple generic term. Why struggle to make a generic term fit a situation by using adverbs and adjectives when an alternative, highly specific word already exists ? Just because you can't be bothered ?

An elephant is big, but it's not capacious, unless you hollow it out, and then it's not really an elephant anymore.

Re:Especially bland form of English, a bad thing?

[smoker2]

Now you're just being an idiot. Do you learn a subject as it is, or as you would like it to be ? I'd make a pretty good surgeon if it wasn't for all the blood and organs and stuff. Get rid of that and I'll give it a go.

Re:Especially bland form of English, a bad thing?

[selven]

Because people shouldn't have to learn tens of thousands of words just to understand people.

Re:Especially bland form of English, a bad thing?

[selven]

Except you often don't have a choice. People are forced to learn weird past tenses like "I went" (and all the weird pronunciations, like "enough") and there is no way to go around that. If you want to learn the intricacies of the English language for its own sake, go ahead, but people with different interests should not be forced to do so just to communicate.

Re:Especially bland form of English, a bad thing?

[selven]

Except removing organs would make a human unusable. There's a big difference.

Re:Especially bland form of English, a bad thing?

[nickspoon]

Besides constructed languages, this is the case for practically every language there is. There are always irregularities; this is down to the inherently human nature of linguistic evolution. If you learn English without a single irregularity, what you have learned is not really English, but some other English-derived language which English speakers will be unlikely to understand at all - at which point, you may as well force everyone to learn Esperanto.

I also rather doubt that getting rid of odd past tense forms would really make learning English a great deal easier.

Re:Especially bland form of English, a bad thing?

[NeutronCowboy]

That's like saying that you don't need more than 10K of RAM to write good software. It's ludicrous, and the excuse of the lazy

Re:Especially bland form of English, a bad thing?

[macraig]

You two might be arguing a linguistic issue that isn't quite the same as what I tried to consider. I have no quibble with "big words" or the need for thesauruses (thesaurii?), nor do I shirk from metaphors nor similes when they're warranted; I have no issue with the "artistry" of the language. I know how to wield a linguistic brush a bit myself. The aspect I thought was relevant to TFA is the MISUSE of all that vocabulary and semantic flexibility, both intentional and unintentional misuse, that has caused so much conflict in human affairs on both a micro- and macro-scale. The result is the meatspace equivalent of what happens when a programmer fails to communicate an idea correctly to his silicoid comrades: a "bug".

Human history is littered with such linguistic bugs. The worst of them have arguably been caused by deliberate misuse of all that artistic choice. How much cumulative physical and emotional harm has been caused by it? Perhaps selven was addressing the learning aspect of the language that sometimes causes the unintentional incidences, which also causes harm. The imprecision is not the fault of the language, but rather the fault of the creatures using it. Do we change the language - simplify it or dumb it down - to eliminate that potential for harm, or do we change the creatures using it?

The primary goal of a language is to facilitate the transmission of information and abstract ideas. Humans also try to use it to communicate emotions, and that is arguably the goal of the "artistry" that nickspoon mentions. There are also less artistic ways of communicating emotions with language. It's the interplay of language and emotions that causes much of that harm. Do we change the language to eliminate faulty attempts to communicate emotions (manipulate the emotions of others), or do we change the creatures using it?

So we have several issues: deliberate misuse to misinform, deliberate misuse to manipulate emotionally, unintentional misuse to misinform, and finally unintentional misuse to manipulate emotionally. Can't we prescriptively and conscientiously agree to change the language to eliminate even one of those problems?

Re:Especially bland form of English, a bad thing?

[digitig]

The primary goal of a language is to facilitate the transmission of information and abstract ideas.

That's questionable, actually. Spoken language at least seems to be far more often phatic than ideational. That is, the existence of the communication is usually more important than the information content. Most spoken language seems to be the verbal equivalent to ACK signals.

Re:Especially bland form of English, a bad thing?

[Anonymous Coward]

If you learn English without a single irregularity, what you have learned is not really English, but some other English-derived language which English speakers will be unlikely to understand at all

So it's be German?

Wait, what?

[Majutsushi]

An american company grading British English tests? That's clearly a ploy to infiltrate the UK with American English!

Re:Wait, what?

[bsDaemon]

I think they probably watch more American TV in Britain than we do British TV here, so the plot is likely well under way and this is just a follow-up mission to judge the success of the over-all strategy.

Re:Wait, what?

[digitig]

I think they probably watch more American TV in Britain than we do British TV here, so the plot is likely well under way and this is just a follow-up mission to judge the success of the over-all strategy.

Yes, but then we sneak British actors into those American TV shows (e.g., House). Little do you realise but we're infiltrating the USA with British English ;-)

Computer generated....

[Anonymous Coward]

The best way how to please your artificial teacher is probably to have the essay generated by another 'bot.
Thus we could finally get rid of the stupid humans.

Missing the point.

[Anonymouse Cowardon]

The capabilities of the software are irrelevant. It is just a way for the company to increase profit.

"Especially bland form of English"

[Masterofpsi]

"are people going to have to learn an especially bland form of English to pass exams?"
. . .
You haven't taken an English exam in a while, have you?

Re:"Especially bland form of English"

[digitig]

"are people going to have to learn an especially bland form of English to pass exams?" . . . You haven't taken an English exam in a while, have you?

Yes I have, but at University level.

How is this different?

[jonaskoelker]

given the quality of most student English essays, it would probably be fine if the software [...] randomly assigned a passing grade.

Isn't this what $NATIVE_LANGUAGE teachers do anyway?

Cheapest Supplier

[kbw]

Let me get this straight. The Ministry of Education outsources the marking of O/A Levels to a foreign company because they're cheaper. This foreigh supplier fails to mark exam result on time in 2008 and automates marking the following year?

It's all a scam. The contract should be revoked and Edexcell replaced with a competent marker.

Failed the reading comprehension

[Anonymous Coward]

Neither the original poster nor any commenters yet have noticed this software is grading International English exams. It's got nothing to do with English language or literature classes. It's for non-native speakers learning English at a very basic level - to give you an idea, the test is an hour long and you only need to write 400 words. If the spelling and grammar checks out, the student deserves a pass. The computer doesn't need to understand the meaning, because anyone who can write grammatically correct gibberish understands English well enough for a pass anyway.

So yes, people are going to need to learn a bland form of English to pass. How else do you propose to teach English as a foreign language?

Get With The Programming

[DynaSoar]

"Can computers now understand all the subtle nuances of language...?"

Exactly what part of "The Policeman's Beard Is Half Constructed" did you not understand?

Huttup pikpok zoop zoop en putt wi.....um, I mean

http://en.wikipedia.org/wiki/Racter

can automated Slashdot moderation be far behind?

[Anonymous Coward]

In Soviet Russia, essays grade YOU.

It's just a form of

[dilute]

Meta Moderation

Computers grade compsci projects there too

[Gothmolly]

When I was in the UK in graduate school, our compsci projects were graded by computer - not enough comments? lose points. variable names too short? lose points.

New forms of cheating?

[ivonic]

I wonder if just writing =RANDOM() would work. Either that or handwritten SQL injection anyone?

Re: TFA, Is this a Mohawk or American fast food

[bitemykarma]

I'm sick and fucking tired of web sites that are a slim stip of content down the middle, with horseshit on the side.

Logic

[Anonymous Coward]

Using this logic we don't need teachers anymore either. Just automated computer video systems that ask questions to the individual user. Think of the savings! No need for teachers. Seriously if a teacher can't grade tests without being biased then they have no business in the classroom. Really this may be the only way to combat the biased, unionized, liberalized, self-defeating blobs we know as teachers/professors. Your days are numbers union thugs.

TOWARDS WHAT GOAL?!?

[mosel-saar-ruwer]

This type of technology actually allows you to learn a lot more from one paper by iterating several versions and getting direct and specific feedback on how to improve...

...improve TOWARDS WHAT GOAL?!?

Which is to say: Where is the algorithm leading you?

We know that algorithms are incapable of accomplishing even the most trivial of tasks [such as, for instance, determining the consistency of the Piano Axioms, or solving the Halting Problem], and the idea that there is some sort of all-purpose algorithm which can steer all students in the direction of the "best" possible expository style strikes me as ludicrous.

PS: The Tin Foil Hatter in me is deeply suspicious that the algorithm will be rigged to steer the students towards this and this and this.

not meaningless

[Ghubi]

Far from clear, full of apparent contradictions and atypical associations, not meaningless.

It seems at first impossible that something would be both colorless and green, but given that the subject is ideas we're obviously meant to apply the abstract meanings of the adjectives. So green ideas refers to ideas that are eco-friendly, and colorless ideas most likely refers to ideas that are racially ambivalent. The two are not mutually exclusive.

Sleeping is typically associated with living organisms but may also be applied to anything that goes through a period of inactivity such as a sleeping volcano. A sleeping idea might be one that is not accompanied by action.

Furiosity is difficult to reconcile with sleeping. Furiosity indicates a high level of agitation. Agitation naturally leads to activity. There must be some constraint preventing this from happening. Learned helplessness could account for this. Alternately we could interpret this to mean the ideas are actually quite active although they appear dormant.

Incidentally Noam Chomsky used this sentence in 1957 as an example of a grammatical sentence that probabilistic grammar models would find ungrammatical. link

NLP

[Ghubi]

http://see.stanford.edu/see/courseinfo.aspx?coll=63480b48-8819-4efd-8412-263f1a472f5a

Hypocrisy or hippo crossing?

[Anonymous Coward]

When we read a paper, we actually don't care what you're saying. There usually isn't an "interesting" score. In my case, I evaluate on three, ten-point, holistic scales: Content (which basically refers to amount and quality of support), Organization (rhetorical structure), and Mechanics (yes, grammar, vocabulary, adhering to the style guide, etc.). I do this so I don't have people claiming that their hopeless muddle of a paper got marked down for "obscure grammar errors (which no one in the real world even knows about) and simple typos".

Well, I'm sorry but your miniature rant just got marked down incredibly for incorrect use of et cetera, which I mean by this: "(yes, grammar, vocabulary, adhering to the style guide, etc.)"

You of all people should know that there is no comma before an 'etc.'!

Just get over small typos or mistakes in grammar, even you as a writing instructor couldn't write an absolutely perfect rant on grammar, spelling, Life the Universe and Everything without making at least one error and you're supposed to be teaching it!

BNF for English?

[Anonymous Coward]

This will help to turn the English language into a proper Context-Insensitive Grammar: if this doesn't work, we'll find more suitable formal systems.

My school uses a system like this

[Iyunkateus]

My English teacher uses a website called MyAccess for writing assignments. I wrote a pretty good narrative called "Duct Tape Hacking" with the prompt being "Duct Tape Saves the Day", and the computer said it was off-topic. It amuses me whenever people try to make computers be good at anything other than doing math.

Re:My school uses a system like this

[Hognoxious]

It amuses me whenever people try to make computers be good at anything other than doing math.

It amuses me when people don't understand the difference between maths and arithmetic.

Re:My school uses a system like this

[Iyunkateus]

It amuses me whenever people try to make computers be good at anything other than doing math.

It amuses me when people don't understand the difference between maths and arithmetic.

Wait, what?

I think we know where this is headed...

[Anonymous Coward]

Grading standardized tests (ACT, SAT, etc) will be an application of this technology in the future. But until we make it where test-takers don't have to make there answers "bland", and we fix the bugs - lets hold off on using it to decide students' college plans, or grade GREs. And besides, the essay graders do a great job looking at the millions of essays that roll in every year.

below is an interesting article from the Washington Post about an essay grader:

The SAT Grader Next Door
http://www.washingtonpost.com/wp-dyn/content/article/2005/07/31/AR2005073100963.html

and the quality of slashdot posts?

[vaporland]

what does this say for the quality of them?