Slashdot Mirror
Indiana First With Computerized Grading
Mz6 writes "Computerized grading has been talked about previously, however, the New York Times reports that Indiana has become the first state to grade high school English essays by computer. The computerized grading process, called 'e-rater', uses a 6-point rating scale and uses artificial intelligence to 'mimic the grading process of human readers'. The system was tested over a 2-year pilot program and produced results virtually identical to those of trained readers. The big question is, will other states begin to emulate Indiana by tossing human grading?"
Funny, because the way I read that is, "Produced lawsuits where the cost is virtually identical to about 20 times the short-term savings."
I see this coming from both sides. The obvious, the grading was wrong, and I lost a scholarship. To other people suing after dropping out of collage level english classes (the test said I was better than I was).
Kinetic stupidity has a new brand leader: Allen Zadr.
it would have been my goal to make the most wrong essay I could that would still generate a good grade from the system.
/bin/fortune | slashdotsig.sh
I bet I could write the other side of the equation: a program to create nonsensical gibberish that always gets A's. What would a teacher do if you handed in something like that? Apply a double standard to the student?
--
My username: hats off to George Carlin, and fuck the FCC. Freedom!
I live in Indiana, and I have taken these. They are not graded fairly, and they determine 10% of the final grade. A computer can obviously not grade essays fairly, so it shouldn't be done. I got a 5/6, which, according to the computer, was extremely well. However, this was an 83%, which brought down my grade significantly. This computerized grading isn't fair at all.
for the time being, i would trust more that program to moderate my comments.
c'mon people i was only joking dont mod me down, not noooo!!
"The quality of life is inversely proportional to the number of keys on your keyring."
SPAM filters are tricked all the time depending on the text of an email. Google was f'd up not too long ago because of trackback linking in blogs screwing up their algorithms. Isn't this a similar situation? If a student can figure out a way to beat the grader, we'll have students learning to write to beat software, not form a well written essay.
- gtaluvit (prnc. GOT-tuh-LUV-it)
Perish the thought should students start writing about the dangers of artificial intelligence. They may very well fail!
He who has no
The GMAT books are already giving formula essays to get you past any writers block that might happpen on the exam day...
You are in a maze of twisted little posts, all alike.
My alma matter graded most of my computer programs with shell scripts and I graduated in 1997. So I don't think India is the first to do that.
Lets just outsource all our test grading to Indiana too.
Best Windows Freeware
Ok, since you know the grading software is going to make it into the hands of the students, here's my scheme for perfect essays:
Step 1: Feed some encyclopedia articles, Wiki pages, and other random material on your subject into a Markoff chain generator.
Step 2: Use a genetic algorithm to generate variations of the text. Fitness is determined by the grade calculated.
Step 3: Repeat step 2 until desired grade is achieved. (And, of course, Profit!)
The result is totally worthless, but at first glance would probably appear legitimate even to a human reader.
Sort of like Slashdot posts.
That's actually a pretty novel way to approach the problem of creating Strong AI. Making smarter machines is hard, so what you do is dumb down the humans until even a coffee maker (or a grammar parser or whatever) would beat them in the Turing test. Damn, this is so sad.
>|<*:=
...until some wiseass figures out a way spoof the grader, probably by sliding under the radar of whatever probabilistic models they've got that pass for spell- and grammar-checkers.
For example:
Flimblarm nif goondatakun, jut sekfar bel shon duc. Seempkin dar goolnac flar tefnek voz toulian; elmpar gef sogquel.
Grade: B+ Your use of double-negatives continues to haunt you, but I'm glad you've gotten over hanging participles.
The only surefire protection against Microsoft infections is abstinence. - The Onion
At least the parent post proves one simple truth: human english teachers can be replaced by simple shell scripts.
Indiana parents are the first to buy (en masse) licenses for Essay Constructor Pro v2.0. The software produces essays that are indistinguishable from those written by real students, using the latest screen-scrape-from-Internet 'n' plagiarism-from-non-credible-sources techniques.
Indiana Director of State Board of Ed comments: "Isn't it wonderful how technology is improving education?"
Fred
"A fool and his freedom are soon parted"
-RMS
Not that there's anything in this post that serves as an example. I guess that's because I was graded by humans. Seriously, I don't recall getting any encouragement in writing back in the '70s in high school, and not much in college. I guess it wouldn't have been any worse if the Grade-O-Vac was inspecting my papers instead of my mostly-marginally-literate teachers. There were several exceptions, but they focused much more on reading than on writing. I suspect they had a lot greater effect that way--I know they had a great effect on me.
If I were a student, I'd want to get a copy of this software and use it to pre-grade my papers so that I could find out what's wrong and fix it before I turned it in.
Slashdot First With Computerized Moderation.
Now with Computerized Moderation the famous Slashdot message site can pre-emptively down-mod 'Redundant' posts long before they are actually 'Redundant.' The computerized modding process, called 'e-modder', uses a 6-point rating scale and uses artificial intelligence to 'mimic the modding process of human readers - including doing stupid shit like modding the first instance of a concept as Redundant'.
Glonoinha the MebiByte Slayer
Wouldn't it just be cheaper to grade the tests at call centers in India? What are those Indians doing when there are no incomming calls? Just slacking off??? They could be grading tests.
Imagine high school students re-writing their essays ...
Actually, anything that would encourage students to re-write their papers and improve their writing would be pretty amazing. Most students jot something down, run a spell checker and turn in their work. If they could pre-grade their work, they might be better motivated to put out more effort and improve their writing.
Fortunately, when people graduate from high school and enter the workforce they become motivated to always make their best effort.
By creating a vernacular consisting of elongated words and sophisticated verbiage, obviously indifferent to definition but simultaneously observing grammar regulations while eschewing colloquialisms, perhaps students may increase individual chances of achieving substantial academic acclaim.
If this works anything like the writing level indexes you find on word processors, it should be easy to fool.
If moderation could change anything, it would be illegal.
Writing is not mathematics. Good writing should not go along some artificial standard. Just because my paper is grammatically correct, has a topic sentence, 3 supporting paragraphs, and a conclusion doesn't mean it is good. Good writing needs a flow of ideas from one paragraph to another. It needs finesse, style, grace. This is like an IQ test for english writing. It would do very well in identifying poor writers - but could never identify a great one. I'm sorry ee cummings, your use of punctuation is poor 1/6. There are examples like this in books on taking the various standardized tests - any truly excellent writer is likely going to do badly. Why? The rules of the english language are guidlines, which may be broken when appropriate. This is just the mechanization of another facet of society, and should be tossed out with the rest of the garbage.
But I think that if a computer grading program which is no worse than humans could be devised, it would be a great learning tool. A lot of people make it to college as borderline illiterates. I'm not kidding. I read a lot of their crap. That's because their HS teachers were too overworked to grade their writing, so they didn't assign much. If a computer program could auto-grade and give detailed comments on how to improve the writing, high school students could be assigned an essay per week, and really get the hang of writing well. Teachers could focus on teaching instead of tedium.
Sure, the first grading applications are going to make a few serious errors. This is the first stage of every application when a computer is asked to interpret rich data. Early voice recognition sucked. Now it sucks much less, and it will just keep getting better. Same with OCR, chess software, machine translation, etc. So the right debate to have is about when this will be good enough for school use, and not whether. I'm prepared to admit that the answer to the right question is "not yet" (I'm sure how deep the current problems go), but I fully support working on this system until it works right.
I live in Indiana (no, NOT India) and took this test. Being a techie, I figured I'd try to fake out the system. This test works out to be 10% of the final grade and since I had a 98 going into the test, I figured I could afford to gamble a little, figuring if it back-fired I could blame it on a computer error since every one would figure the kid with a 98 MUST be telling the truth.
I almost wimped out. I wrote about 80 percent of the essay (about influence of pop-culture on society - and silly me I always thought society influences pop-culture but anyway). I had 5 paragraphs - 1 intro, 3 body - 1 half-assed conclusion. I reoreded the paragraphs, copied the one I felt was the best written and pasted it into the body 3 times.
Guess what I got.....6/6 (six point grading scale which is pretty messed up because a 5/6 is an 83%). Hopefully they won't audit mine....
All your base are belong to us!
A good essay always consists of an introductory paragraph, three body paragraphs, and a closing paragraph.
It is essential that every paragraph begin with a topic sentence. The first paragraph should state the thesis, or point of the essay. Since computers cannot actually understand the entire essay, you can assume that it will only be judging the local coherence of writing which is free to run like a river, past Eve and Adam's, from swerve of shore to bend of bay, taking us by a commodius vicus of recirculation back to Howth Castle and environs.
The second paragraph should make a point that present a countervailing view, the antithesis. Once again, spelling should be correct, the essay should be capable of passing a Microsoft Word grammar check, but after that we pass through grass behind the bush where a gull calls, coming far, ending here. Finn again? Take, but softly memory till thousands are given the keys to a way a lone a last a loved a long the river runs.
The third paragraph should synthesize the material covered in the first two paragraphs. It is, however, important that any material obtained from external sources be modified so that it cannot be detected as an exact match for anything on the Web. So, she went into the garden to cut a lettuce leaf to make an mince pie; and at the same time a great wolverine, coming up the street, goes into the store. "What! No laundry detergent?" So he died, and she very imprudently married the barber, and they all fell to playing the game of catch as catch can till the gunpowder ran out at the heels of their boots.
In conclusion, the final paragraph should recapitulate and summarize what has gone before: since you can be sure that a computer is capable of counting paragraphs, a good essay always consists of five paragraphs. If it has the right number of paragraphs and every word is spelled correctly, you are almost certain to get at least a passing grade.
"How to Do Nothing," kids activities, back in print!
This issue cuts deep into the heart of what grading is for -- it's possible for smart people to reasonably disagree, depending on what they think the intent of the grade is. Since grades are put to many uses, there are many answers to the question.
As a college instructor, I tend to use a strict grading protocol -- and then "bump up" a few of the students. If someone comes in to my office every week and really struggles to understand the concepts, but the computer tells me that they earned a "C+" -- they're likely to find a "B-" on their transcript. But if someone who's smart enough to get an "A" blows an exam from being hung over, that person gets little or no sympathy.