Slashdot Mirror
Computer Program Makes Essay Grading Easier
phresno writes "c|Net is running a short article on Prof. Bent at the Columbia, Mo., University. The Prof. has developed a computer program which he now uses to grade his sociology students' essays. He claims the program can discern content, and argument flow within sentence and paragraph structure, and has saved him over two hundred hours of reading per semester. How long before he's replaced entirely by his own program to cut down on staff costs?"
Qualrus doesn't operate using a set grading criteria, but trains based on the users' grading markups. Therefore, you'd need the teacher's copy (complete with its "learned" patterns) to fool the system. Actually Ed Brent encourages the students to use Qualrus to write rough drafts, as it gives instant feedback - arguably a better learning technique from a usability standpoint (faster feedback == more retention).
I was required to take a lot of writing classes for my college (and still haven't finished them all) and I've observed the quality of my writing go up appreciably since I began school. However, the reason I've become a better writer is because my essay graders write copious comments about where I'm going wrong in my papers and what I should do to improve - and they read the next paper I write for the class with those things in mind, and tell me whether I've improved sicne the last one.
The article didn't say anything about what kind of feedback the program provides, but I can't imagine it's anywhere near as helpful as the paragraph-long evaluations of my logic, style, and structure, which I got back with every paper I ever turned in, and I'd be impressed but surprised if his program took each student's previous weaknesses into account in the course of the evaluation. In writing, practicing can only do so much - the real help is in constructive feedback, and I just can't imagine where these students are getting it if not from the human graders of their papers.
According to ETS, the e-rater agrees with the human grader 98% of the time.
My userid is prime!
From TFA, which apparently no one has read yet:
"The final papers, which he does read, are usually much better as a result of Qualrus, too."
There you go! For the reading and comprehension impaired, here's a summary of what's actually happening, which even the reporter didn't get:
1. Students write a draft of their essay, which they then upload via a Web form to this program
2. The program gives them a score on various parts of their essay, giving them valuable feedback on what needs to be improved.
3. Students improve the pieces of their essay that the program suggests.
4. Students submit the final draft to the professor, who reads and grades each one by hand. Due to steps 1-3, the quality of the final draft is much higher.
This sounds like a great thing to me. Wish I had something similar for my students. I don't have the time to read through dozens of drafts for every student. Too bad I'm not in sociology.
Actually Ed Brent encourages his students to use Qualrus on earlier drafts of the papers. This provides immediate, extensive feedback. And by "extensive" I mean more detailed and descriptive comments than those that a single teacher/TA could supply for each and every paper in a large lecture. The immediacy of this feeback is what is really important - immediacy is KEY to learning.
Actually that's already been done. Quite effectivly too.
This is similar to what happened in the essays in the Civil Service examinations in Ancient China. All essays needed to follow an exacting formulaic structure and as a result the system, though complex, was easy to manipulate if one knew the proper structure. This can be a problem since unique thought and creativity are not given a great value.
Hmm...the NSF grant provisions I recall seeing the last time I submitted one discouraged, or even ruled out, supporting commerical activities. This does sound a bit funny to me, too.
Professor of Astronomy, Author of Spider Star & Star Dragon (Tor)
I wrote a midterm and a final paper for an anthropology class in college. The Prof talked about how he wanted us to do in-depth analysis and blah blah blah. For the midterm, I focused on a a few details and examined them closely, but did not cover all of the facts that were relevent, and got a C. For the final, I spent an approximately equal amount of time and basically rephrased the first sentance out of every paragraph in the relevent book chapters and got a near perfect grade.
An important lesson to learn freshman year: the TA grades the papers, not the prof. (in this rare case, the prof might have actually graded differently -- he was pretty good).
My father is a professor at the Australian National University (Physics Dept).
He gets paid US$60k a year, works 8 hours a day at work.
Then comes home and spends his evenings on his laptop working for another 1 to 3 hours. And then on weekends spends another 3+ hours a day working.
None of which he gets paid extra for, as he is on a fixed salary.
Don't taint all professors with one what professor did.
Well, there are some professors that meet that description, but at a reasonable university, those tend to be in the minority. At a reasonable university, most faculty work more like 60-80 hours a week, particularly if they are active in research. I certainly have pulled many more all-nighters as a professor than I did as a student and I pulled a lot of them as a student. A few things that students tend to overlook:
There are terrible professors and great professors at every university- the fractions may change from place to place, but with some seeking out and strategy, usually it's possible to do well.
It's psychosomatic. You need a lobotomy. I'll get a saw.
I taught first year eng lit to a class of 30. I read each paper 3 times. 1. To order the papers from bad to very bad. 2. Read, marked and made comments. 3. Read again to make sure the marking spectrum was fair (worse papers got worse marks ect.)
I'm an English teaching assistant, and I can tell you that every TA and professor I know reads each paper carefully. One of the main purposes of handing back students' essays, especially in a first-year English course, is to give them personalized tips on how to improve their writing (argumentation, flow, grammar, etc.). It takes care to give each student useful comments.
If we end up sorely overworked it can be hard to maintain that standard of marking, but even then we're certainly not just skimming for keywords.