Slashdot Mirror
Computer Program Makes Essay Grading Easier
phresno writes "c|Net is running a short article on Prof. Bent at the Columbia, Mo., University. The Prof. has developed a computer program which he now uses to grade his sociology students' essays. He claims the program can discern content, and argument flow within sentence and paragraph structure, and has saved him over two hundred hours of reading per semester. How long before he's replaced entirely by his own program to cut down on staff costs?"
How long until some students get hold of this program and tweak their essay until it's puifect? It's similar to spammers using spam filters to test their emails first.
Rock that crushes, Paper & Scissors that don't matter.
I'm not so sure I like the idea of a computer grading my work.. I spent hours making it, but the guy doesn't even give it the time of day....
Angst
"The program analyzes sentence and paragraph structure and can ascertain the flow of arguments and ideas."
So it measures structure and argument.
How's it going to measure creativity of thought? Are we going to just pump out logic machines from colleges?
The only way that a computer program can possibly analyze a paper fully as a paper is if it read it as a human. heuristic algorithms, however sophisticated, just aren't enough for things of this sort of importance--after all, the profs are paid to grade (well, them or the grad students). It doesn't seem like it would be too difficult to just throw in keywords and make sure that you use proper syntax in order to fool this thing, albeit the prof says keywords alone aren't enough. I find the claim that his program can "analyze argument flow" quite dubious. I'll stick to getting my several grands' worth out of my courses, thank you very much.
Now if we could code a program to write those essays, the whole process would be automated. Press enter for another A. If you know how the program grades the essays, it should be possible to write a program that generates essays that comply perfectly to those rules, right?
Sure, you could be great at grammar and sentence structure. You could be an ace at using proper english.
But how would hidden talent and creativity be found? How will the teacher know if his students are actually trying hard to write their papers when all he does is check the thing with a computer program?
It's a really terrible idea and I think it's really cheezy. Ohh, he saved some time. So does that mean he now gets paid less? Does this automation get the students a discount? Yea, right.
If I'm going to put a lot of work into writing an interesting paper about something, I want someone to read it.
- It's not the Macs I hate. It's Digg users. -
What does this say about the field of Sociology? :P
Why not give the program to the English department and use it for teaching?
Would be great for high-school students. Have students write an essay or paper and analyze it right in front of them. Then the program highlights their errors (or what the program perceives as an error). Even better, complaining students would help fix bugs in the software because they know their intent - they could send off a highlighted error-ridden version to the developers with an explanation of why they think they are right.
Better yet, give it to everyone! It's not like you can cheat, you still have to rewrite and resubmit your papers. Shit, I say build it into text boxes on slashdot and wikipedia to start!
please do not hold this post to the standard of the Qualrus (real page of the software)
Get your Unix fortune now!
It works by scanning text for keywords, phrases and language patterns.
Which is to say, this prof is asking students to regurgitate data. Given, a certain level of base knowledge is necessary in any class and topic, and regurgitation (aka parroting) is an easy way to check that base knowledge. If a paper is assigned on a particular topic that they've been studying, then this sort of program can easily check for base level ability to spit back key words and phrases.
But, I seriously doubt that the class is ONLY about that base knowledge -- or that the program can reasonably check for anything more. I've had classes where the prof or graders did basically the same thing that this program does (i.e. check only for key words, phrases, and patterns they want to see), and I have little respect for those profs.
If you don't want to put even a basic amount of effort into checking a paper, don't assign it -- find some better way to check students' progress.
This one is just nuts. Why on earth am I writing essays which are going to be marked automatically by a machine? It's bad enough that scantron cards have found their way into subjects where they're totally irrelevant (a multiple-choice test for a university level Shakespeare course?), this is just another reason why post-secondary education has become increasingly less complete.
If he's allowed to use a machine to save him the effort of reading an essay, I should be able to use a machine so I don't have to go through the effort of writing one. Trust me, as arduous as it is to read a 20 page essay on the relative merits of liquid rubber concrete compound fasteners, writing it takes a lot more effort, a lot more time, and it damn well deserves to be read by the professor who assigned it.
I have lots of programs that save me hours of "doing my fucking job". They're called scripts, and it's called efficiency.
If I can write a program to automate a menial task so I don't have to do it, then by all means, I should do it. If grading undergrad papers is a menial task that can be automated, then it should be automated.
I mean, just because a freshman writes a bad paper doesn't mean a professor has to actually read it.
paintball
Unless it's an English class, they're just looking to see if you've learned the material for that course. Especially for the essay questions in the midterm and final, they will always just skim the papers for key words, phrases, and dates. They underline them and add up the points. That's just the way it works. The "B" student, who writes something interesting, but doesn't cover all of the relevant material, loses points.
For a term paper, you additionally have to use correct grammer and spelling. Also, do not try to argue something stupid. Don't take a contrary opinion to the professor or to popular opinion on the college campus. You won't be able to convince the grader, and they'll think that if your argument isn't convincing, then it must be flawed and you deserve a bad or mediocre grade.
These are things that I wish someone had told me when I was an undergrad.
My other first post is car post.
Announce to the class that they have two options. If they want their paper to be computer-graded, the maximum they can get on the paper is a B. Only hand-graded papers can earn an A - but they will be judged on original thought, well-chosen sources, and total structure. If they don't meet those criteria, they will be penalized (and as such, would score lower than the computer would give). That way, students who aren't confident in their capability for truly excellent writing and novel thought can choose the computer grader for a safe B, while the truly thoughtful and creative students can be justly rewarded.
Wait - what the fuck am I saying? ALL college students should be graded by the non-computer criteria I just listed, and those who can't do (or at least attempt) that kind of work shouldn't be in college.
If you mod me down, I shall become more powerful than you can possibly imagine.
I'd be pissed off, too, but don't let one bad professor sour you on all of us. My base pay is less than $60k a year, but with summer salary from grants I'm in your range. And I work way more than 20 hours a week. Way, way more. But what is this "union" you speak of? I know of no such union (although tenure can provide significant job security).
I used to give short answer/essay questions to my astronomy students the first couple of semesters I taught the big non-major course. It took a tremendous amount of time to grade which was one reason I stopped, but not the primary reason. I'm a novelist, and I know how to write, and there was a consistently high fraction of exams written so badly it was very painful to read. Perhaps I should have kept at it, with the idea that it's good for the students. But a few essays in a science class won't dent the problem that starts in k-12 education.
Professor of Astronomy, Author of Spider Star & Star Dragon (Tor)
One of my teachers at school tried that. Unfortunately what happened is that in each group the rest of the members (the groups were arranged so each group was mixed ability) ganged up on the 'smart but weak/shy' one to do all the work then goofed off.
Actually, now I come to think about it, that's exactly what happens in the workplace.
Stephen
"Don't write down to your readers, the only people less intelligent than you can't read" - Sign on Newspaper Office Wall
arguably a better learning technique from a usability standpoint
... but only for a content-free "discipline" like sociology or other pseudo-sciences.
Yes, it might well be a better learning technique from a usability standpoint
Cargo cults a la Feynman are all about form, and this tool can indeed detect the presence of form and even distinguish form that is considered "good" by some metric from form that is considered "bad".
But unless it actually understands what is being written through deep semantic analysis performed against a thorough database of relation-interlinked concepts, then there is no way the tool can detect hard scientific content (to the small extent that it occurs in sociology) from gibberish that just obeys the right forms.
As Brent himself says, "In sociology, we want them to learn the terms." And that pretty much sums it up.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
Same here. Base pay here is not much more than 50k but I'm not complaining. I work 40+ hours a week too. Most nontenured faculty do, as do most tenured who still want to get promoted or who actually want to stay active in the field. I know of a few faculty who just show up and teach but they are a year or two from retirement.
With regards to students' written work: My field is meteorology. I too used to give students in my survey-level meteorology class opportunities to "express themselves" via short answers (a paragraph or two) on exams. I stopped because it was so hard to grade many of them because they were written so poorly. In addition to that, it is very difficult to grade short answers in a consistent way. For many of the short-answer questions I would usually end up just writing a number down ("Hmm.. this feels like a 3-points-out-of-5 answer") which real doesn't feel right... but what do you do when the concepts are confused, spelling and grammar are terrible but they have expressed some knowledge of the material?
I have talked with professors who have been doing this stuff for a much longer time than I (some of whom are into the latest trends in teaching etc.) and many of them are gravitating towards all objective tests (multiple choice and true false) for their survey level classes (and some upper level). A well-written objective test should adequately test a student's knowledge of the material in a fair way, especially in the sciences where there truly are right and wrong answers. Still, I don't like giving these kinds of tests - it just doesn't feel right - but like grading the others even less.
In my upper level classes all of my testing is subjective, and I do assign papers such as case studies where a storm system is described and analyzed. Some of my seniors can write well, most of them are so-so and a few are truly terrible. I tell them up front that spelling, grammar, style etc. counts on these assignments, and I find that if you tell students that these things are part of their grade they will put in an effort to write well.
I suppose I could just "blame the high schools" but I think the problem is deeper than that. In the US grade inflation is a huge problem in many universities and at the college level, student evaluations of faculty are often very highly regarded (and if you are evaluated poorly it can keep you from getting tenured or promoted). So a logical response is for faculty to go easy on students, rightly assuming that this will return higher evaluations. I don't know if that is a part of the writing problem, but I know an A today isn't an A 20 years ago at many universities.
A squid eating dough in a polyethylene bag is fast and bulbous, got me?
If the profs at your school mark grades down because they have a different opinion you should change schools. No offense.
I had a professor that said he would give his opinion but he would grade based on how well the argument was crafted and backed up by facts/references.
ANother professor complained that too many papers where just repeating what he had said in class and he had marked those down for not having enough original thought.
Grammar and spellling are alway important..