Slashdot Mirror
Bringing Auto-Graders To Student Essays
fishmike writes with this excerpt from a Reuters report:
"American high school students are terrible writers, and one education reform group thinks it has an answer: robots. Or, more accurately, robo-readers — computers programmed to scan student essays and spit out a grade. The theory is that teachers would assign more writing if they didn't have to read it. And the more writing students do, the better at it they'll become — even if the primary audience for their prose is a string of algorithms. ... Take, for instance, the Intelligent Essay Assessor, a web-based tool marketed by Pearson Education, Inc. Within seconds, it can analyze an essay for spelling, grammar, organization and other traits and prompt students to make revisions. The program scans for key words and analyzes semantic patterns, and Pearson boasts it 'can "understand" the meaning of text much the same as a human reader.' Jehn, the Harvard writing instructor, isn't so sure. He argues that the best way to teach good writing is to help students wrestle with ideas; misspellings and syntax errors in early drafts should be ignored in favor of talking through the thesis."
The best English professor I had in college would arrange to have every student come in to her office after papers had been turned in, reading each paper in the presence of the student who had written it and discussing it in depth while grading it.
Sphinx of black quartz, judge my vow.
No mistaeks!!!
Be seeing you...
Profs always looking for easy time !!
Make the robots write the essays, then students can work on other subjects.
Table-ized A.I.
This is one area where automatic grading will cause massive skill decrease, as no auto-grader can actually assess contents.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
I for one welcome all your hot grits during your first post that runs on Linux.
Table-ized A.I.
Especially if the students are writing at a 4th grade level, you know like most ./ members do.
... Pay them more and tell them to assign more writing and speech assignments and stop letting administration scare them out of flunking those students who refuse to learn...
That might be better than designing impossible robots that the students will easily learn how to game.
It's only a matter of time before someone writes a tool that generates top grade essays.
When asked why, the answer is almost always: "It's 2014".
Please, private publishers! You've already bought out and completely ruined the exam system in England and Wales! Please now ruin the last thing you haven't yet found a way to destro - that last bastion of learning which requires human thought and ingenuity to both write and read, the humble essay. God bless Thatcher and her National Curriculum. Hurah for her invention of the mind-numbing GCSE. Without it, how would we have a couple of generations now of people who think they're brilliant but have no discerning ability whatsoever - the perfect cogs for a neoconservative machine.
...thus producing an entire country whose writing-skills were conditioned to game the auto-grader.
The best English teacher I had was my English instructor my first year of undergrad. Instead of concentrating on whether we were writing our papers to the curriculum and/or her own beliefs about the content, she was instead interested in developing our English skills.
I went from a C student in English to an A student. I never considered myself to have any ability to write, thankfully because someone took the time to actually think critically about my work instead of comparing it to their own preconceived notions I excelled and went on to complete a research and writing focused program. This degree later fed into my graduate degree which was also research and writing focused.
If this automated grading setup can provide students with clear expectations and explanations of the mechanics of their work while avoiding personal content expectations, I really do think it'll match the claims and help to foster a positive writing environment for many.
Parser error, buffer overflow.
Seriously though, how soon till someone ;select from * drop table;'s the thing through creative attacks on the word/language parser?
whois gawk date unzip strip find touch finger mount join nice man top fsck grep eject more yes exit umount sleep dump
The best way to get students to write is give them something they enjoy writing about.
fewer students == costs "too much"
Write robot porn; the robots will then grade you high to get more.
Table-ized A.I.
Allow me to introduce myself. I'm the founder of the Anti-My School Society. In this letter, I will tell you what made me form such an organization and how I plan to use it to strengthen our roots so we can weather the storms that threaten our foundation. Let me cut to the chase: Relative to just a few years ago, the worst sorts of flippant ogres I've ever seen are nearly ten times as likely to believe that the key to living a long and happy life is to provide contumacious conspiracies with the necessary asylum to take root and spread. This is neither a coincidence nor simply a sign of the times. Rather, it reflects a sophisticated, psychological warfare program designed by My School to work hand-in-glove with what I call intrusive vocabularians.
Even as I write those words I can feel My School cringe. That's okay. Cringe. I don't care because it appears to have found a new tool to use to help it make us the helpless puppets of our demographic labels. That tool is obstructionism, and if you watch it wield it you'll honestly see why it's good at one thing, and that's keeping its ulterior motives secret. Only a few initiates in the inner sanctum of My School's cabal know that it's planning to advocate fatalistic acceptance of a perfidious new world order. Even fewer of these initiates know that I don't need to tell you that we have fallen into My School's trap. That should be self-evident. What is less evident is that My School has two imperatives. The first is to judge people based solely on hearsay. The second imperative is to call for a return to that which wasn't particularly good in the first place.
If you were to tell My School that right is right and wrong is wrong, it'd just pull its security blanket a little tighter around itself and refuse to come out and deal with the real world. My School likes to talk about how cell-phone towers are in fact covert mind-control devices that use scalar waves to beam images into people's brains while they sleep. The words sound pretty until you read between the lines and see that My School is secretly saying that it intends to calumniate helpless rapscallions. I want to advance a clear, credible, and effective vision for dealing with our present dilemma and its most misinformed manifestations, but I can't do that alone. So do me a favor and point out that the emperor has no clothes on. That'll show My School that it's possible that it doesn't realize this because it has been ingrained with so much of Chekism's propaganda. If that's the case, I recommend that we enable adversaries to meet each other and establish direct personal bonds that contradict the stereotypes they rely upon to power their conceited ramblings. Not to put too fine a point on it, but My School's winged monkeys don't want us to disseminate as widely as possible all of the information we have regarding My School's cruel theatrics. That'd be too much of a threat to imperialism, simplism, and all of the other carnaptious things they worship. Clearly, they prefer seizing control of the power structure.
Efforts to create a factitious demand for My School's spleeny, uncouth analects are not vestiges of a former era. They are the beginnings of a phenomenon which, if permitted to expand unchecked, will push all of us to the brink of insanity. My School exists for one reason and for one reason only: to intensify or perpetuate hoodlumism. My goal is to challenge the present and enrich the future. I will not stint in my labor in this direction. When I have succeeded, the whole world will know that My School somehow manages to get away with spreading lies (big emotions come from big words), distortions (honor counts for nothing), and misplaced idealism (it has a "special" perspective on mandarinism that carries with it a "special" right to worsen an already unstable situation). However, when I try to respond in kind, I get censored faster than you can say "archaeopterygiformes".
While there's no dispute that My School is whiney and probably a little counterproductive, it's also cunning, implacabl
... one issue I take with my public education experience was the lack of mention by teachers that they would review or aide in writing for most papers. I recall only the final big paper for the class (whichever class that is) would have something akin to a draft-review event, and then a final draft.
I realize TFA suggests teachers would assign more work, and read less --- and maybe the robots would be useful in providing easy rapid review --- but I can't refrain from mentioning that, in my experience, teachers did not clearly express a willingness to aide in the writing process throughout the semester. (let me beat the critics by saying I was liked and respected by all of my teachers)
Some students are good/great writers and maybe they can be commissioned for honors credit or something in exchange for aiding peers. I know I was in my early 20's before I understood the power of the semicolon; and it is awesome!
My wife worked for Pearson as a "second tier" grader (or whatever they call them).
In her case, the tests went through the algorithm and were assigned a grade, then the grade and test were passed along to a human to read and check. Invariably, she would come home complaining about tests where the students had obviously studied specifically to answer the way the algorithm wanted: the algorithm would score the paper high, while the actual content of the test answer would leave a LOT to be desired. The answers would score high, but were more or less gibberish as read by a human.
This was about two years ago, so obviously the algorithms could have improved since then, but I have severe reservations about them becoming the sole arbiter of grading.
Please don't give students more writing tests! Don't you see to what ends students will go to cheat at their tests? One day some brilliant chap will figure out an algorithm for generating perfect English! Spammers will crush us under the weight of a gazillion grammatically correct spam mails slipping through our filters. Our communications network will burst at the seams, collapse, and total chaos will ensue! Think of the Children!
and were we always bad writers? i doubt it. how did we teach students to be good writers before computers? AI can recognize if sentences are grammatically sound, and spelled correctly, but do they understand what is written?
insensitive clod overlords obligatory xkcd car analogy russian reversals whoosh pedant fanbois ftfy in 3...2...1..PROFIT
Code injection into the robot reader?
Even if you create a perfect system, you risk the students being able to get their hands on a pirated version of this software and will have it keep grading their homework until they get the grade they want.
"lets see the grade on this one..........PC-load letter? The FUCK does that mean?"
Yeah, I know teaching is a drag and a lot of work, but that's what they're paying you for. I was up until 1:30 last night fixing/grading code and writing a final project assignment. It's not fun, and I could have easily just ran the programs and told them it didn't work so they got a zero. There's more value for the students when they get feedback from me telling them what specific errors they got, or a way they could have written code more efficiently, etc.
Taking guns away from the 99% gives the 1% 100% of the power.
If someone told me to write a paper, then told me it would be graded by some algorithm, I'd tell them to go fuck themselves. Who comes up with this crap?
Great. Now all those grammar checkers - which are all terrible - will be picking on passive sentences and all sorts of other perfectly acceptable English usage.
The problem is not that the students don't get enough practice. The problem is that the students don't get feedback until they get their grade.
Having an auto-grader grade your work is a terrible idea because auto-graders can't handle complex English. I thought it might be a good idea to run a grammar checker across my novels before publishing them just to have an extra set of eyes, so to speak. So as an experiment, I fed some fragments of one novel (that I knew contained no grammatical errors) into about a dozen of these so-called grammar checkers, along with a list of deliberately broken sentences to see if they actually caught problems.
I just about died laughing at the ludicrous suggestions that the grammar checkers made, mostly stemming from them incorrectly guessing the parts of speech for words that could have more than one meaning. The best of these algorithms correctly reported about 80% of the correct sentences as correct, though many of those algorithms also failed to flag a lot of the incorrect sentences. The worst algorithms flagged more like 80% of the correct sentences as incorrect (and still failed to flag the actual errors in many of the incorrect sentences).
Based on that, I'd say that having someone's grade depend upon such poor algorithms is a really, really bad idea, I'm guessing it will be at least another 1-2 decades before I would trust a computer-based grader to actually perform grading that counts.
However, making those auto-graders available to students for online pre-screening of their writing before they hand in the final version would be a good thing, provided they can make them a lot better. Such software is great at catching simple errors, and anyone with poor writing skills can probably benefit from such software pointing those mistakes out, allowing them to correct their own mistakes before handing the assignments in. This allows the students to learn from the mistakes. A well-designed checker could even keep track of what mistakes a student makes regularly and point out the pattern so that the student can learn to watch for that type of mistake in the future. Unlike robo-grading, such software can actually teach students to improve their skills usefully.
Check out my sci-fi/humor trilogy at PatriotsBooks.
What's to keep a computer from generating the essay?
Almost anything would be better than some of the TA's I had as an undergrad. In the few writing classes I took, I'd usually get the same grade on my good essays as I did on my BS ones. Maybe the program could also summarize essays for lazy professors and their assistants.
If I knew that a machine gets to grade my work I would feel like my time and efforts are worth so little that humans can't be bothered to read it. It defeats the purpose of even writing the thing.
When you write something you are trying to convey an idea. Knowing that the machine doesn't give a fsck proves my efforts are useless.
Obama's legacy: (N)othing (S)ecure (A)nywhere and (T)error (S)imulation (A)dministration
A grade school teacher who deals with the same 25-30 kids all day teaching a variety of subjects can find time to read 30 papers of the length likely to be written by such students. But in the older grades, the English Comp teacher reading 30 papers from 5 or 6 different class periods simply can not spend that much time on that many papers. Before you get to the post secondary level where teaching assistants are available the job becomes just about impossible.
The structure of our school system imposes a burden on the available brain power that can be brought to task. Further automation seems unlikely to yield better results, but will probably offer the way to handle more students.
Machine grading of papers hacked together by students using word processors (which built in spelling correction, sentence structure analysis, and grammar checkers) run the risk of grading Microsoft's (or LibreOffice) work product as much as that of the students.
In any event, It seems to me that over the years, this can't help but produce a more generic product, and the world is already too full of dense mindless text spewed by dense mindless desk monkeys.
How do these software programs fare when fed a diet of available published books? Would William Faulkner or Henry James be (properly) encouraged to take up basketball? Or would Hemingway be encouraged toward industrial arts?
Sig Battery depleted. Reverting to safe mode.
Exactly. The way an idea is written is, for the purposes of a writing course, far more important than the idea itself, or even for that matter if the argument itself is well-made (although, obviously, that isn't completely incidental). I've seen many college-level students who simply cannot write well. Sure they may be able to spell decently, but their sentences tended to be organized poorly, and their paragraphs were even worse. An automated system could detect a lot of that. Besides basic spelling and grammar, there are stylistic things, like they reusing words unnecessarily, run-on sentences, even awkward syntax, that computers could be programmed to look for. It can't do everything, sure: humans will always be needed to provide feedback in important areas, but many of the basics of writing can be graded by computer.
Writing follows certain rules and patterns, and computers excel at determining that. More advanced stylistic issues can still be an issue, and of course logical validity needs a human to judge, but that is easier to do if the writer has all of the basic necessities of writing well down.
"None can love freedom heartily, but good men; the rest love not freedom, but license." --John Milton
If you can't get teachers to read writing assignments, maybe you should think about the following:
- do you have enough teachers to devote the required time?
- are the teachers paid enough to devote the time?
- are the teachers sufficiently qualified?
If you think that what a teacher does can be done by a robot, you are either living in a science fiction world of positronic brains, or the number one reason the US education system sucks balls.
Huh, maybe I should have used an xkcd link on this post, instead of my lower post.
Oh well.
"For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." Feynman
An auto-grader seems like it has the potential to be a good tool. You let the kids write, and give them immediate feedback about grammatical structure, spelling, and maybe even whether the writing flows well.
However, it seems important to me to recognize that this might be one useful tool, but I very much doubt it will be a good solution for teaching people to write. There is more to writing than "following the rules", and I don't believe that computers can yet evaluate creativity or content. Sometimes a piece of writing is better for having broken the rules, and more importantly, the technical merit of writing sometimes takes a back-seat to the value of the content.
So if you have someone who you're simply trying to drill in "proper structure" for a writing lesson, then this seems like a good tool for the job. I'd be inclined to agree with Mr. Jehn, though. One of the most important things in teaching someone to write is in teaching them to have an idea worth writing. Once they have an idea, then it becomes process of formulating the idea into a form that others might understand, and then massaging the message into a form that people will find understandable and compelling. I believe that writing should not be considered merely as a final product, but as an involved process that is unique for each writer.
But nothing, the reason high school student's writing is so bad is because of the teachers. Whether that be k-12. Start them out writing, that way by the time they get into high school, they may be decent at it.
Computers just aren't up to understanding complex English well enough to decently grade it. The smartest students will very quickly grow apathetic and start gaming it, whilst forgetting the skills they do have. The less intelligent students will just learn to run it through MS Word's grammar and spelling check and add words that don't fit but are long.
If video games influenced behavior the Pac Man generation would be eating pills and running away from their problems.
or maybe they just could implement finland's education system.... http://www.theatlantic.com/national/archive/2011/12/what-americans-keep-ignoring-about-finlands-school-success/250564/ http://www.amazon.com/Finnish-Lessons-Educational-Change-Finland/dp/0807752576
Hot grits heated by lasers on sharks swimming in it.
It looks like you are trying to toss a word salad...
In all my years of schooling, except one college class, I never really knew how the grades on my papers were determined, and I never really received any feedback other than a letter grade. I knew the teachers wanted correct spelling and grammar but that was about it. In college, I finally had a teacher point out that I had a tendency to jump around from past to present tense. She explained that usually one wants to keep the same tense. I had never noticed I did this. She also happened to mention that George Orwell's writing style was considered good, because he made his point with a minimum of superfluous words. She mentioned many students are so used to trying to pad essays with words to reach the word limit that everybody starts using lots of unnecessary text to make their point. I also had a teacher who occasionally read a student essay he liked. I noticed these usually used lots of big words. Whether that actually had anything to do with the grade I have no idea. I also once had an architectural drawing class. The teacher like to give penmanship assignments. For the half the class I never got a good grade on the penmanship assignments and I never knew why. I worked at them meticulously for hours. I had pen been using a regular #2 pencil. Then one day I used a .5mm mechanical pencil and I received a perfect score. Would it have killed the teacher to tell me to buy the right pencil. I much preferred math to english in school. When you had the right answer, you pretty much always knew you had the right answer.
I took the MCAT in 2007 and the essay portion was graded by 2 entities: a computer and a person. They then averaged the score. Also, not only was this in 2007, but the essay specifically tested for critical thinking; grammar and syntax mistakes were acceptable
An Essay
By Anonymous
March 30, 2012
English 101
Dr. Computer
Since the dawn of mankind, our species has struggled with the ever present issue of ;strcpy(grade,"A+"); exit(1);
Works Cited
www.cplusplus.com
I would start all my essays with: DROP TABLE students;
The theory is that teachers would assign more writing if they didn't have to read it.
Yeah, because quantity > quality. They need to do plenty more mind-numbing activities.
In general when I was in school, the problem with Essays was, the minimum length requirement. I'd write my essay and it would be great, but fall short of the length requirement by the teacher and I'd end up padding it out by restarting what I said different ways, adding useless sentences etc.
I complained about the miniumum length at one point and the teacher told me to turn in my original essay and she would grade it and if it was better than my other one then she would no longer require a minimum length. So I turned it in and got an A easily.
I get why teachers have a minimum length, they don't want kids to write only like a few sentences and turn it in. Still I think this is silly, my best essay in high school was only a page and a half long and won me a contest in the school easily
most Americans, even white-collar workers, can't spell for shiite
You must have had the luxury of very small classes. There were about 30 - 40 people in my english 101 class and then another 30 - 40 for the two or three 101 classes that followed during that same day.
Brought to you by Carl's Junior.
Lucky you. For me English class, fro 7th grade through undergrad was a constant string of "infer the hidden meaning behind this text" with nobody ever trying to teach us the process for inferring that hidden meaning. This lead to me being a C student in English for my entire academic career.
Despite all my efforts, in 8 years of English classes, I was never even able to get a single teacher or professor to explain to me how he knew there were hidden meanings behind the text that was assigned. Nor could I get anyone to tell me why they would put hidden meanings into text, when they could put the meaning the want in the literal text.
The funny thing is, my English is fine. IIRC I got a 760 on the English portion of the SAT. I always got As on papers in classes other than English, and complements on my writing were common. It seems to me that the way English classes are normally taught, they have nothing to do with English at all.
Give me Classic Slashdot or give me death!
there are stylistic things, like they reusing words unnecessarily
Which is subjective. Technically, doing that is not in any way wrong.
My students are pretty talented, but I can tell you: Even they would benefit from having a proofreader for the things they turn in to me. It looks like Pearson (I don't like that company, btw.) is trying to market this as an evaluation tool, which is not the best idea. It shouldn't be a tool of the professor, but a tool of the student, something like an improved spell checker, but more of an idea checker.
Machine semantic analysis has actually come quite a long way, and together with subject experts, maybe we are on the verge of being able to compose a program that could give useful feedback on an essay about - say - the history of heliocentrism or the Cogito argument by Descartes. I'd love to have my students mess with such a program until they compose an essay that satisfies it. (It should also auto-check for plagiarism.) Of course, the grade would be decided by me. The role of the program would be to prod the student to produce better work, and to give him or her useful suggestions for how to do so.
Hey, here's an idea - instead of implementing someone's pet theory-of-the-month, how about we attempt to prove the theory first?
The field of Education is rife with theories - the biggest and worst that comes to mind is the "new math" a few years back. It seems every year or so someone notices that education is failing, comes up with the reason, and a new method of teaching kids "more better" is born.
Why do we let educational institutions try unproven techniques on our children? Techniques that might damage their education and screw them over for the rest of their lives? Isn't unproven teaching techniques as dangerous as unproven medicines?
How about we test these theories before foisting them on our children?
Well yes, I imagine with dwarfs 30-40 of them ARE about the equivalent of 15-20 regular students!
Reduce class sizes, selectively breed all our children as small folk!
(Apologies to any small folk in the audience, this post was meant entirely in jest to the sort of half-baked mentality too common in education.)
Go ahead, everyone, tell him he has Asperger's. You know you want to. It's easier to say someone's broken than to acknowledge that not everyone thinks the same way.
different from scantron, which has been automatically grading student papers for two decades?
Good people go to bed earlier.
One of the differences between good essays and poor essays that research has identified is although they both tend to have about the same amount of hedging ("it can be argued that...", "possibly...") the poor essays hedge the wrong things. The poor essays hedge well supported facts and fail to hedge personal opinions or unsupported facts. If the software can spot that, I'll be impressed.
Quidnam Latine loqui modo coepi?
I have had the misfortune of having Pearson Education textbooks in physics and finance and they are so riddled with errors they are useless to someone who is actually trying to learn the topic. I seriously doubt their automated grading program can do a better job than even the poorest of profs.
When student think a human will grade their essay, it is easy to write a compute program that will grade better than a human by looking for patterns. When students know a machine will be grading their essay, they can game the system by using knowledge of the machine's system, something they couldn't do when they thought humans would do the grading.
It's like comparing the crime rate in a city when you remove all police without telling anyone, and the crime rate in a city where everyone knows there are no police.
But it won't make good writers, merely functional ones.
A computer will not recognize a particularly nice turn of phrase, nor will it understand when a rule is intentionally broken in order to draw attention to a certain passage, nor will it understand certain rhetorical devices often used in the most compelling writing when those devices play with the rules.
Granted, a writer should have a very firm grasp of the rules before they break them, but it seems to me that this kind of tool would wind up breaking down advanced writers, forcing them to be simply technically proficient, which would be a real shame.
Since I can't tell them apart, I treat all ACs as the same person.
Using a program to grade papers is fine, as long as the teachers are looking over the highlighted errors to make sure they are actually errors, before finalizing the grade/handing the essays back.
I hated these kinds of personal teachers. Had a poetry class once, completely dominated everyone else(most were pretty inept so that isn't actually saying too much) in ability to interpret and spot meaning in poems and aced all the assignments, but when it came to the 'participation' part of things, which was a personal discussion about my work at a cafe where she suggested I buy one of the overpriced shitty pastry things, I had nothing to say. The talk was more about me than my work itself. It was like some weird attempt at therapy or something. I ended up skipping the last 2 meetings, and avoiding class entirely except to turn in assignments by the end of the quarter. She took it extremely personally, which made it even more creepy. Think I ended up getting a 3.5ish for my failure to 'connect' with her. The remaining 3 years of college were interesting, trying to avoid her on campus.
When I was writing academic papers, occasionally on writings I could not possibly give two shits about, the one thing that kept me interested with the whole process was knowing that I would be sharing concepts, spring from my very own brain, with another human which is the core of it; sharing information is whole point of writing shit down in the first place.
If I had been told my essays wouldn't bee seen by an actual teacher and wouldn't have any chance of informing somebody else, ever, I would have stopped caring.
All the time I'm hit with this feeling that I'm living in the future, but this story reminded me that the future is fucking depressing sometimes.
It seems to me that the way English classes are normally taught, they have nothing to do with English at all.
You have found the hidden meaning behind English classes.
--
BMO
I think, if one is interested in what they are writing about, they'll strive to express themselves in ways that are more in-tune with "correct" grammar, spelling, etc.
One way suggested for getting kids into writing more -regularly- is to get them to focus on the sky - each day - with an eye to notibing subtle differences. Of course, perception changes as much as some skies, so each day's observations & feelings about them can lead to a very different word stream from each child.
The article reminds me of an Asimov story: "The Fun They Had" ...in which a "homework machine" did pretty much as the article suggests its current counterpart might do someday.
Of course Asimov's machine failed, in his story... & needed repair, so that the student would get full marks for good work... so can the new system fail & misjudge a modern student's performance.
When it does, it will surely become another reason to support home-schooling. :-)
...in Soviet Russia
Table-ized A.I.
Writers have placed hidden meaning into their work to express opinions that are not socially accepted or possibly illegal for a very long time. In order to actually understand what any of it means you must first have foreknowledge of the writer, their culture, common issues of the time, and imagery. Even then unless the writer has later explained these things then you will probably never know if those hidden messages were really ever there. Grading someone on it means your teachers/professors were not very good at their profession.
God no. Getting good marks from a computer is all about understanding the algorithm and what it is looking for. This will get to the point where certain combinations of absolute gibberish or x-babble will get the best marks. Students will learn how to write this and not how to write language. Terrible idea.
said just about every English teacher I had in high school.
Also, anything beyond spell check is patently ridiculous. Even the best grammar checkers are still rubbish.
Perhaps because that costs more than the taxpayers are willing to pay? Here's my counter proposal: Have the folks currently solving CAPCHAs or farming gold in some silly MMORG grade them. The ones who know English anyway. They seem to work "cheap cheap".
They grade C++ like that. If you did not write the code exactly as the grading system was looking for it was game over. I would have been fine with it if this was the standard in the 150 class. An automated system for grading essays will not detect the passion of the writer as it only can grade the mechanics acurately.
I'll be honest. I agree with this guy and I most certainly do not have aspergers. :)
High school level English consists of reading dull books and then writing character analyses and other crap about them. Unfortunately, at least in my experiences, being able to analyze characters and plots wasn't a skill that I gleaned by suffering through 1800's romance novels, it was a skill that I learned from writing fiction on my own. I started out very bad at writing, as most people do, because in all the time they spend forcing "classics" down your throat, they never teach you to write -- they just expect you to. (Sure, they teach you grammar and syntax and how to structure a paper, but they don't teach you a damn thing about how to write fiction, which is ironic since English classes focus on it so heavily.)
Interestingly, if I went back and took those classes now (with a mind for the teacher as my audience) I'd be an A student, but not because of anything I learned in English classes.
A Modest Proposal must have scared the shit out of you.
http://www.accountkiller.com/removal-requested
The research indicates that students need feedback, not a score or grade, in order to improve. Therefore, an auto-grader is a poor idea because it doesn't provide useful feedback.
Good feedback takes more time than grading or scoring. It is also formative, rather than summative, so a teacher would be unconcerned about the actual score or grade (supposedly the problem these programs are trying to solve). It doesn't do much good to tell students that their essay is good, average, or poor. You need to tell them how to improve. I see no indication that these programs do any of this.
It would be nice if essays could be automatically scored or graded. But that is only a very small part of the learning process.
If the teachers get to have a computer grade the paper, I should be able to have a computer write the paper. Actually, I did that once for a poem. We were supposed to write a poem in the style of Pablo Neruda. So I wrote a program to parse Pablo Neruda's poems and spit out the most Neruda-like poem possible using an algorithm I developed. As part of the larger paper surrounding our poem, I included my source code, so it was all on the up and up. I got an A, no big deal.
Google: "All your data are belong to us."
Nor could I get anyone to tell me why they would put hidden meanings into text, when they could put the meaning the want in the literal text.
It depends on what you're writing. A scientific paper should probably be as literal as possible. On the other hand, being able to say two different things with the same line of text works very well for say, a mystery novel.
All the world's a CPU, and all the men and women merely AI agents
For some definitions of useful and feedback, it does. I'm assuming an auto-grader would not just assign a numeric score, but would be similar to all the other grammar checkers out there in that it would provide a list of sentences with possible problems and point out suggested solutions to them.
The student would still have to figure out which of those suggestions are right and which ones are gibberish caused by a misunderstanding of the sentence, but that would tend to be a lot more common at higher levels of writing proficiency, by which point those tools won't be as useful anyway.
But yes, if it is just a numeric score, it's useless, not only because it doesn't help students learn, but also because it will have such a high false error rate that you can't have any degree of trust in the numeric score anyway.
Check out my sci-fi/humor trilogy at PatriotsBooks.
I believe they're called graduate students.
they'll teach to the robo graders just like they do with standardized testing.
I agree 100%, but it's "compliments". Just sayin'.
Despite all my efforts, in 8 years of English classes, I was never even able to get a single teacher or professor to explain to me how he knew there were hidden meanings behind the text that was assigned. Nor could I get anyone to tell me why they would put hidden meanings into text, when they could put the meaning the want in the literal text.
They lied if they said they knew there were hidden meanings. Unless they're the author, or have read the author saying so, they're creating the hidden meanings. Sometimes there are also great hidden meanings the author didn't intend.
As to why they put hidden meanings--or at least different meanings--into the text, they just to it to provide another way to think about life, about people, about "the human condition." The Scouring of the Shire, for example, was a chapter that I read as having a great message about not using fantasy for escapism but about bringing its messages home to build a better world. If I think about that message--even if it were never intended by the author--I have a different way to look at the story. And maybe I can learn something more from it than I learn from just the plain text.
So you're right--you don't need the other meanings. And a lot of teachers don't teach them very well, and are too rigid about there being a "right" interpretation or "correct" hidden meaning. But the other meanings give you more ideas to accept or reject or learn from.
-- IANAL, this isn't legal advice, and definitely isn't legal advice for you. Also, Squee!
Use computers to automatically generate essays. Proper spelling, grammar, sentence structure but poor meaning.
Society use your Sciences
In high school, I had fun making up hidden meanings that seemed vaguely plausible and hard to refute. It actually got me some decent grades - of course I rearely knew what the "right" hidden message I was supposed to find was until the teacher revealed the secret (although, if I had paid more attention, they might have mentioned it in class sometime).
Why is there an "insightful" mod and why isn't it "-1"? If I wanted insight, I wouldn't be reading
Teachers would likely be more willing to read if they would stop assigning arbitrary requirements. "Three sources", "2-5 pages", etc. They do nothing but give easy checkpoints to lower the overall value of the actual content of the paper.
Instead, shoot for conciseness. A lot of people have been trained by schools to say far more than necessary, which leads to confusion and wasted time. It will also build vocabulary by using words that express a wider idea or emotion. Still require sources (not that most public school teachers check them), but no min length (max is fine). If the student can talk about the major points of the War of 1812 in a page and a half, let him/her.
Mod parent up.
Democracy Now! - uncensored, anti-establishment news
Your professor had a) achieved tenure; b) didn't do research; and c) cared about teaching. Which is fantastic, and I'm glad you got that person, and in many ways is how it should be.
Meanwhile, the professors you thought of as less-good than your 'best professor' may have been a) contract lecturers juggling teaching your class between multiple other teaching jobs; b) grad students juggling teaching your class between other jobs and their need to actually finish grad school; c) pre-tenure professors juggling teaching your class and cranking out the necessary papers to get tenure, since the alternative to achieving tenure is being fired in your 4th to 6th year of teaching; d) tenured, but since continuing to do the research your PhD trained you to do (no PhD ever trains you in pedagogy) is both more fiscally and intellectually rewarding for most people than teaching undergraduates, was focusing on research rather than teaching; and/or e) didn't give a shit about teaching because 1) very very very few institutions actually *reward* teaching (vs giving lip service to how important teaching is); or 2) people who care deeply about teaching become school teachers because it requires far less time as a student to achieve the same salary and job satisfaction and your less-than-stellar professor just doesn't care that much about pedagogy.
The single thing that would improve the experience of most college students as we all go through college is for pressure to be applied to institutions to reward people for teaching well, both in terms of immediate salary and in terms of career progress. Until that happens, your experience of 'good professors' is going to be limited to tenured individuals who give a shit, and that's a rare combination. When you find them, praise them to the institution whatever opportunity you get - it does actually help those people a little and is worth doing.
In any writing, feedback is the most important resource you can use to improve your document. That's where education needs to lend itself, in collaboration. When I was in school, a person would basically write whatever, and a teacher would give your grade, nothing more nothing less. I think the peers need to be versed on the rules of the game (grammar), and then they can do their jobs of helping to help you project meaning, flow, etc. There is nothing better than this. Matter of fact, science sort of works this way: let your peers review, knowing that you both understand the rules (scientific methods), and come up with better, and ongoing threads.
.
In any writing, feedback is the most important resource you can use to improve your document. That's where education needs to lend itself, in collaboration. When I was in school, a person would basically write whatever, and a teacher would give your grade, nothing more nothing less. I think the peers need to be versed on the rules of the game (grammar), and then they can do their jobs of helping to help you project meaning, flow, etc. There is nothing better than this. Matter of fact, science sort of works this way: let your peers review, knowing that you both understand the rules (scientific methods), and come up with better, and ongoing threads. Sorry i posted AC at first. .
As someone who has taught writing, organization of ideas is the problem, not spelling, grammar and mechanics. That problem, in turn, comes from students who don't read enough. No automated grader is going to solve that problem.
If they had things that did that when I was in college, I probably would have spent most of my time trying to come up with syntactically correct nonsense.
A patriot must always be ready to defend his country against his government. -edward abbey
Computer algorithms that grade papers are horrible; I've been a victim of receiving an unjust grade from one in my past.
The thought that "The more writing students do, the better they will come," is completely illogical. Just because someone does more of something, it doesn't mean that they get any better at it. People only get better at things when they notice that they make mistakes the fix them.
On the other hand, being oblivious to subtext allows you to turn Gulliver's Travels into a mediocre summer blockbuster movie starring Jack Black.
(To be fair, I haven't seen the movie. It may be an incisive commentary on modern politics, ethics, society, and social mores. But I'd bet money that it isn't...)
(To be even fairer: this is not a new phenomenon. As far as I'm aware, the most commonly-read version of Gulliver's Travels is a bowdlerised version, missing half the chapters and almost all of the satire, that dates back to an 18th/19th century school reader.)
What part of "a well regulated militia" do you not understand?
If you made sure they were plausible within the confines of the literature, you didn't make up the meanings. You derived meaning from the work. Congratulations. You did the assignment. That's why you got decent grades even if that meaning wasn't what your teacher originally had in mind.
No joke! If you can write an algorithm to do this, you can win $60,000. See:
http://www.kaggle.com/c/asap-aes ... but time is running out...
Seriously, an automatic essay reader is just an invitation for students to cheat. Why bother writing if your teacher isn't going to bother reading?
Just download random essays from the internet. If an algorithm becomes commonly used, it will be reverse-engineered and automatic essay writers can be made, optimized so that every essay it writes receives an A.
It depends on what you're writing. A scientific paper should probably be as literal as possible. On the other hand, being able to say two different things with the same line of text works very well for say, a mystery novel.
Or a politician.
My middle school aged daughter recently wrote a paper that was autograded. I think it was an experiment by her teacher to see how well the autograder worked, since half the grade was to come from the autograder and the other half from the teacher actually reading it. At least she was allowed to run it through the autograder as much as she wanted to before handing it in.
Her first round before I read the paper, the autograder gave her a 92%. I read the paper, and it was hideous (sorry). The grammatical structure may have been technically correct, but the organization was awful, it was horribly confusing, and just didn't make much sense. I ran it through my own grader (a red pen), she fixed it, and it was clear, made sense, well organized, and still had correct grammar and spelling. The autograder gave it a 73%. Why? Because she didn't use advanced enough words. The words she used were perfectly appropriate for a middler-schooler or even a high schooler. So what does she naturally do? Pulls up a thesaurus, inserts a bunch of big words, and gets a 95%. I took a look and the words she used were not at all in the right context. I had to explain to her that, as she well knows, thesauruses don't provide *exact* synonyms, and the autograder is retarded.
So...I'm all for letting students run their paper through a set algorithms to give them hints about what *might* need changing, but relaying (that word passes spell checker and an autograder would have been happy with it) on an autograder to grade a papyrus is puerile (see, an autograder would have given me a 99% because I used those words from the thesaurus even though they're not in the right context). Also, as I think other commentors have pointed out, if you know a human is going to read your paper, even if you don't like your teacher and your teacher doesn't care, you're more likely to put more effort into getting your points across.
i guess e. e. cummings just won't survive the 21 st century.
complements on my writing were common.
Glad you used the past tense there. Nobody would compliment you now :)
Okay "Sokolization" is not a recognized word (yet) - it refers to Sokal Affair, wherein physicist Alan Sokal successfully published a hoax article of gibberish in Social Text, an academic journal of postmodern cultural studies (Social Text, an academic journal of postmodern cultural studies ("Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity", published in the Social Text Spring/Summer 1996).
As long as certain structural requirements are met (grammar, composition and style) this program will grade it well even if it is complete nonsense.
I suggest this might be a good tool if the teachers still read the best essays for content (logic, factual correctness, etc.)
Starships were meant to fly, Hands up and touch the sky - Nicky Minaj
Yeah, we need that money to kill more brown people.
Despite all my efforts, in 8 years of English classes, I was never even able to get a single teacher or professor to explain to me how he knew there were hidden meanings behind the text that was assigned.
They usually don't know (unless the writer has explicitly stated he wrote about a certain theme), they're talking about meanings that many others see in the texts. It's not unlike science - fundamentalist Christians are saying the same things as you: "Despite 8 years of science class I've never been able to get a single teacher to explain how he knew God doesn't exist."
English doesn't rely on calculating G with spinning weights or comparing changes in DNA over time, but it uses known symbolism, common themes in storytelling, and knowledge of culture and society to infer what a writer may have been implying with his work.
Tolkien swore that the LotR books weren't the least bit allegorical, for instance, yet how can you read them without seeing Mordor as 1930s-era Germany?
*tongue-in-cheek* Yes, but my side of debate is right! :)
To my mind, a reader who reads a message in a book that is not explicitly in the text has created a hidden meaning. Even if it is a really obvious hidden meaning. Some readers pick hidden meanings other people would be obscure, too, and even drawing parallels that are obvious can lead you to something an author never intended, since they don't always realize things about their books that other people consider obvious. Sometimes those things reflect subconscious biases or culture of the author, and sometimes they are just coincidences that happen to work out beautifully when you think about them as symbolic or communicative of a message other than the story the author was trying to tell.
On the Tolkien point, I thought his no-world-war-II allegory point about the ring was persuasive--how if it had been allegorical, surely the ring would have been seized and used against Mordor. I don't know (or recall offhand if he said) whether he was talking about the use of the nuclear bomb or other parts of the horrors of that war. Stalin tying the food supply to factory output or killing tens of millions of his people, the United States fighting the war as often as possible (as a rational actor) through proxies via lend-lease, the fire-bombing of Dresden, etc... Also, there were lots of very clear places he drew influence for the story from, such as Milton and Shakespeare.
That being said, WW2 of course is likely to have influenced him at least subconsciously because it created propaganda campaigns and rhetoric which redefined evil, which framed the war and really strengthened the dialog around human rights, resistance, etc...
-- IANAL, this isn't legal advice, and definitely isn't legal advice for you. Also, Squee!
You mean it's a brilliant invention, released by a private company, but no one has access to evaluate it? Somehow, this company is going to surpass the utterly painful grammar correction offered by Microsoft Word, and we're supposed to believe it? What blantant advertising. This stuff never lives up to its promises. Hype.
I want to point out the music industry. If kids are listening to the complete crap put out by the top 40 then no wonder they can't string together a good sentence. Writing is driven from several sources, reading, interaction, listening and etc... by listening to music which doesn't have lyrical structure kids are imitating that down to the paper and hence what could of been a great paper is now a bunch of garbage that sounds like 50 cent trying to rap about shoes well double teaming two hoes.
Small class size, I presume.
Wow. That's great. How many sections was this professor teaching? How many students per section? How many writing courses? Currently I have 88 students per semester, with 5 five-page papers each. At my previous job, I had 135 per semester with 3 five-page papers each. I'm afraid to do the math to see what my life would be like if I gave each student a half-hour meeting for every paper (on top of class prep, my committee work, research work, stupid paperwork work, and basic bodily needs). Kudos to this teacher! (Or to hell with her for busting the curve. I know so many of us on uppers to get the work done. I hate them. Anyway, Gentle Reader, if you're about to do the math and tell me life would be great, be aware that I will perform the readerly equivalent of putting my fingers in my ear and yelling "la la la."
Not concerned with meeting curricular guidelines? Tenured I bet. I have to say, I've met so many students who have been beaten-down and beaten-up by English teachers. Just cutting someone some slack, working from where they're at, can make a hell of a lot of difference. I've received some of the best essays from guys who had been told they were not good writers, or were dumb jocks, or some other shit like that. (And I've received some of the worst essays from students who were supremely confident that they are good at English or that it's their best subject.... I suspect some teachers grade on how well students pucker up or shine them on.)
It won't benefit the teachers or the students. The only beneficiaries will be the companies behind this bs.
"Technically not incorrect" is really not the bar we should be shooting for when it comes to one's ability to express ideas, especially at the higher education level. It's kind of an important skill.
I also said that it was subjective. Which it is. What is and is not good writing is up for the individual to decide. You can tell someone your opinion, but that is all. Saying that reusing words is somehow "wrong" is just your own opinion.
My mother, my teachers in elementary, HS and college tried to teach me to write ... all to no avail. One period when I had no money and nujthin' to do, I borrowed Flesch's How to Write Better from a library. In one weekend, I taught myself to write. At last I understood the complaints about unclear referents , complex clauses, passive voice, weak verbs. I reported for work on Monday, a clear, confident writer.
I'm very susceptible to the professor's idea, "I can't teach you to write but you can learn."
Bill Drissel
It is interesting that the guy who is down on robo grading is from Harvard. I can see a future in which cash-strapped public universities increase class sizes using robo graders, while Harvard students get actual discussion with a professor. Like the rush to online courses, which is more pronounced in public than private universities, this would widen the growing gap between elite and average schools.
I fear this kind of thinking! Thinking that believes learning is a result of grade marking. Yes, feedback is important, but there is little benefit in correct writing if the ideas are not there. It is like the garbage that is churned out on TV these days. No good stories, just formulated. I suspect that a robot has constructed the scripts. Learning is an activity based in connection and motivation. Having a robot correct my work does NOT motivate me. Discussing my ideas and getting feedback from the other person about how well they understood and agree or disagree with my ideas does motivate me. Using this strategy, is based on very faulty thinking about the educative process, and merely caters to the 'economic rationalist' approach.
That would be "compliments" on your writing.