Slashdot Mirror


Australia To Grade Written Essays In National Exam With Cognitive Computing

New submitter purnima writes: Australia keeps on giving and giving. Each year school kids in Australia sit The National Assessment Program (NAPLAN) which in part tests literacy. The exam includes a written page-long essay aimed at examining both language aptitude and literacy of students. Of course, human-marking of such essays is costly (twenty teacher-minutes per exam). So some bright spark has proposed that the essays be marked by computer. The government is convinced and the program is slated for the 2017 school year. Aside from the moral issues, is AI ready for this major task?

11 of 109 comments (clear)

  1. No, but... by Capt.Albatross · · Score: 4, Interesting

    AI is not ready to do this task properly, but, at least in the US, human grading has sometimes been dumbed-down to the point where you would not even need current 'AI' to do as well, as prof. Perelman of MIT has demonstrated - e.g: http://www.bostonglobe.com/opi...

    1. Re:No, but... by ShanghaiBill · · Score: 3, Interesting

      AI is not ready to do this task properly

      Neither are humans. The question is not whether an AI can do it perfectly, but rather whether it can do it as well as a typical human grader. The human graders are under time pressure to increase throughput, and spend little time considering the logic and cogency of the students arguments. They are just looking at spelling and grammar, just like the AI would. At least the AI will be consistent. Human graders tend to give lower scores just before lunch, and better scores just after. Is that really fair, considering the importance of these scores on the student's future?

      Anyway, this discussion is silly, since it is happening in a data-free environment. It would be far more meaningful if we could see the human and AI grades given to the same papers, side by side, preferably in a blind test, and then decide with is better. AI has advanced rapidly in the past few years, so I wouldn't be surprised if the AI won.

  2. Testing literacy by oodaloop · · Score: 5, Funny

    Each year school kids in Australia sit The National Assessment Program (NAPLAN) which in part tests literacy.

    Can we get this AI to test Slashdot summaries?

    --
    Tic-Tac-Toe, Global Thermonuclear War, and relationships all have the same winning move.
  3. Exclamatory sentence! by meta-monkey · · Score: 4, Funny

    Adverb clause, independent clause conjunction independent clause dependent clause. Subject, adjective clause, verb prepositional phrase? Participle phrase subject verb conjunction dependent clause!

    Emoticon.

    --
    We don't have a state-run media we have a media-run state.
  4. Can we submit a poem? by WillAdams · · Score: 5, Funny

    Eye halve a spelling chequer
    It came with my pea sea
    It plainly marques four my revue
    Miss steaks eye kin knot sea.

    Eye strike a key and type a word
    And weight four it two say
    Weather eye am wrong oar write
    It shows me strait a weigh.

    As soon as a mist ache is maid
    It nose bee fore two long
    And eye can put the error rite
    Its rare lea ever wrong.

    Eye have run this poem threw it
    I am shore your pleased two no
    Its letter perfect in it’s weigh
    My chequer tolled me sew.

    --
    Sphinx of black quartz, judge my vow.
  5. it will be gamed. by Anonymous Coward · · Score: 3, Insightful

    Since machines cannot yet understand the semantics of complex English text, they will use some simplistic rules as a substitute. These rules will be things like "average sentence length" and other such metrics, which as soon as they are discovered by students, will be used to game the system. Instead of producing essays born of rational and coherent thought, they will instead make them to match the things being measured while being utterly devoid of meaning.

  6. So ... by BarbaraHudson · · Score: 4, Funny

    written page-long essay aimed at examining both language aptitude and literacy of students.

    So, the same technology used SO effectively to rank resumes will be used with students. Okay, kiddies, remember to stuff a lot of fancy-pants words into it.

    Fail: This is sh*t. Go f*ck yourself. I'm not kissing your ass.

    PASS: Subjectively, it is blatantly obvious to this observer that the new paradigm, as a cost-saving measure, was inspired by, and mimics, the the natural environmentally safe process of translating organic matter into nutritious compost. This has the outcome of allowing everyone who is in a paid position to devote the time saved to stress-relieving activities such as self-pleasuring, resulting in both a higher awareness of the need to practice good hygiene by such prophylactic procedures as more frequent hand-washing, and use of tissues to properly dispose of organic residue, though it could also negatively impact on their visual acuity over time.. Affected students should refrain from overtly engaging in behavior with superior's inferior posteriors to avoid being perceived as having a brown proboscis by their peers, with the associated negative impact on their social placement in the student hierarchy.

    --
    "Transparent" is a shit show that trades on every stereotype going. A man in drag is NOT a transsexual.
  7. Human profs already use AI tools by sandytaru · · Score: 3, Interesting

    Husband is currently grading final papers for college classes. He slaps them into software that detects plagiarism, then another software that picks out vocabulary level, typos, etc, and assigns a grammar score. Only then does he read it, quickly skimming over it and seeing whether there are citations on the "plagiarized" parts, if there are any, and whether he agrees with the AI score. Nine times out of ten, he does, and he uses the grammar score assigned by the AI. If someone plagiarized whole paragraphs without citations, they get an incomplete and need to do a rewrite. If someone didn't write the required number of words or pages, they get points knocked off the grammar score. It's faster than manually marking 150 papers, but still takes him about 15-20 hours of labor over the course of 2-3 days.

    --
    Occasionally living proof of the Ballmer peak.
    1. Re:Human profs already use AI tools by Dutch+Gun · · Score: 4, Interesting

      Does he check the grammar score before he reads it himself? I would worry that it may bias him before he can make his own judgment. Another potential problem, of course, is that if students have access to the same software, they'll be able to "tune" their papers to ensure the AI gives them the highest possible score. While this may not be "cheating" per se, it does tend to devalue the AI somewhat. This is the same process that's been happening forever with "Search Engine Optimization", or put less nicely, trying to "game" the search engines.

      Minor issues aside, it sounds like a reasonable integration of AI and human judgement. This probably sounds like the future direction educators will be taking more and more. Use AI to handle most of the tedious work - that's what computers are good for anyhow. The professor can then use his own judgement to make the final call, using the AI as a tool and not necessarily as a final arbiter. Moreover, it's going to be a long time before AI can evaluate the worth of the content of the paper, of course.

      --
      Irony: Agile development has too much intertia to be abandoned now.
  8. Why not. Just get it over with: fire everyone by Catbeller · · Score: 4, Insightful

    Hell, why not. While we're at it, why don't we automate the student process. Dump the students and educate AIs instead. Computing solutions always work, just ask any nerd about self-driving cars.

    At some point, and it seems that that point is arriving now, people will realize that the driving force behind technological change, as far as money people are concerned, is to eliminate jobs, and that the good jobs are not realy being replaced, and cannot be replaced. AIs grading papers gets rid of more pesky teachers who make a living wage. A self-driving car doesn't fit the picture until you realize that millions of people make a living *driving trucks*, and self-driving trucks will eliminate their jobs (in theory, if it works, and I don't see it working) and make oodles of money for capital and kick millions of truck drivers, along with all the taxi and Uber car drivers, out without a dime. (Uber is VERY interested in self-driving cars. Guess why).

    Some jobs are being made. And capital is desperately trying to commodify and cheapen such labor, to the point of demanding governments force coding classes on all kids. There are such jobs, but no where near enough, and those are mostly dropped onto cheaper kids, not newly dumped middle-aged workers.

    Asimov was on point, decades ago, when he wrote that inevitably automation would eliminate most jobs, and that the biggest problem - in his view, opportunity -- would be finding something for people to do. I would say that people without purpose are the most dangerous force for destruction and stupidity on the planet - worse than global climate change.

    Capital and people who work for capital, and neoliberals and business conservatives who support capital, tend to have well-paying white collar jobs and live among other people of their class, and don't see anything amiss. They're fine. Step outside into the vast middle grounds of the world, and you'll see a growing sense of we're-being-fucked that will require an endless army of pepper-spraying drones and surveillance to keep from erupting into riots someday soon.

  9. Oddly by Greyfox · · Score: 3, Funny

    The winning entry will be a heart warming story about a robot that kills all humans.

    --

    I'm trying to teach myself to set people on fire with my mind... Is it hot in here?