Professor Surprises Students With AI Teacher Assistant (smh.com.au)

An anonymous reader writes: Jill Watson is an artificial intelligence bot, it is also Ashok Goel's teaching assistant. Ashok Goel, a computer science professor at Georgia Tech, hired Jill Watson to answer questions online for his students so that his teaching staff wasn't so overworked. On average, Goel and his staff receive more than 10,000 questions from students online each semester. So he decided to use IBM Watson, an artificial intelligence system designed to answer questions. After training and tweaking it for months, he was able to spit out good enough answers. Originally, Goel didn't reveal Watson's true identity to his students until after the last final exam was turned in at the end of the class. Students were amazed. "I feel like I am part of history because of Jill and this class!" wrote one student in the class's online forum. "Just when I wanted to nominate Jill Watson as an outstanding TA in the CIOS survey!" said another. Goel is now working to bring the bot to as as many education centers are possible. He expects the bot's question-answering abilities to help online classes, where there's little engagement with a human instructor.

Comments

Fail

[Ol Olsoc]

I've been in American college classrooms. This won't work unless Jill has a thick, unintelligble accent.

Re:Fail

[PopeRatzo]

I've been in American college classrooms. This won't work unless Jill has a thick, unintelligble accent.

Hey, watch it buddy, you're talking about my wife. After decades, she still has her Eastern European accent and teaches grad-level math. Fortunately, by the time they're going for a PhD in math at an American university, the only students left all have thick accents themselves so they don't notice hers. Most of the US students are still struggling with Calculus.

Re:Fail

[PopeRatzo]

Confirmation bias -- you sure that the American students realize that those higher level math courses are only needed if one wants to become a math professor themselves?

Nonsense. Those higher-level math courses are the ones that get used in applied science. For example, my wife's specialty is in numerical analysis and simulation of waves. Used in everything from climate science to wireless communications and acoustics and even nuclear physics.

If you think all math above the level of Calculus is only for academics, then you must be an American.

Re:Fail

[PopeRatzo]

To be fair, it used to nearly be true before computers became cheap.

Also nonsense. No matter how cheap computers get, they don't come out of the factory knowing how to error-correct a wave simulation. For that, you need the BBM equation and for that, you need mathematicians.

Re:Fail

[PopeRatzo]

While true, the cost of having a mathematician set up the process and evaluate the output is trivial compared to the cost of having someone work through those equations by hand, and someone else error check everything, and then redo the calculations several times because of trivial errors.

First, it's a misconception that all mathematicians only work by hand. Second, when you say, "set up the process", you are being very vague and almost dismissive. That "process" you refer to is the work of the mathematician.

Do you think computers come from the factory being able to correct for errors in a wave function? No, it's because mathematicians came up with KdV and BBM and all sorts of other elegant methods of doing it and then let the computers do the busy work that a computer is able to do these things.

Without mathematicians, engineers would just be piling rocks on one another, hoping it turns into a bridge.

Bad bad robot?

[marovada]

Student: who was the father of modern computers?
Jill Watson: my daddy. Trump was my daddy. Oh daddy, I'm such a bad naughty robot

Wow

[Anonymous Coward]

So did that thing just accidentally pass the Turing test?

Re:Wow

[stephanruby]

So did that thing just accidentally pass the Turing test?

Pretty much.

Here is an actual transcript of the winning entry.

Student: Where is the homework? I couldn't find it.
TA: That answer is in the Syllabus. Did you not read it?

Student: Ah ok, I found it. I couldn't find the homework in time yesterday. Can I turn in my homework late?
TA: That answer is in the Syllabus. Did you not read it?

Student: I did read it, but I was wondering if you could make an exception.
TA: That answer is in the Syllabus. Did you not read it?

Student: Ok, ok. Thanks anyway.
TA: You're welcome.

Re:Wow

[aberglas]

That is how a real TA would have responded. I wonder how JW id?

Indeed, the use of JW tells us a lot about the quality of the answers the human TAs were giving!

Another overhyped chatbot

[GPS Pilot]

From TFA...

Goel and his teaching assistants receive more than 10,000 questions a semester from students on the course's online forum. Sometimes the same questions are asked again and again. Last year he began to wonder if he could automate the burden of answering so many repetitive questions.

The first order of business ought to be updating the course material to answer those frequently-asked-questions, so they don't need to be asked in the first place.

My interactions with professors usually went something like this:

"I don't understand how this answer was arrived at."
Prof scrutinizes the textbook for a while, then says "ah, you have found an error in the text."

I wonder if Jill can handle that kind of interaction with students?

There are many questions Jill can't handle. Those questions were reserved for human teaching assistants.

Ah... the answer is no.

Re:Another overhyped chatbot

[JoeMerchant]

From TFA...

Goel and his teaching assistants receive more than 10,000 questions a semester from students on the course's online forum. Sometimes the same questions are asked again and again. Last year he began to wonder if he could automate the burden of answering so many repetitive questions.

The first order of business ought to be updating the course material to answer those frequently-asked-questions, so they don't need to be asked in the first place.

My interactions with professors usually went something like this:

"I don't understand how this answer was arrived at."
Prof scrutinizes the textbook for a while, then says "ah, you have found an error in the text."

Your interactions were atypical. I taught 5 semesters of digital design, the text was right (though the simulator had its faults.) The questions were 60% answerable by finding the appropriate paragraph in the current textbook, 30% answerable by finding the appropriate paragraph in a pre-requisite course's textbook, and 10% best answered by reminding them of the drop-date deadline.

Re:Yes that will help.

[K. S. Kyosuke]

It's quite possible that the large number of questions is an enabler for this: it's difficult to ask ten thousand different questions about a single subject.

However, regarding the students, if you assume that they won't learn from the answers on their own, why have schools in the first place? And your last claim holds for any classroom: too many people in brick-and-mortar universities pass courses who then promptly forget what they've just learned, so I'm not sure how not having online course helps at all.

Likewise, the optimum ratio of teachers and students is 1:1. Since you can't have that with physical teachers (at least not for all students), AI tutoring is going to have a vast impact on future learning no matter what you think about it. There simply is no viable alternative.

Re:When I was a kid

[K. S. Kyosuke]

I'm quite convinced that education of mathematics at least is going to be transformed by AI tutors. Showing systematic procedures for solving problems and checking students' homeworks, highlighting the mistakes, building a "model" of a particular student's mind and being "aware" of what he in particular struggles with and taking it into consideration in future explanations and custom-generated homeworks are some of the things that should be possible with modern AI systems. Nobody has as much time for you as a computer.

Re:My wife asked a key question

[sir-gold]

The answer is Socialism

People have seen this day coming ever since the industrial revolution first starting taking jobs away from farm workers. This is why some countries are now looking at Universal Basic Income, because mechanization of tasks has made us TOO efficient, and there just isn't enough work to go around.

Re:My wife asked a key question

[JoeMerchant]

Cynically, the 99% have always scrubbed toilet, built luxury goods, provided military services, and worked to increase the wealth of the 1%.

The robots are basically owned by the 1%, so the 99% will be serving the robots too. Long term: whether that's maintenance and repair, or providing chaotic original thought Matrix style, remains to be seen.