Machine-Guided Learning Matches Teachers In Study

New submitter dougled writes "A study at six universities found that students taught statistics mainly through software learned as much as peers taught primarily by humans. And the robots got the job done more quickly. '... our results indicate that hybrid-format students took about one-quarter less time to achieve essentially the same learning outcomes as traditional-format students.' They add, 'There is every reason to expect these systems to improve over time, perhaps dramatically, and thus it is not foolish to believe that learning outcomes will also improve.'"

Comments

Was the teacher tutoring a single student?

[perpenso]

Was the teacher tutoring a single student, as the machine was? How does the machine do when teaching a group of 30? I suspect that all we have really learned is that individual tutoring is better for some topics.

Of course computers can be less expensive tutors so the approach does have merit.

Re:Was the teacher tutoring a single student?

[wanzeo]

The most valuable part of machine assisted learning is the ability to move at your own pace. There are some OCW lectures I had to watch 3 or 4 times before I got it. Now matter how good a teacher is, no student is going to ask them to repeatsomething four times. The student will just nod and feign understanding, and the teacher will move on.

Re:Was the teacher tutoring a single student?

[l0ungeb0y]

Why would we need to have the software teach a classroom? Isn't the ability to have teaching software able to deal with individual students better than a single teacher dealing with a class full of students? Why ruin it by saddling it with unnecessary burdens?
That's sort of like bashing a nice sports car because it wont do any better than a Honda Civic in a traffic jam.

Re:Was the teacher tutoring a single student?

[davester666]

Too expensive to have one computer per student. We'll just get a cheap big-screen TV hooked up to a single computer at the front of the class, and each student has a 'go back 30 seconds' button.

What could go wrong?

Re:Was the teacher tutoring a single student?

[rtb61]

Actually doing both makes the most sense. Were the students unsupervised or was someone present to ensure the odd student did not abuse the computer they were using.

In the end, is not thus why computers are being introduced to the classroom. The computer does the bulk of the teaching and the teacher supervises and helps students stuck and not proceeding the with computer driven teaching. So no the teacher must understand the computer education program as well as the subject being taught.

If you think having all children sitting at home connected to internet with no teachers will work, I have a US only teaching for profit teaching program to sell you and screw the resultant chaos, where both parents work and children are free to roam the streets during the day. I will electronically fudge the results to 'er' prove the success of the program so that the government will keep sending me millions and 10% of which I will religiously donate to the campaign of the crooked politician who signs off on it. This should euphemistically work for a minimum 3 maybe as long as 5 years then no matter how much PR=B$ via mass media is applied the majority will know it is all bull shit.

Re:Was the teacher tutoring a single student?

[perpenso]

The most valuable part of machine assisted learning is the ability to move at your own pace. There are some OCW lectures I had to watch 3 or 4 times before I got it. Now matter how good a teacher is, no student is going to ask them to repeatsomething four times. The student will just nod and feign understanding, and the teacher will move on.

Depends on the tutor. Some will ask a question that is designed to demonstrate your understanding of the concept. It wasn't one-on-one but I've had professors who were notorious for posing questions to students to gauge their understanding of the topic at hand during a lecture. Nodding and feigning was hazardous to your grade, admitting your confusion was not.

Computer based lectures can also be helpful for those who are getting the material. You can get through it faster than if sitting in a classroom lecture, no interruptions from someone who is not getting it. And if you can play the lecture at 1.25 times normal speed then there can be a big win. Most of my classmates and I found that speed to be entirely equivalent to normal speed.

Re:Was the teacher tutoring a single student?

[nbauman]

So why not use a textbook?

When I don't understand a chapter, I can repeat it as many times as I like.

Re:Was the teacher tutoring a single student?

[RivenAleem]

It's great to see the robots are joining in on the discussion.

Re:Was the teacher tutoring a single student?

[cvtan]

That's easy for you to say.

Re:Was the teacher tutoring a single student?

[Hognoxious]

I for one cannot disagree with any of that.

Re:Was the teacher tutoring a single student?

[Architect_sasyr]

The old adage of practice makes perfect would seem to apply - it only makes perfect if you're practicing correctly from the beginning. I ran into a particularly (for me) difficult piece of relativity equation in a text book some time ago, even now when I look at it it makes little sense to me. I was, however, able to go to a lecturer and have them clarify it for me based on how they understood my previous perceptions.

Bring on the robits.

Re:Was the teacher tutoring a single student?

[Nethemas the Great]

A good teacher will never have a single student. It doesn't matter how much juggling they do it is impossible to achieve the same level of synchronization with the individual student as is possible with a piece of software. If humans "teachers" still exist in the future, it is inevitable that their roles will dramatically change as software steps in to augment and eventually provide the lessons.

And this is a success?

[c0lo]

I can't believe how low the teaching level must've got if a machine receives better outcomes than a teacher. Or how low the assessment of learning...

Gosh, from the Fermi's way of teaching to this? In a space of... what??... last 20-30 years?

Re:And this is a success?

[sFurbo]

A teacher can understand what essential part of understanding you are missing and make an analogy that explains exactly that. He can then change the analogy if you don't get it. This, of course, assumes rather few students per teacher, and good teachers. But by all means, let's find out where humans do well and where computers are better.

At the university where I work, the experience seems to be that class discussions work better in e-learning courses. This could be because even quiet types will join in, or because people spend longer time thinking about their answers. This is, of course, only anecdotal.

Re:And this is a success?

[williamhb]

I can't believe how low the teaching level must've got if a machine receives better outcomes than a teacher. Or how low the assessment of learning...

Gosh, from the Fermi's way of teaching to this? In a space of... what??... last 20-30 years?

Four things to say on this one -

1. It looks like essentially the same result that's been repeated over and over again since the 1990s. Technology-enhanced learning papers from a host of universities have been occasionally reporting gains of up to about 1.1 standard deviation over classroom teaching alonefor a couple of decades.

2. Beating university lectures is an extraordinarily low bar. We've known since the 1980s that non-interactive teaching, as typically happens in lectures and used to happen in high school classrooms too, is rubbish. See Bloom's "Two Sigma Problem" paper for extensive details (from high school classroom studies), and how simple things like "setting homework" can give a gain (actually marking the homework gives another gain; teaching students where they went wrong in the homework yet another gain! Wow, who'd have thunk it! What a surprise that a computer doing some of those things that we found 30 years ago provided a gain ... provide a gain over doing nothing). There is a more difficult challenge of matching human tutoring -- the intensive small group teaching by experts that routinely beats classroom teaching by two standard deviations but is much more expensive to do.

3. Unfortunately for the field, a lot of mixed-mode experiments are flawed because the groups can't practically be isolated properly, and there's usually very little way of knowing how much of the learning is due to what part of the technology. It varies from case to case, but one of the common problems (from an experimental, not a learning, point of view) is that students are pretty much required to subvert your system -- no student is ever told "you mustn't find some other way of learning this". So, even if some part of the teaching is crap, if the system has given the students a clearer indication of what's going to be on the test, students will find another way of learning how to answer the test (ask their friends in the other stream, read a worked answer to last year's exam, Google...). That means there's often a hidden variable of whether setting computer practice tests is making the students better at the guessing game of knowing what's going to be on the real test. In some ways that'd actually be fine (yay, our students are doing better), but not if it means they neglect any learning that isn't on the computerised test (Joe can calculate eigenfactors til the cows come home; he just can't do anything else and has no clue when it's useful to do that)

4. It's also important to note these are not machine-only learning methods. There's been plenty of lionisation of entirely online teaching, but the subtle truth is that universities have never thought it's just knowing the material that matters. Which is why they don't mind giving away all the material for free. After all, you've been able to go to the library and get the material for free for a few centuries now, but not that many people choose the Good Will Hunting route for their education. It turns out there's value to the soft skills you develop from being stuck in with a bunch of bright kids and (hopefully) bright faculty and put through the academic rigmarole, and to the credibility of having come out successfully at the other end. Or if you want to be really cynical about it, it turns out that employers also value some of the less glorious things that universities teach you:
* Being able to navigate ridiculously over-complex bureaucracies and still get things done
* Being able to learn what you need and accomplish what you've been required to do even when faced with setbacks such as the unintelligible academic gibbering away at the front of the class being actually pretty useless at teaching
* Being able to manage your workload even when every damn subject lands a 40-page assignment on your plate in the same week of semester
etc

Re:And this is a success?

[BitterOak]

I can't believe how low the teaching level must've got if a machine receives better outcomes than a teacher. Or how low the assessment of learning...

Did you say the same thing about human chess players when computers started beating the world champions? Maybe it isn't that teachers have gotten worse, but that computers and instructional software have gotten better.

Read the PDF

[solarissmoke]

They actually make some pragmatic conclusions in the report itself, and don't claim that machine-guided learning is some sort of panacea:

The findings in this study warn against “too much hype.” To the best of our knowledge, there is no compelling evidence that online learning systems available today—not even highly interactive systems, of which there are very few—can in fact deliver improved educational outcomes across the board, at scale, on campuses other than the one where the system was born, and on a sustainable basis.

...

We do not mean to suggest—because we do not believe—that ILO systems are some kind of panacea for this country’s deep-seated educational problems, which are rooted in fiscal dilemmas and changing national priorities as well as historical practices. Many claims about “online learning” (especially about simpler variants in their present state of development) are likely to be exaggerated. But it is important not to go to the other extreme and accept equally unfounded assertions that adoption of online systems invariably leads to inferior learning outcomes and puts students at risk.

Another bogus study...

[ToddInSF]

That measures success not by ability to think critically and solve problems, but instead by the ability to regurgitate garbage back to the robots.

Which is all good and well, since that's mostly all that teachers have been doing anyway.

Way to shoot for the bottom of the barrel and diminish any real improvement in education !

Let teachers *teach*

[White Flame]

By offloading the rote and basic informational dispersal to the students, that would hopefully free up the teacher to focus on walking through real demonstrations and examples, interacting with students, and helping out with some of the difficult-to-understand areas, instead of spending most of their time doing the same lecture-style material over and over.

Re:Let teachers *teach*

[bcrowell]

By offloading the rote and basic informational dispersal to the students, that would hopefully free up the teacher to focus on walking through real demonstrations and examples, interacting with students, and helping out with some of the difficult-to-understand areas, instead of spending most of their time doing the same lecture-style material over and over.

In the subject I teach (physics), what you're describing is standard modern pedagogy. What I mean by the word "standard" is that anybody who pays attention to the published empirical evidence knows that this is what you have to do in order to get decent results. It's not really new. A lot of the relevant work was done by Richard Hake (see this paper) in the early 90's, and it was popularized by Eric Mazur in his 1996 book Peer Instruction. What Hake and his colllaborators have shown is that in traditional lecture-style courses, the amount of conceptual understanding that students gain (compared to what they had entering the course) is always extremely small, and there are no exceptions to this rule. The findings apply even to lecturers who have won awards, get wonderful student evaluations, etc. Techniques like the ones you're describing have been shown to do significantly better.

The problem is simply that most teachers don't pay attention to the empirical evidence -- which is pretty pathetic for someone teaching a subject like physics, which is supposed to be an empirical science. Rather than doing what works according to the evidence, they do what their own professors did when they were undergraduates.

A secondary problem is that students typically prefer traditional lecturing, because it doesn't make as many demands on them. They come to class without reading the book, sit passively in their seats, doodle in their notebooks, and think about sex.

Hybrid

[fiziko]

This was a hybrid approach. How would they have done without the face-to-face hour each week to get questions answered that the machines couldn't answer?