Slashdot Mirror
Software Goes Through Beta Testing. Should Online College Courses? (edsurge.com)
"Testing online courses is not standard practice at traditional colleges," points out a new article at EdSurge -- though beta-testing is part of the process for other online learning sites. jyosim summarizes their report:
Coursera has recruited a volunteer corp of more than 2,500 beta testers to try out MOOCs before they launch. Other free online course providers have set up systems that catch things like mistakes in tests, or just whether videos are confusing. Traditional colleges have shied away from checking online course content before going live, citing academic freedom. But some colleges are developing checklists to judge course design and accessibility.
"It would be lovely if universities would consider ways of adopting the practice of beta testing," says Phillip Long, chief innovation officer and associate vice provost for learning sciences at the University of Texas at Austin. One factor, though, is cost. "How do you scale that at a university that has thousands of courses being taught," he asks... How much beta testing makes sense for courses, and what's the best way to do it?
A senior instructional designer at the State University of New York says "On most campuses, instructional designers have their hands full and don't have time to review the courses before they go live... We're still trying to find the magic bullet that motivates people to review other people's courses when they're not being paid."
"It would be lovely if universities would consider ways of adopting the practice of beta testing," says Phillip Long, chief innovation officer and associate vice provost for learning sciences at the University of Texas at Austin. One factor, though, is cost. "How do you scale that at a university that has thousands of courses being taught," he asks... How much beta testing makes sense for courses, and what's the best way to do it?
A senior instructional designer at the State University of New York says "On most campuses, instructional designers have their hands full and don't have time to review the courses before they go live... We're still trying to find the magic bullet that motivates people to review other people's courses when they're not being paid."
Traditional colleges have shied away from checking online course content before going live, citing academic freedom.
What the hell sense does that make? That's like saying I don't check my texts for errors before hitting send because "Freedom of Speech, bitch!"
I am posting as AC for -reasons- (not sure what all is NDA, etc)
But Coursera courses go through a beta test before they are launched. I'm not sure how many 'learners'(they don't call them students) go through it, or if it is just Coursera Staff... but there -are- _some_ people who go through the course(s) early.
Their new platform sucks IMHO. It is difficult for Instructors to actually produce meaningful content since their platform is 24/7 on-demand only now and new course sections start every 2 weeks. So it is impossible for a lot of good academic practices because of the platform design.
Can no longer: release an assignment, set a due date, have it submitted by that date, release the answers, do peer review based off the answers, and then have the peer-review be a mechanism by which 'learners' not only see the 'right' answers, but also are required to evaluate others' answers.The reason for this is that the platform requires that the -same- assignment be released again in 2 weeks, so no way of ever releasing answers publicly. And there is no way to produce _quality_ assignments for every assignment for every course every 2 weeks (unless that is all you are doing & we have University Classes, etc. still)... And that still ignores possible updates to course material, planning new courses, etc...
This is very serious when it comes to [redacted] assignments like I like to produce for my courses... I've also heard that it is even a bigger issue when it comes to Computer Science, Engineering, or Maths classes because they can only ever tell you what are 'wrong' answers (due to auto-grading) but never tell you the RIGHT answers...
In many cases, the beta testing of classes taught in traditional settings is just to teach them. As someone who is teaching an online science course and doing so for the first time, I'm building my class based on lectures when I taught the same class in a traditional setting. Although it's a different instructional format, I'm not sure that beta testing would have been helpful. My class is an elective requirement, and although my students might never care about meteorology again, I try to teach them a bit about how science works (in general) and force them to do higher order thinking (~80% of the grade is based on levels of thinking above recall in Bloom's Taxonomy).
It's a legitimate science class, and any good instructor should be constantly testing their teaching and trying to improve. Every semester I've taught, I've learned from what I've done well and what needed improvement throughout the semester. There isn't a single beta test, but any worthwhile instructor should always be working to improve the quality of instruction. That's been no different this semester, even though it's an online class. I have made a couple of changes to my instruction this semester, which I believe are for the better. You can't simply prepare a class from the beginning and expect that you won't have to change anything during the semester. Instead, you need to have clear objectives for your class, and make changes as needed to either assess whether those objectives are being met (assignments and tests) and how to prepare the students to meet those objectives (instruction).
Any good instructor is always working to improve a class, even during the class. Instead of a single beta test, each week of my class is like Patch Tuesday, where I learn from how the students did the previous week, and always make minor or major tweaks. It might be as simple as explaining concepts again that students didn't understand as well or as complex as making changes to my learning assessments.
For a good instructor, a beta test isn't as important as the continual revisions during the semester. I can certainly say that I've gotten better at teaching this class as the semester has gone on. If I'd prepared the entire class prior to the semester and beta tested it, I might have missed some things that could be improved upon during the semester. Beta tests are no substitute for periodic updates to software. And they're no substitute for continually improving courses.
I'm in a new online course at the University of Maryland. There are definitely some pain points (for me, at least) that could be eliminated with some beta testing, as well as some standards for what its reasonable to expect students to do in terms of work outside the classroom. Beta testing (or at least offering the course at a reduce cost for the first one or two offerings) would make being a guinea pig a bit more palatable. Instead, I have a grade in my degree program dependent upon the whim/good sense of an instructor. I don't know what outside observation there is to ensure that the treatment is reasonable.
In some cases, I'm guessing when a college or a state writes a contract for creating these courses, the length of time of the video content must be the most important clause in the contract. My son signed up for Texas' online high school physics class last summer in order to avoid taking it during the school year. It was very clear that the objective of the course materials was to consume required amount of time and not really to teach physics. It was more like a remedial drivers ed class one takes as punishment for speeding than anything that resembled a real class. I imagine that quite a few online classes come into being based on this "time content" model.
I think I've spotted the flaw in this plan. Anyone else?
Confucius say, "Find worm in apple - bad. Find half a worm - worse."