Slashdot Mirror
Resources for Programming Course TA?
cndrr asks: "I'm a Teacher's Assistant for intro to Java at my university again this fall. The last time I taught, all the TAs had students turn in their assignments through email. I'm thinking about scripting a site that will let students turn in their programs automatically and in some cases, run the program and (based on the output) automatically grade it. Has anyone else TAed and found a good solution that they would recommend?"
I'd rather you manually grade it and provide valuable remarks to your students about their programs.
If universities were all about automated stuff, students can very well learn from course textbooks such as those prescribed by ocw.mit.edu by themselves. They go to university so that they can interact with their professors, get their amateur evaluated properly to shape their future work, and collaborate with their classmates.
Banu
Have any of the previous posters actually attended a large university in the computer science program?
Every class I've taken at UCSD does some sort of automatic grading on the programming assignments. It would be impossible to grade everything otherwise... Last quarter we wrote a compiler that ended up being a few hundred KB of source. There were over 200 test cases ran by the autograder.
All of the assignments are turned in from a unix prompt using a TURNIN command after prepping for the appropriate class.
The class I'm working on now (Operating Systems, using NACHOS) even has autograder() methods in the skeleton of the code that are used during grading...
I agree for a first class some feedback and hand grading might be necessary, but even with autograding you can add comments after looking at the code that causes similar test cases to fail.
If part of the assignment requirement is that, upon being run, your program should produce a specific output, I don't see where there's a difference between having a human run the program and determine whether the output is correct, and having a script do the same thing.
Sadly, automating any part of the grading process means you end up giving a lot less feedback to the students concerning their errors, but that's the prerogative of the instructor and assistants. In some classes, however, the student-to-TA ratio is so large that it would be impossible to grade every homework manually.
Also keep in mind that some TAs don't actually get paid for their efforts - and those who do usually make peanuts. At some schools, and in some academic programs, being a TA is a required part of your graduate degree. And since they're working on their own degree, the TAs actually have a lot of their own work to do in addition to grading hundreds of homeworks every week.
For you and all the other people saying human eyes are better than a computer for grading:
They don't need to be a substitute, and they can help a great deal in doing a lot of the manual checking you would be doing anyway, as well as the organizational part. I'm a TA, and I use a script to check for just about every simple mistake I can think of, and then I go over every assignment by hand, with a printout of my test-script's results. The script doesn't so much grad ethe work, but point out any output that might not be exactly what I expect, over an assortment of tests. The students often comment that these scripts, which I also hand back with the assignments along with output from the scripts, help a lot to identify not only what their problems were, but what sorts of things they should have in mind when writing their programs. I then *carefully* go through their code by hand to insure they were using good style and didn't coincidentally happen to pass any test with code that isn't technically correct.
When you have over 60 students in a class and have to grade long programming assignments every week, these scripts are essential to getting my work done in a timely manner. My personal attention and comments are not replaced by my scripts, but are enhanced by my scripts. My time spent grading is made *more* effective.
Also (for cndrr): I might be able to provide the main part of the script we use, I'd just have to check with my instructor. The test scripts we use are fairly easy to write using it, if you don't mind doing it partially in scheme... (our department is in love with that language). Let me know if you're interested.
"Question with boldness even the existence of a god." - Thomas Jefferson
In COMP 314 (Rice University's sophomore-level programming & algorithms class, taught this past spring by Prof. Dan Wallach) we TAs solved this problem with Subversion.
An important part of real-world programming is teamwork; in 314 we embrace this and randomly sort students into groups of two for pair programming. The groups change for each project, and each project group gets a spot in a svn repository set up for the course. ACLs keep groups from peeking at one another's changes (for example, see this team list, which is actually just a slice of our Subversion access control file). Students were required to tag their submissions for each of the project milestones: specification, prototype, final; this was how students submitted code for these deadlines. From timestamps we could easily see which groups incurred "slip hours" for late turnins.
There were a number of reported incidences of lost work or conflicting changes which would have been disasters if not for svn, which saved their bacon. Groups that learned to check in early and often knew that accidental deletions or disk failures posed no threat to a successful project submission. A few enterprising teams even used tags and branches to help organize complex development efforts. In all, it was quite a successful adventure and we'll probably do something similar in the future.
Fully automatic grading breaks down quite badly if there is a possibility of students making small mistakes that cause large amounts of tests to fail (or, conversely, big mistakes that cause few tests to fail); in this sort of scenario (and concurrent programming is one of them), you really want a human to assign the final grade by identifying the underlying mistake/bug instead of the symptom/failure. However, having automated tests that are designed to expose common problems is a good thing.
Of course, if you like giving out failing grades for messing up input or output formats while letting blatant errors in mutual exclusion pass by with minimal impact, you can do automated grading based directly on test results. Trying to make the computer deduce the actual mistake from the test results, on the other hand, can be quite painful (this is essentially automated debugging, which may be possible to some extent; see Andreas Zeller's work on Delta Debugging).