Slashdot Mirror
Resources for Programming Course TA?
cndrr asks: "I'm a Teacher's Assistant for intro to Java at my university again this fall. The last time I taught, all the TAs had students turn in their assignments through email. I'm thinking about scripting a site that will let students turn in their programs automatically and in some cases, run the program and (based on the output) automatically grade it. Has anyone else TAed and found a good solution that they would recommend?"
Is very much automated-submission and grading -- or at least was from 00-04. Some professors do the grading by-hand (i.e. they run scripts by hand) but the majority of classes I took where you turned programs in, there were scripts that graded you that you could run part of yourself. (i.e. there were "public" and "private" scripts that used the same interface.)
In a large part, it depends on how big your class is, and if you plan on continuing teaching. One professor, (whom I idolize) has been doing this long enough to have set up an extensive and all-inclusive framework. Can't go (too badly) wrong when you roll your own, but YMMV.
-b
When in doubt, parenthesize. At the very least it will let some poor schmuck bounce on the % key in vi. (Larry Wall)
Berkeley's been doing this sort of thing for years and years now. I'm sure someone there can help you identify software that can do what you're looking for. Unfortunately, I never TA'ed while attending, so I don't know what they use.
One of the grad students at my school developed this system as his masters project...
Don't know if it's of any help but you might try contacting a Dr. Sun since he was the advisor.
It doesn't automatically grade the homeword, but it does let you set time limits and since it uploads directly to the server, you won't have to deal with students saying they got the wrong email address.
The system, currently, is only for homework and old code retrival but many other things are planned for it.
At RIT we used a program called "try", developed by one of the professors there. You can download it from his page: http://www.cs.rit.edu/~kar/software.html
Search first, ask questions later.
I am also a TA for an introductory course to JAVA. I am not sure how many others here are but I have found auto-marking to be essential. I am a single TA and I have 180 students. Each assignment has 3-4 substantial programs. Trying to mark them all by hand would be impossible given time limitations. I do have to agree with others though in that feedback is always nice. Therefore what I have done is I mark half of each assignment and the other half is automarked. I have automarking compare output given input and allocate this to a mark called 'Correctness'. I then mark certain functions (usually the harder ones assuming that if they could do these ones they could do the others). This way the students still get feedback. I also encourage them to come in and see me if they have questions so we could go over it together in more depth. The test cases are also put online (after all assignments are in).
The submission system I use however is unix based. So the first or second tutorial/lab we run is all about how to submit your assignment through unix. We have had a website submission system in the past but we found this negativly affected some students who in third year had yet to touch unix at all.
Just as a warning though, I have found with introductory courses, no matter how many times I tell them to output something in a specific way, the students will not do it. I usually have my automarking scripts do a 'diff' with multiple possible answer or ways of formating an answer. They will still fail all the test cases forcing me to hand mark anyways. I have even given them Strings in provided code to use and they will either change the output, ignore it, or have extra output. Sometimes I just give them 0 in the correctness area. The main reason for this is the students don't care. Some are just in the course because there program requires them to have one CS course.
Therefore, if you plan on doing this then you might want to consider having final results stored in a variable, and have them make a public get method. That way you can simply redirect standard out to something like /dev/null when you run their code to get the final result, and then set it back to the screen and use the get method to output just the result. This only works in the case of computing numbers though.
Anyways, good luck with whatever you decide to do
As we try to teach our students, use the best tool for the job. In my opinion, that means only using automated testing when necessary. I am teaching a course (Programming in C & the UNIX Environment) the first time this summer. (I was fortunate enough to be granted a research assistantship when I was accepted to grad school, so I was never a grader.) I was offered the use of some automatic collection and grading scripts developed by someone in the department decades ago, but I decided to avoid them.
In my case, grading does not constitute a significant percentage of my teaching and prep work. I only have 5 students in the class. Although this system would have saved me some time, I remembered my own experiences with a professor when I was an undergraduate. I do not know whether or not this professor used automatic testing, but he was rather draconian about things being written exactly the way that he would have written them and that they produce exactly the output that he envisioned. Unfortunately, I have learned from trying to complete his assignments and from my experiences in industry that it is extremely difficult to completely and unambiguously write a specification document. Furthermore, I wanted my students to be able to use their creativity in their assignments -- I instruct them, for example, that their program should produce certain pieces of data as their output, and that these should be organized in some tabular form, but I do not require that their output look exactly like mine. Thus, automated testing would be of little use.
That said, you asked for resources, not to be talked out of it, and I assume your situation is significantly different from mine that the benefits of automation outweigh the drawbacks. The collection part is easy; either write a script they can call to copy their files to some specified destination, or write one that goes out at the submission deadline and copies from a standard location in their home directories. On UNIX, require the students to include a makefile with the all target. I am not very familiar with Java, but I presume they have a similar cross-platform approach. Assuming that all of the programs read from standard input and write to standard output, the testing part is also rather straightforward. Write your own implementation of the assignment. (An important step in any case!) Write some input files as test cases, capture your implementation's output for each of them, and then compare them to the output produced by the students' code with diff. For added security, do the entire compilation and testing steps in a chroot environment.
If you adopt this though, remember that you still have to look through everyone's code, even the ones that pass all tests. Hopefully you are grading for style in addition to correctness, and you need to verify that students aren't plagiarizing each other.
Good luck
Stanford's CS 106a class is famous not only for being a popular class (1/3rd of all Stanford students take it) but for producing amazing TAs as well. When I took the coursea long time ago not only did the TA had grade a print out of your code, but they then scheduled a 15 one on one session with you to talk about what you did right and how you could improve. This sort of mentoring in the intro to programming course preped students for writing more complex code for other courses and meant that TA resources in later classes could concentrate on the topics at hand and not have to worry about spaghetti.
The TAing program became so famous that Silicon Valley companies would recruit people that had been TAs and pay big bucks for someone that had been head TA.
Lasers Controlled Games!