Are Written Computer Science Exams a Fair Measure?
me! asks: "I seem to have this inability to write substantial chunks of code (500+)
in exam conditions (for uni). I have been
writing code for years for open source and commercial applications, so I know a thing or two. There is just something about exams and code that does not work for me. I find that I need to be sitting in front of a computer to
get a problem out, to get in the 'vibe', have you will. I have done exams on computers (closed environment) that involve coding, and it work so much better for me. So what I am asking is...how do people tackle exams that involve solving problems on the fly, on paper, in exams?" I have this exact same problem, and I've never thought written tests were a fair way to measure someone's knowledge of coding. It's fine when you are asking questions about design and structure, but when you need to write code it falls way short. How do you feel about it?
Yeah, that works really well in the current job market. I dare you to try to find a job right now with what you had on your resume when you first got into the industry.
Moo
I have no resect for any exam that involves writing actual code on paper. Asking about fundamental principles is fine; asking for pieces of code or specific API details is IMO rediculous.
Granted, you can never design a perfect test. But you can at least do away with the totally obvious imperfections like asking people to do things in ways they never do and never will do in the future (except on other tests).
The few times I was subjected to these tests, I sure felt primitive erasing entire chunks of code after having realized I needed to insert something in the beginning. And it's a hard thing to avoid; I don't think any reasonably experienced programmer writes code entirely from top to bottom. Just a simple thing as placing braces becomes a hurdle in writing unless you know the length of the code you need to put between them!
/ Peter Schuller
--
peter.schuller@infidyne.com
http://www.scode.org
I personally hate it when they require programmers to write code on an exam. Every programmer knows that part of programming is debugging, and it's very difficult for many people to get chunks of code correct the first time. I can see, perhaps, requiring a small snippet of code, but large chunks are difficult to deal with in writing.
Due to complexity, programs simply cannot be written 100% perfect the first time around. Not only that, programming is very individual and specific to the programmer. Just because one person writes something one way, and the other writes it another way, doesn't mean either of them are wrong or right.
Written exams should be limited to syntax and concepts, to see how well you know the language. If you want to test a programmer's skills as a programmer, you should at least allow them to excercise their debugging skills...
Hopefully the instructor will realize that everyone drops a ; occasioanlly (however forgetting ; altogether is something else), or misspells a variable. I think that as far as written tests go they should be more based on making sure you understand how the concept works and not so much on not forgetting a ;. Questions like are better suited for questions like "What is wrong with these 4 lines of code" imo.
--"Karma is justice without the satisfaction"
This was my least favorite thing about CS at school...
I am much better at solving a programming problem in front of a computer.
Also, they are nothing like real world conditions... Do you know any programmers without reference books on their desks? That have the whole Java API memorized?
RateVegas.com - Vegas Reviews
Written exams are fine for computer science; Coding exams are dumb.
Remember, computer science is about methods and algorithms, not about learning syntax. If you forget a semicolon when you're writing a program, you'll remember about it as soon as you try to compile it; if you code a bubblesort where a quicksort would be more appropriate, you're going to be stuck with a slow program until someone more clued fixes it.
Tarsnap: Online backups for the truly paranoid
... because you don't have a compiler to tell you when you make mistakes.
This is why it's an outstanding way to test people's knowledge. Anyone can make a program work given a compiler and sufficient time. If you can do it with just pen and paper, and remember the syntax without having other code in front of you, then you know your shit.
This is why it's a great test for interviews. You'd be amazed how many "senior C developers" we interviewed at my last company who couldn't write push() and pop() for a linked list.
This is a skill that you need to learn. When you go out on a job interview, you will most likely be given a marker and white board and told to code something. If you're going to freeze up without a monitor and keyboard in front of you, the employer will be very hesitant.
It seems that you've associated syntax and semantics with the visual cues of a computer. That's not the same part of the brain that answers exams. So you need to retrain yourself. Here are some things you can do to relearn programming:
A) Stop using IDEs. Use the plainest text editor that you can find to write your code. Turn off any syntax highlighting and code completion.
B) Design your code before you write it. Use UML, flowcharts or whatever, but design it first. Then when it comes time to code you will know where you are going.
C) Write code away from a computer. Use a pencil and paper. I didn't have a computer when I was in college (very few people did). So I coded by hand on paper, then went to the lab to type it in.
A Government Is a Body of People, Usually Notably Ungoverned
That said, I remember one of my worst exams ... Pascal. Got the paper back with a mark around 30% ... which, after talking to the Prof, jumped to a 90 ... since the damn markers didn't actually know what routines were on the system, and my code used them extensively.
Like an english paper, marks should be given primarily for content, with spelling (and grammar) subtracting from that slightly.
The best advice is to do a question in three steps:
1/Shetch out the flow of what you want to do.
2/ Write the code, and
3/ At the end of the exam (assuming you have the time), go back over each answer checking the spelling
The greatest thing about college for me was NOT the CS, Engineering, and math courses that eventually got me the job I now enjoy...
It was the total mind-opening academic scene that surrounded me 24/7 for 5 years, the chance to really go out and party my brains out to relieve the stress, the hour long conversations with professors who are true geinuses in every sence of the word.
You might get paid a lot of money, but if you see college as simply a means to that end, I strongly suggest you take a year off and attend full time. Because you certainly do not get the point.
And, through so many experiences, I learned that your final career and salary doesn't define you, your worth, or how much you can get out of life.
but congrats on your job, though. Pretty sweet deal.
I have run into this same problem in job interviews. Usually it comes down to the fact that, on the job, I have resources at my command; books, web sites, code examples, etc. So, rather than memorizing every API (or even every keyword), I become very facile in knowing where to look up what I want very quickly.
But when I am asked questions about the same thing away from a computer and my books I start drawing blanks even about things I know in detail. This is why, IMHO, computer programming exams and even job interviews should be 'Open Book'. Memorizing details doesn't make you a good programmer -- knowing how to use the tools does.
Jack William Bell
- -
Are you an SF Fan? Are you a Tru-Fan?
If the test is asking you to use obsure Motif function calls or NT sockets, then you have a point. But if you're being asked to write a linked list or balance a binary tree, then it is perfectly fair. These are skills you are expected to know as a software engineer.
Just because you have a calculator is no excuse not to learn how to add.
A Government Is a Body of People, Usually Notably Ungoverned
When you're on the job, you WILL have those kinds of time pressures. One of the things the interviewer wants to know is how well you can cope with it.
It's Friday at 5:00 and you're leaving on a weekend trip at 7:00. The software is handed off to SQA on monday. There's one more bug in the queue and it belongs to you. Question: A) Do you bail on your trip? B) Do you sneak out the back door? C) Do you go fix that bug in 57 minutes?
A Government Is a Body of People, Usually Notably Ungoverned
nearly all of my exams were pseudo code... that is the entire point of a 4 CS degree: theory!
if you understand the problem and solution, then you are an 'A' student.
now, if this was DeVRY or something, syntax is the important thing cause you are just going to be a code monkey your whole life.
MARIJUANA, SHROOMS, X: ONLINE?! - E
One of the things that universities are supposed to do is give you a chance to express your knowledge in new contexts. It's relatively easy to apply things you've learned in familiar contexts, but how good are you at applying them in unfamiliar environments?
Making you work with pencil and paper forces you to plan ahead and architect your solution, not just jump in and start writing code.
When you write by hand, the logic and aesthetics of your code are especially important -- since you (and the reviewers) can't run the code to see whether it works or not, you need to make an extra effort to make it transparent and comprehensible.
I might take issue with an exam that was nothing but one big handwritten coding assignment, but I don't think this is at all out of place as a part of an exam or course. It's a little like essay questions in a soft-science class; they're unrealistic because in the "real" world you have access to reference materials and time to spell-check and so on... but at the same time they are a useful gauge of your ability to articulate a subset of your knowledge, to think on your feet, etc.
Also, in courses I've graded where such exercises where required, we usually didn't worry about things like minor syntax errors that didn't obscure the intent of the author. The goal was to look for an understanding of and ability to solve the problem, not to be human compilers.
"Biped! Good cranial development. Evidently considerable human ancestry."
Questions like, here is a recursive function, write the code to turn it into a loop.
Write some code to rotate/scale an image.
Fine an element in this sorted list (of say 10000)
Give this xxx write a class model for it.
Now as long as they don't expect perfect code, eg. forgeting to +1 because of a rounding error, or a binary search can result in a high or low result(for a non-perfect match), these seem fair enough questions to me because:-
They all test your abilty to write good code, not just code that works.
I know loads or 'programmers' who don't know how to turn a recursive function into a loop.
Would use sin/cos to rotate each point on the image, instead of using an interger line drawing algo to scan accross the image to rotate it (or even floating point)
Would write a n order search instead of a binary search.
And would use a poor or unflexable design pattern.
So long as they mark your programming skills and not how perfect your code is, i don't see a problem.
Debugging is also a key skill, but is very hard to test in an exam, a good debugger uses a hell of a lot of past experiance and insperation to locate, fix and check that there are no simila bugs.
thank God the internet isn't a human right.
I used to teach a class to engineering freshman, and part of the course was writing code in Matlab. There are in fact a bunch of matlab questions that you can ask that aren't just writing code, but you really don't get complete coverage unless you have the student put some code on the page.
Additionally, there probably was some cheating going on with the homeworks (this was a large class taught to all freshman, with relatively small programming assignments. It would be difficult to prove someone was cheating.) We needed a controlled environment where this couldn't happen.
However, we all realized how crappy this was for the student, and we graded the assignment more like it was psuedocode. If the code had obvious typos, we ignored those. If they were close on syntax, but a little unclear on the API, we usually let it slide, too. (We understand when online you can look that stuff up with the online help.) The thing we stressed was the overall algorithm itself, as well as a demonstration that the student knew how to use matlab. (After all, as a TA, I'm taking programming courses myself; I have some idea of what's reasonable to expect and what's not)
If you do have a written CS test, and the prof does mark you off for syntax errors, I would probably go in and (politely) complain. Say, "I made a typo which would have been caught by the compiler, but I got the gist correct." If you demonstrate that you know what you're talking about, and you're not a jerk about it, and the professor's not a total jerk, I bet you'll get most of the points back. (We would do this all the time, especially if the student could clearly explain what they were trying to do.)
If the professor or TA is a jerk, all bets are off, though. But if they're a jerk, they aren't interested in being fair anyway, so you were screwed from the start. We've all been there. Sorry.
I half agree with this. Syntax doesn't matter. If they can't get the syntax right, who cares. That's like saying that people who can't spell can't think - those are unimportant details. The important thing is that the know HOW TO WRITE push and pop. Not that they be able to type it once and have it compile.
:-).
I can't write a java main() statement. Just can't remember how - don't do it very often. But my emacs expands when I type 'main', so it's not a problem
I couldn't write push and pop and get it to compile, either. But I know how. And I can communicate the design.
If you're talking about "can't design push and pop", then my sympathies. If you're talking about "can't get the {}'s right", then I don't want to work with you anyway.
When I took a required test for C when I was an engineer, a very large chunk of the test consisted of writing code. One part required making the shell of an API, while another asked for a program that (if memory serves) did some sort of computation or sorting. I ended up getting a 99/100 on the test.. Why? Well, on the last problem of the last page on the last test of the year (the final), I neglected the final closing left brace " } ".. doh!
Anyhow, the code wasn't nearly perfect.. to the contrary. I had to erase several times, draw arrows (see below) to places, and other things.. but in end, what works, works. Here are some suggestions to help people write code during tests.
1) Think how you want to structure your solution. On the back of a piece of paper, write down each [method/function/procedure] and the variables involved.
2) Write down the [methods/functions/procedures]. Leave 2 'spaces' between the declaration and the start of your code in case you need to insert variables or code here.
3) Leave a little space between lines. If you find it necessary to 'insert' code somewhere, write it near the bottom of your page and draw an arrow to where you want to insert this code. Most TA's/Professors are very forgiving and understand that you won't erase 15 lines to insert 2 in the middle.
4) Do required editing.. that is, check your 'code', and if you need to insert lines, see #3. Those arrows work wonders.. really.
-Matt
Yes! I'm glad somebody finally brought this up.
Paper is just plainly the wrong medium for CS exams that involve programming. That became plainly obvious in my first CS exam and only got worse.
CS students should be tested on computers in at least a simulated development environment. (Controls would of course be needed to prevent cheating). Reference manuals should also be allowed as using them is a vital part of being a programmer. Forcing students to remember the parameters to fopen() or whatever is just pathetic.
If athletes were tested like computer programmers, the teams would be made up of those who could write the best description of how to play the sport, not of those who could actually play it the best. The worst part is that the intersection of those two groups is probably not very large, especially in CS, so I think some truly good programmers are being punished.
In first year CS our prof once asked us how to make the course better and I suggested the exams be held on computer. She was actually quite receptive to the idea, but as we both knew it was impractical. Computers cost $$$ and take up extra space, and testing 400 or more students on computers is just too ugly.
However, we are getting to the point where it's starting to become a lot more feasible, so I dearly hope educational institutions will start to upgrade their evaluation methods.
In the meantime, I hope instructors treat handwritten code more like a sketch than a masterpiece. We were lucky that the profs here didn't worry too much about syntax in handwritten code and instead looked for understanding of what we were doing. If we were a bit off on the syntax that was okay as long as we had the concepts down well. But we still did have to memorize a lot of stuff that was quite unnecessary and that's just Wrong.
... of someone's knowledge of fluid mechanics?
Once again, Slashdot continues to perpetuate the (Computer Science == Programming) myth.
Programming IS NOT "computer science" any more than telescope design is Astronomy (with apologies to Dykstra).
Programming is a trade skill. Certainly, a challenging and rewarding one.. but it is not a science.
Being the first guy to figure out quicksort... thats science... thats research. Coding a quicksort in LISP is not.
I've ranted about this before, but it upsets me to see such a misleading headline.
I think *computer science* exams are quite fair... if you understand the data structures and algorithms associated with B-trees.... you shouldn't have any problem describing them on an exam. Likewise, if you understand the halting problem, reducing a simple contrived example should be doable on an exam.
On the other hand, I very much disagree with the "code this up in PL/I, using proper syntax" type of exam question. What an enormous waste of time...
The bottom line is that if you've been through a computer science program and understand the underlying principles, I should be able to give you a manual for 'trendy language x', a few examples, and 48 hours. After that, you should have no trouble coding in Jav^H^H^H 'trendy language x'. Will you screw up the syntax occasionally? You bet. Is it fair to dock points on a formal exam for being human and forgetting a semicolon? I sure don't think so; and I've *never* done that to my students.
But I stray from my point. Once more, for the record, since no matter how often I rant about it, no one seems to listen:
COMPUTER SCIENCE != PROGRAMMING.
I've already finished my CS degree and one thing I like is writing code on an exam makes me happy. I mean of all the other exams I have to write with various degrees of memorization, writing code is *thinking*. So it means for me, less memorization bullshit.
Also another thing I would like to say: many would like online exams almost exclusively. But they are missing one crucial point: TAs and Profs mark such tests in "black & white", meaning if it compiles and passes the test cases you get most of your marks.
But if your program doesn't even compile, your mark starts at 0. And depending on the mercy of the marker, they *may* go back and look at your code and give you a mark here and there.
In such tense situations, I've seen people literally cry 'cause a) the program was too hard and b) they can't get it to compile. Where you literally get in a hack-peck/compile frenzy to get your program to spit out some correct output before the test is over.
In such tests we usually have 9 questions and gotta do about 6 of them. And the worst thing: they issue such online tests during the first year, where many are having their first crack at programming. Thank god, I was able to do the questions, but alas some individuals who struggled couldn't.
At least if the test was written you could get the core logic of the program done and you'll get most of the marks anyway.
But, I did enjoy online tests they were fun. The positive of such tests: if it compiles and spits out the correct output you've got 90% of your mark if not 100%.
Oh boy....talked too much...
The coolest professor(this was a EE class) I ever had did it this way - he made the tests extremely hard, but allowed open book, open notes, and calculators. His philosophy was that on the job you would use these tools, so you might as well use them on the tests. You were competent in the subject if you made it out with a C, and perhaps 2 people in a class of 20 would get an A.
Actually, I think CS tests are pointless, period. They have very little to do with your skill as a programmer or knowledge of the subject. Its much better to give hard assignments that require lots of work before they are due(i.e. a two week assignment that is virtually impossible to finish the weekend before it is due). I can see concepts on tests, and perhaps a few functions to demonstrate knowledge of certain algorithms, but a 500 line program is ridiculous.
No, Thursday's out. How about never - is never good for you?
I feel the same way -- every written test I had to take, I was proficient on the underlying theory, and would be able to code it in front of a computer. Most of the time I miss the 100% grade due to an off-by-one error, or an `if' clause testing the exact opposite of what I intended to. These sort of errors are easily spotted if you have a computer to aid in debugging, and that's why I believe coding tests should be done on a computer, period. Especially with students unfamiliar with the mechanics of coding -- after a few years of daily coding, a person seldom commits these silly errors.
As a side note, having learned touch typing, it's really hard to write on paper. Thoughts flow much faster than handwriting, and you end up losing yourself sometimes. I'm positive that also affects other coders when doing written tests.
Join the NFSNET. Our prime goal is making little numbers out of big ones. http://www.nfsnet.org/
You think that programming is a completely deterministic process? That there is only one way to model a given problem? You're the moron.
Write code so the machine will NEVER do ANYTHING it doesn't need to.
That means NEVER EVER use ANYTHING but hand written assembly language, that uses every shortcut in the book. It is assumed that maintainability has ZERO priority.
Cache every execution result that you will need again. RAM is cheap.
It is assumed that you have an infinite amount of RAM. Google should optimize by storing both the page cache and the HTML output of every single query ever executed in RAM, in case the exact query is repeated twice.
But RAM is not so cheap that you need to waste money and time filling it unnecessarily. Only cache what you absolutely need again.
Wait, what I just said, do the exact opposite.
I loathe moron who keep saying that software development is an individual's preference.
It's not all individual preference. But your attitude that there is only one way to solve any given problem, and that optimization is always the only priority, is equal bullshit. And your posturing and ranting bespeaks a religious zeal that kills development teams in their tracks. You'd be off any team of mine in a matter of days.
I, too, find it easier to write code on a computer, but there are times when you need to be able to write something out longhand. If you have a lot of difficulty doing so, I suggest you practice it. It can be a very useful skill to have.
First, it's something that will come up during an interview, and you can look really bad if you can't do it. I have given more than one interview when an otherwise impressive candidate was turned down because they couldn't write a relatively simple algorithm. I've also been on the other side of the table, where it really gave me an opportunity to distinguish myself. (Btw, from an interviewer's perspective, what's more important there is the approach, not the end results. I wouldn't be surprised if many of your teachers feel the same way.)
Also, it's often necessary to be able to write out code when you're working in a team. It's not always enough to just describe your approach, especially if another programmer wants to go a different way. It's like any visual aid, it helps you get your point across.
Finally, it can be useful when working on your own. I sometimes find that it's helpful for writing complex programs if I sketch it out longhand first. It helps me guarantee I haven't left anything out, so I don't have to go back over my code as much.
So, is having students write code on an exam a good way to test their coding skills? I don't know. But maybe coding skills aren't all they're testing.
Unlike fluid mechanics, where you can learn, and do research without knowing plumbing, you cannot be a good computer scientist without being very good at programming. Exams, which require you to write correct code, are usually done in introductory classes where the goal is to teach you programming in a specific language.
Learning the exact syntax and idioms of a particular language in an introductory class is necessary, as it teaches you how to learn a new language as much as it teaches you the basics of programming.
Is there any type of CS Testing that is fair. Me personally will do better on the coding questions then on True False statements because I personally over read the answer and I miss enterprit the queston and put the wrong answer. But the trick for coding questions is to read the question. Make Psuto Code then convert it to the language that the class is teaching with. The problem I find with a lot of CS Students they enter the degree with the mind set that sience they can program and make advanced programs they assume that they are good at it. While they may be able to make it work it may not be the best algorithem. I get this from the parent post because 500+ lines for a test is way to much. I would think it may be 60 lines or so. And you havent studied the algorithm that they wanted to teach you.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
The best professor I ever had could do everything in his head, hundreds of lines of code. He knew exactly how the compiler would react to anything.
He didn't just teach us the syntax and the languages, he taught us how to do *that*, and *that*, to me, is a f-ing awesome skill.
Dr. Caviness. We used to joke about him being the "human compiler". He ruled.
I'm a 2000 man.
I think you left out part of your answer:
package dataStructures;
public class FinalExam {
public boolean pass() {
return true;
}
public int grade() {
return 100;
}
}
So, what you're telling me is that there is only one way to store data, and that personal preference has absolutely nothing to do with it?
Wow, I'd really like to know what kind of programming you've done lately. Perhaps in procedural programming there is a set place to put things, but as far as OOP goes, I'd say there's quite a few ways you could arrange things. People have different reasons for coding things differently, both syntactically and in organization. One person may find it more convenient to store something in a simply ASCII text file, someone may find a small but efficient database system better.
I agree, you shouldn't use more resources than you have to, but there are multiple ways of accomplishing things, each with their own benefits and drawbacks. If you can't see that, I'd really hate to see what happens when you have to work together with someone on something.