Slashdot Mirror
Ask Slashdot: Should Coding Exams Be Given on Paper?
Slashdot reader Qbertino is pursuing a comp sci degree -- and got a surprise during the last exam: being asked to write code on paper.
Not that I'd expect an IDE -- it's an exam after all -- but being able to use a screen and a keyboard with a very simple editor should be standard at universities these days... I find this patently absurd in 2018...
What do you think and what are your recent experiences with exams at universities? Is this still standard? What's the point besides annoying students? Did I miss something?
A similar question was asked on Slashdot 16 years ago -- but apparently nothing has changed since 2002.
Leave your best answers in the comments. Should coding exams be given on paper?
What do you think and what are your recent experiences with exams at universities? Is this still standard? What's the point besides annoying students? Did I miss something?
A similar question was asked on Slashdot 16 years ago -- but apparently nothing has changed since 2002.
Leave your best answers in the comments. Should coding exams be given on paper?
Intellisense is the death of programming.
Everything should be done on paper... or VI. Whatever makes life more miserable.
You modern kids and your erasers have it easy. We had to write our exam code using only 1's and 0's and the 1's we had to make by squishing the 0's.
You are going for a computer science degree. You must be able to express your ideas on paper, a white board, napkin, back of your hand, ....anywhere.
The poster apparently needs to transfer to a code monkey program.
I hear DeGree Mill will take anyone with the $$$$ or student loans.
Not that I'd expect an IDE -- it's an exam after all -- but being able to use a screen and a keyboard with a very simple editor should be standard at universities these days... I find this patently absurd in 2018...
Yes you know better than the university when it comes to evaluating your skills. And I find you patently annoying.
Online you could do stuff like -
Here is 10K lines of code that is supposed to do XXX. Debug and fix it.
It would be able to test if the person actually knew what was supposed to happen, how it was supposed to do it and their ability to actually understand structures and interactions way more than you can do on paper.
I graduated in early 2000s myself. The finals were all on paper. I had a lot of programming related classes. 3-4 of those 2+ hour tests back to back.
Writing small apps, quick sorts, manipulating data structures, you name it. As much as hundreds or even thousands of lines of code handwritten over the course of a few days, every year. My finger had a mark from the pen. And if you made a mistake, not all professors were ok with just drawing arrows to "insert" code, so there was a lot of starting over too.
The challenge in those exams was not figuring out the solution. It was writing it down. I still have nightmares from it to this day.
"Computer Science and Software Development" graduate of 2004 here,
To the question asked: "Should Coding Exams Be Given on Paper?"
Yes.
Does "Coding Exams" mean "being asked to write code on paper"?
No.
The majority of the "coding" portion of my classes was evaluated through assignments, and practical classes.
The exams ask specific things to test your knowledge, and didn't require "writing a program on paper", and if they did, only asked for psuedo-code, not something that would compile.
This question is pointless.
While written exams might be considered torture, the real issue is that giving an exam on computer opens up too many opportunities for academic dishonesty.
I don't know about the author's instructor or course, but there's generally a point to this.
* Can the student write a correct algorithm, bug-free without the crutch of running the code. Running code often leads to writing code by trial-and-error without much thinking.
* Does the student know the language well enough to write code without the suggestions of an IDE?
* Avoid all technology problems. My computer crashed. My battery's dead. I accidentally deleted the file. I have the wrong version installed.
There are a lot of rules to make sure that exams are fair, auditable and so that it requires at least a modicum of effort to cheat. The University is likely set up so that all exams are paper so setting new rules for one or two classes is going to be an uphill fight at the best of times. Specific issues you will need to resolve:
1. How do you prevent people cheating (USB sticks, Bluetooth, getting access to the Internet, etc.)? Who's time is going to be used to set this up and enforce it?
2. What if someone's computer crashes? What happens if they accidentally knock the reset switch (I have seen this happen in a "practical exam" of this sort)? What if they "accidentally" knock the reset switch?
3. How do you support students who use assisstive technology on computers (screen readers, specific high-visibility colour desktop designs, desktops in other languages, etc.)? How do you deal with students who sent skieing in the holiday before the exams and broke both arms (again, I have seen this -- and we sorted it out)?
4. Where do you draw the line on "reasonable" support? Text editor? IDE? IDE with documentation? IDE with documentation including code samples?
Critically : it depends on what you are trying to assess. I've had interviews where they want code written on the whiteboard -- how is this different?
Next those spoiled brats'll expect to have access to stack overflow before they can answer their questions.
but it's sort of a moot point since a real computer science curriculum is mostly about math and math can be done on paper just fine.
I suppose if we still had programming vocational schools, but between the H1-Bs and the offshoring they're really just scams at this point. You can count the number of jobs available for that kind of code monkey on one hand of a retired shop teacher.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
So paper is ok
Hereâ(TM)s how Iâ(TM)ve adjuated my teaching practice for a world with smartphones: 1) All my tests are open book/note. In a world where you can lookup anything anytime, knowledge is no longer a valued commodity. However, using knowledge to solve problems is. So all my questions involve higher level thinking, with students able to use their notes. 2) My tests and exams are all done on computer through Google Classroom (when in a lab setting). So thereâ(TM)s always a keyboard and screen. 3) I have my students write code in their assignments with an IDE, not on tests. Writing code without and IDE has always pissed me off. So, my tests/exams involve things like analyzing code, eg: hereâ(TM)s some code, tell me what it does. Or, hereâ(TM)s some code with problems, what are they and/or how can they be fixed. Or, when would you use a while loop vs a do loop? Or a local variable vs a class variable? I must note that this form of teaching is entirely my own, after spending much time reflecting on how I think teaching and schooling must adjust for a world with instantaneous access to endless information. This is not endorsed by my government (Ontario, Canada) or school board, and even many of my colleagues disagree. However, something has to change in education which still uses methods developped over a century ago for chalk and slate! I do espouse these methods when possible, and continually strive to evolve my teaching for an ever evolving world, but I certainly donâ(TM)t think this is the final answer in education methodology. Any further ideas you have would be welcome! At some point I would like to evolve my tests/exams to be âoeopen Googleâ, but frankly have no idea how that would work. But thatâ(TM)s my thoughts on next steps from where I'm at currently.
To write code from scratch with a pen and paper indicates true mastery of the subject. (if it's good)
But, yeah, it's a bit like requiring fountain pens for exams.
If it's just a case of lack of resources that's a whole different thing.
[I'm a college professor that does a lot of science-related programming, but not a CS prof]
If you get an IDE and/or compiler to test your program, then the exam changes in kind, turning into a practical assessment more than a theoretical one. This is good in some cases, and bad in others, but I would guess that most programming exams with a functioning compiler might as well be a "take home" software project. If you don't get an IDE or compiler, getting an editor is mostly convenience, but the considerations are the same between a cs paper-code exam and a sociology open question exam.
At our university, I would say 90% of our exams are pen-and-paper based. This is annoying for everyone involved, from the student (RSI, more time spent writing than thinking, no longer used to think "linearly", e.g. without quick editing), the professor (bad handwriting, lugging stacks of paper around that you really shouldn't lose, more difficult o spread the work or grade by question) and support/admin (more difficult to archive, more difficult to check samples post hoc, etc).
The reason why? A lot of our university's exams are in the same weeks (to make it easier to mix and match courses), so there is a peak demand for probably well over a thousand exam seats. Normal computer labs aren't really suitable, both in terms of layout and software (to prevent cheating). They have facilities for digital testing, but it's too expensive to provide enough, so the higher priority courses get served first, i.e. exams like applied statistics where you actually use a computer program for the test.
So, yes you're right that it would be a lot nicer if more exams would be computer based, but in most cases CS is no different from e.g. sociology in this respect.
I've tried this all ways: on paper, computers allowed for help (use your IDE, why not, but answers on paper), and I have some quizzes that are purely on the computer. There are two issues to consider here:
- First, and most obviously: cheating. As soon as you allow student-owned electronics, you open the door to connectivity. If not WLAN, than via mobile phones or ad hoc networks or even bluetooth. This is very difficult to control, and is the primary reason that my school still officially prohibits electronics during exams.
- Second, if you're going to allow a computer, you had just as well allow an IDE and make it more "real world". The thing is, this makes exams more difficult for all but the best students. People who are not (yet) very good a programming might be able to show a believable (but uncompilable) concept on paper, and get reasonable partial credit. As soon as they have a computer, it is natural to expect a program to run. The poorer students will lose lots of time trying to get their program to actually work, and are therefore more likely to fail such an exam.
For the last point: I'm not sure this is bad. Personally, I think the world needs a lot fewer mediocre and lousy programmers. However, while that would improve overall software quality, it would mean less code written overall and make software even more expensive than it is.
Enjoy life! This is not a dress rehearsal.
I think the idea of making student write code on paper comes from countries that don't use the Latin alphabet. If you're in China or Russia or something, making them write the code shows they know how to write Latin characters by hand. It was slightly annoying, but on the other hand it meant we could have a quiz with actually coding rather than multiple choice picking possible outputs to some given code.
Downside: My fingers hurt from writing so much.
Upside: I'm actually spending time on learning to optimize my code and make it more simple, easier to write. Start with something complex, rewrite and make it less redundant and repeat.
I try to stop before it starts turning into write only code.
A chisel and a big slab of granite please. In addition to write the code, the student should also shape it like a tablet. Preferably with a notch!
Years ago, at Georgia State U a couple of us going for MS degrees had to take the C++ programming course.
My classmate had an MS EE and had over decade of C++ programming experience. I had a similar amount.
Both of us tried to get out of having to take it and showed our resumes with our years of experience.
Nope. We HAD to have a class in it from an accredited school.
I sat back, did my assignments and only participated when called upon. My classmate had a tendency to point out the instructors (Ph.D. CS) where he was wrong.
Easiest 'A' I have ever got - but pissed at the money I had to spend for the complete waste of time.
So, yes, sometimes we DO know better.
Good times.
(NOTE: I don't know why an f-ing iPhone can't submit text properly, but here's my response properly formatted.)
Here's how I've adjusted my teaching practice for a world with smartphones:
1) All my tests are open book/note. In a world where you can lookup anything anytime, knowledge is no longer a valued commodity. However, using knowledge to solve problems is. So all my questions involve higher level thinking, with students able to use their notes.
2) My tests and exams are all done on computer through Google Classroom (when in a lab setting). So there's always a keyboard and screen.
3) I have my students write code in their assignments with an IDE, not on tests. Writing code without and IDE has always pissed me off. So, my tests/exams involve things like analyzing code, eg: here's some code, tell me what it does. Or, here's some code with problems, what are they and/or how can they be fixed? Or, when would you use a while loop vs a do loop? Or a local variable vs a class variable?
I must note that this form of teaching is entirely my own, after spending much time reflecting on how I think teaching and schooling must adjust for a world with instantaneous access to endless information.
This is not endorsed by my government (Ontario, Canada) or school board, and even many of my colleagues disagree. However, something has to change in education which still uses methods developed over a century ago for chalk and slate!
I do espouse these methods when possible, and continually strive to evolve my teaching for an ever evolving world, as I certainly don't think this is the final answer in education methodology. Any further ideas you have would be welcome!
At some point I would like to evolve my tests/exams to be "open Google", but frankly have no idea how that would work. However, that's my thoughts on next steps from where I'm at currently.
Cram based tests are stuck in the past and big hand written ones just make it easy to make spelling / spacing / etc errors.
It all depends, are you testing thinking and code fluency or are you testing for IDE automaticity?
I would suspect that through the programming course, the students have already written several programs using the IDE and turned those on to demonstrate their skills.
The paper exam shows their thinking, familiarity, code fluency, and code grammar and syntax proficiency.
As one who writes code, I still think about it on paper before I type it into the computer, especially when writing algorithms.
Planing on paper really reduces bugs and logic problems because it doesn't require technology. If you just sit down and start coding, it is easy to go down rabbit holes of "why doesn't this work" vs. thinking about the problem.
To me it is similar to writing an outline before writing a paper. You know where you are going and how you plan to get there, Sometimes if I am really pressed for time I will just write comments for the code blocks and use that as my plan, then go and fill in the code blocks.
Alternately, should English exams be given on paper or should students use an IDE like Microsoft Word?
That's great! You're reaching your students that knowledge is useless because of course there are never bad answers to anything on the net. They should just cut n paste whatever shit they find in Google from an h1b posting 8 years ago.
I think we should just remember that writing with pen/pencil and paper is an important life skill, and we as humans should try to keep that skill well rehearsed for the sake of the human race. The skill will come in handy if the lights ever go out.
Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
How do they get around disability laws with this? There are a fair number of people that go into computer assisted professions because they have reading and/or writing disorders such as dyslexia or dysgraphia. I'm at a serious disadvantage on a paper test because I have both. Spell check, grammar check, and syntax highlighting are fundamental requirements for me.
That makes the case for being allowed to test out of a class. Which should be allowed for any class that's not core to the curriculum.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
Fine if they want it on paper.
But expect bad pseudocode. Any other expectation is not relevant to the medium!
+----------------- | What is the question!
Not only that but also the coursework in general is bad. They teach "PSP", don't teach git, still use textbooks from the 90's. And I go to a high ranked public college. Often when you're allowed to use IDEs it's something like Eclipse and all other ones get the axe or using IDLE to program in python. The tests of course are done on paper still.
Ridiculous.
Except for my martial arts exams and my driving license all exams were on paper. ... because we only had 3 or 4 computers in school, and waiting till one was available seemed unplausible ...
And actually when I was young I wrote many programs on paper first
if you can not explain something on paper (I'm not talking full UML 2.x) you likely have not grasped it enough.
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
>The point of the paper exams was to check if I understood the *concepts*. Yeah, except when the sadistic professor doesn't actually test you for the concepts and fails you for minor errors that you would call a typo on a computer screen. This was my experience.
You should test how you learn. How many folks program without ever looking at different web sites. Tests should be real world.
"Gentlemen, you can't fight in here! This is the War Room!" -- Dr. Strangelove
For my programming exams I got to write out C, LaTeX, Racket, bash, SQL, and assembly on my exam papers. Only 1 class used online. I liked paper exams for this. Makes you think harder.
Qbertino is pursuing a comp sci degree
Really? Why? You're just flushing your time and money down the toilet. There aren't any jobs for computer science graduates. There are exactly zero jobs. Go on. Spend years applying for fake jobs that don't exist. Come back when you've given up and realized that your degree is completely worthless.
If you go through with it and get the degree, you're totally fucked.
There. Are. ZERO. Jobs.
If you know the material you should be able to write it down with paper and pencil.
The question is whether the exam can be completed in a reasonable time. So the problems should be compact enough. It should be acceptable to use pseudocode, flow charts, etc to express the ideas clearly and show you know the material.
If you cheated your way through the semester by using others code ideas, you won't be able to complete the final exam. That was my feeling - if you studied all semester and did the assignments yourself, the final should be a cakewalk. You already studied it, now prove you know it.
Otherwise, why are you at university studying courses in computer science, if you just want to be a code monkey?
I have a computer science degree - obtained circa 2001.
None of my exams during the pursuit of my degree used a computer - all exams were done on paper. Demonstrating understanding of data structures, algorithms, complexity analysis and other CS topics is not coding.
Coding is the implementation work of computer science, much like construction is the implementation work of Civil Engineering.
Would you test a Civil Engineering student by asking him/her to build you a bridge?
The test is not fair.
If the partner you are paired with doesn't have the necessary logical-thinking skills, no amount of explanation will make them competently understand an algorithm with any degree of sophistication at all.
And anyway, your ability to explain things well to others is a separate skill from your ability to create, analyze, optimize algorithms. While it is true that you must understand it well in order to explain it well, it is not true that you must explain it well in order to understand it well.
... if the purpose of the exam is to determine coding capability?
I had a slightly different issue.
Last semester I had a written test in my CS program as well. I was thoroughly aware of the subject material, and was acing the homework assignments, even using somewhat novel techniques, just for fun.
But I struggled more with the written variety because it was less forgiving. My development style relies on feedback from the system for things like type checking or other conventionally supplied features.
So, I can program with little problem in the real world (been there, done that), and understand the algorithms fine, but the written test put an unexpected premium on being "mistake free" which I found to be troubling, because it wasn't representative of the real world.
Long story short, I had problems with the first test due to this, then paid attention to mitigate it for later tests, and did fine. But the way it prioritized "perfection" was a surprise.
The answer to "Did I miss something?" in any article is always: "Yes. You missed the whole point."
I agree with the above advantages of paper over typing. However we have to consider clear handwriting, to know if the test-taker wrote the correct operator. The operators less than, open curly brace, open parenthesis, and open square bracket look similar to each other. And a period and comma look similar. So do ! and | .
If a test is on paper, one way to get around clear handwriting problems is to have a multiple-choice question:
How do you express the ith element of myArray?
a) myArray[i]
b) myArray(i)
Hey I need you to paint a house, but I am going to not let you use a paintbrush.
And I'm going to blindfold you.
While throwing spitballs at you and scribbling all over your plan drawings.
-----------------
Seesh, let us use our freaking tools. My brain is optimized to only remember what is needed. I say this as a Vim-using C++ coder on Gentoo who can install it without a manual. Without google for quick lookups, I'm paralyzed against those who wasted their time learning things no one ever is not able to lookup (except when artificially paralyzed by contest rules).
Might as well practice crawling fast since I won't be allowed to run upright during that next marathon..... I'm sure we'll find the best runners with those restrictions!
You are going for a computer science degree. You must be able to express your ideas on paper, a white board, napkin, back of your hand, ....anywhere.
Not only that, but it's a college course run by a professor. The exam format should be the professor's choice, with a very few exceptions mandated by the university. (For example, blind grading most obviously.)
However, the professor should be up-front about the requirement at the beginning of the class, before the student is locked into taking it.
Real lawyers write in C++
Back in highschool I mostly did math homework in English classes. I was pretty bummed that university wasn't any better at organising this sort of stuff. Adopting some parts of MOOCs could be god here: Just follow the lecture videos, hit up the lecturer when you're stuck, work on the assignments, do in-person exams whenever you're ready.
(I also was really annoyed you could in fact get dispensation for a core class in the second year of university CS if only you'd done accounting in the second-to-last year of highschool. That really ticked me off at the entire university course.)
* As a second language, since it was for all of us, in fact most everyone in the country. But I digress.
Same here, all my exams were on paper. One we had to write out both Python and C++. Basically it was just memorizing syntax. Very tedious and probably worthless. Be prepared to be very disappointed in CS from universities. You will need to teach yourself to code while passing their memorization tests.
I am a community college instructor and hope the audience will forgive me for posting AC.
I give paper tests, quizzes on an LMS (Blackboard/Canvas/Moodle/et cetera), and programming labs. Exams are all open (paper) book, no electronic devices. Students usually have access to a PC with Visual Studio and the Intel architecture manuals (for Computer Architecture classes) during tests, but the lab is not connected to the internet. The final may be split into a paper-only, no resource segment and a segment with the PC available.
It is my experience that there is a certain segment of every class which can answer multiple choice questions OK, but flounder when asked to write any code. I am very lenient regarding syntax errors. But giving exams on paper, requiring problem analysis, design, and coding is essential to identifying those who should be counseled to change majors.
Learning disabled students have more time in a separate, quiet room to complete their exams.
Let the flames begin!
Im job interviews, meetings, etc you have to be able to express yourself without a computer.
You should know enough code to write something that maybe has syntax errors but will describe clearly what you are going to do.
The low level code monkeying might be outsourced to India when you as a developer has solved the problems and used 3 times the coding time documenting it.
Well it seems your education was also a waste of money. Since you clearly cannot read between the lines or comprehend a text which is longer than two sentences. The teacher above didn't mention that they should copy something without thinking it over if it works or not. But lexical knowledge is really overvalued in our world and the only advantage you have with your lexical knowledge is that in some corner cases you might recall some lexical facts faster then I can google it or find it on stackoverflow which is an extremely small proportion of the cases.
And actually one of my job interviews in a reputable London based company was exactly like this. We sat down to a laptop together, they opened the IDE with some code in it and they asked me to refactor it and find bugs in it. I could ask questions from a domain expert and a software developer to finish the task. I really enjoyed it. The task was challenging but fun and very much relevant to the job. All tests should be like this. Not like asking trivia on whiteboards or paper.
I do my OOP exams on paper because I don't care if you can or cannot write correctly in $LANGUAGE. The IDE and stackoverflow will eventually correct your code syntax. What I want to test is whether you can understand what a piece of complex undocumented code does, how do the OOP concepts work in practice* and whether you are capable of designing a software architecture that uses said concepts to solve a non-trivial problem.
If you think a good programmer is somebody that can translate a pseudo-code algorithm into his/her favourite language blindfolded, think again.There will be an AI available on the market that does that quicker and better than you before you hit 40.
* Here's an example in Java:
class A {
public int x;
public A() {
this.x = 0;
inc();
}
public void inc() {
this.x += 1;
}
}
class B extends A {
public B() {
super();
inc();
}
public void inc() {
this.x += 5;
}
public static void main(String[] args) {
B b = new B();
System.out.println(b.x);
}
}
What does that code print and why? You'd be surprised how many students fail a question like that. Now, this is typically a paper question that doesn't make much sense if computers are allowed. However, I think it's a question that makes a clear distinction between the students that truly understood what polymorphism is and those who did not completely, which is a pretty good indication of who is going to design correct programs and who is not....
Video of some good progressive thrash music
The exam was OOP1 and pure code on paper. So your critique doesn't compute.
We suffer more in our imagination than in reality. - Seneca
I graduated in 2009 and all of my CS finals ever involved a paper exam. Given that over the course of each term I would have to write multiple programs that had to compile and pass the grader's testing programs, it seems perfectly reasonable to me to have a paper test. They already know by that point whether or not you can code in an editor. They want to know if they can think clearly and rationally about algorithms:
1. Under a tight time constraint
2. Without anything more sophisticated than a pen and paper (imagine doing a literal back-of-a-napkin design)
3. On your own without help
A paper exam is the best way to do it. Bear in mind that the questions in these paper exams are going to be appropriate to the medium - 20 lines at most, including comments and closing curly braces.
Because hey, guess what, sometimes you'll need to do this in the real world. You're in a conference room with other developers and all you have to work with is a whiteboard and marker, and you need to outline your algorithm to them.
This is computer science. Not coding 101. If you are asked to write code, it is to demonstrate versatility in mapping algorithms to arbitrary code vernacular which was part of the syllabus. By not knowing the rote usage of constructs, you are forcing synthesis of a solution in code instead of a solution in the algorithm. We are social learners. We should not relearn the how every time we have a problem or we are constantly having to relearn as we forget. You are the problem. You are the reason junior team members on my staff are routinely aghast when I ask them to use something they haven't looked up a in a while and I have to reteach it to them from memory. And then they are shocked that I've retained it.
All you are doing is training a group of kids to always need a crutch and feel that they are entitled to it. You are not doing them a service. You are forcing them into a lifelong relearning cycle because you did not teach them to value knowledge.
Knowledge is power. Shame on you for teaching otherwise. Some things do not have to synthesized each and every time. Some things should just be remembered as muscle memory. You are the reason why kids can't do simple math any more because they are taught to reason it out each time. The human mind is designed to acquire and chunk knowledge because the speed of our perception is limited. A chess master cannot use your method. A plumber cannot use your method. Only a blank slate that has to work everything out each time can use your method. You are hobbling your students for life.
With you as a teacher back in the time of Galileo, the students would be dropping cannonballs to calculate gravity. Or redoing Leibnitz sums instead of integrating.
Teach how to value knowledge and when higher thought is needed. Not everything online is worth having to look up. Stop taking the easy way out with your students. Demonstrate why knowledge itself is power instead of worrying that they will not engage. Do your real job instead of getting paid. My Ontario teachers from high school would be appalled at your method.
If you know the language, a solid text editor is the best way to go. For professional programmers, if you take the time to really learn a great editor the code can flow effortlessly, almost as if your brain is wired directly to the output. Think "delete this block" and your fingers automatically do the keystroke to delete a block.
If you DON'T know the language, having autocomplete make suggestions can help as you guess your way through it.
I do turn on syntax highlighting mostly because it provides an obvious cue if I miss a quote or something - half the screen turns red, which looks obviously very different than how it should look after each statement is written. I don't pay any attention to the details of the syntax colors - there is just a difference in what the screen as a whole looks like when there is a syntax error.
Wait, he's complaining that he has to use paper in order to get a piece of paper?
"National Security is the chief cause of national insecurity." - Celine's First Law
Speaking as someone that teaches second semester CS to 300 undergrads a semester, there is value in both.
Advantage Paper: The computer is a crutch. If you know it, you can write it. You are going to to have to do this in coding interviews. Requires no infrastructure to give an exam without cheating. (I actually don't have the space/computers to give a computer exam to hundreds of students at a time.)
Advantage Computer: I/TAs don't have to grade it, I can just use gradescope.com and students won't argue with me for partial credit.
>Let the flames begin!
Ok, here goes. Why are you "identifying and counseling" students to change majors away from CS? Let me guess, the students you identify to change majors away from CS are women and underrepresented minorities. Sorry, if some student blows your exam, that doesn't give you license to try to change the course of their life.
Coding, no. Big picture stuff, YES!
When I took programming in school, the tests were about pseudo-code and big ideas. Homework was graded either by showing the code and running it in front of the teacher or by turning in the code, compiler output and runs.
15 yrs ago, I was applying for a senior systems architect position at a large financial services company. My resume had C/C++ on it, but I hadn't done any of that in 3 years and mainly on Unix systems. After the interview, I was sent to another room for testing, which was a shock. They put me behind a Windows computer, which I hadn't programmed on in 5+ yrs, and handed some paper with coding questions.
After looking through the test, I stood, told the proctor that I clearly wasn't the guy they wanted, and left. I wrote a thank her to the VP explaining where I was coming from. At that point in my life, I wasn't going to code anymore. I'd been happy coding for 15 yrs, but wasn't ready for the 50% pay cut or to be a code monkey anymore.
I can completely understand why testing someone's skills for a position is needed. Nobody mentioned that there would be any testing involved. My resume was loaded with technologies in a list for everything I'd used over 15 yrs. I wasn't prepared to be tested on OSI models or writing an outer-join in SQL at that point either.
I still code a little, but mainly in bash or perl. Wrote some bash this morning. The C and C++ standards have all changed since I last did anything non-trivial. Heck, std templates were new and we were migrating from RogueWave tools when I stopped.
Are taught to write their code away from a computer. It leads to better code. If you can't code away from the machine, you can't code on the machine.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
A lot of my beginning classes in college were only on paper for the exams. We would use the computers during class and kinda peer-program with the professor (if we were learning something new) or program by ourselves after he taught us the basics using a projector (the camera-based ones, not the old-school ones that you need to print on clear plastic).
Only during my final year was I able to write my programs on a computer, and they were much harder.
If you want to make it harder for the students, give them access to a computer that contains no software at all. See if they will write an editor before starting work on the exam questions.
In my time the students had access to computers where tney could do programming assignments. The theoretical exams used paper. Today, programming tests in job interviews are often taken on paper.
You think your better than the rest of the world? Goodluck.
[($)]
It would be interesting to do a study on the extent to which autocomplete is a teacher vs a crutch.
My guess is that the ideal would be a slow-response autocomplete, slow enough that it never appears while typing things you know, and it gives the learner a second to try to remember. If you stop typing mid-word for a few seconds (because you don't know) it prompts you with reminders.
It's language-dependent too. For example, JavaScript was written in a just a few days. Because of that, the order of arguments and such is inconsistent in places. Reminders are needed more than in a more consistent language such as C. C is also a much smaller language, and therefore should be easier to learn quickly.
Mostly because of how slow ACTUAL information flows back to educational institutions, compared to the Internet-Speed changes in the workplace, educational courses are going to lag actual realities of working with technology by 10-20 years minimum.
Hell, to take computer classes back in the late 80's, they were still REQUIRING you to take a typewriting class!
Never mind that you don't use a computer the same way you use a typewriter or even a dedicated word processing machine...
And, after leaving the Army in the mid 90's, I found that college-level courses were barely more sophisticated.
If I wanted to take programming classes, I had to learn how to use OFFICE?
REALLY?
Chas - The one, the only.
THANK GOD!!!
If you know how to code properly...why do you care about the medium used to do so? Writing stuff down actually forces you not to use shortcuts and demonstrates your actual knowledge. Suck it up! :)
Tired of my customary (Score:1)
I hear what you're saying, many of my colleagues initially react in a similar way (though with more professional language, mind you). They make the same initial, and faulty, assumption -- that my students aren't being taught the skills necessary to analyze information for its authenticity/relevance. If I could pick one skill which is most important in our world of FAKE NEWS and biases galore, it would be that.
Although I suppose we could stick to the government-supplied textbook model of information acquisition. Heck, it's worked for over 100 years, why change?
Back in those days, you didn't have a computer in your pocket. I had no computer access until school started in fall (and then it was really limited). So I wrote a threaded-code FORTH kernel in 6502 assembly on lined paper. Then I debugged it and rewrote it a couple times in between milking cows and attending band camp. By the time I had semi-routine computer access again, I had lots of other stuff I wanted to get to, so I never actually saw it in action.
IOW, big deal, a computer scientist makes do with the tools available to build solutions. Even today, decades later, when I want to _really_ get a problem solved rather than just dicking around with it pretending to work, I take some legal pads to a quiet spot with no computer access and get shit done. When you go interview for a job, they also are probably not going to give you an IDE. When you get a job and have a project proposal, they aren't going to sit and watch you screw around in an IDE for a few hours until you start making sense.
lol, slow-response...so that is a "feature" in Eclipse?
That would be interesting. Mr. Miyagi would suggest that you'd learn better by being forced to go look it up repeatedly. There is some research that suggests you do learn best when things are difficult, but not impossibly difficult.
I was just discussing with the always insightful Angel'o'sphere the possibility that the proliferation of IDEs with autocompletion is at least partially responsible for some of the poor interfaces in libraries and languages. Java apparently has at least three Date classes, all in different places!
Context: i graduated with a CS degree in 1983, and have been doing systems-ish stuff ever since. Compilers/Runtimes/Binary Translation/Interposing&Intercepting user space calls for security purposes. No web-ish stuff all all. Thus: when interviewing candidates for similar positions, debugging skills are key. Which means that general problem solving approaches are critical. Pseudo-code is just fine. Goofs are acceptable, and if the candidate can adapt to changing requirements (aka annoying product requirement changes) that is a big plus. I've recommended "HIRE NOW!" for one guy who got stuck on one part of the puzzle presented, but admitted it, and clearly had an approach outlined to attempting to solve it with more time. Fortunately, we hired him (great contributor)
Yes, final exams are on the horizon.
As an instructor who is considering this idea for CS1/CS2 courses, one major motivation is exam integrity. We teach with cloud based tools and everything goes into GitHub. I have no control over lab machine network access, etc. Other than a paper exam, how can I be sure of testing their mastery of relevant knowledge and skills? Seriously-- suggestions are welcome.
(I wouldn't nitpick minor syntax issues, but I would expect `objectReference.propertyName` types of expressions to be correct, etc.)
I teach at a community college, and all of my exams in programming class have been on paper, except one. Before this exam, I showed students on the overhead that I could see their desktops on my computer. I required them all to leave the IDE maximized (I use the VPL plugin in Moodle). One student did extensive browsing and copying from the net anyway, and I witnessed it. He dropped shortly thereafter, and I returned to paper-only exams forever.
The trick is to condition your students to writing on paper. If you don't, most will fail because going from hours of staring at a computer screen to writing on paper is very unsettling. So I give students a 5-10 minute written quiz every class meeting (twice weekly). It seems to work, and although almost every student fails some quizzes, the exam grades are pretty good, on average, compared to semesters when I didn't give daily quizzes. (Quizzes are weighted 10%, exams 50% (2 during term, 1 final), projects 40%.) Of course, this quadrupled the grading I normally do (50 students), but I think it is very rewarding for myself and the students.
If the point is to write a substantial amount of working code that does stuff, it's a project, not an exam, and it should be done on a computer.
If the point is to write a small amount of correct code that illustrates a concept, it's an exam question and should be written down.
In projects, all but egregious deficiencies in application of the theory should not be emphasized. Instead, quality of design and implementation, process, documentation, user experience and satisfaction of project requirements should be the goal.
In exams, all but outrageous errors in execution of code should be ignored if the theoretical principle is adequately demonstrated.
As a student this also annoyed the heck out of me. After all, you'll never be without a computer when you're the programmer -- whether it's 1995 or 2018.
As a professional programmer I found great value in being able to write "stream of consciousness" code which compiles, runs, and does what I want it to do.
Over the years, I realized that the only way to get my code to work each time is to be able to write and debug it on paper -- and to master the syntax of the language to the level of not needing the compiler.
So now I give exams on paper and have recently taken away the cheat sheets which I used to allow.
That being said, most of my intro programming classes focus on projects, assignments, and actually writing code during class. It is ABSOLUTELY pointless to have a paper only CS class. I also don't take off too many points for syntax and allow my students to define helper functions during exams without writing the bodies of the functions.
You can (very inefficiently) get code to work by trial and error -- and you HAVE to do this when you're learning.
However, if you can't at least provide the basic idea of what you're trying to do on paper, you haven't mastered the language or the content of the class.
Why not head over to their departments in your university and see how those degree candidates write up their exams:
English Litrrature - paper
Physics - paper
Chemistry - paper
Math - paper
Psychology - paper
Astronomy - paper
Biology - paper
Etc.
Why is Comp Sci different?
Ken
First thing I turn off is automatic auto complete, I.e. the waiting game thing. I do use auto completion on keyboard shortcut a LOT though. I know what I want to type and I know when exactly to ask the auto completion to type the rest out for myself. If I'm in vim I use vim auto completion (usually just for local strings like variables but I have used some plugins that make vim an IDE where it populates auto complete from included header files and such as well (used it for C back when).
In eclipse in java as an example I use some other auto completion features to my advantage too like adding a variable I use without declaring it as a method parameter including the correct type. There's a keyboard shortcut for that. Why would I go add the parameter first and then use it? Takes much longer to type.
Another great IDE feature is opening the right file. I don't bother learning where things are package wise. Thanks to consistent naming of classes I know how to open most important pieces of the code base I work on by typing just the CamelCase version of the class I want. I've learned most of the ambiguous cases as well where I need to type maybe one or two other characters. If I had to open all those from the command line and know which folders/packages they were in that would take ages. Of course my use case is that I rarely actually develop anything any longer but I need to debug or look stuff up in our code base.
The good point for paper is that students cannot destroy their work by issuing a wrong rm command (with the intent of cleaning up their directory from object files).
I am working on the next feature for an ongoing software project and my first step ... Sketch out how it is going to work on paper. This might include pseudo code because I have found this is a good exercise for identifying issues.
My exams were on paper - and my data structures course was also open book, open note.
The professor wanted open book, open note because it let him ask more interesting questions. He wanted to have the exam in the lab so he could ask questions that were even MORE interesting; however, there wasn't a lab big enough where everyone could take the exam at the same time. (I think this is better than having people take it remotely as it helps prevent cheating.)
Also, exact syntax mattered on our exams. Which didn't make things harder - it just made you a LOT more careful about what you put down. If you wanted to write pseudocode, you could take an algorithms class.
Reading code is like reading the dictionary - you have to read half of it before you can go back and understand it.
Why not allow testing out of a class that is core to the curriculum? Perhaps the "test out" test should be harder (deeper and broader and probably longer) than the tests that would be taken by someone taking the class and/or perhaps the "passing score" should be an "A" just to make sure that the person really wouldn't benefit from taking the class because they would likely learn something from it.
Why is there an "insightful" mod and why isn't it "-1"? If I wanted insight, I wouldn't be reading
"It would be interesting to do a study on the extent to which autocomplete is a teacher vs a crutch."
The other problem with autocomplete (and IDEs in general) is that they can get you to the stage of having code that compiles and runs without error, but unless you know what you are doing, it can give something completely different to what you expect.
NEED A HACKED ATM CARD WITH PIN?
We Sell Physical Loaded ATM Cards . It Is A Crooned Card That Can Be Used To Withdraw Cash At Any ATM Machine. This Cards Comes In Visa/MasterCard. Therefore It Works At Any ATM Machine That Accept Visa/MasterCard Worldwide.
CAN I USE THIS READY MADE ATM CARD TO BUY STUFF IN STORES? OR ONLINE SHOPPING? PAY BILLS?
Yes, With This Physical ATM Card, You Can Use It To Pay Stuff At Stores Through POS. With This ATM Card Information, You Can Use It Online To Pay Bills Or Do Online Shopping. When You Order For This Card, Full Information About The Card Will Be Given To You. We Also Reload Your Card When Funds Exhausted.
DO I NEED TO ORDER NEW CARD EVERY TIME THE FUNDS IS FINISH?
No, If You Have Already Ordered Our Card, There Is No Need To Keep Ordering New Cards, Just Contact Us For A Reload. We Shall Easily Reload The ATM Card Already In Your Possession
HOW LONG DOES IT TAKE TO RECEIVE ATM CARD IN MY COUNTRY?
If You Are In ASIA, You Will Receive Your Card In 2-3 DAYS With Guaranteed. If You Are Outside Asia Your Card Will Arrive To You Between 3 – 5 Business Days Guaranteed.
HOW SAFE IS THIS CARD?
It Is 100% Safe To Use This Card. Because It Will Be Shipped To You As A Gift Card.
DO YOU ALSO RELOAD ANY OTHER CARD NOT FROM THIS CLONED CARDS?
Yes, We Can Reload Any Active And Valid Cards, Any Type Of Card Just Contact Us For A Reload (Prepaid Cards, Credit/Debit Cards).
HOW DO I ORDER FOR THE ATM CARD?
EMAIL: legithackers@outlook.com
HOW DO WE MAKE THIS CARD?
We Use A Machine MSR To Crone This Cards . You Can Also Buy This Machine From Us Also. You Can Order For The ATM Card Either The Designed Card Or The Blank Card But Still Same Information On Them.
EMAIL: (legithackers@outlook.com)
Approx. 1996, I took and passed an exam in assembly code on paper. It was the kind of paper with a carbon back that made a copy of what you wrote.
Ha! I had a horrible vision of the future where a descendant of Clippy from MS Word asks something like "It looks like you're trying to write some code that I already have an open source library for, would you like me to just use that instead?".
They should be allowed to use typewriters.
As a long-time CS professor, let me add this: The entire purpose of a university education is to advance your understanding about whatever topic you are studying. To develop mental models. The purpose of a test is to see what you understand, and I want as little as humanly possible between your brain and what you produce for the exam. For the vast majority of students, that means brain to paper, with no distractions in between. My exams never have a lot of writing -- they do tend to have more problems to solve than many students like to solve in one sitting, but they are designed to get to core concepts with as few distractions as possible. I have had many students over the years that struggled with some disability or other that made paper exams difficult (whether dysgraphia or even a blind student in one situation), and so we have accommodations for students.
But again, the entire purpose is to see what you understand. I don't want to see how well you can use a tool, or how fast you can google things, or whether you can do some task like a trained monkey. Do you understand (and can you explain) why a red-black tree is balanced? Do you understand the balancing operation well enough so that you can write down a basic rebalancing operation on paper? Can you analyze the balance property to justify why the tree is O(log n) depth? If you have a clear picture in your mind, and if you really *understand* red-black trees, you should be able to do those things.
That sounds great. I have some co-workers that could use that!
Recently I fixed up some Perl where one of our most senior guys had forked a process to run /use/bin/time. Uhm, did you mean time()?
Actually /usr/bin/date +%s
Which returns the same thing as the Perl built-in time()
Yes for paper. Otherwise you end up testing for ability to use the editor/IDE. Eclipse (for example) is so byzantine that one can get lost in it trying to find the correct submenu for setting up a runtime environment.
The one that drives me bonkers isn't coding. It's Linear Algebra tests on paper, without even a four function calculator. Get one of dozens of arithmetic operations wrong, and the confused teacher tells the class /they/ are missing the forest for the trees.
In Linear Algebra, the important thing isn't doing dozens of arithmetic operations correctly - that's what computers are for. The important thing is knowing what formula or transformation to apply when, which pencil pushing just obfuscates.
When I took a Computer Graphics course, the one that talked about Beziers and Splines and stuff, we used a lot of Linear Algebra. I did my homeworks on paper, as requested, but I checked it with symmat, which I wrote just for that class.
Really, AS a student you should be able to do some coding in paper, cardboard, whiteboard etc. This is how we communicate primarily with each other and customers. Also it is an exam not a piece of software going to run somewhere. BTW even after the exam you should still aim for knowing everting the course thought you.
Pseudo code on paper is okay. But compilable code on paper is a throwback to the 1960's, when computer time was limited, and a computer program had to be desk checked on paper before being compiled. As long as this university is being so lame, why not require a flow chart on paper before the student writes code on paper. And make sure that the flow chart has goto arrows to insure that the program will be filled with goto statements.
I sympathize with the student, the university is being absurd.
I like your idea on how to condition the students to take exams on paper with regular quizzes. Thanks for sharing!
Wow -- they're using these techniques at the job interview level! I had no idea -- thanks for sharing. The disadvantage of having become a teacher and leaving the programming industry, is that I've left the programming industry. So all my skills have aged, and keeping up has been a challenge. Thanks for the insight!
I love the idea of having a lab which is offline, but at the (public) high school level that's unfortunately not an option for me.
My "online" tests/exams seem to involve all the same things you do, but without code writing. It's taken a lot of time and effort on my part to figure out how to do that -- as I still believe in the validity of tests/exams, just not writing code without an IDE. So, for example, I'll give them code and have them explain what's happening in English. Or I'll give them broken code and have them explain what's wrong or how to fix it. Or I'll ask what situations you'd use one construct vs another. I just don't like asking them to code in tests/exams. It seems like a skill that doesn't mimic the real-world, and to me that's what education is supposed to be preparing them for. Though, I realize that MANY disagree with me on that point.
I'm afraid I respectfully disagree. Without trying to shame you for your opinion, I'll try to explain why I strongly believe the complete opposite of you.
Knowledge WAS power, that is true. In the time of Galileo, in which you have placed your argument, knowledge was hard to come by. One would have to hunt down the proper books, and spend countless hours reading through them to acquire the knowledge necessary for some task at hand. But we don't live in that world anymore. Now, a properly formatted Google search will instantly take you to several resources somewhat-related to what you need. Learning how to synthesize that knowledge into a solution is now where the time can be spent, instead of spending the hours finding and memorizing knowledge which will soon be out of date.
But, I get where you're coming from. It's hard to recognize that a fundamental paradigm shift in society has occurred, and even harder to change oneself as a result. I could have happily gone on teaching as my forefathers have, all the way back to Galileo. It certainly would have been easier! But we're not in Galileo's world anymore, nor will we ever be again.
I still have to write exams on paper where they are grading on syntax in courses like Introduction to C, Data structures and algorithms, heck we even had to write assembly and VHDL with correct syntax on paper, and I'm fine with them grading the syntax, my only problem is what if I make an error or I have bad handwriting or I realize I forgot to add a line somewhere.
I think exams written on paper should be better organized. Some courses had exams where you just have to write certain functions or short snippets of code and im all for that but don't make me write a 30+ line program on paper.
Does English department expect students to properly punctuate their twisting, or are they allowed to answer questions in pseudo-English?
A computer program in a major programming language generally uses a larger variety of punctuation marks than comparably long prose in English. Even in something like Python that has keywords and indentation for things where C++ and JavaScript use punctuation operators, you need to use all three kinds of brackets (round, square, and curly) in any nontrivial code that handles dictionaries. Nor does English class generally give a grade for handwriting past primary school. That's something you typically see only in logographic languages like Chinese and Japanese.
Nor should they be expected to be writing fully functional programs on a CS test
They are if the question is "How can this be done without mutation operators?" and the language of choice is Haskell or the subset of Scheme without set! or other mutators.