CollegeBoard: Analyses of CS Study Benefits Shouldn't Be Interpreted As Causal

theodp writes: Code.org, backed by some of tech's wealthiest individuals and their companies, is this close to getting computer science declared a 'core subject' in K-12 public schools. So, when the non-profit recently asked CollegeBoard for more evidence that learning computer science is linked to improved learning in other subjects, it must have been disheartened by the study results. "The purpose of this brief note," wrote the CollegeBoard, "is to document some exploratory analyses linking participation in AP Computer Science to subsequent performance in SAT Mathematics and AP Calculus and Statistics. None of these analyses should be interpreted as causal. Although there appears to be a relationship between AP CS participation and subsequent outcomes, it is highly likely that this is the result of one or more omitted and confounding characteristics of students that are not able to be controlled for given this research design."

Comments

They should make them all core subjects

[Overzeetop]

Learning doesn't happen in a vacuum. Every subject, addressed properly, will have spillover - even if it's just as an expansion of the curriculum to create a feeling of value to a student concerning the learning environment.

But, of course, when they're all considered "Core" subjects, none of them are core subjects - they're just curriculum. Pixar said it best - when everybody is special, nobody is special. And then we're back to where we started.

Re:They should make them all core subjects

[TsuruchiBrian]

How many people use trig in their daily life? If we were to go by the standard that core subjects are those that are used by a majority of people, I think we will just have reading, and maybe typing.

Core subjetc my a$$....

[dablow]

In this day and age, everybody should have a basic understand of how computers work and how to use them. Know what a file is, network etc. Same with social media.

But I get the feeling what theses clowns are aiming to do is get people to learn basic coding in order to flood the market with code monkeys that know how to write an if-then-else statement in order to deflate CS salaries......Make it so that anybody with a high school diploma can apply for entry-level coding jobs.

Re:Core subjetc my a$$....

[LaurenCates]

That's one theory (and the more realistic one).

Another is that coding won't be the primary job of most of the college graduates in the future, but that it will be a necessary subset of skills required to do a job (not realistic until it's sold that way).

We're probably already at the place where anyone with an Sparkfun account feels like they're an expert, and coding is something anyone can and should pick up.

Anecdote incoming, which I fully acknowledge is not the singular of "data": I was having an eye-roll moment recently when a young woman I follow on Facebook (who isn't in tech for a living) espoused using Arduino to learn about computer security as a way to tell people to "educate yourselves" (as if, somehow, YouTube isn't an acceptable tool for learning things, and far more versatile...it was far more a play to say "hey, I know Arduino, pat me on the back for being so smart, would ya?").

I imagine, though, she didn't pick up that attitude in a vacuum.

I acknowledge that Arduino is quite a powerful tool for a lot of things, but fiddling with one makes you about as much an expert as picking up a baseball bat makes you a professional ballplayer.

Point being that the idea of "coding" and "putting your hands on the metal" is so much more accessible to the normal person nowadays that anyone who has access to something that looks a little more technical than a toaster is within reach of a job as a coding ninja.

Re:Really?

[Nidi62]

I would expect it to improve reading, reading comprehension, written language skills, and logical thinking. That is what the student is learning!

The problem is, and I think CollegeBoard is saying this, that anyone who has the ability to take AP CS and then take the test should already have significantly developed reading, comprehension, and logical thinking skills. From my experience (I did go to a school with a magnet program but AP classes were open to all students) most students who took an AP class took several; it was very rare to have someone take just one class. So it was a bad idea to have CollegeBoard do a study anyway because there is no way to isolate any potential benefit with AP CS from the student's general ability/interest. Unless Code.org was counting on this so that they could obfuscate the results to show whatever they wanted (a distinct possibility).

Why children should NOT be taught to code

[XxtraLarGe]

I had submitted this post about two weeks back, but it never made it to the front page. FTA:

I've written about this at some length in my book Beyond Technology. The argument depends upon assumptions about learning transfer -- the idea that learning in one context will automatically transfer across to others. This is to conceive of the brain as a kind of muscle: a good workout in the coding gym will have payoffs when we need our logical thinking skills to solve problems elsewhere. Similar claims are often made for learning the game of chess, or Latin. Yet there is no convincing evidence that learning computer programming enables children to develop more general problem-solving skills, let alone that it will 'teach you how to think', as its advocates claim.

While it seems intuitive that programming develops logical thinking, it may be the case that people who program already possessed that skill and programming merely reinforces it.

Re:Why children should NOT be taught to code

[AthanasiusKircher]

While it seems intuitive that programming develops logical thinking, it may be the case that people who program already possessed that skill and programming merely reinforces it.

Indeed. Additionally, we need to consider that the performance of a self-selecting group of students taking a course voluntarily may NOT necessarily reflect a general trend that would be applicable to ALL students when a course is required.

In other words, even IF programming does help develop logical thinking in students who are interested in it, it does not necessarily follow that these performance gains would happen with all students.

We need only look at the history of geometric proofs in high-school curricula to see the large-scale failure of another attempt to teach logical thinking indirectly in a high-school course. From the mid-1800s until the late 1900s, a full-year course in Euclidean geometry with emphasis on proofs was a standard part of most high-school curricula in the U.S. Yet a number of studies done in the past few decades have concluded that the logical skills actually developed in such courses were nearly non-existent outside a small group of students, most of whom probably already possessed significant logic and abstract thinking skills before taking the course. (Many studies concluded that the majority of students left such courses with little to no abilities to actually do mathematical proofs, and -- more disturbingly -- they also left the courses with profound misunderstandings about the nature of logic.)

I'm NOT saying that teaching logical thinking is hopeless, but it requires a combination of a good teacher and good student engagement, as well as a curriculum that is not "dumbed down" to accommodate "the lowest common denominator" of student. (This was a problem that many critics raised about geometrical proofs -- that they were taught in a way to make them "accessible" to everyone, but in the process they were dumbed down to a point that they no longer taught critical thinking very well. And the exercises were boring to those students who actually had an aptitude for such things.)

In sum, even if the measured skills show real improvements (not just selecting students already good at these things), it may be quite difficult to extend such improvements to uninterested students required to take a course, or to all students of all ability levels. We have loads of data on such attempts to teach abstract thought from math curricula reform over the past century or so... it rarely works as intended.

AP CS students already have advantages

[ErichTheRed]

I'm glad the College Board is showing a little academic objectivity here, considering the fact that they have the potential to make lots of money off AP exams, increased SAT usage if more students are herded into college, etc. There are several things that AP CS students most likely have going for them that explain any causation:
- They're probably at least halfway decent at math and science courses already, or they wouldn't be on the AP track.
- They go to a good high school, as lousy high schools have lower AP course attendance / exam administration levels.
- They probably have semi-involved parents, or at the very least aren't having insurmountable home front problems preventing them from benefiting fully from school.

On top of that, I'm not sure it's a good idea to force every reasonably logic-minded student to be a "coder." I'm not a coder, I work in IT and use my problem solving/troubleshooting skills to fix things. Yes, I write scripts and automation tools, but it's certainly not Internet-facing stuff. Other people with the gift for logic would make good doctors, traditional engineers (civil, chemical, etc.) or dare I say it, lawyers. Even in a severely changed employment world, I don't see millions of people clustered around cafeteria tables in hipster San Francisco office lofts coding up the next Tinder or Uber. In fact, I'm amazed about how much this latest tech boom is like the dotcom boom...people are running around saying "this time it's different," companies are IPOing with valuations based on the modern equivalent of eyeballs, and no one apparently learned anything from the last boom. There was an article on here last week about how CS enrollment has hit its pre-dotcom crash peak again...hang on tight folks!

I think that if we turn out a whole generation of Java coders who know little about actual computer science, which seems to be the majority now, it'll be the equivalent of the Soviet Union or China trying to rapidly industrialize without having the necessary skills in place. In those cases, it worked but there was a significant skill mismatch, famines, etc. The only reason it worked was because it was forced. I doubt every single smart, talented person in the US is going to want to sit cranking out JavaScript, Ruby or PHP code all day for some phone app...it's just not a sustainable market, especially when wages are headed down and offshoring is constantly being used.

Re:Really?

[myowntrueself]

I would not expect computers and/or computer science to improve the performance of students in SAT Mathematics, AP Calculus, and AP Statistics.

We use computers so we dont have to remember all that crap. The computer does the math.

I would expect it to improve reading, reading comprehension, written language skills, and logical thinking. That is what the student is learning!

Computing teaches any problem domain that you are asked to code solutions for.

The problem with initiatives like code.org is that they generally try to engage kids by making things move on the screen. Most of that means doing very basic arithmetic in an esoteric firmat surrounded by Byzantine library calls.

If you want kids to do better in statistics, you shouldn't start with the paradigm of interactive entertainment, but with the far less abstract view of a computer as something that computes stuff. Kids might not like their schoolwork, but it's certainly relevant to them. Part of the problem teaching complex maths is that the mechanics of carrying out the underlying computations diverts attention from the "big picture" view. Procedural computing was designed specifically to address the problem of "can't see the wood for the trees" by separating the general algorithm from the specifics of implementation.

I think they are conflating programming and computer science. I think theres a lot of this confusion surrounding the discussion of this article, and indeed in teaching computer science at this level.

Computer science is only tangentially related to programming.

Most of computer science involves things like logic and discrete maths; state machines, turing machines, computation theory, set theory, algebra of functions, big-O notation and efficiency of algorithms. I majored in computer science and did very little actual programming. In fact I hated programming, still do. I can't comprehend people who program 'for fun'. But I did enjoy delving into the math that lurks beneath computing, discovering the limitations of algorithms (and hence of any methodical approach to a problem ie there are some problems that no computational process can solve, not because they are NP hard, its worse than that). Computer science introduced me to things like Chaitins algorithmic information theory and the first known uncomputable number (the halting probability), Goedels incompleteness theorem etc.

Re:Not so really

[Nidi62]

As someone who returned to a community college in 2009, I was surrounded by kids who had the whole litany of AP classes. Many of them were bragging about their 5.0 GPA in HS. However, a LOT of them had very poor spelling, reading and grammar skills.

These things do not add up. First of all, considering only AP classes get the added 1 weight it is impossible for anyone to get a 5.0. I went to school with several people who aced the math portion of the SAT, one person who taught himself Chinese, and many other very smart individuals: none of them ever got a perfect score in every AP class nor did any of them go to community college-but plenty went to Ivy League schools, top research/engineering schools, we even had one guy go to Juilliard for violin (and who is also now a cop out in Aspen, CO, which shows the prospects for classically trained musicians in the US). And given the amount of writing necessary in many AP courses (especially History and Language/Literature) it is highly doubtful that someone could get a 5.0 in any of those classes with deficiencies in spelling, reading, or grammar. Either you or your classmates are lying their asses off.

CS should not be a core subject

[prgrmr]

CS, much like blacksmithing, is a combination of art and science; as such, while anyone can learn the basics, only a minority of people are ever going to be good at it--let alone understand it enough to be good at it from the start. To put it another way, anyone can learn to play a musical instrument, but only a minority of people can be described as being musicians. There are many CS jobs that work this way, programming, database admin, and system and network administration being the obvious examples.

CS courses in elementary and even in middle school are generally a waste of time. The amount of accretive knowledge to be gained at that early an age isn't going to put any student so far along the learning curve that doing it all again in high school would be so repetitive as to be a waste of time. So just do it at the high school level, when kids are actually at the point of making career choices and the corresponding college selections to follow those choices. And don't make every kid take the CS course, when it's obvious far from every kids will be pursuing a CS-type career.