Slashdot Mirror
"Clinical Trials" For Programming Languages?
theodp writes "High school junior Charles Dawson's New Year resolution is to write a new program in different language each week. It's an ambitious project for someone of any age, and while it won't give him an in-depth appreciation of programming language differences, it'll certainly give him greater insight into the strengths of certain languages than would perusing the Hello World Wikipedia article. Lots of claims are made about the comparative productivity of programming languages, but have there been any landmark studies that measure the efficacy of a programming language's productivity claims in a 'clinical trial' of sorts? Would head-to-head tests against other languages be a better way of sorting out Popularity vs Productivity vs Performance claims, or is relying on more nebulous claims of superiority the best we can do?"
long flamewar
There is already a pretty good collection http://www.99-bottles-of-beer.net/
There is also a website with the implementations of the Perl cookbook in a bunch of languages: http://pleac.sourceforge.net/
...Some languages are good for some things, and other languages are good for other things. Think LISP.
My New Year resolution is to design and implement a new programming language every week.
If Pandora's box is destined to be opened, *I* want to be the one to open it.
And therein lies the problem of comparisons. An extreme case: a person writing a program that involves concurrency among hundreds of processes will probably be more productive in Erlang than in Perl, but a person writing a program that does massive amounts of text manipulation will be more productive in Perl than in Erlang, because of what the languages were designed for. It's somewhat like asking which is a better tool, a hammer or a screwdriver. A lot of it depends on what you're trying to build.
In any such trial, it is important that aspects such as maintainability, reliability and securability be considered. The ability to hack out a-lot of functionality is not the only criteria that is important, unless you are building a home hobby project.
I have this same problem -- there are a lot of interesting languages out there that I'm interested in trying, but I always keep going back to languages I already know because:
I was thinking that the solution to this is to have one program that I understand very well implemented as well and completely as possible in a language that I feel proficient in, and have that be my reference. Then, over the course of a couple of weeks (a month?), re-implement the same program in the new language and strive for the implementation to be as idiomatic of the new language as possible. After all, if you're still thinking in the old language but just using the new one's syntax, what's the point?
I feel like this would give you a lot of data to make a reasoned decision -- you can compare language features and how the implementation works in one versus the other; time to implementation (LOC, maybe?); how much of a mental shift the new language requires; the toolchain around the new language; etc.
The problem is figuring out what the reference app is, and having the stomach for implementing it over and over again. Tetris, maybe? ;)
But, back to the resolution (and partially touched on) -- I don't think a week is enough time. A month is even cutting it close, IMO.
C
The Sun is proof that we can't even do fire properly.
do you relly want apps that need 2-5+ run times and are very bloated?
The answer is research is sparse in this area, but the few times it's been tried (using competent programmers in each language rather than conflating learning the language and productivity in it), is LISP and LISP-like languages win when measuring programmer productivity and ability to express complex algorithms in small amounts of code, and C and C++ like languages win for ultimate ability to make the program run fast. But the variability from programmer to programmer in how fast the program runs can exceed the variability between languages so it pays to get high quality programmers who have an intimate understanding of both efficient algorithms and the underlying machine architecture, rather than think "the language will make the program run fast".
With everything, there are professional users and amateur users. For amateur users, it's important to get reasonably good results with relatively low effort without much learning. For professionals, what counts is the effort for projects the size a professional does, after learning a lot.
Trying a new programming language every week cannot give any useful information to a professional user, because the language can only be judged on how well it works for inexperienced developers on tiny projects. That's not what professionals do.
This is a myth. There is only suitability to solve a problem within a given context. At first, it's "how fast can I bring this to market", then "how does this scale" (in terms of execution efficiency). Finally, "can I hire people to do this?"
The starting point is inevitably what the initial implementer is most familiar with (or infatuated with) at the moment.
A few weeks ago, I wrote an article about how asinine this technology argument is.
Some key features of "gold standard" clinical trials: 1) large enough sample size to draw statistically significant conclusions; 2) real illnesses, not a simulated laboratory setting; 3) a double-blind control group; and 4) long enough duration to measure real-world outcomes.
The programming-languages version would be to have teams randomly assigned to perform major (6+ month) programming projects in different languages, and then see their outcomes. For example, 40 game studios will continue to write their games in C++ as the control group, while you'll have the other 40 write them in Haskell. You probably want to iterate a few times as well to make sure that there's no first-game-in-a-new-language effect and to ensure that everyone is actually knowledgeable in the language being tested.
Oh, and it should be blind, so neither the teams nor the researchers know which language they're using.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
it's my observation that programming languages have become the equivalent of fashion and bands.
it's as if choosing and using one is like saying "oh, im listening to arctic monkeys and calvis harris these days, and that makes me feel liberated from those plebs still listening to kings of leon and david guetta..and MAYBE if im lucky someone will be impressed by my trendiness"...now that so many 20-somethings code, they all seem to feel the need to break from the "boring old bonds" of existing languages and define themselves among their peers by how esoteric their language-de-jour is.
what this guy is doing is illustrating the point, he isn't going to learn anything about the benefits of each "language" or how to maximize productivity...it's just a "notice me notice me" stunt.
it's just a silly exercise in syntax-swapping anyway for 90% of it...
please don't get me wrong...it's totally fine by me whatever language you want to use, as long as it's maintainable and there is a large enough pool of existing programmers who know the language so when you leave my company because your bored with maintaining the code you wrote, i can find someone quickly and affordably to replace you.
never bring a twinkie to a food fight.
Repeatability. Productivity must be repeatable if any measured claims can be made, and no one does the same thing twice.
Even if you write similar code for a similar purpose, you have the backing of experience. The results are different even if you end up with the same code.
And, there are the os and third party libraries. I can save time by using such features, if I take time to learn them.
There is a post below about maintainability, securability, etc, all of which are good points, but will rarely be done exactly the same way. Especially since some are intended to be server side, where obscurity is possible, and some client side where you may need real security.
Any such study will have questionable conclusions due to the number of variables. Writing new code vs inheriting, where the choice might be maintain or rewrite in a new language.
But none of these will be applicable to an entirely new situation, and any study results irrelevant. Repeatability in code means studying something that will probably never happen again, or something so basic it will never represent normal professional usage.
Look on craigs list. You can get it for $1-$2,000.
Do you even lift?
These aren't the 'roids you're looking for.
There are good practical reasons for writing the same program in different languages. (1) People often make the same mistakes in the same language, because they are often taught/trained in similar fashions. (2) Different programming languages have different issues and failure modes.
I'm not an expert, but I worked as an undergraduate research assistant on a project way (way) back in college (1985-87) for a professor on a NASA contract to study issues related to N-version fault tolerance - like used on the Shuttle, where several computers solve the same problems and vote on the answers. One problem (if I remember correctly) was that the different programs, written by different people, in the same, or same type of, language often failed on the same or similar edge cases.
The experiment was to implement the same solution to a problem using much different languages. In this case, Pascal and Prolog. The "gold" Pascal routine was already written and my task was to write the corresponding "gold" routine in Prolog. [ I also did work for a related study in the automatic analysis (and execution) of abstract data types in LISP.] I graduated before the experiment was run, but found the idea as least plausible.
Note that I might be remembering some of this incorrectly as it was a while ago and I was only an undergrad. (They wanted a graduate student, but I was the only one with LISP and Prolog experience... And $9.50/hour wasn't bad for 1985.)
It must have been something you assimilated. . . .
Because the productivity doesn't scale well as projects get larger. Dynamic languages are amazing at 20 line programs. They are pretty hot at 200 lines programs. At 2000 lines you are starting to feel the minus but they still work well. At 20,000 things start going badly wrong. By 2m, well there are 2m line programs mostly because all those things that are good about a dynamic language for 20 lines turn into disadvantages when you need hundreds of programmers to work together.
It's a good lesson, but for different reasons. Here's why.
In the real world, you pick the right tool for the job. You never pick a language because it's the best language. There is no such thing. Factors going into language selection where technical merit plays no role include what the other developers at the company and/or the project are using, what environment you're using (if Apple, then Objective C; if Android, then Java), what language the code you are maintaining was written in 5, 10, 20, or 30 years ago, and (hopefully if you are a great programmer this will be a minor issue) what languages you're comfortable with using.
After 30 years I've learned that basic computer science concepts are helpful, but only to a point. Google may want you to know specifics about certain types of trees for their interview process, but if you need to know that level of detail for a job, you spend a few hours on Google and learn it. The same goes for languages. Figure out what you need with a bit of research before you start the job. You should have a great idea of what environments a language is nearly always used, so you don't try to do something weird nobody can maintain. If you're going to write an iPhone app, you're going to adhere to whatever specific Objective C thing Apple is doing. Maybe I'm slightly out of date and Apple is doing something else, who knows? I don't work in that space.
Python everywhere, be damned with you, is a quick way to make enemies of people 10-20 years down the road who have to maintain your code. I was doing web development in the 1990s, and everybody used Perl. For everything. Now I work with a legacy Perl code base, and mod_perl seems to be completely abandoned, and it certainly hasn't been released for apache httpd 2.4 yet. We're using Perl because we have to, but not for new stuff. But for the Perl part of our system in bug fix maintenance mode, it is the appropriate language. We didn't have the attitude that we'd continue to use Perl for everything just because that's the way things were done. We were flexible enough to slowly switch over to PHP for certain things that we had been using Perl for.
Avoid fads like the plague. After 30 years of programming, I just ignore marketing. I have no gee whiz attitude about anything. I focus on perfecting my craft, learning how to program better, to debug better, to test better. Learning how to deliver code that works now and five years from now. All that is way more important than figuring how how some language is subjectively the best.
Hi everyone. Thanks for all of your comments. I just want to quickly elaborate on why I'm doing this and what I'm trying to do with this project: I am not trying to start a flamewar, annoy programmers, or hog attention. I am not aiming to master any of these languages after only a week. I'm not even trying to get "okay" at any of them. All I'm trying to get out of this project is a more broad understanding of the languages that are out there, and what, in a very general sense, each language is good for. My thought is that if I can get even one cool new idea or concept out of each week's programming language (like functional programming concepts from this week's foray into Haskell), I'll consider this project a success. Also, it will allow me in future to be able to quickly reference a new language if I need it for something I'm working on (not necessarily remember any syntax or anything, but at least to save myself some googling time). Thanks for your interest. Sincerely, Charles Dawson
As someone who pretty much only codes in python lately, I would agree except that I hate the fact that whitespace actually matters in python. I feel that this limits the utility of python while gaining only a modest increase to readability (according to some people).
I tend to agree. One thing I find annoying is that in order to temporarily comment out the body of a conditional you have to add a "pass" line. The ability to add members to classes at runtime has also created some situations for me that were very confusing to troubleshoot.
Account -> Discussions -> Disable Sigs