Beginning Perl for Bioinformatics

babbage writes:"As the banner above the title of James Tisdall's Beginning Perl for Bioinformatics indicates, this book is 'an introduction to Perl for biologists.' What the banner doesn't mention is that it's also an introduction to biology and bioinformatics for Perl programmers, and it's also an introduction to both Perl *and* biology for people that have never really been exposed to either field. The author has clearly thought a lot about making one book to please these different audiences, and he has pulled it off nicely, in a way that manages to explain basic topics to people learning about each field for the first time while not coming off as condescending or slow-paced to those that might already have some exposure to it." Read on for the rest of his review. Beginning Perl for Bioinformatics author James Tisdall pages 400 publisher O'Reilly & Associates rating 8 reviewer babbage ISBN 0-596-00080-4 summary Well-balanced approach to applying Perl's sorting and analytical abilities to the field of bioinformatics.

Superficially, this book isn't all that different from a lot of introductory Perl books: the Perl material starts out with an overview of the language, followed by a crash course on installing Perl, writing programs, and running them. From there, it goes on to introduce all the various language constructs, from variables to statements to subroutines, that any programmer is going to have to get comfortable with. Pretty run of the mill so far. Tisdall starts with two interesting assumptions, though: [1] that the reader may have never written a computer program before, and so needs to learn how to engineer a robust application that will do its job efficiently and well, and [2] that the reader wants to know how to write programs that can solve a series of biological problems, specifically in genetics and proteomics.

As such, there is at least as much material about the problems that a biologist faces and the places she can go to get the data she needs as there is about the issues that a Perl programmer needs to be aware of. The author introduces the reader to the basics of DNA chemistry, the cellular processes that convert DNA to RNA and then proteins, and a little bit about how and why this is important to the biologist and what sorts of information would help a biologist's research. The main sources of public genetic data are noted, and the often confusing -- and huge -- datafiles that can be obtained from these sources are examined in detail.

With the code he presents for solving these problems, Tisdall makes a point of not falling into the indecipherable-Perl trap: this is a useful language, well-suited to the essentially text-analysis problems that bioinformatics means, and he doesn't want to encourage the kind of dense, obscure, idiomatic coding style that has given Perl an undeservedly bad reputation. Some of Perl's more esoteric constructs are useful, and they show up when they're needed, but they're left out when they would only serve to confuse the reader. This is a good decision.

Rather, the focus is on teaching readers how to solve biological problems with a carefully developed library of code that happens to leverage some of Perl's most useful properties. The result is pretty much a biologist's edition of Christiansen & Torkington's Perl Cookbook or Dave Cross' Data Munging With Perl. The author presents a series of issues that a working bioinformaticist might have to deal with daily -- parsing over BLAST, GenBank, and PDB files, finding relevant motifs in that parsed data, and preparing reports about all of it. If a bioinformaticist's job is to be able to report on interesting patterns from these various sources, then following the programming techniques that Tisdall explains in clear, easy-to-follow prose would be an excellent way to go about doing it.

And when I say "programming techniques," note that I'm not specifically mentioning Perl. The code in this book is clear and organized, and all programs are carefully decomposed into logical subroutines that are then packaged up into a library file that each later sample program gets to draw from. Each new program typically contains a main section of a dozen lines of code or less, followed by no more than two or three new subroutines, along with calls to routines written earlier and called from the BeginPerlBioinfo.pm that is built up as the book progresses. Each sample is typically preceded by a description of what it's trying to accomplish and followed by a detaild description of how it was done, as well as suggestions of other ways that might have worked or not worked.

This modular approach is fantastic -- too many Perl books seem to focus so heavily on the mechanics of getting short scripts to work that they lose sight of how to build up a suite of useful methods and, from those methods, to develop ever-more-sophisticated applications. It isn't quite object-oriented programming, but that's clearly where Tisdall is headed with these samples, and given a few more chapters he probably would have started formally wrapping some of this code into OO packages.

If I have a complaint with the book, in fact, it's that Tisdall doesn't go any further: everything is good, but it ends too soon. Seemingly important topics such as OO programming, XML, graphics (charts & GUIs), CGI, and DBI are mentioned only in passing, under "further topics" in the last chapter. I also have a feeling that some of the biology was shorted, and the book barely touches upon the statistical analysis that probably is a critical aspect of the advanced bioinformaticist's toolbox. I can understand wanting to keep the length of a beginner's book relatively short, and this was probably the right decision, but it would have been nice to see some of the earlier sample problems revisited in these new contexts by, for example, formally making an OO library, showing a sample program that provided a web interface to some of the methods already written, or presenting code that presented results as XML or exchanged them with a database.

But these are minor quibbles, and if the reader is comfortable with the material up to this point, she shouldn't have a hard time figuring out how to go a step further and do these things alone. It's a solid book, and one that should be able to get people learning Perl, genetics, or both up to speed and working on real world problems quickly.

You can purchase Beginning Perl for Bioinformatics at Fatbrain. Want to see your own review here? Read the review guidelines first, then use Slashdot's webform.

Comments

if only it was in italian...

[joss]

then I could learn perl, biology, and Italian all at the same time.

Heh

[British]

"You got your Perl in my biology!"

"You got your biology in my perl!"

Two great interests that interest great together!

statistical approaches

[ciole]

I felt the same about the lack of statistical approaches. While this book is probably great for biologists just learning to write code, for coders entering the field (bioinformatics) it contains too little biology or math to be really educational. My opinion.

What I'd love would be a dissection of the construction of various motif analysis tools, critiquing various impl's of HMMs, really going into detail. This seems like a perfect complementary work to OSS, so I might even find one, someday...

Re:statistical approaches

[sql*kitten]

. While this book is probably great for biologists just learning to write code, for coders entering the field (bioinformatics) it contains too little biology or math to be really educational. My opinion

A question for practitioners: why would you want to use Perl over a flat file data set, rather than loading the data into Oracle and using professional data mining (OLAP/DSS/DW) tools? Surely the latter are more mature and comprehensive.

Re:statistical approaches

[babbage]

Well of course loading data into an DBMS is the ideal here, it's the loading of the data into one that's the tricky part :)

Generally, a lot of biological data is publically available from sources such as NCBI (US national computational biology lab) and EMBL (European molecular biology lab), but it could be coming in as SQL statements ready for loading into your database, CSV or TSV files, any of several annoyingly flexible standard biological data exchange formats, or worst of all something like an Excel spreadsheet or just scraped from a web page somewhere. There is way too much of this stuff to pump it all into your local storage system by hand, so you need something like Perl that can munge it into an intermediate format that can be loaded properly. Once it's actually in there then yeah, you only revert to some sort of flat file system if you want to redistribute data.

A related but more central problem is in looking for interesting patterns in these huge datasets once you have them locally, whether in flat files or a database or what have you. This is a huge area of research right now, because modern bioloogical lab technques can slurp up data extremely fast, we have the whole genome decoded but uninterpreted, etc, and now we need computational techniques that can chew through this fire hose of information efficiently.

A lot of this seems to be unsolveable at the moment, because the algorithmic complexity is up there with the Travelling Salesman problem (e.g. protein folding), so every little bit that can chip away at the difficulty of it helps. Perl is good at this, and a lot of places are using it heavily right now. Being able to work with flat files is only one aspect of it; it just happens to be a useful one to teach with, which is why it was used so heavily in the book, but in actual use the applications of Perl go way beyond simple file maniipulation.

Re:statistical approaches

[Marcus Brody]

why would you want to use Perl over a flat file data set

Good Question. Answer is yes and no.
Flat Files are really quite useful in biology (btw, when a biologist mentions a "database", he almost certainly mean a "flatfile"). DNA/RNA/Proteins are just a long sequence of letters, and therefore these are perfectly represented by good 'ol ASCII. This is particularly useful for means of distribution etc. When annotations are added to the data, they are traditionally added to the flatfile by way of an "annotation table", to keep the simple ease of ASCII.

However, more advanced ways are used to store annotations of biological data, although traditional databases arent allways that good at expressing the rather messy, randomness of biology ;-) Therefore, specialised databases such as acedb are quite useful and intuitive to the biological mind. Furthermore, projects such as ensembl (which ambitiously attempts annotations on the whole genome) store their data in an SQL database. However, they still make extensive use of perl to interact wiht the database.

I haven't read it myself but

[Theodore Logan]

I have a number of friends in the business who have read that book. In summary:

1) It is good for biologists who wants to learn how to program

2) It is not good for programmers who want to learn biology

Obviously, my friends disagree with reviewer Babbage on this point. However, a quick look on Amazon reveals that most reviewers who found the book interesting are biologists with no programming experience instead of the other way round.

Flashbacks

[keiferb]

Seeing a title like this, aiming a particular language at a particular discipline makes me flash back to the college days (last year) where the engineering classes all used fortran. God forbid, if perl gets outdated in another few years, are all the Biologists in the world going to lock themselves into a dead language like those stuffy engineers?

Re:Flashbacks

[glwtta]

I've worked in bioinformatics for the last few years, and I can say that there's a bit of a difference between bioinf and perl, and engeneering and fortran - perl is suited for bioinformatics far, FAR better than any other language. And so far the benefits of modern languages just can't seem to outweigh this innate suitability.

Traditionally almost all bioinformatics tools have been done in perl, and they continue to be so, for one very simple reason - bioinformatics, when it comes down to it, is just plain text processing.

Anyway, about the book itself - it's nice for biologists who want to learn something about programming, but I neither learned much about biology from it, nor am I afraid I will lose my job because all the bio people are gonna start doing their own programming :)

Re:Flashbacks

[glwtta]

I think I was a bit too general in what I said - yes many widespread "standard" bioinf tools are not Perl (I can' remember what GCG is written in, at the moment, but it's probably a mix of C and Perl) - but "on the job" day-to-day work is still mostly done in Perl. You usually don't see too much GUI stuff (and therefor java/swing) because bioinf is still a very command line oriented world (probably because of things like BLAST, mview and so forth). When we did things in java to fit in with everything else at the company, the bulk of work was being done by org.apache.oro.text.perl :)

The challenge of Bioinformatics

[nesneros]

Bioinformatics is probably the biggest challenge facing the biological sciences in the next few years. Its becomming more and more apparent that even slight changes in very small elements of a system (i.e., a small sequence of a protein, the behavior of a single neuron within a group of 10,000) can have a drastic effect on the behavior of the entire system. As a result, to really study the problem, you have to aquire massive amounts of data. For example, in our lab we routinely collect data from 64 channels of 16-bit data (monitoring neuron firing in culture) at 1KHz, in addition, we're simultaneously taking calcium imaging video at 100fps at 256x256 (at 256 colors). This results in about 200 MB of data gathered every second. Considering we run tests for over 10 minutes, just aquiring and storing this data is a challenge, but finding useful methods to analyze it is even more difficult. Its refreshing to see texts being written on how to bridge the gap between comp. sci. and biology. I've been working in the area for about 4 years now, and its really great to see the field growing and getting more mainstream attention.

Re:The challenge of Bioinformatics

[babbage]

Ok, so you can work out about how much data is coming out of that machine. Now assume that the lab in question has several such machines, and that labs all over the world are churning out this degree of output, and maybe your lab needs to keep a local copy of all the relevant data. Go ahead & make up your own numbers if you'd liike, but keep in mind that this is a huge field these days, so there are probably hundreds or thousands of such groups working on it all, and they're churning out, by your math, 7mb per second per machine per lab.

Now take a step in a different direction, and realize that we don't know what *any* of this stuff means (much less than 1% of it, at a rough estimate). We've got a completed genome project that has produced another mountain of mostly undecoded data.

Or to go to the central issue, we understand that DNA translates trivially to RNA, then to chains of amino acids that fold up into balls of protein, with secondary, tertiary, (etc) levels of structure. Largely this is determined by how the chemical bonds between each amino acid twist together, and how disparate segments of the chain come close together or far apart. And the effect of this protein chain biologically is determined by which segments of the chain end up at which parts of the knot: the same sequence of amino acids can be neutral or active depending on whether it's near the surface, for example. Finally, go a step further and realize that all these proteins in the body are in a constant state of flux, constantly changing each other, catalyzing each other, restricting each other, and so on. The number of active variables very quickly hits a point that becomes incalculable, and we're down to a new version of the travelling salesman problem, which no contemporary computer system can even dent, nevermind solve.

Laugh at the stream of data if you want to, but keep in mind that it's not like we're just talking about a piece of network hardware that needs to be able to shuffle this much data around more or less blindly. Rather, any & all of it could be biologically relevant in any given context, and so each bit of that data stream has to be scrutinized, often more than once in different contexts. It is, simply, a *huge* amount of computational work.

try it

[oo7tushar]

As a CS person about to switch into Biology I found the reviewed book interesting. Even if you have a good handle on Perl and Biology you will find certain elements in the book intruguing.
On a personal experience side note, Perl does seem to handle genetics problems with quite a bit of ease. The ease seems to stem from Perl's obfuscation. (it also seems to confuse my Biology profs quite a bit since my answers are legitimate answers on the exams)

And

[NMerriam]

They also don't mention it's a great introduction to books for those familiar with perl, biology, and bioinformatics, but not the written word!...

As a biologist...

[Ubi_NL]

We were just discussing programming languages recently.
We use so-called micro-arrays frequently, which yield so much information it is not possible to go through all that manually (on average you get about 10.000 "genes" that show changes in expression, after which you have to check the intertesting ones for functionality).
At the moment we can either mess around with MS excel or buy some serious software which is so incredibly expensive only companies can afford it.
Still I doubt whether Perl should be the language of choice due to it tending to be "write-only code". Maybe this book will change my mind though.

Re:As a biologist...

[mfarah]

Still I doubt whether Perl should be the language of choice due to it tending to be "write-only code". Maybe this book will change my mind though.

FWIW, in my personal experience, I find Perl to lend itself to some very obscure code, worthy of the IOCCC [*] just as easily to extremely clear code - the latter, though, requires a disciplined programmer and some effort (not much, though) directed to that goal.

[*]: so, when will the first International Obfuscated Perl Code Contest will come? Perl poetry is getting kinda old.

Re:As a biologist...

[Jeremy Erwin]

biopython is fairly primitive (but their goals are laudable.) bioperl is more advanced, but microarray modules are still on the todo list. I've found bioperl modules to be fairly easy to write code around.

Coming Soon, "ML for Philosophers"

[Anonymous Coward]

This could spawn a great trend in cross-area programming books. Ada for Historians? Smalltalk for Hairdressers?

Alternative book

[Theodore Logan]

Instead of just whining, I should really recommend an alternative book for people who (like myself) have their background in CS.

Algorithms on Strings, Trees, and Sequences: Computer Science and Computational Biology by Dan Gusfield is usually very liked for people with a computer science background. And it's not only of use if you want to go into bioinformatics: most algorithms on strings are usable in everyday coding too.

More for your library

[chundercanada]

I just spend a couple of days trying to choose a few books in this area. My interest was as a computer guy needing to get filled in on the bio side of things. Here are the books I ended up ordering:

Human Molecular Genetics 2: Looks to be a great primer on all the biology background.

Bioinformatics: A Practical Guide...: This book is a detailed tour of the online databases and existing tools for analysis of genes and proteins.

Algorithms on Strings, Trees and Sequences: This is a book for real computer science types who want to do high-performance implementations of new tools.

Universities going this way

[Marx_Mrvelous]

At Purdue University, there is a class specifically meant for CS majors and Biology majors, to address this same issue. I wonder if they use this book in the class.

For those interested in Biology and Perl

[SloppyElvis]

The BioPerl project (http://bio.perl.org/) has been going on for some time.

In their own words they are, "The Bioperl Project is an international association of developers of open source Perl tools for bioinformatics, genomics and life science research."

There bioinformatitians can find a wealth of useful Perl scripts and modules to use in their efforts.

Yet another example of an open source initiative serving the needs of science!

Re:For those interested in Biology and Perl

[glwtta]

yep, BioPerl is good, I've used it extensively in the past.

Also check out BioJava - same concept (probably mostly the same people), for those times when you can't use the bioinformatics language of choice (e.g. if you want someone to maintain your code at some point ;) )

Not all biologists are doing genomics!

[RevAaron]

This book seems to equate biology with genomics/bioinformatics, when that is simply not the case. There are a fair amount of scientists in the general school of biology who *are not* bioinformaticians. As a person who does computational ecology, this book really wouldn't help me- and I am a biologist. Sure, DNA is swell, but it won't tell us about the complex interactions between a number of populations of organisms and the environment in which they live; it doesn't provide strategies and formulas (or references to perl modules?) that *other* kinds of biologists use. ...sigh.

Re:Not all biologists are doing genomics!

[SloppyElvis]

From Gray's Lab Dictionary on medical sciences:

Bioinformatics: The use of computers in solving information problems in the life sciences.

This says nothing about bioinformatics being used solely for genomics, though I hear your gripe, as many think of the two as the same. No doubt, this author has made the same assumption. I speculate it has something to do with money, since genomics are a "hot topic". The point is, you may be a bioinformatician and not even know it.

Re:Not all biologists are doing genomics!

[jfrumkin]

Agreed - I happen to work on a phylogenetic project, which heavily uses PERL and other Open Source technologies. I believe O'Reilly's other book, "Developing Bioinformatics Skills" makes some mention of phylogeny, but it is rather limited, to be sure.

On the other hand, my guess is most of the big money is in genomics at this point, so I can understand the heavy emphasis in that area at this time. Perhaps the increased attention given to this area will allow for increased interest in other biology-related arenas....

Re:Not all biologists are doing genomics!

[RevAaron]

Oh yes, I am a bioinformatician, and I know it. However, with this big trend with many dollars behind it, any bioinformatics worth mentions outside of real biology has to do with computational genomics or molecular biology. It's like telling people you're an anarchist; they think you mean you're one of those 14 year old kids who smoke their dad's ciagrettes when he's not home and have the circle-A patch on their pack-packs.

Re:Not all biologists are doing genomics!

[RevAaron]

You'd think they'd have a little more on phylogeny, since so much of that is comparisons of different proteins among species. But yeah, not much money in "real" science. ;)

Perl Bioinformatics for AI Neuroscience

[Mentifex]

Anyone who wanders into the use of Perl for bioinformatics ought to consider the ultimate plunge into the use of Perl for neuroscientific Artificial Intelligence. Since v.t.y. Mentifex here has been coding the AI Brain-Mind in JavaScript for tutorial purposes and also in Forth for Intelligent Mind Roboinformatics, the switch-over to Perl is advancing so slowly that I must first promulgate some candidate AI module proposals for inclusion among the object-oriented Perl 5 Module List.

The Comprehensive Perl Archive Network (CPAN) contains some not-yet-implemented, suggested AI module namespaces for those who read the Beginning Perl book reviewed here on SlashDot and who may then wish to do some really exciting, wave-of-the-future Perl neuroscience theory and practice work.

Maybe there's a reason...

[pongo000]

If I have a complaint with the book, in fact, it's that Tisdall doesn't go any further: everything is good, but it ends too soon. Seemingly important topics such as OO programming...are mentioned only in passing, under "further topics" in the last chapter.

Mabye that's because Perl's OO support is an extremely kludged-together ugly beast that's undergoing a much-needed facelift in Perl6.

The author actually does the world a favor by not mentioning Perl and OO in the same sentence.

Re:For fun or for work?

[babbage]

I would say that it's a crash course in two linked fields, targeted at an audience of people lookiing for bioinformatics work who might be familiar with one or the other of these fields, but need to get up to speed on the other one quicky.

And I *do* think it does a good job at this -- I'm a Perl hacker that hasn't taken a biology class since my freshman year of high school (ten years ago, oy vey), but the genomics & proteomics covered in this book did bring me up to speed to the point where I understand the terminology and have a decent grasp of the computational issues involved in doing work in this field, as well as some techniques that can be appled to these issues. After reading this book, I read The Cartoon Guide to Genetics by Larry Gonick -- it's a better introduction to the field than you might expect from a title like that -- and felt satisfied that I had already been exposed to 95% of the material in there, with a significant portion of that coming from this book (and O'Reilly's other bioinformatics book, and skimming over web sites).

No, it isn't a masters degree by a long shot, but it's a solid start at learning the field, and if I choose to follow it that far. And it is enough of a crash course to land you a job, if you feel comfortable with the Perl stuff. You might not be expected to understand all the subtleties of DNA and proteins on your first day on the job, but you will at least come in knowing what your colleagues are talking about, and you'll be able to begin workiing with it immediately.

Give it a chance, it's a good book for starting out with. Yes, there's more to learn -- I understand that James Tisdall is doing a followup that'll be more like a "Perl-Bioinformatics Cookbook" for more advanced users, and there are of course other books out there besides the O'Reilly stuff -- but it's a worthwhile & solid start.

Re: IOPCC

[sab39]

[[ so, when will the first International Obfuscated Perl Code Contest will come? Perl poetry is getting kinda old. ]]

<tongue-in-cheek>Wouldn't that be rather like having a International Wet Water Contest?</tongue-in-cheek>

Stuart.

Re:out of touch

[pclminion]

The problem with doing OO is it requires you to understand software engineering. Biologists are probably more interested in crunching their numbers than in good OO design. Or am I wrong?

If biologists are learning how to do OO, maybe I should get out my old chemistry set and try some gene-splicing :)

What's with that last couple of sentences? Did you fall out of bed this morning?

Why a scripting language?

[pclminion]

Why do scientists gravitate to these scripting languages? My guess is that scripting languages avoid several common things that non-programmers usually have a hard time with:

• Variable declarations
• Memory allocation
• Type conversion

Unless you're using Python in which case you have to do type conversion sometimes...

Really, why scripting languages? It seems like some of these scientists are getting really good at it, using OO and everything. Why not switch over to a native language like C++ (which isn't actually that hideous if you avoid all the stupid features) and do the calculations 50 times faster?

Anyone have input?

Re:Why a scripting language?

[babbage]

Why do scientists gravitate to these scripting languages?

For the same reasons that people gravitated to them for internet programming: there is so much ad hoc work do be done that it isn't worth the effort to work "that close to the metal". Perl's text analysis capabilities are so sophistocated that it would be hard to match them with custom written C code -- and if you did manage to pull it off without getting ensnared in infuriating memory leaks and so on, a well designed system will end up approaching Perl anyway. Yeah, Python is well suited towards modularizing systems and reworking bottleneck components in something like C, but Python just isn't as slick at text analysis as Perl is, and this kind of genetic/proteomic work is essentially a text analysis problem.

I mean, look at it the other way around -- Perl isn't actually that hiideous if you avoid all the stupid features, and you can do the development 50 times faster. If it really runs that slowly -- and usually the execution time won't be a problem -- then sure, redo parts in C (or XS), but 99% of the time that really doesn't help very much.

Re:Why a scripting language?

[Phillip2]

Because bioinformatics grew out of a bunch of people writing small tools, to do small things. After a while we had a whole load of small tools doing small things, and we wanted to stick them together. So we write small perl scripts to tie them together. Perl is very good at this. Unfortunately it tends to also hide the fact that if we had written some decent libraries in the first place, we wouldn't have need to stick bits together with perl.

Bioinformatics is in a mess, and its slowly crawling out of it. To be honest, I think that the last thing that we need is more biologists with a working knowledge of perl.

"It seems like some of these scientists are getting really good at it, using OO and everything. "

Really, what actually using OO?

The reality is that if you can work out how a cell works, its easy to write computer programs. The problem is that too many people feel that because its easy to write programs, its also easy to write programs well. Which is why we are in such a mess now.

Phil

Re:Why a scripting language?

[scottcain]

It's easy to explain really: text manipulation. Bioinformatics is really about moving text around. What are DNA and protein sequences? Text. What are the reports generated by the plethora of analysis programs? Text. And Perl has outstanding and easy to use text manipulation tools. Add to that CPAN and BioPerl, and you have the makings of excellent Bioinformatics tools.

Re:Why a scripting language?

[jslag]

scripting languages avoid several common things that non-programmers usually have a hard time with:

* Variable declarations

Actually, most perl programs more than a few lines long (hopefully) use strict; thus requiring variable declarations.

* Memory allocation

Seems like plenty of programmers have trouble with this as well, based on the number of memory leaks out in the wild.

Really, why scripting languages?
Why not? Hardware is fast and cheap compared to programmer time, so slightly slower (but written!) programs are often better than super-optimized programs that are only half done.

Scripting languages aren't necessarily slower, anyhow. Perl programs, for example, tend to do all their heavy lifting in libraries, with performance-critical parts coded in C. If you're into benchmarks, you can dig some up showing perl outpacing java and c++ at various text-processing tasks.

Re:Why a scripting language?

[Mike Buddha]

Hmm. I agree with you that Perl is an excellent choice for this task, but I'm wondering if a lexical analyzer generator (like flex or lex) might make a better choice even than Perl? I suppose it would all matter on what exactly was being recognized.

Fortran

[wiredog]

The reason engineers, and physicists, use Fortran is that, until recently, it was the best number crunching language around. C and C++ didn't get math libraries that could compete with Fortran until a couple of years ago, and no one with any sense is going to use an interpereted language for serious number crunching.

Re:out of touch

[Phillip2]

"The problem with doing OO is it requires you to understand software engineering. Biologists are
probably more interested in crunching their numbers than in good OO design. Or am I wrong?"

It depends what you want to do. If you are writing a few hundred lines of code, then good software enginnering is not that important. Nowadays biologists and bioinformaticts are writing very large code bases, to do very complex tasks. Under these circumstances software engineering becomes necessary.

Programming is not my area of research. Nor is enginnering, or architecture. It is however my plumbers spanner. Of course us biologists are learning how to do, and in many cases how to do it very well.

Phil

Perl and Bioinformatics

[fasta]

I would like to answer several questions that were raised in this discussion.

(1) How does a CS person learn biology? I recommend "Recombinant DNA, A short Course", as an accessible (Scientific American style) introduction to the cloning breakthroughs and discoveries that lead to genome science.

(2) How does a CS person learn "Bioinformatcs"? I strongly recommend "Bioinformatics - Sequence and Genome Analysis" by David Mount as an accessible and extremely comprehensive survey of current approaches in Biological Sequence Analysis.

(3) Why do Biologists use Perl? Much of the information Biologists want is on the WWW, and Perl's LWP makes it extremely easy to get it. We don't use Perl for sophisticated text analysis (similarity searching, motif searching, etc) because the algorithms that are appropriate are typically not exact (or even regular expression) matches. But it's difficult to beat Perl for getting stuff off the WWW.

(4) Why do Biologists use Flat files? Several reasons - (a) the most useful information is sequence information, and it can be read much more quickly out of a flatfile (esp. one that is memory mapped) than a DB; (b) flat files solve some versioning problems that DB's make very complex and slow. (c) Most data providers only provide flatfiles. This will change, however, over the next 2 - 3 years, mySQL and postgresQL are moving into biology labs.

It is very exciting that Bioinformatics has high visibility now, and many people with CS background are considering bioinformatics problems. Unfortunately, many of the introductory books on bioinformatics (particularly the O'Reilly books) do not adequately present the substantial foundations of bioinformatics that have been build over the past 15 - 20 years, and some newcomers are mislead into believing there are simple problems looking for a few good programmers. Most of the simple problems have been solved; many of the complicated problems are challenging not because we do not know enough CS, but because we do not know enough biology.

Re:out of touch

[pclminion]

Have the bioinformaticists (now there's a word) ever thought of outsourcing it? Putting together all the experience gained over the years of scripting, pasting, gluing, etc and writing up a nice requirements document, and handing it off to some software people to write a system?

Or are bioinformaticists so paranoid about others using their techniques that they don't want such a tool to be available commercially? I bet there are a ton of "trade secrets" in some of that Perl code...

Another language for bioinformatics

[Jon Howard]

Since I'm a Lisp fiend: while we're on the subject of programming for bioinformatics, I'd like to point out that Allegro Common Lisp has been used by a few folks in the field. Here are two links:

Pangea Systems Inc. (now DoubleTwist) for EcoCyc.

MDL Information Systems to design new drugs.

Re:Another language for bioinformatics

[Jonathan]

Well, at least in the case of Pangea, that's because Larry Hunter was in charge, and he came of age in the Symbolics LISP workstation era. Most bioinformaticians tend to be a bit younger and therefore missed out on the whole LISP-as-mainstream-tool era. Whether this was to their benefit or loss is of course subjective.

PubMed Books online

[NullSpaceKid]

A selection of possibly relevant books (_Introduction to Genetic Analysis_, Molecular Cell Biology_, etc) can be found at: www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=Books NSK

Re:Neuroscience is bioinformatics too!

[RevAaron]

I use Squeak Smalltalk (see link below) for the same reason you use Java. But luckily, in my research, the software doesn't need to be really maintained, I just need to use it personally to visualize and generate graphs and HTML.

Re:God, this book is hard

[KidSock]

It is a very good book but it is *hard* for people without strong math background

Skip the proofs (I believe you Gusfield :~). You don't need to understand the proofs. I think he does a rather good job at propviding a high level description for each algorithm. And you can complement this other material from Algo books. I do NOT have a strong math backgroud but I managed to understand what was being discussed enough to implement the algorithms and understand why you would use one or a permutation of one to solve a particular problem. It's hard read but not too hard.

Re:out of touch

[Phillip2]

Outsourcing is no magic wand. Some one has still
got to write the software, and some one has still got to do the research. In the case of all research requirements analysis is extremely hard. If you knew what you required from the software in advance, then it probably wouldn't be research!

But yes there are lots of computer scientists in bioinformatics these days. Its a hybrid discipline. Even though my origins are as a bench biologists I consider myself to be a computer scientist at least in part these days, as well as a programmer.

Research code is often a little flaky. We are not writing finished products to sell to people. We are writing code to do research!

Phil