Slashdot Mirror
Creating a Computational Linguistics College Degree?
$random_var asks: "I am an undergraduate student currently studying Bioengineering. However, I am growing more and more interested in programming and linguistics, which leads me to think that I should define my own major, Computational Linguistics [mit.edu], which google defines as 'a field concerned with the processing of natural language by computers.' When I present my proposal for this degree to the school's advising staff, I would like to have a complete list of all of the topics this major should cover. Having only little experience with computer science and engineering, I'm not sure what parts of that field I should include. Beyond the basic lower-division courses, what specific fields of computer science do you think should be emphasized in a practical undergraduate study of Computational Linguistics?"
you suceed it! schnits.
Computational Linguistics, or Natural Language Processing (NLP) as it was called for me, is one of the many areas that traditional Computer Science is exploring, in addition to things like biology (bioinformatics), etc.
I'd say the first two years in the major need to be very similar to the first two years in the Computer Science curriculum and last two in the humanities/linguistic area. The reason I say this is because a lot of the math, basic computer science, etc that is needed in the field will be the same as the core Comp Sci/engineering classes. However, the first two years in the humanities are very generic, and you specialize a lot later. So build up the core of Comp Sci and linguistical study early.
Then, get deeper. A lot of the NLP work is being done in Maching Learning/Data Mining classes. Make sure you take those, we had a whole class called Textual Data Mining at my university. Take algorithm design and some of the common advanced Comp Sci classes too, since a lot of the techniques are very advanced, being developed, cutting edge and will require research. Take advanced statistics classes too, much of the field is built on statistics.
NLP is a very interesting area, but I don't know if it deserves its own major yet. I would advise majoring in comp sci, with a concentration in Machine Learning/Data Mining through your technical electives. And a minor, or perhaps second major in Linguistics. I'd say minor because then you could take only the classes relevant to the field, instead of all the other stuff related to a humanities major that you may not want.
Anyway, that's my take. I did a bunch of NLP research, even getting work published as an undergrad, and I was a Computer Engineering major. The field is as such that it's so new and emerging that not much formal linguistics study is required right now, if you are a native english speaker you are probably good. But, it doesn't hurt to get a more formal background in it, that's why I suggested the minor.
-"Those who fought today will die tommorow."-
The only thing as a CSE student that pops out at me is Artificial Intelligence-related courses. Though if you ask me, the best way learn how to do something is by doing it (assuming you have taken the entry-level courses like C, etc). I'm doing an internship (co-op) right now that will help me in at least three classes I have to take still.
Also, just thinking and re-reading, some language classes would be handy. Though the elementary school english about nouns/verbs/etc will help, you'll need more if it's to process natural language.
Another thing is some data structures classes. One project in one of my data structures classes dealt with the computer making up a sentance based on a template (the "grammar") and the stuff to plug into the template (the dictionary). You're looking at the exact opposite but with a difficult to define "grammar" and a huge dictionary.
Plus anything that would help you in implimentation in software such as database usage, etc.
Peace, Chris
http://www.cstit.cl.cam.ac.uk/lms/pages/current/ho mepages/index.html.
Computer Speech, Text and Internet Technology has to be the lamest course name in history but the course itself is very good with both computation and linguistics aspects, as you would expect from Cambridge...
Not as an undergrad. You're still learning, still ought to be figurin g out what you want to do. Get a good, well rounded CS education. Maybe take a few courses in linguistics if the subject interests you. Then specialize in computational linguistics in grad school if you decide its really what you want to do.
Basicly, undergrad is to learn the field. A masters is to specialize in one domain. A doctorate is to research a single problem in as much depth as possible. They're in that order for good reason- it leaves you plenty of room to change your mind and interest as you grow older, while keeping you marketable in both industry and academia. Don't specialize too early, or you may regret it later.
I still have more fans than freaks. WTF is wrong with you people?
To me, this sounds like a good idea for your specialization for a masters degree (in computer science), but I don't really think it justifies it's own undergraduate degree. If that's what you're really interested in, I recommend majoring in CS as an undergrad, perhaps with a minor in english, and then submitting this your linguistics stuff as your masters thesis when you get there.
Of course, there are a lot of classes to steer you in that direction. From the computer science field, you have to take compiler structure. That may not sound like a relevant class, but it is. Learning to write good lexers and parsers is going to be vital, and that is the class where you will learn them. Also, seeing how a high-level programming language is parsed can give you a lot of insight into how natural language will be parsed.
Of course, an AI class would be helpful, too. And machine learning, but that's not always offered at the undergraduate level.
Outside CS, I would also recommend taking at least one foreign language to give you a little more basis in how syntax and grammar vary across languages. A non-latin based language, like Japanese, might help.
i don't believe that your chosen major is sufficiently broad to be useful. i also believe that, unless you intend to immediately pursue a graduate degree, you're likely to find it very hard to explain to potential employers what, exactly, it is that your degree says you can do. i don't know that anyone can develop the degree of maturity in the fairly wide set of disciplines that you'll have to master (general comp. sci., intro a.i., "soft" a.i., statistics, linguistics, and quite possibly some signal processing) to really succeed at this major as an undergrad. i suggest instead that you do a major in comp. sci. (or e.e., or linguistics) with a concentration in computational linguistics and then pursue further study at the graduate level.
It involves taking a canon of text (for example, the complete works of shakespeare, or every example of written english you can get your hands on, or every example of transcripts of spoken english you can get your hands on) and subjecting them to a statistical analysis of what chucks (that is, words, phrases, what have you) are likely to follow which other chunks.
while the outputs tend to make little sense, it is "interesting" to see what kinds of "statistically probably" examples of language a computer can make based on the training (input) you've given it
But I am a Master Debator
The courses you want are: Intro to AI; NLP; Linguistics; Machine Learning; Discourse Processing; Dialogue Processing; possibly Speech Recognition or Machine Translation.
For more ideas, see the list of courses offered at CMU's Language Technologies Institute, and pick the introductory ones.
To all those saying "why do this in undergrad?", here's why: because a Bachelors degree doesn't matter a whit. You could have a BA in English, and go on to get a job in software engineering; you could have a BS in Chemistry, and go on to work as a journalist. The point is that you put together a major that shows you've got the smarts. Doing an NLP major is not premature overspecialization, but rather a demonstration of interdisciplinary skills and interests.
Something to keep in mind whenever you want to have a group of people approve of something is to already know that they will. Start talking to the advising staff. In particular start talking to professors. They would need to approve, and for a custom major you'll likely need an adviser. Who on the faculty do you think would be appropriate? Approach them. If the first one is a pompous ass (as some academicians are), then find another. And another. A good prof will recognize you as a potential grad student, and therefore a potential resource to be exploited as is their job (I'm only half joking), and therefore a resource to be developed.
Maybe you don't need a new major, just some undergraduate independent study. My first research publication started as a special project after a class (it was a cogsci paper). It showed up in print at my school's library the day I completed my last undergrad final in compsci. By that time I was already enrolled in grad school where you really do make your own personal major. Either way it is the same process.
"This mission is too important to allow you to jeopardize it." -- HAL
That's the first thing I thought when reading this question. This seems way too concentrated for a bachelor's degree. I'd say either doing computer science or computer engineering would be the way to go, followed by grad school. Find the right school with the right research group and/or professor and make that your thesis research. I'm not too familiar with the topic of NLP, but if there aren't any good books written on the topic go for your PhD and turn your dissertation into a book on the subject (a couple people from my graduate research group did that).
My main advice though is this: don't get too specialized for your BS. If you have lofty goals for research (as I'm assuming you do), go to grad school (as I said above). If you don't have much experience with programming or engineering, make your major CS or computer engineering. With computer engineering you'll get the programming background along with a solid understanding of engineering principles, and you'll more than likely be able to take advanced CS classes like AI or something. Oh, and some sort of linguisitcs would seem (to me) to be necessary, as the parent stated (somewhat).
notes: I admit that the parent obviously know way more than me on this subject. I'm just adding my own $0.02. I'm biased towards the engineering (I do have an MS in EE). The best programmers I've ever met were not CS students, they were physicists and engineers (I'm not a great programmer, I barely know C++).
"Physics is to math what sex is to masturbation." - Richard Feynman
Shouldn't it be Linguistic Computation?
i think you're a bit off the mark. normally as you say someone
would get a cs degree, and do additional work in comp ling, then
apply to grad school with an specific bias towards that kind
of work.
however, if you really know thats what you want to do, there's alot
to be gained by putting together an interdiscplenary degree
based around your interests and getting it approved. it would
mean you would be able to take more courses in linguistics and
philosophy of language, or statistical methods, or logic, or
whatever you think you need and apply them to your degree. it means
as you suggest more work, because you probably have to develop
a basic grasp of cs as well as all those other things. but if you're
sufficiently motivated, why not? it may also help to get approval
to take grad level courses in the areas you think are necessary,
given the general paucity of undergrad level materials at that
degree of specialization.
however, if it turns out that that isn't what you end up wanting
to do forever, you have kind of painted yourself into a corner.
i also disagree about the job issue. if you're willing to accept
a narrowed field, i can think of a few companies who would love
to have comp. ling. people on staff. i've even seen job advertisements.
this person is setting a higher bar for themselves and committing to
it.
Make-a-Degree programs are for underwater basket weaving. Computational linguistics is an advanced topic. You won't touch it, nor should you, until a graduate program. You don't know enough about computer science to do anything advanced yet. Get yourself a CS degree and take some linguistic, anthropology, and psychology electives, then apply to a graduate program and do CL as your thesis. Read some CL papers and apply to the schools that publish in the CL journals. University of Washington has a program in CL consider applying there, or at least read their prereqs.
I think that it wouldn't be terribly difficult to move from a computational linguistics job to a software engineering job. As long as you get a fair amount of training in software design (what do you think you'll be doing with a PhD in computational linguistics? Creating databases by hand for a grad student?) and general theory of computation, you should be able to transfer most of your skills.
The only issue is language specialization--if you only know Prolog very well, you might have trouble switching to Java or C. The solution, of course, is to make a systems programming language do what you need for computational linguistics--it might be clunky sometimes, but it offers much better integration.
I had a prof for discrete math that work heavily on linguistice reseach... check out some of his papers and contact him for more info.
http://www.csc.ncsu.edu/faculty/rodman/
"When I die, I want to go quietly, like my grandfather, in his sleep... not screaming, like the passengers in his car."
Why hijack your future in something that you might only have a passing interest in at the undergraduate level? If you are going to switch your majors, switch to computer science and then specialize in whatever field you wish. Choosing a very specialized major can hinder your job search later on.
I'm in my last year before becoming a Chemical Engineer, and I'm specializing in polymers but I have taken an interest in CS also (mostly as a hobby). I'm ok about learning a programming language by myself... but some pointers as to which language is best wouldn't hurt me.
What do scientists or engineers use when they want to simulate a reactor? How about design of equipment? Does everyone use Excel and Macros/Solver? Or Aspen???
Today I handed in a paper which did the design and pricing of a distillation column separating a water-acetic acid mixture. I refused to do it in Fortran, didn't know if OO2 has something like 'solver' in it, don't know much about Matlab (and don't want to be tied to working only at school in order to use it, and I don't like the idea of 'stealing' it), so I turned to Excel.
The 'program' turned out to work great, but I still feel like it's a bit n00b-ish. Any fellow ChemEngs that wouldn't mind sharing $0.02?
The University of Regensburg, Germany (and some others in .de, e.g. Konstanz) offer "Information Science" which may be pretty close to what you want. It contains scanning and parsing, information retrieval, information analysis on syntactical, semantical and pragmatical level as well as associated courses. I'm writing my PhD there right now and don't regret it.
Feel free to mail me for more information!
Speaking as the director of a new professional MA program in computational linguistics at the University of Washington, I'd say the answer depends on what you plan to do next, and what specific computational linguistics courses are available there at MIT.
Successful computational linguists have strong programming skills, a deep understanding of algorithms and/or systems architecture, and a linguist's perspective on language, linguistic patterns and linguistic structures. An understanding of machine learning and the probabilistic methods typically employed there is important, too.
I think the best preparation you can get as an undergraduate to go into this field is a double major in computer science and linguistics, where you choose any offerings available in computational linguistics as elective courses. You might also investigate the offerings of your local library/information school as a source of possible electives (think information retrieval). Likewise, if speech processing interests you, check out offerings in signal processing in the EE department, and be sure to take the prerequisite statistics courses. If a double major is not feasible, I think majoring in one field and minoring in the other would be a suitable alternative. Designing a custom major only makes sense if you can identify required courses in one major or the other as clearly not useful to you --- which is really only something you can do in consultation with an advisor.
What I doubt you'll find at the undergraduate level (and indeed at the graduate level at most universities) is in-depth courses specifically investigating computational linguistics. I'd encourage you to take whatever courses are available (including graduate courses once you have sufficient background), but you'll probably find that a career in computational linguistics requires further training, either a graduate degree or on-the-job training.
If you plan to go on for a graduate degree, there should be plenty of time to do the in-depth courses then. Your undergraduate degree should serve to give you the foundation you need on both sides to go on in this interdisciplinary field, while giving you enough of a taste of the intersection to be sure it's what you want to do!
take tons of it. learn first hand why the rules of language are far too complex to machine...
The problem with specializing too early is that students risk getting bored and wanting to change to a different degree. You said it yourself, you're doing bioengineering and you want to do programming. Doing a solid CS degree as mentioned earlier then specializing later on either by choosing final year electives or doing masters with an linguistics focus would be more flexible.
Your job prospects with a degree like this would probably be a lot worse than most CS graduates. It would be very hard for you to get into other CS fields if you did not do any of the more advaced CS subjects in your final years. I have just recently finished my CS degree with a major in computer security and though computer security is a lot bigger (both in sub-fields and in practical applications) than computational linguistics, I still would not recommend a separate degree for it.
Again, a CS major would be nice but now a whole degree.
I recommend looking for interships as soon as possible that relate to the field. If you're interested in Japan at all (even if you don't know Japanese), there are incredible opportunities at the companies NTT and ATR through the Japanese government (look up JETRO internships - these are often set up through your university).
Personally I have taken up an internship in Japan with NTT for one year and am currently working actively on acoustic modeling for speech recognition. While I took many courses in linguistics and a decent amount of computer science courses, I almost completely left out college level math and statistics. In retrospect I see that statistics is one of the most important subjects to study before doing actual work in the field - linguistics is used quite little in practice (to my dismay). I took CS courses because of a general interest in computers, and fortunately it's paying off now.
Another poster mentioned it may be too early to decide what you really want to do, but if you have an interest in computational linguistics, throw in a couple basic CS and statistics courses, and make sure to have done at least calculus, and take at least an intro to ling. course. I believe that's pretty safe and if you can land an internship within the next one or two years you'll really know how you feel about it all.
It turns out I love nearly all of it (despite having to review and learn some things I left out in university), and after graduating I'd like to continue this work. I didn't think I had the drive to go for a master's or doctorate, but now my interest is piqued.
So anyways, you're in the right place for what you want to do. My university is just starting up its computational linguistics major, so there's hope for you. I say go for it, but give yourself a little bit of time before taking this idea to full fruition. Let people know what you're thinking, take safe classes (like the ones I mentioned before), try hard for a good internship in the field, and after spending some time there propose your ideas in full.
Do not do it. Bioengineering is a way better degree, in fact, ChemE is even better.. get a CS minor or double major. Study linear algebra and dynamical systems. You can always do computational linguistics in grad school (and it is dying field anyway so far)... Anyway, how do you think you compute all that stuff anyway? Matrices, Markov chains, and other mathemtical algorithms. GET AS MUCH Math
training as you possibily can. Linear Algebra is essential.
You are really looking at a subset of the area of man-computer interfaces, which is itself a subset of man-machine interfaces which ranges from knife handles to the current experimental devices for translating neuron firing patterns into machine control commands. You can study computer language compilers and natural language structures until the cows come home and all you will achieve is some understanding of why we have not progressed much beyond mouse-menu systems in the last 30 years of computer use. (Granted, computers are still very primitive...) If you really want to accomplish something, study the nature of how computers and people think and what the differences are in depth. Maybe then you can find new ways to cross a vast barrier. For example:
Computers think like this (binary logic) - If A=true and B=true then C=true.
People think (subconsciously) more like this (fuzzy probability risk logic) - Source Joe (90% reliability) indicates A probably (80% confidence) true and source Tom (70% reliability) indicates B probably (90% confidence) true while experience logs indicate A=true&B=true implies C=true (90% test cases), therefore C=true (75% confidence level with maximum risk factor $5).
When I went back to school a few years ago for my Masters, I described the levels of degrees like this:
Undergrad: Do XYZ.
Masters: Find out something interesting about XYZ.
Ph.D.: Based on XYZ, invent ABC.
Yes, I found out interesting things.
...laura, B.Sc, M.A.Sc.
I worked in the field for quite a while, with a formal background in physics and a personal background in linguistics, and I agree with the parent. Major in CS, emphasizing the mathematical side over the technological side (e.g., algorithms vs. memorizing C++ trivia). Minor in linguistics (not "humanities", but real linguistics, which can be quite hard core.) Don't bother trying to officially create a "major". If you do what I (and the parent) suggest, you can always claim to have gotten your B.S. in computational linguistics and your transcripts will back you up.
You can follow an officially published "medieval literature" track in the "French Department" from the "School of Fine Arts and Humanities", take every required course, follow the rules for that track to the letter and graduate. Will your diploma say "Bachelor of Arts in Medieval French Literature"? No, it will say something like, "...successfully completed the requirements for the degree of Bachelor of Arts from the School of Fine Arts and Humanities", (though if you graduate from Harvard College it may say it in Latin.) So what degree do you have? You have a B.A. In what? Pretty much whatever you want to call it as long as it describes the classes on your transcript.
So, you want a bachelor's in Computational Linguistics. Get a BSCS, minor in linguistics, and that's what you'll have. (And it won't be lying. That's what you really *will* have.)
"Those who have never entered upon scientific pursuits know not a tithe of the poetry by which they are surrounded."
I minor'ed in NLP for my Masters at edinburgh this year, the statistical stuff is where its at, specifically hidden markov modelling of speech units. Its truly remarkable what you can do with only TNT (trigrams n tags) which as the name suggests is a trigram tagger.
Stay the hell away from formal semantic modelling..its horrible.
You can access all edinburghs course informations via http://inf.ed.ac.uk/ if you want to see the kinds of things covered in each course.
I did Introduction to Computational Linguistics, Natural Language and Speech System Design and Semantics and Pragmatics of Natural Language Programming.
I have discovered a truly remarkable sig which this post is too small to contain.
I have similar education in CS and CL, worked in CL and AI/OR. IMHO, and as mentioned in others posts you're missing the more formal subjects. Scanning, parsing, translation, logic specialized to formal semantics, syntax and discourse should be in any basic Computational Linguistics (masters) course as well.
Some of the worst programmers I've ever met *were* CS students, though. :)