Advice for Building a Multi-Platform Lyrics Database?

AntonOnymous,Cowherd asks: "I am in the process of designing an application for general public use. The application will allow end users to search and display a large collection of songs (both lyrics and tunes) with annotations, all in text format. The intent is for this application to run cross-platform (Linux, Windows, Mac, and whatever else), so I want to avoid platform-specific binaries as much as possible. I also believe that the program should be Open Source. The end users will not necessarily be computer experts, so I want to avoid as much additional setup on their computers as possible. The application (data and program) will all be stored on a CD or DVD, and it should be able to be run locally. The most important part of this application is the data, not the program, so the guts of it should be fairly simple with a decent user interface. Does anyone have any suggestions as to general approach to setting this up, or have any pointers to existing open source programs which already perform a similar function?" "One way to implement this would be to set up each song (with lyrics, tune, and annotations) as a single record in a database. I would like to avoid the inherent security issues and overhead of setting up and running a database on a user's computer.

Another possibility, which is fairly appealing, is to use a Web Browser to provide the user interface, and to use Open Source text indexing/searching programs (such as Lucene or Egothor) as the engine. It is probably safe to assume that most users have a Browser. However, most users probably would not have a web-server (even a local one) on their computer, and going by the principle of as little messing around with the user's computer as possible, I would like to avoid having to set one up, even a local one."

Comments

Internationalization

[CRCulver]

Whatever you do, please store everything in UTF-8 encoding, since most of the lyrics of the world's music are not in English. I was outraged the day I discovered that the old CDDB system required everything to be in ISO-8859-1. What is someone to do with music in foreign scripts? ISO-8859-1 doesn't even have the necessary characters from standard Latin transliterations (such the characters with carons for Cyrillic transliteration).

If you don't have any experience with Unicode issues, a problem shared by a regrettable number of developers, try Gilliam's Unicode Demystified .

web service

[fuct000]

store all of the data on a server and write either a .NET or Java EE program to share the information as a web service. Then just have a desktop client people download, which contacts the webservice to request the information.

Wiki on a stick

[Glonoinha]

Sounds like a perfect application of Wiki on a stick. I set one up in a few hours, most of which I wasn't even sober - and it can install with a zero-footprint (designed to run from a thumbdrive.)
I have a little more write-up in my Journal, along with links.

Copyrights

[AuMatar]

Music lyrics, unfortunately, are copyrighted. Every db on the web thats gained real size has been shut down by the RIAA. Whatever you do needs to be hosted out of a country that doesn't do copyrights, or you're dead in the water.

Re:Copyrights

[Kelson]

You're assuming he's going to be distributing copyrighted material -- and that he's going to be distributing it without permission. (You can distribute copyrighted material all you want, if you've gotten permission to do so. Otherwise, the publishing industry would be vastly different, and GPL software wouldn't exist.)

The story doesn't tell us anything about which songs he's going to be including. For all we know, it could be a collection of folk songs or hymns that are already in the public domain.

Advice to watch out for copyright issues is good. Assuming he's violating copyright (even if it's the safe way to bet) is still jumping to conclusions, and ignoring the technical questions posed in the story doesn't help anyone.

Heh...

[djsmiley]

Webservice.

Lots of websites already do this, why bog your self down with something that has already been done? Unless its for some kind of research project for university/college of course.

Open source solutions which do the same? Amarok has a "lyrics" tab which brings up the lyrics to the playing song, i think they are pulled from wikipedia but im not sure.

Also musicbrainz has a huge database of music too, this is why they are seemingly linked in amarok.

So basicly your not onto a winner with this unless your going to offer something all the hundreds of others fail to offer.

Amarok, wikipedia and musicbrainz are all open source.

Im not sure however, how all of these cope with non-english alphabets, which is something lots of people tend to bring up.

Start with the MusicBrainz code

[Matt Perry]

You can start with the MusicBrainz codebase. The schema already supports albums, tracks, and annotations. You could extend it for your purpose to add lyrics. A daily dump of the database is available as is the source code to the server application.

The Mozilla Platform

[Feneric]

Copyright issues aside (I'm assuming that you're talking about lyrics that you have the legal right to use) I'd say that there's a pretty simple answer to your problem. You're thinking through the pros and cons of using a back-end database versus a browser front-end, and you're not keen on running any flavor of server.

You can get both the database and browser advantages without having to set up a separate server by building your app on the Mozilla platform. You can utilize its built-in RDF capabilities to store your data in a clean, extensible way, and fairly quickly put together a user interface using XUL and CSS that can work with Firefox, Seamonkey, Flock, etc., or even just the XUL app runner for a more stand-alone user experience.

Because all of your data (and even interfaces) will be XML-compliant, you'll even be making it easier for third party apps to work with your stuff.

Re:He didn't say anything about a service

[vorpal22]

I can't recall the name of it (PearLyrics, perhaps?) but there was an excellent program for OS X that integrated with iTunes and would query several sources and download lyrics to songs you were listening to. The author took it down under threat of being sued by the RIAA, if I recall correctly, and it didn't violate copyright in any way imaginable.

Even if the guy would have won in court, there's likely no way he could have afforded the legal costs, unfortunately, and his programming time was wasted :(.

Good story about a Lyrics Server and the Lawyers

[PixelJonah]

So, a friend of mine wrote one of the first online lyrics servers.

Here's his story.

Re:Internationalization / UTF-8

[Kelson]

Well, "script" doesn't really make sense in the context of your original post, but I'll take you at your word that you don't see the appeal of mixing scripts on one page.

To start, I'll direct you to the Japanese codepage 932, which includes at least four scripts: basic latin alphabet, katakana, hirigana, and kanji. People seem to have thought it was necessary to be able to use all of those on one page, perhaps because Japanese tends to mix three of them together on a regular basis and likes to throw in English words for flavor. (No doubt, Latin characters helped to write computer programs as well.)

Unicode just extends the principle so that you can do things like:

• Aggregate titles from articles in multiple languages
• Use one language for content and another for labels (or, in the case of the web, navigation)
• Write something like a Japanese/Russian dictionary intended for readers, that displays words the way you would see them in actual Japanese or Russian text

...and so on. The Unicode character set is just a big flat space, just like ASCII except with a lot more code points.

The point about internationalization perhaps shouldn't focus on UTF-8 specifically -- one could use UTF-16 instead -- but both encodings give you access to the Unicode character set, which allows you to, as you put it, "define the code page once per document."