Slashdot Mirror


Specialized, Open Source Databases?

PyTHON71 asks: "I've been asked the head of the Wichita State University Anthropology department to help fix his growing biological anthropology collection database. It's currently in Access (insert boo's and hisses), but he is willing to have it developed in a reliable open source format. Now, there are a lot of university departments out there that need to have specialized work done on a small budget. They can't rely on just any developer, because the developer has to know biological anthropology (in this case) as well as MySQL, etc. And since it's not in the Hacker Code to duplicate work that's already been done, I was wondering what specialized database projects are out there & available for general use (not the data, just the structure)."

34 comments

  1. Why? by LordNimon · · Score: 2, Insightful

    Why does the developer have to know biological anthropology? Can't the developer just work with the scientists to understand the data formats, and create the database for them, and then teach them how to use it?

    --
    And the men who hold high places must be the ones who start
    To mold a new reality... closer to the heart
    1. Re:Why? by perlchild · · Score: 1

      I might offer a suggestion... Because a developer who already knows the field will need less training
      Also the relationships between the data will be more obvious to a developer already familiar with the background... And usually never happen to show up during training... Nobody ever thinks about them until problem X occurs, this client already seems to know that, and wants a solution where developer A writes something, and BANG they don't have to see him again... instead of the ongoing relationship a solution that gets gradually defined would be(this approach might yield a superior product in the long run, but would be much more expensive)

    2. Re:Why? by AndyElf · · Score: 2, Informative

      I guess his question is mis-phrased: he wants a ready application/database for that very purpose that is not proprietary.

      --

      --AP
  2. What about Access? by GTRacer · · Score: 3, Insightful
    While I don't have a great love of the "Evil Empire" that is MS, what's so bad about Access? It's certainly not perfect, but tell me what Windows-based alternatives exist that will:

    * Let me create front-end forms populated and controlled by code and query

    * Support relational tables, indexes, complex JOIN queries, aggregate functions, etc.

    * Design complex reports with page/section formatting and behind-the-scenes code controls

    * Can access ODBC data sources

    If you can show me the way, I'll take the first step! I'd love to cut some of the distribution costs here!

    GTRacer
    - Needs help with gcc + PS2...

    --
    Defending IP by destroying access to it? That makes sense, RIAA/MPAA. Go to the corner until you can play nice!
    1. Re:What about Access? by SLiK812 · · Score: 3, Interesting

      The problem is Access's data storage is in a flat file format. Which means it'll be slower than dookie after it gets to be a certain size and complexity. Especially when sharing the database with many users. I'd suggest SQL Server, with a vb frontend or even an Access frontend with the SQL Server tables linked into Access.

      Unfortunately the question was pertaining to open source, probably because the owner of the data, doesn't want to drop a chunk of change on a major database. Yes Access comes with Windows, but the performance issue is too big.

    2. Re:What about Access? by Anonymous Coward · · Score: 2, Interesting

      Most of the time, whenever anyone posts about database, I like to point out Adaptive Server Anywhere (from Sybase). It's ....

      - inexpensive

      - probably the easiest DB to administer

      - fully SQL compliant, supports ESQL/C, ODBC, OLEDB, JDBC

      - available in linux, windows, mac, aix, etc. versions

      - available for free download (as an evaluation copy) off the sybase website

      - advanced query optimizer (blows most open source engines out of the water .... really)

      - fully ANSI transaction-oriented; stable in the event of disaster (which is really what you pay for when you buy a database)

      - comes with all sorts of other goodies in the suite (lets you make forms like Access [via "Infomaker"], database/UML graphs, etc) -- mostly things i don't use, but probably some things that an Access user would like

      - can run on anything from mainframes to palm pilots (ASA has technology that lets it generate specialized database engines that run on handhelds in as little as 80kB)

      start your first step here:

      http://www.sybase.com

    3. Re:What about Access? by metacosm · · Score: 2, Interesting

      I am not sure about the data storage of access. It was my understanding it was NOT a flat file -- but it is very limitted.

      I recommend you use the "upsize" wizard (which will automatically relink to your existing forms) and goto MS SQL 2000.

      This does a few things for you. #1. Huge number of people familar with Access. (Better chance of finding a "biological anthropology" major with experience [I should note, I find this requirement a bit silly, have a guy from the CS department set it up, and you use it]). #2. The app appears to work exactly like it did -- only without the preformance hangups.

      Once you get thru that basic change -- and you have de-coupled the interface (access) from the backend (sql server 2000) -- you can slowly but surely start fixing the design of the database without interupting the USAGE of the system too much. The forms will be consistant, and you just have to ensure that you keep them up to date with the database changes you make.

      I hate to be putting forth an MS solution on /. -- but I think it is probably the most sane answer. :)

    4. Re:What about Access? by jbolden · · Score: 1

      I'm not sure where you get this from. Access is fully relational. The file format is just a packed collection of files stored as a single file. Similar to an oracle database which uses a single tablespace for all tables and stores the entire tablespace in a single datafile.

    5. Re:What about Access? by Anonymous Coward · · Score: 0

      As my grandmother would say "flat - shmat", who cares. Got any hard numbers or benchmark results to support your statement?

    6. Re:What about Access? by Anonymous Coward · · Score: 0

      And not only the points noted, but Access does support Unicode/UTF-8 encoding, a critical element of any database that needs internationalization.

      And which support is notably absent from seemingly everyone's darling, mySQL.

    7. Re:What about Access? by GTRacer · · Score: 1
      I recommend you use the "upsize" wizard (which will automatically relink to your existing forms) and goto MS SQL 2000.

      I agree wholeheartedly, but I don't have any money to spend and I'm trying to make the most of what I have. I may be able to convince I/S to give me SQL Server space, and if so, then I'm all set.

      But they may tell me to take a flying leap, and then I'll be forced to roll my own solution. My boss supports some small expenditure for this effort, but I'd rather spend the money on a true programming environment when data solutions already exist in-house.

      If there was an inexpensive and comparable alternative, I'd consider it. I'll be taking a look at ASA soon, and would love to have some other products to include in the eval.

      Thanks, and no, I'm not involved in this bio-anthro project.

      GTRacer
      - Anthropomorphises machines from time to time...

      --
      Defending IP by destroying access to it? That makes sense, RIAA/MPAA. Go to the corner until you can play nice!
    8. Re:What about Access? by ispeters · · Score: 1

      Perhaps this is (-1, Offtopic), but why suggest SQL Server with an Access front end, when you can suggest an Open Source/Free server, such as PostgreSQL, or MySQL, and an Access front end?

      I've never tried it myself, but I've heard that PostgreSQL can be an ODBC datasource on Windows machines.

      If you've already got Access, and you already have a viable data structure and forms/reports that work with it, you should be able to move the data to Postgres, change the tables in Access to ODBC links, and pretend like nothing changed....

      Of course, I've never done any of this, so YMMV

      Ian

    9. Re:What about Access? by teasea · · Score: 1

      All you need is available with powerbuilder. Quick, easy reports, connect to most databases, use standard or 'enhanced' SQL inline, and you can compile it to native code for Windows or Unix. Easy language; main thing is learning what not to do in powerbuilder. kinda quirky. Maybe the latest is better.
      Not a plug by any means, but Access is nasty. My experience with it is when they need to port apps to a more robust, less buggy environment.

    10. Re:What about Access? by ed1park · · Score: 1

      I agree. If the jump to MySql or Postgres is too great, then Access should work out fine.

      Currently our Access 2000 database weighs in at 740 MB's with 4-5 users. And it's running fine. However, I'm planning to migrate to Postgres very soon.

      Proper choice of Primary/Foreign Keys, Indexing, and NORMALIZATION of the tables is *critical* to size and performance.

      3rd Normal form will do wonders for size and data integrity. Indexing/keys will do wonders for speed.

      If the developer is not properly versed with these concepts, you will end up with a database that runs at a small fraction of it's optimal speed.

      I've had a query that took 45mins. After proper indexing/normalization, the query took only 3 minutes.

  3. Sure by jsse · · Score: 1
  4. database? by Kevin+Stevens · · Score: 2, Informative

    Why dont you explain what you mean by 'database' first. A database can mean different things. I am not sure if you want a complete database application, front end and all, or if you just want a database design- A UML model or E-R diagram, from which SQL DDL can be created. (or perhaps by open source you meant SQL code to Create a database). Your requirements are kind of blurry to me. Doing something like this should not really be all that difficult though. If you have a developer look at the kinds of data you are trying to store, and the relationships between those data (in depth knowledge is not required of what the data actually means, though I guess it helps) a developer should be able to come up with a Data model and front end relatively easy.

  5. Do you really have to have this answered? by dacarr · · Score: 2, Insightful
    Asking for this beast is like asking for a version of M$ Excel (or, to be fair, Gnumeric) that is specifically designed toward balancing a company's books.

    A database is a database. There is no "specialized" database for any purpose, with the possible exception of design for platform (IE, DB/2 for AS400) - you build the structures in the DB program and go. It's what makes apps and engines like MySQL really cool.

    If you want a DB that is geared for bio-anthropology, fire up MySQL or PostGreSQL or insert your favorite engine here, build the structure for the DB you need to run, put a pretty face on it for the users, and kick back as it does the rest.

    --
    This sig no verb.
  6. Frontend? by aspjunkie · · Score: 1

    The requirements seem kind of vague, but if you don't want to go all the way, you could always just export the data to PostgreSQL or mySQL, and use the the existing access interface for a frontend, connecting over ODBC to the Open Source alternative - at least until you've transitioned it to something else.

  7. Maybe I'm missing something by fean · · Score: 2, Insightful

    Here's what I understand:
    you already have everything up and running on an access database
    you want to use a different database platform

    doesn't sound like you need to worry about designing anything, its already there.... especially if you want to go from access to MySQL (or any other of the same family), just build the database structure to match the old one, export to .csv from word, and import to your chosen database... interfacing should be similar, it depends on how you're accessing it (if web based, and you have admin privs on the host machine, you just change the database type in the control panel, google:asp+mysql )...

    so unless I've over simplified something, sounds like this is a classic mountain != molehill problem

  8. SQLITE by ddriver · · Score: 2, Insightful

    It supports almost all sql92, is blazingly fast, it has an ODBC interface and native api's for c and c++, is has a python db api 2.0 compliant interface. It has a really cool and versatile command line tool. It is imbedded though, so if you want to you it as a server you will want to write the implementation. I don't think that it would be too hard though, depending on what you wanted to do with it. I have just started a payables application withit and it looks like it will work out quite well.

    Oh, and it is public domain.

    --
    I found my inner child, then I got caught abusing it...
    1. Re:SQLITE by ddriver · · Score: 3, Funny

      try www.sqlite.org

      Here is the licensing terms from source: /*
      ** 2001 September 15
      **
      ** The author disclaims copyright to this source code. In place of
      ** a legal notice, here is a blessing:
      **
      ** May you do good and not evil.
      ** May you find forgiveness for yourself and forgive others.
      ** May you share freely, never taking more than you give.
      **

      Dude I so dig that!

      --
      I found my inner child, then I got caught abusing it...
  9. Wrong forum by rubinson · · Score: 1

    A generalized forum such as "Ask Slashdot" probably isn't the best place to ask this type of questions. The problem is that the domain is too specific. What you probably want to do is talk with other biological anthropologists and see if anyone has a schema that you can adapt to your needs.

    That being said, one thing that you might want to do is search Google for XML schemas. I do a lot of (relational) database design and I've noticed that there are XML schemas for just about everything. (Although I generally dislike XML, a real benefit is that it promotes the idea of sharing information and schemas. In the relational database world, we tend to horde our schemas because they are viewed as competitive advantages.) Even though I typically don't use XML in my own projects, I've found that reviewing relevant XML schemas can sometimes help in the database design process. (If nothing else, it can help point out what not to do!)

  10. future support by Parsec · · Score: 1

    You will also have to ask who will continue to support the app/db combination in the future. With any dedicated database server, who will keep it patched and secure, and backed up in the future? I see the following options:

    1. Stay with Access:
    Security is based on file sharing, and up to the Network Administrators. However, the problems with Access, you are probably well aware of; a biggie is that the db can lock and require reboot of the server (which can really piss off your Network Administrator if they have to take down a major file server for one application). Anthro may have already decided that Access is a bad thing(tm), but it may be good to gather a few quick arguments against it as justification against any MS flag wavers.

    2. Migrate to MySQL (or other free DB) back-end with Access front-end:
    You will have to figure out who maintains the server setup. Access does not like to migrate to non MS databases, and you may have to totally redo the db structure and Access front-end.

    3. Migrate to MySQL (or other free DB) and rewrite front-end:
    Again, you will need to figure out server maintenance, as well as finding an open-source front end. I'd be positively giddy to find/learn about an Access/Oracle Forms-like open-source tool.

    4. Migrate to Microsoft SQL server with Access:
    There may be other SQL servers on campus or with your IT department that you could rent space on. They would perform all maintenance with the server and the Anthropology department would just need to maintain the front-end. I would especially recommend server consolidation if you consider a Microsoft SQL server back end. Keeping up with patches, security, and other considerations with this server greatly outweigh the all-eggs-in-one-basket and local control factors.

    5a. Does you university have an Oracle site license?:
    You may be able to rent space on an IT Oracle server. However, I do not yet have experience migrating an Access DB to Oracle.

    5b. If your university has an Oracle site license, does that include Oracle Forms Developer?:
    Look at Oracle Forms developer as a replacement for Access (try version 6 before the latest 9 version, it's a bit more mature/easier to deploy until 9's bugs get worked out). It does have a steeper learning curve than Access, though.

    Of these choices, 2 is my pick for ease, 3 is my pick for open-source, and 5a/5b would be my pick for an industrial strength solution.

    1. Re:future support by Electrum · · Score: 1

      2. Migrate to MySQL (or other free DB) back-end with Access front-end:
      You will have to figure out who maintains the server setup. Access does not like to migrate to non MS databases, and you may have to totally redo the db structure and Access front-end.


      Actually, it's very easy to migrate Access to MySQL. Install the MySQL ODBC driver, then import the database and data into MySQL. You can then use Access as the frontend to the MySQL ODBC data source.

    2. Re:future support by Parsec · · Score: 1

      There are a few caveats depending on how a particular Access db is set up. He may have to re-structure some tables and re-write portions of the Access front-end to accomplish this. You're right that some people may not experience any problems. See this for a bit more information.

    3. Re:future support by La+Camiseta · · Score: 1

      3. Migrate to MySQL (or other free DB) and rewrite front-end:
      Again, you will need to figure out server maintenance, as well as finding an open-source front end. I'd be positively giddy to find/learn about an Access/Oracle Forms-like open-source tool.


      I know that PostgreSQL has a semi-decent front end program called pgaccess (I know that it's in Linux, but I'm not sure about Windows), which you can use to create forms/reports for. And the best thing about it is that all of the form data is stored in the databse, so you don't have to go around updating everybody's software every revision you make to the front end. It may take a bit more work than just using simply Access, but it seems to be easy to use, and it's completely GPLed and written in TCL/TK.

      Hope that was of some help.

  11. Dammit, we've evolved past "inexpensive" by Anonymous Coward · · Score: 0

    "Inexpensive"? Haven't we graduated to "free" yet? Why waste time looking for an "inexpensive" database when one could get a free one?

    1. Re:Dammit, we've evolved past "inexpensive" by Anonymous Coward · · Score: 0

      There are some things still worth buying.

      When it comes to databases, the people with the degrees -- who contribute to the majority of the research in the field, and who are at the forefront of database (and computer science) theory -- work for database companies like IBM, Oracle, Microsoft, and Sybase.

      From a performance, stability, and feature standpoint, products offered by these companies (IBM, Sybase, etc.) are years ahead of open source competition. In fact, they're so far ahead, that open source doesn't even appear on their radar screens when it comes to databases.

      I think that from a:

      1) performance
      2) stability (and data integrity)
      3) lawful responsbility

      standpoint, some databases are still worth buying. Not all the time, and not for everyone, but for a lot of people. (Arguably, there are a few more reasons, like features and security, that make commercial databases a success.)

      e.g., I wouldn't trust mysql to "run" a nuclear reactor. Similarly, I can think of a number of other cases in which I would value performance and data integrity over cost.

  12. You'd think that... by 0x0d0a · · Score: 1

    A generalized forum such as "Ask Slashdot" probably isn't the best place to ask this type of questions.

    You'd think that...but amid all the noise and chatter of people pimping their own pet databases, they got your actually useful comment.

  13. Matt Dillon's Backplane database by cpeterso · · Score: 1


    Matt Dillon (of FreeBSD fame, not the actor :-) has a new database startup company called Backplane . They are creating a high-performance, distributed database which should be massively scalable (using many small servers ala Google's web farm). It's open source, too.

  14. *BSD is dying by Anonymous Coward · · Score: 0
    It is official; Netcraft now confirms: *BSD is dying

    One more crippling bombshell hit the already beleaguered *BSD community when IDC confirmed that *BSD market share has dropped yet again, now down to less than a fraction of 1 percent of all servers. Coming on the heels of a recent Netcraft survey which plainly states that *BSD has lost more market share, this news serves to reinforce what we've known all along. *BSD is collapsing in complete disarray, as fittingly exemplified by failing dead last in the recent Sys Admin comprehensive networking test.

    You don't need to be a Kreskin to predict *BSD's future. The hand writing is on the wall: *BSD faces a bleak future. In fact there won't be any future at all for *BSD because *BSD is dying. Things are looking very bad for *BSD. As many of us are already aware, *BSD continues to lose market share. Red ink flows like a river of blood.

    FreeBSD is the most endangered of them all, having lost 93% of its core developers. The sudden and unpleasant departures of long time FreeBSD developers Jordan Hubbard and Mike Smith only serve to underscore the point more clearly. There can no longer be any doubt: FreeBSD is dying.

    Let's keep to the facts and look at the numbers.

    OpenBSD leader Theo states that there are 7000 users of OpenBSD. How many users of NetBSD are there? Let's see. The number of OpenBSD versus NetBSD posts on Usenet is roughly in ratio of 5 to 1. Therefore there are about 7000/5 = 1400 NetBSD users. BSD/OS posts on Usenet are about half of the volume of NetBSD posts. Therefore there are about 700 users of BSD/OS. A recent article put FreeBSD at about 80 percent of the *BSD market. Therefore there are (7000+1400+700)*4 = 36400 FreeBSD users. This is consistent with the number of FreeBSD Usenet posts.

    Due to the troubles of Walnut Creek, abysmal sales and so on, FreeBSD went out of business and was taken over by BSDI who sell another troubled OS. Now BSDI is also dead, its corpse turned over to yet another charnel house.

    All major surveys show that *BSD has steadily declined in market share. *BSD is very sick and its long term survival prospects are very dim. If *BSD is to survive at all it will be among OS dilettante dabblers. *BSD continues to decay. Nothing short of a miracle could save it at this point in time. For all practical purposes, *BSD is dead.

    Fact: *BSD is dying

  15. Berkeley DB... by curunir · · Score: 1

    As I understand it, the *NIX equivalent of Access is Berkely DB. Small, lightweight, stores all its information in a single file. They've got 4 different versions depending on your needs and support pretty much any OS.

    --
    "Don't blame me, I voted for Kodos!"
  16. Support down the road by ClayDowling · · Score: 1

    Something that's important to look at is who will support this application down the road. Funding at universities can be dicey, and this thing may have to be supported by inhouse flunkies. What can they reasonably support? This doesn't necessarily mean that it needs to be in access because the inhouse flunkies only know access. If you could write a C++ front end that was stable and reliable, with an easy-to-maintain connection to a backend that they can support, that might be a better solution. That said, some great recommendations have already been made. SQL-Server on the backend, if there's already someone maintaining one, would be ideal. I actually don't recommend putting an Access front end on things though, because a user with a little bit of knowledge can do a lot of damage.

  17. go for more - by Anonymous Coward · · Score: 0

    OpenOffice + MySQL

    see the Trail of Tears article - write up what you learn so the trail getts better

    http://www.linuxworld.com/site-stories/2003/0207 .b arr.html