F/OSS Flat-File Database?

Leemeng writes "I'm looking for a simple, free, and F/OSS flat-file database program. I'm storing info about Wi-Fi access points that I come across, maybe 8-9 fields per entry. I've outgrown Notepad. This info is for my own reference only; it is not going on a Web server. Googling was unhelpful, with results skewed towards SQL, Access (MS), and Oracle, all of which would be overkill for my purposes. My criteria are: it must be simple, F/OSS, must work in Windows Vista, preferably use a portable format, must not be an online app, and must not require Java. Does such a beast exist?"

Comments

Re:Python?

[goombah99]

You need YAML

I think is exactly what you are requesting.

Re:Python?

[goombah99]

YAML meets the flat file requirement and can be made to be the persistent DB behind Python, Perl, Ruby.

If it were not for the Flat File requirement then a simple Python shelve or Perl Tie would be the most logical solution since they are both part of the standard library so don't require installing libs on random computers you might use or port to.

of those two Perl Tie is probably the most suited because it's backed by a real DB operating off the disk not fully in memory.

But why not do both: use YAML as the DB backing the Perl Tie.
  then you get a nice human readable flat file.

Re:Python comes with SQLite

[fyngyrz]

You may think you're keeping it simple by using a flat file, but you're really not. It may be somewhat easier to manually edit, but it's also easier to screw up, and I've never heard of one with the ability to undo changes.

Class dbtxt includes the ability to undo changes, and features human-readbility for the flat database files. It's trivial and not error-prone at all to mod a database file to delete, or undelete a record, and of course you can do it through the database engine as well.

Re:Python?

[moderatorrater]

I'm missing your point I believe. Or being purposely obtuse which you (probably rightfully) feel you can get away with because this is a very GPL friendly forum. BSD, MIT, etc licenses allow you to do whatever you want with the code with none of the strings that the GPL has. Your false dichotomy between GPL and closed source doesn't really help promote a useful dialog about freedom of source.

Re:Python comes with SQLite

[dkf]

mydb.execute("insert into contacts values('Spooky','Monster','spook@spammity.spam')") Couldn't you have at least done an example that doesn't promote bad practices with quoting? My experience with sqlite (admittedly using the Tcl binding) is that it gets this stuff far more correct and easy to use in practice than any other database API that I've seen. I don't know if the python bindings are of that same quality; if they are, they're top notch. The code I would have written? Like this:

set first "Spooky"
set last "Monster"
set email "spook@spammity.spam"
# In practice it's really easy to put values into variables
mydb eval {insert into contacts values (:first,:last,:email)}

The advantage? That code is now totally armour-plated against SQL injection attacks as well as being fast. Which is nice, really really nice.

You really do NOT want a DBMS, use a spread sheet.

[ChrisA90278]

Your problem is that you are looking for a "database". This is a very complex kind of program that is total overkill for your purpose. Will you really be storing hundreds of thousands of records and wanting to do some complex querries? Ifnot why not just use a speadsheet? If you want a free spreadsheet look at OpenOffice.

Re:Python?

[fyngyrz]

Having a db that is human readable can be a good thing--and a bad thing (anyone can change its contents manually?). Being small is a good thing. But even if you are doing embedded programing, is 800k such a burden that you would give up this feature set?

Sure, it can be. For instance, adding 800k of stuff to your application means that you are giving up control. You might have the source (which is going to be somewhat larger in this case)... but do you have the time to dig into it if there's a problem? Does reporting the problem necessarily mean it's even going to get fixed? For instance, I've reported many problems to various authors, some of which got fixed, some didn't. Likewise, adding a feature can be non-trivial with a large (source code) project. Compare that to a very lightweight thing that, let's say, almost does what you want but you just need feature X. Might be a whole lot easier to add that to a 20k "almost-there" set of source than megabytes of source, is that not possible?

Distribution of a class module with a python app is an entirely trivial thing to do. For that matter, you can take the class and put it in the same file with your application if that appeals, giving you the advantages of atomic distribution -- harder for an end-user to separate things and make whole app, as you handed it to them, fail to work.

And I suppose the elephant in the room is that the end user may have no need for the features. I sell a commercial program with features not available in other programs in the same general application space, functionality-wise; that doesn't mean it will sell to anyone on that basis unless they actually need those features, right? In this specific case, based on the article, the needs described seemed to be met by what I'd written, and hence my post.

PostgreSQL whips the living hoo-ha out of SQLite; it is larger, but offers more features (sound familiar?) Yet SQLite has a place, as your advocacy clearly indicates. I would venture that the difference between SQLite and class dbtext is similar, just occupies another place on the size/features scale. In the end, the user, based on their requirements, may go "what a POS" and step up the ladder. From either one.

OK, dbtxt looks interesting, but SQLite is very stable

dbtxt also seems to be very stable. It's never failed for me, nor have I ever received any indication that it wasn't doing what it was supposed to do when used as indicated. Also -- as a user myself -- I'm quite interested in fixing it, should it turn out to have problems.

Also, just as an aside, If someone can explain cross-platform file locking in python to me in terms I can understand, and if python's ability to lock can be made to implement a reasonable form of "wait for lock to release", I'd like to add that as a feature, too. I found python's docs on locking to be more than a little opaque. I'm sure it's just me.

Re:Python?

[legutierr]

He said he wanted a flat-file database. Why, on this Earth, would he ever need "standard SQL including joins." SQL is designed for relational databases and, in particular, joins are only EVER used in a multi-table, relational database. In fact, the guy even said "SQL is overkill". First of all, my point was that in comparison to a python-only tool like dbtxt, sqlite is probably worth the additional memory overhead of a couple hundred kilobytes.

Second, I'm not sure that you know any better than I do what the poster really needs. Open office calc might be an option, but certainly not if 800k is too much space; open office is what, like 200 megs? And dbtxt might also be an option, if doing everything in Python is what the person wants; I love Python, but there are plenty of people who don't. I'm sure that all three proposals are equally valid.

Now, to defend SQLite: in most cases, dealing with a relational database is "overkill" not because of SQL per se, but because of the fact that software needs to be installed and configured*, and because complex sets of processes need to be managed and maintained. SQLite is useful, in part, because it provides the functionality of a relational database without the complexity of the engine. Now, if you don't want or need SQL, then you might not consider that particular feature to be of particular benefit. Nonetheless, there are many situations where SQL might not be required, but where it might be useful, where in the past SQL might not have been used, but where now, because of SQLite, it will be used. There are very few cases where real-world information is not most accurately represented as relational data (in comparison to flat data). SQLite allows you to avoid making compromises regarding the way that that data is stored. And SQLite is fast.



*(Although some interpreted languages might require that sqlite's libraries be installed on a system for it to be used by a program, that is not the case for compiled languages like C, C++, Obj-C, etc., where the sqlite libraries can be compiled directly into the code distributed to end users.)