Slashdot Mirror
F/OSS Flat-File Database?
Leemeng writes "I'm looking for a simple, free, and F/OSS flat-file database program. I'm storing info about Wi-Fi access points that I come across, maybe 8-9 fields per entry. I've outgrown Notepad. This info is for my own reference only; it is not going on a Web server. Googling was unhelpful, with results skewed towards SQL, Access (MS), and Oracle, all of which would be overkill for my purposes. My criteria are: it must be simple, F/OSS, must work in Windows Vista, preferably use a portable format, must not be an online app, and must not require Java. Does such a beast exist?"
Can't be Java... well, how about Python?
Here is a completely free (PD, not GPL-style "you're free to do as we tell you") database engine that will do what you have described thus far.
The database engine is about 19k bytes (not a typo), has no dependencies (other than Python itself), supports a useful subset of SQL so you can actually create flexible queries that produce well-sorted results from your database, and it works everywhere Python does, which is to say, it works pretty much everywhere. It's just as happy operating on a command line as it is on a web server. The results (the actual databases) are 100% portable from OS to OS. I use it on various linuxes, OS X, and Windows for tasks very similar to yours.
Comes with tutorial examples, sample databases and extensive docs. In a 13k (not a typo) archive.
I've fallen off your lawn, and I can't get up.
Comma Separated Variable Text Files, as exported and imported by Excel. You can get libraries to read and write these, and search these in most languages.
Otherwise what's wrong with a simple database like MySQL or PostgreSQL on your computer?
No comprende? Let me type that a little slower for you...
Sqlite is used in many apps (including Firefox), it's small, and it's efficient. It also has bindings to just about every imaginable language.
I find it amazing that you didn't come across it in Googling...
Get it http://www.sqlite.org/ here.
There are GUI clients that work fine for this sort of thing, SQL is simple for doing basic things. One file, one database.
Obligatory blog plug: http://www.caseybanner.ca/
Have you looked at SQLite?
C based, no client-server. Very small, quite fast. And a very permissive license.
I think this would be perfect, I found it when I was looking for a DB which met similar criteria. I don't care about windows, but it does work under windows AFAIK.
http://www.sqlite.org/
Use it all the time for everthing from trivial databases to several 10s of megabytes. Since query's can be entered on the command line it's quite simple to write an ascii results grabber.
Absolute statements are never true
In addition to Calc, there is also Base, the alternative to MS Access. Check it out here: http://www.openoffice.org/product/base.html It's gotta be way easier to work in OpenOffice than a whole database system (making up accounts, tables, reports, and all).
Open office should do the trick.
BDB is *not* a relational database though, it's just a storage/indexing engine, which is used most notably by MySQL as a backend. SQLite on the other hand is a full file-based RDBMS with a small runtime, so it might be a bit of overkill for you in this particular scenario.
But both of them run on Windows, Linux and the BSDs, so you won't have portability problems. And most languages have bindings for them.
The twitter monologues. Click on my homepage and be amazed.
If that doesn't satisfy your need, take a look at Berkley DB. It offers a more sophisticated interface than DBM.
For a small number of fields like that, why re-invent the wheel? Grab OpenOffice and just create a spreadsheet. Easily searchable, sortable, extendable, and all with zero maintenance on your part.
I recently moved my collection of serial numbers out of a defunct proprietary program and into a spreadsheet - couldn't be happier.
I don't know what kind of crack I was on, but I suspect it was decaf.
Have you used OO Base?
I was using it about a year ago and I found it buggy and hard to use. I was using it to access
PostgreSQL.
I like the other parts of OpenOffice that I use, Write, Calc, Draw.
Of course, I have not had time to look at it since then.
"We can't solve problems by using the same kind of thinking we used when we created them." -- Albert Einstein
Just get Python, and use the version of SQLite that comes with it:
import sqlite3
mydb = sqlite3.connect('sample.db')
mydb.execute("create table contacts (fname text, lname text, email text)")
mydb.execute("insert into contacts values('Spooky','Monster','spook@spammity.spam')")
mydb.commit()
mydb.close()
You can then use the free and open SQLite database browser to browse, edit, and print your table.
You may think you're keeping it simple by using a flat file, but you're really not. It may be somewhat easier to manually edit, but it's also easier to screw up, and I've never heard of one with the ability to undo changes.
- Despite popular opinion, I am not perfect.
Metakit is a small footprint database that might fit your needs. Metakit
echo "badger, badger, badger, badger, snake" >> my_file
Technology -- No Place For Wimps! Grateful Dead and Jerry Garcia Chatroom -- http://www.wemissjerry.org
There are a number of XML databases, several free and open source, that will rely only on "flat-files". You could probably get by with Microsoft's xml libraries, though there are a number of ways to manipulate and query a set of xml documents. Several of these XML databases implement XQuery which may help if your dataset grows beyond effective queries by visual inspection in Notepad.
eXist is one alternative; while I haven't personally used it the home page indicates it's a fairly capable project.
Sedna also appears to be feature-rich.
There was a similar discussion on Slashdot specifically with reference to XML databases, here.
Happy hunting -
If the Government becomes a lawbreaker, it breeds contempt for law;
CSQL only supports in-memory databases. It doesn't even have an on-disk representation -- it's strictly a memory-only cache thing.
1993 called to recommend PCFILE!
http://www.umich.edu/~archive/msdos/database/pcfile/
Mind you, you'll have to toss Jim Button $10 as it's shareware!
AT&ROFLMAO
Nobody has yet mentioned that MySQL has a CSV storage engine - just create a table with type CSV and away you go. It does require the MySQL engine though.
Otherwise, I agree with most posters who say just use a simple text and/or xml file if the data volume is relatively small. That should be more than enough!
SixD
The poster didn't want SQL, so I'm guessing they ruled out SQLite on those grounds. QDBM and Oracle/Sleepycat/Berkley DB-4 would be the smallest, fastest "pure" flat-file databases that are also cross-platform - at least, that I know of.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
I'm still using it for a customer database, it's two main problems are it's buggy and hard to use, and slow.
...
So really it's 3 main problems are being buggy, hard to use, slow* and totally bloated
Actually it's not half as buggy as it was a few releases back - I can actually use it now to fill in a form without it crashing. Just tried opening a windows XP made odb file in Ubuntu (OOo 2.4) and it's slow as hell and bugged to death, so you might add portability to that list of main problems.
[Yes I've filed bug reports].
---
* slow for searching, takes about 2 mins to do a straight text search on 500 records.
He doesn't need python. He just needs a database. He can download a precompiled binary for windows that allows one to work with the database at the command line. Python is not necessary.
And if the command line is too much, as others have noted there is already a convenient firefox extension for graphically interacting with a sqlite database.
Why not the pickle/cPickle module? Not a flat-file but easier to manipulate since your just serializing native Python objects.
e.g.
import cPickle
class Record:
def __init__(self, fname, lname, email):
self.fname = fname
self.lname = lname
self.email = email
# ---
# Write out the data
records = list()
records.append(Record("John", "Doe", "jd@email.com"))
output = open("records.db", "wb")
cPickle.dump(records, output)
output.close()
# ---
# Read back the data
inputFile = open("records.db", "rb")
data = cPickle.load(inputFile)
for record in data:
print record.fname, record.lname, record.email
inputFile.close()
Berkely DB?
The sqlite3 module as shipped with Leopard says that it supports question mark substitution, so his example would be better written like:
BTW, the same could be written for PostgreSQL like:
Hopefully sqlite3 will get the same standard substitution in the future, but at least it is possible to do it safely today.
Dewey, what part of this looks like authorities should be involved?
Been searching flat files with this open source VB5 program since 1999. Searches flat files at 20,000,000 cps displaying hi-lited hits in matching lines only or full context mode. Results can be exported to a results file. Append txt files together. Search and replace option. Run multiple versions with various default settings. I have been yapping and yapping about flat files, only to get great amounts of resistance from the Geeks. The search uses lots of defaults and quick keys (for typists). From startup to shutdown nothing is faster. Channel9 and thedailyWTF.com have the best threads. Check it out. It's the only program you'll ever need
Challenge: I have better access to my Video, Music, Pics and Text than anyone on Earth.
Actually, the Python SQLite adapter supports the standard Python DB API (PEP 249), and so you can just follow its recommendations (which includes using placeholders).
Take a look at Metakit. http://www.equi4.com/metakit/index.html. Its single file, small, fast and it has proven itself over many years. It is written in C++ and bindings for Python and Tcl are available. Instead of tables, rows and fields it uses similar concepts called view, index and property. Interestingly, a property may also be a subview thus allowing a mix of a relational database (flexible) and a hierarchical database (fast).
I use hsql. It has many options: flat file, memory just to name 2.
I have also heard good things about H2 and apache derby. These have been especially valuable for embedded apps