Slashdot Mirror
F/OSS Flat-File Database?
Leemeng writes "I'm looking for a simple, free, and F/OSS flat-file database program. I'm storing info about Wi-Fi access points that I come across, maybe 8-9 fields per entry. I've outgrown Notepad. This info is for my own reference only; it is not going on a Web server. Googling was unhelpful, with results skewed towards SQL, Access (MS), and Oracle, all of which would be overkill for my purposes. My criteria are: it must be simple, F/OSS, must work in Windows Vista, preferably use a portable format, must not be an online app, and must not require Java. Does such a beast exist?"
Can't be Java... well, how about Python?
Here is a completely free (PD, not GPL-style "you're free to do as we tell you") database engine that will do what you have described thus far.
The database engine is about 19k bytes (not a typo), has no dependencies (other than Python itself), supports a useful subset of SQL so you can actually create flexible queries that produce well-sorted results from your database, and it works everywhere Python does, which is to say, it works pretty much everywhere. It's just as happy operating on a command line as it is on a web server. The results (the actual databases) are 100% portable from OS to OS. I use it on various linuxes, OS X, and Windows for tasks very similar to yours.
Comes with tutorial examples, sample databases and extensive docs. In a 13k (not a typo) archive.
I've fallen off your lawn, and I can't get up.
I would have recommended HSQL, but you don't want Java. Frankly, usually, when we're talking databases I won't say "use a spreadsheet", but with 10 fields, you might as well use a spreadsheet. Of course OpenOffice.org Base is out, because it uses HSQL.
Something like CSQL might fit, but I have no experience with it.
Ahhh...the great dumpster continuum. Many a free computer will be found there. -- sowth (748135)
Comma Separated Variable Text Files, as exported and imported by Excel. You can get libraries to read and write these, and search these in most languages.
Otherwise what's wrong with a simple database like MySQL or PostgreSQL on your computer?
No comprende? Let me type that a little slower for you...
Sqlite is used in many apps (including Firefox), it's small, and it's efficient. It also has bindings to just about every imaginable language.
I find it amazing that you didn't come across it in Googling...
Get it http://www.sqlite.org/ here.
There are GUI clients that work fine for this sort of thing, SQL is simple for doing basic things. One file, one database.
Obligatory blog plug: http://www.caseybanner.ca/
Are you sure you are looking for FOSS?? I think you would have a far easier time finding what you are looking for if you gave up on either the VISTA requirement or the FOSS requirement in this situation.
This is my sig. There are many like it but this one is mine.
Have you looked at SQLite?
C based, no client-server. Very small, quite fast. And a very permissive license.
First thing that came to my mind was SQLite. It's used by FireFox. I have a couple of projects lined up I plan on using it for, when I get the time....
"We are not always what we seem, and hardly ever what we dream."
Schmendrick the Magician
a comma delimited file?
Umm... just write a few one-liner perl scripts to get info out.
findstr, the windows equivalent of grep.
I think this would be perfect, I found it when I was looking for a DB which met similar criteria. I don't care about windows, but it does work under windows AFAIK.
http://www.sqlite.org/
Use it all the time for everthing from trivial databases to several 10s of megabytes. Since query's can be entered on the command line it's quite simple to write an ascii results grabber.
Absolute statements are never true
Since the majority of the comments so far have pointed at SQLite, I'm kind of surprised that the post didn't come "from the sqlite dept." :)
Use perl with a tied hash like BerkeleyDB http://search.cpan.org/~pmqs/BerkeleyDB-0.34/
Easy, fast, simple.
Trying to solve such an issue as a flat file is a poor design characteristic. This says "I don't want to learn SQL" or "I want the output human readable." If you just want to store information for programmatic use, use SQLite and quit worrying about data storage format. If you want it human readable, grab libxml2 (works on Windows too) and store it as XML. Decide which problem you are solving because if you don't need (i.e. it's not helpful) it human-readable and/or want the ability to search it quickly without loading the whole thing into memory, then SQLite is probably a better solution.
Support my political activism on Patreon.
In addition to Calc, there is also Base, the alternative to MS Access. Check it out here: http://www.openoffice.org/product/base.html It's gotta be way easier to work in OpenOffice than a whole database system (making up accounts, tables, reports, and all).
Open office should do the trick.
BDB is *not* a relational database though, it's just a storage/indexing engine, which is used most notably by MySQL as a backend. SQLite on the other hand is a full file-based RDBMS with a small runtime, so it might be a bit of overkill for you in this particular scenario.
But both of them run on Windows, Linux and the BSDs, so you won't have portability problems. And most languages have bindings for them.
The twitter monologues. Click on my homepage and be amazed.
The only thing that comes to mind is Mac only. Bento (which from the write up is exactly what you want). I really think the only thing on Windows that really comes close is Microsoft Access and even that isn't what you're looking for.
If you are just doing tabulated data in a piece meal form, Excel with an Access back-end will do the trick, I'd be willing to be you're going to be able to find templates that help you start. You could also use File-maker (which is overkill, but easy to use) and go to the user community for starting templates.
Burn Hollywood Burn
Can be used by all spreadsheet programs (Excel,Gnumeric,OOCalc probably even Google's online offerings) for complete portability
Depends what you want to be able to do with your data? If it's just quick searches to find local wifi points then it seems overkill to use a multi-platform/FOSS database
---- There are 10 types of people in the world. Those that understand binary and those that don't
Something that maybe just up the ladder, and quite useful for all kinds of Database work, would be the spreadsheet in Open Office.
People all over use spreadsheets for simple list based database stuff.
It is often easier to use the tools that are built in to do what you want.
You can start just by entering the data, then you can increase functionality with custom data entry dialogs, charting functionality etc.
All not online, all f/oss.
- Small persistence layer in school project (Java).
- In custom Ant tasks to store metrics and various information.
- Even used it to replace the storage of an MS Access based application in C++ we were trying to port to other OS.
So SQLite is quick, portable, self-contained andIf that doesn't satisfy your need, take a look at Berkley DB. It offers a more sophisticated interface than DBM.
I think a wiki sounds like what you need instead of a "database", if you're using it for personal reference and not hooking it to a web server or any kind of client application, you don't really need a database.
For a wiki "engine" that will run locally on a PC try googling for "wiki on a stick". I'd recommend DokuWiki on a Stick, as DokuWiki uses flat files for storage. You can keep it locally on a PC, or as you may have guessed you can run it from a USB thumb drive on any computer. There are other wikis that run in this fashion as well, some cross platform.
A database seems overkill.
Read the XML records into an STL container for easy access.
For a small number of fields like that, why re-invent the wheel? Grab OpenOffice and just create a spreadsheet. Easily searchable, sortable, extendable, and all with zero maintenance on your part.
I recently moved my collection of serial numbers out of a defunct proprietary program and into a spreadsheet - couldn't be happier.
I don't know what kind of crack I was on, but I suspect it was decaf.
You're looking for Tomboy.
It's like a personal, offline Wiki and you could easily have links to all of your wifi-related notes.
Have you used OO Base?
I was using it about a year ago and I found it buggy and hard to use. I was using it to access
PostgreSQL.
I like the other parts of OpenOffice that I use, Write, Calc, Draw.
Of course, I have not had time to look at it since then.
"We can't solve problems by using the same kind of thinking we used when we created them." -- Albert Einstein
Just get Python, and use the version of SQLite that comes with it:
import sqlite3
mydb = sqlite3.connect('sample.db')
mydb.execute("create table contacts (fname text, lname text, email text)")
mydb.execute("insert into contacts values('Spooky','Monster','spook@spammity.spam')")
mydb.commit()
mydb.close()
You can then use the free and open SQLite database browser to browse, edit, and print your table.
You may think you're keeping it simple by using a flat file, but you're really not. It may be somewhat easier to manually edit, but it's also easier to screw up, and I've never heard of one with the ability to undo changes.
- Despite popular opinion, I am not perfect.
Metakit is a small footprint database that might fit your needs. Metakit
echo "badger, badger, badger, badger, snake" >> my_file
Technology -- No Place For Wimps! Grateful Dead and Jerry Garcia Chatroom -- http://www.wemissjerry.org
If it's just for your reference, you don't need a database at all. Databases don't become technically worth-while until you get to indexing. And until you have fifty-thousand records, or complex queries, you don't need indexing.
So why not simply write out a tab-delimited file? Retrieval is as simple as reading every line, splitting by the delimeter, and regexp'ing whichever field you're examining. 15'000 records takes about two seconds on a modern machine. Of course, for anything more complicated, that same file gets easily imported into your favourite spreadsheet application.
You don't need to worry about locking because you're the only one using it. And otherwise, your application simply locks the file handle, or creates & destroys a traditional lock-file.
It's a thirty-line perl script (of legible perl). You can do it in JScript as a local HTA if you want the benefit of html etc. interfaces.
I've only had a passing encounter with powershell, but if command line searches are all you need for searching, and something like 'echo $newrecord >> flatfile.txt' will work for adding records, then you probably have everything you need already on the computer. If you need searches to return 'ranes' of records, for example 'all APs within 200 yards with public access' you probably will need something more complex, and a db platform like sqlite is probably what you want to start looking at.
You never know...
I was about to recommend that and then realized that the the database engine that OO Base uses (HDBC), as well as the interface with other databases (JDBC) both require Java.
YAML and Ruby play well together. (or perl, or Python)
CSV has a perl DBD driver for it.
http://www.mathematik.uni-ulm.de/help/perl5/doc/DBD/CSV.html
Keep It Simple Silly Principle- Something that every programing language can use and is very open source:
CSV - Comma Separated Values.
"This is one text entry follows by date, integer and float",20080520,2,3.222
Compatible with nearly everything out there. Crunch number in a spreadsheet if you need - or with any number of SQL compatible programs.
How is this news?
needs REBOL but there are free use version.
the database program can be found here
I have used ODBC to both CSV and fixed width text files with good results. Try a Google Search
The weathers here - Wish you were beautiful
NoSQL, half way between standard SQL and unix command line, stores data in plain text files
I mod down anyone who says "I will be modded down for this", regardless of the rest of their comment
If you have only just outgrown notepad, why not use XML? There are some nice tools to create and query the files, it is cross platform, tools are available for every language, and you can import it straight into a real database when you feel you need to upgrade.
How many records are we talking about anyway?
--
My sometimes helpful blog
There are a number of XML databases, several free and open source, that will rely only on "flat-files". You could probably get by with Microsoft's xml libraries, though there are a number of ways to manipulate and query a set of xml documents. Several of these XML databases implement XQuery which may help if your dataset grows beyond effective queries by visual inspection in Notepad.
eXist is one alternative; while I haven't personally used it the home page indicates it's a fairly capable project.
Sedna also appears to be feature-rich.
There was a similar discussion on Slashdot specifically with reference to XML databases, here.
Happy hunting -
If the Government becomes a lawbreaker, it breeds contempt for law;
KeePass might work well for this, and has the added benefit of encryption. I use it to store a lot of networking-related information at work, and with its search capabilities it has been quite useful.
Will get the job very quickly, I can say that from experience. If you must have it it human readable, Python + XML.
"Thanks for all the money you paid to us. We've used it to buy off ISO among other things" -Microsoft
The question seems slightly misguided, and the answers suggesting sqlite3 seem very good. However, if you want a literal answer, I would think that Python's csv module does what the poster requested: http://docs.python.org/lib/csv-examples.html. Well, if you're inclined in a different way, you could use Text:CSV: http://www.cpan.org/modules/by-module/Text/Text-CSV-0.01.readme
Buy Text Processing in Python
As the original poster had been utilizing Notepad previously, vi/vim should both be considered significant upgrades.
It would help if the OP had posted the current number of records and projected growth. Size of the database and how it will be utilized goes a long way to determining a 'best' solution.
Most of the posts are going to be about a lightweight RDBMS like sqlite. I suspect however, what you are actually looking for is a fully fledged GUI frontent ala MS Access, but lightweight and F/OSS. Is that what you a really asking for?
To the question poser: use SQLite. Learn how to use a search engine too, for goodness sake. To the Slashdot editor who approved this question: improve. This is utterly woeful.
Class dbtxt includes the ability to undo changes, and features human-readbility for the flat database files. It's trivial and not error-prone at all to mod a database file to delete, or undelete a record, and of course you can do it through the database engine as well.
I've fallen off your lawn, and I can't get up.
I don't Free Base... I use (cue angelic/Tantric/Buddhist music...) da Lotus Approach.
DBF IV, relational front end... works with multitudes of backends, imports to and exports from them... WYSIWYG GUI.
Just sucks because IBM won't open up what it DOES own, and Approach (while is or at one time was sold separately in Japan, where SmartSuite was "Super Office") is stuck in SmartSuite, requiring windoze, won't run under WINE (not with my lowly skillz), and has no capability of stand-alone executable binaries...
But, for those who have half-a-leg in windoze, the other in Linux, and want a hella nice WYSIWYG GUI that can open MySQL (and other) tables, need decent (but basic) table linking... it comes with a slew of single database tables as well as linked projects samples. Even on-line there are free or near-free medical, rental, mental health and numerous other user-created apps, and user-supported and professional (free and paid) developers who moderate the support groups. Just query Approach Users Groups.
Approach is what ooBASE should have looked like-- 4 or 7 YEARS ago... Thanks, IBM, oo.ooo (not-created-here) and others...
Previously: "Linux... Toward the Sunrise..." Now: "Linux... Toward the-- No, now, part of Every Sunrise"
Why mess about with filty batchfiles, bashscripts and clunky sqlite, mysql. How about a full Oracle Database 11g Enterprise? o;)
I don't think the poster actually wants to *develop* something using a database, so all this replies like "use (sqlite|mysql|whatever)" are completely off. He probably doesn't want to have anything to do with SQL or just how the data looks like on the byte level!
As a reply to the poster: it's impossible to find a nice user-friendly F/OSS database that will mimic the ease of use of Access (and Access isn't really an overkill if you want something better than Notepad and less complicated than a "real" database). OpenOffice Base is the only reasonable contender (though it sucks worse then Access if you try doing advanced stuff in it) and it fits the requirements: works on Windows, file format is portable, and doesn't require Java (it *can* be used with Java but it doesn't require it).
Othen than that, just use Excel or OpenOffice Calc.
-- Sig down
Using the file system as a DB meets all your requirements.
Place one record in each text file, group according to a sensible directory structure.
BenefitsOne downside is you maintain the schema yourself, just like you are doing now, (or would do with a spreadsheet). The other, rather pedantic, downside is that it is not F/OSS unless you store the files on a file system that is.
Sounds to me like a better notepad might help as regular notepad is slow to load and use with large files. Also, learning SQL seems out of the question as it can a bit advanced for a hobby project. Perhaps the features of Notepad++ (free) or TextPad (free to try) can help as they open large files quickly and still work fast (Google for them). Note, the amount of system memory limits the file size that the notepads can work with. Perhaps learn to use the Vi text editor or Vim which can open and work with very large files and are not limited to system memory. Albiet, the user interface of Vi or Vim can feel frustrating to a new user.
If you've outgrown notepad you might as well save yourself some trouble down the road and switch to MySQL immediately. It has a program that comes included that allows you to import the contents of your text file almost instantly and all you need to do is give it the name of your file and the delimiter character you used to separate your data, | (pipe) is a good one. Seriously though, might as well just use a relational database. Microsoft Excel 2007 supports four billion rows if you absolutely must use a flat-file database (aka a spreadsheet) please, for the good mankind, just use a relational database like MySQL. It's really easy to learn and a marketable skill to boot.
1993 called to recommend PCFILE!
http://www.umich.edu/~archive/msdos/database/pcfile/
Mind you, you'll have to toss Jim Button $10 as it's shareware!
AT&ROFLMAO
Cardfile was a free program distributed with with Windows 3. Basically, it is a flat database that gives you a title line/sort key and around 400 free form characters per "card". It ran fine in Windows 9 and I think in XP. Might not work with Vista as it was a 16 bit program. There used to be a couple of freeware cardfile clones around, maybe one of them would do. Cardfile also would allow one OLE object per card, but that needs 16bit OLE which is not implemented in WINE and I expect not in NT based Windows either.
You can't see ANYTHING from a car, You've got to get out of the goddamned contraption and walk...Edward Abbey
I think what the submitter is looking for is something akin to the old Card & Field based consumer databases that were quite commonplace 10-15 years ago. e.g. there'd be a little GUI, you'd drag fields of various kinds onto a "Card" which would be the template for all data entry. The database would consist of filled cards. (I assume this idea came from the way old mainframe databases/UIs used to work.)
I was unable to find any modern implementations of this idea with a quick google search, which is annoying as they were a nice way to set up a little database of your CDs or books etc. Anyone know of projects like that? The only one I remember is from when I was a kid; "Supercard" on the Atari ST.
You can use mdac to create an excel sheet and both update and query it. It is running under Vista and you can do this in c# which is free like you don't have to pay for it if you own Vista. It can be done from pretty much an script host too. MS has examples, but it is pretty much just an ADO connection.
http://support.microsoft.com/kb/q195951/
---- aut viam inveniam aut faciam
I'm guessing he wants a FOSS version of FileMaker Pro; something that is an application with a GUI. All you guys are suggesting various frameworks he can use to program his own. No, I don't have an answer either, other than to suggest the spreadsheet in OpenOffice.
awk, sort, cut, paste, grep, sed All these utilities, or awk alone can be used to process dbs stored in flat files. That is their main purpose. With them, you can write procedures to edit and view your records interactively. And your dbs will still be accessible to any text editor.
Oops, poster didn't want Java-based stuff. Never mind. Open-Office is Java-based.
Table-ized A.I.
how about using NeoOffice or one of the openoffice variants to edit a spreadsheet format such as excel? I use excel for my checkbook and I had to turn off the auto calculate for my complex set of conditional sums after it hit about 4,000 entries, so I don't think performance as a database will be an issue for your uses.
Then if you want to export it to something else later or just to run numbers, CSV or other delimited format is compatible with most other database and spreadsheet apps. We've been testing neooffice for a bit here and it seems to be the best of the openoffice derivatives. I don't know if that particular one has a windows variant, but I'm sure you can find at least a few that work under windows.
I work for the Department of Redundancy Department.
set first "Spooky"
set last "Monster"
set email "spook@spammity.spam"
# In practice it's really easy to put values into variables
mydb eval {insert into contacts values (:first,:last,:email)}
The advantage? That code is now totally armour-plated against SQL injection attacks as well as being fast. Which is nice, really really nice.
"Little does he know, but there is no 'I' in 'Idiot'!"
See "Managing data text files like database tables", and also "How To Use RDO and ODBC Text Driver to Open a Delimited Text"
MOD THE CHILD UP!
try Sqlite or Qdbm ( http://qdbm.sourceforge.net/ )
-- Programming with boost is like building a house with lego. It's a cool but I wouldn't want to live in it
try couchdb, if all you really want to do is just store data....
It was very new a year ago.
"Thanks for all the money you paid to us. We've used it to buy off ISO among other things" -Microsoft
Nobody has yet mentioned that MySQL has a CSV storage engine - just create a table with type CSV and away you go. It does require the MySQL engine though.
Otherwise, I agree with most posters who say just use a simple text and/or xml file if the data volume is relatively small. That should be more than enough!
SixD
Many have suggested yaml, which can obviosuly be read on any programming language.
You are obviously going to invest some time on coding the persistance layer of your app. Take a look at http://datamapper.org/, an ORM for ruby. If you need a more scalable or robust data storage system in the future, just get another adapter (for a DBMS, for instance) and change zero code.
This would work on any OS, and from the sound of it pretty much solves your problem...except for the "having to learn ruby" if you don't already do.
The poster didn't want SQL, so I'm guessing they ruled out SQLite on those grounds. QDBM and Oracle/Sleepycat/Berkley DB-4 would be the smallest, fastest "pure" flat-file databases that are also cross-platform - at least, that I know of.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Kexi should be able to hande that pretty easily, and can even create a nice form for the data entry:
http://www.kexi-project.org/
Erik Dalén
I'm still using it for a customer database, it's two main problems are it's buggy and hard to use, and slow.
...
So really it's 3 main problems are being buggy, hard to use, slow* and totally bloated
Actually it's not half as buggy as it was a few releases back - I can actually use it now to fill in a form without it crashing. Just tried opening a windows XP made odb file in Ubuntu (OOo 2.4) and it's slow as hell and bugged to death, so you might add portability to that list of main problems.
[Yes I've filed bug reports].
---
* slow for searching, takes about 2 mins to do a straight text search on 500 records.
He doesn't need python. He just needs a database. He can download a precompiled binary for windows that allows one to work with the database at the command line. Python is not necessary.
And if the command line is too much, as others have noted there is already a convenient firefox extension for graphically interacting with a sqlite database.
A spreadsheet seems perfect for your purpose. I won't say Excel specifically, since there are lots of other free alternatives, but Excel is fine if you already have MS Office. (Since you mentioned Notepad, I'm guessing you're on Windows). If you need to access it with software (and you're not just using it as a list for yourself) you can export/import tab delimited versions of your spreadsheet. PHP, Perl and other languages have standard modules for handling tab delimited data (or just do it yourself if you're comfortable with that). From a spreadsheet you can filter, sort, tally, etc. and adding fields is easy.
A beginners' guide to Portland, OR?
Your problem is that you are looking for a "database". This is a very complex kind of program that is total overkill for your purpose. Will you really be storing hundreds of thousands of records and wanting to do some complex querries? Ifnot why not just use a speadsheet? If you want a free spreadsheet look at OpenOffice.
That's kind of one of the things that Tcl was designed for. You can easily embed an interpreter in your application and then your data is stored as a tcl file that can be parsed by the interpreter. Each line in the file would be one of your access points with a command that hooks your data into and out of your program.
Very simple, more than adequate performance for your data, readily hooked into pretty much any exiting code/language, with an easily extended "standard" file format. Love or hate the language itself, it's one of the things it does rather well as the "tool command language".
Cheers!
Sean
Comment removed based on user account deletion
You can use SQL queries through OLEDB onto an Excel spreadsheet and still be able to edit the spreadsheet manually if necessary. Yah, it's not F/OSS but I'll bet it's already installed.
Is it just me, or have people completely gone nuts?
How about a freaking text file, CSV?
Cobol even?
XML even?
If you just need a FLAT DB, meaning more than a 'format' and an 'interface' then freaking even make your own, write a few lines in Pascal.
Heck, on Vista you could write an application using the Windows scripting built in, and in like 5-10 lines to do everything you want, and stuff the data in every format ODBC can talk to from XML, CSV, to even MDB formats.
(OS level scripting today is not like the old *nix or DOS/bat world. They are extremely rich platforms on their own using common language syntax from JScript,VBScript,C#,etc..)
Holy scary cow. What do people think we did before dDase, Oracle, Access, etc for small localized non-relational data storage?
I swear SlashDot has gone to the script kiddies, or should I say, the unable to script kiddies.
Why one earth would you use a database to store one table on information? Save youself some trouble and use a spreadsheet.
- ZooLib C++ cross-platform application framework
It includes a database implementation in which the databases are kept entirely in a single file. One reason for doing so was that the databases could be used for end-user documents.It has two options for the low-level storage, one of which is fault-tolerant.
The database has been proven in real-world use, for example in Knowledge Forum, a multimedia client-server educational database.
It's not SQL - it's a C++ API. So you'll need to write C++ to use it. But it would be easy to write an application that would handle your user interface and that deals with the database internally.
If you use it, you'll want to subscribe to the ZooLib-Dev mailing list. Tell Andy I sent you.
And yes, ZooLib's terrible website is all my fault - I didn't know much about web design when I did it. I've been meaning to redesign for years.
ZooLib has the MIT license.
Request your free CD of my piano music.
Does anyone else think it's ironic that a person wanting to run a database on Vista has an issue with Java? Yea, must be Java's massive memory requirements, right?
Why not write it in awk? Grab a copy of "The AWK Programming Language", there's loads of examples of flat-file databases in there.
For that matter, some would say perl would do the trick as well.
But also, I'm puzzled by the "must not be written in Java" aspect. Why an exclusion based on the programming language - surely a tool that does the job does the job regardless of the language it's coded in. I can see the "not web-based, not complicated", but for a simple app that does the job, it shouldn't matter what it's written in.
"I want to build a house, but without using a hammer or a saw of any kind" - I just don't understand that thought process.
Insanity is a gradual process; don't rush it.
The current google search returns mostly entries referencing the Ruby version, but there's also a Python version.
Kirbybase stores its data in text files, so I think that counts as flat.
http://www.netpromi.com/kirbybase_ruby.html
http://www.netpromi.com/kirbybase_python.html
(s.a. http://www.netpromi.com/index.html )
Caution: I've never used this product. I just noted that the specs seem to fit your request.
I think we've pushed this "anyone can grow up to be president" thing too far.
Not one but two Tcl-based flat file database systems exist:
/rdb system (but does not require it). The data files are just ASCII tab delimited tables. You can use either Unix file utilities or a pure tcl interface program.
TclVSDb (Tcl Very Small Database) http://sourceforge.net/projects/tclvs/
Provides multiple hierarchical tables (with rows and fields) per database and multi-user concurrent access with locking. Database files are standard ASCII and are portable between platforms.
Starbase http://wiki.tcl.tk/3444
A simple relational database system. The basic table manipulating features are similar to the
How about a graphite-on-cellulose based solution? Advantages include tactile feedback, choice of open-format-specifications which everyone can read if you wish it, and the possibility for safe data-deletion through applied combustion just in case the pigs show up?
If you quote this signature there'll be 72 copies of Windows ME waiting for you in Heaven.
how about just a spreadsheet? Open office?
Why not the pickle/cPickle module? Not a flat-file but easier to manipulate since your just serializing native Python objects.
e.g.
import cPickle
class Record:
def __init__(self, fname, lname, email):
self.fname = fname
self.lname = lname
self.email = email
# ---
# Write out the data
records = list()
records.append(Record("John", "Doe", "jd@email.com"))
output = open("records.db", "wb")
cPickle.dump(records, output)
output.close()
# ---
# Read back the data
inputFile = open("records.db", "rb")
data = cPickle.load(inputFile)
for record in data:
print record.fname, record.lname, record.email
inputFile.close()
This is what I do for license keys that I keep on my thumbdrive. You don't need the speed and management of a database, so start in any spreadsheet app (you can use OpenOffice), and save the file to CVS format. If you don't have something like a spreadsheet app available at the moment, you can still just open it in a text editor. Thanks, Chaz
Berkely DB?
Firebird (http://www.firebirdsql.org/) works with many programming languages so it is not tied to java, works fine with windows vista and operates without an installed server in its "embedded server" mode.
http://www.firebirdsql.org/
I'd recommend Open Office BASE but I still sort of consider it a hack job. Too flakey, and may still be overkill. At least it's not a server program. - Kc
-- Kevin C. Redden kcredden@ gmail 392992
Seriously, if Notepad was all you were using before, and that was good enough, give Wordpad a shot. I've had to use Wordpad to open multi-megabyte .txt's when Notepad failed me.
Why not use Excel? A spreadsheet has a 65,000+ record capacity and can be sorted by the fields. As to portability you can save it as a csv comma-separated-values file. Don't have Excel?... I think all Windows systems come with free MSWorks. And it will do the same.
For a guy whose looking for a F/OSS app you sure have a lot of non-open requirements. You wanna make a private-only database on Vista?
I don't mean to be a total downer (in coach-z's words) Why is F/OSS even a requirement for you?
On a more helpful note you might consider that mySQL doesn't take much to run if your dataset isn't huge and as long as you've outgrown notepad you might consider it for super easy manipulation of data, versatility, ubiquity and simplicity. It meets all of your requirements except !sql. It's not overkill- it's easy. Setting up a local database is really really simple. If you were on a Mac I'd recommend MAMP- it's a self contained installation that includes and Apache/PHP for easy web interfaces.
Or use Python. And if it scares you, have a look at Google's App Engine. It include the GQL database API that will write to a local database in the development environment. The local database is persistent until you erase it.
Sorry, that label is already reserved for getting the guy who decided to suppress filename extensions in Explorer. But it can be a supporting example.
Shapefiles use a dbase file along with the shp file. Tons of software can read the dbase file and you have the added feature of storing the UTM locations of the access point and graphing it.
There is a c library called shapelib http://shapelib.maptools.org/ that allows for you to write the files easily. The c code can be compiled and run on vista and almost any other OS.
How about that.
W3rdna
After trying all sorts of PIM software and PDAs over the years, that's the one solution that I've found most useful. The only real drawback is the steep learning curve before one gets comfortable with vim's more powerful features.
SQLite works from most languages and on most platforms (even embedded ones). It's perfect for most single user things -- although, it's ... er ... skewed toward SQL.
Imagine if you weren't allowed to use roads because a bus company complained about your driving 3 times. --skunkpussy
If you're partial to Lua, it has SQLite bindings (and various other DB engines).
I've not used LuaSQLite personally so I can't give you a code sample, but I wouldn't imagine it would be much more code than the Python example.
As a bonus, Lua was primarily designed as an embedded language (of course Python can be embedded too).
Homonyms are fun!
You're driving your car, but they're riding their bikes there.
From your requirements, looks like a spreadsheet would be sufficient.. Should be better than notepad for sure.. You can always export to CSV and import to database programs later..
I'm not even going to go into the issue of whether it's a good idea to store a database in XML, but you're kind of proving the other guy's point for him when you call that "human readable".
Why not use Oracle Apex. Its Oracle's Free version of there database. Stable as a rock, is pretty light on resources and it also comes with Apex which is a RAD tool for developing quick web based apps for the database with little or no programming knowledge.
...because he wants to hack his own system...
It's not too expensive, but I would love to find a non-Microsoft equivalent, just because, well, just because.
I've never seen anything like that, though. Has anyone? It seems like most db programs want to be told which fields to search and not simply search all fields. On the other hand, I could be totally and irrevocably wrong about that.
Meanwhile, for a small amount of cash, the Works database module would be ultra-simple and appropriate for use by the original poster.
It isn't free, no, but older releases of File Maker Pro can be found on eBay for dirt cheap. It is flat file in its older incarnations. DBs are simple to create, as are forms, searches, etc.
As a bonus, it is cross-platform with Mac OS versions of File Maker Pro. It is well-supported with a rabid following.
Bearded Dragon
NT has a pretty powerful DOS batch scripting program. I'm not entirely familiar with Vista, but I suspect it still retains that capability. Here's how the Vista (and XP and Windows 2000 and NT) batch scripting will suit your needs: 1) simple 2) works in Vista 3) portable 4) not an online app 5) does not use Java It does fail miserablely in the F/OSS requirement, however - it will be proprietary to your needs. But on the upside, there are no libraries to install -- it's all built in. --------------- DOS scripting can do variable expansions, for loops of numbers, for loops of directory entries, and such. It can be configred to read in entries from flat files, and parse each line and split arrays, and such - perfect for your needs. --------------- Failing that, you can write simple Visual Basic files (.vbs files) that you can execute from the command line. These things are great, because you can use the Windows Scheduler to schedule execution of these batch files, the batch files can spawn other batch processes, kick off other jobs, etc. http://www.computerhope.com/sethlp.htm http://www.robvanderwoude.com/variableexpansion.html http://www.maem.umr.edu/batch/dadd.htm
What issue are you trying to solve? Is it a matter of managing the data, or querying/reporting on the data? If it's managing the data, then a spreadsheet is probably a good choice. If it's querying/reporting, since you've already got the flat-file with Notepad, you might look at LogParser from Microsoft (http://www.microsoft.com/technet/scriptcenter/tools/logparser/default.mspx). While not F/OSS it's very good at parsing data from various file types for queries and reports.
Lotus APPROACH was a simple data base back when. It had a good GUI front end for a really good and simple Report writer section and could be either a simple relational database or Flat file. I still use it for little personal type things and you can still find it by some searching.
Yes, it's a shame. I can honestly say that I use the OpenOffice spreadsheet every day
and it works great. I also really like OpenOffice draw and use the Writer and presentation
portions all the time.
But Base has a long way to go. (IMHO)
"We can't solve problems by using the same kind of thinking we used when we created them." -- Albert Einstein
The sqlite3 module as shipped with Leopard says that it supports question mark substitution, so his example would be better written like:
BTW, the same could be written for PostgreSQL like:
Hopefully sqlite3 will get the same standard substitution in the future, but at least it is possible to do it safely today.
Dewey, what part of this looks like authorities should be involved?
- Despite popular opinion, I am not perfect.
Yup, that's it.. While not FOSS, I really like Brilliant Database. It's cheap, brain-dead simple to use, supports one-to-many relationships, and even has a 30 day free trial. I used it to put together an simple app for managing voicemail numbers at a previous job (we had a pool of 3,000 DIDs we assigned to consultants as they came and went - a real db would have been overkill, but a spreadsheet was cumbersome). I created the DB in one afternoon and never had any problems with it.
Not affiliated with them at all, just a happy user.
Today I didn't even have to use my AK; I got to say it was a good day -- Icecube
Why the F/OSS requirement? You clearly don't care about the "Free, as in speech" ideas. Your requirements are:
1. Run on Vista. Vista?
2. Doesn't cost me anything.
3. Can't use an open source runtime called Java.
Just go out and pirate a copy of Office from the same place you pirated Vista and Grand Theft Auto. Then you can use Excel. I think it will meet all of your requirements ($0).
If you just want to serialize your object data from an array of objects you can simply use Perl to tie to a DBM file. It will allow you to load your object data when needed, process it, add to it, modify the objects then serialize again back to the DBM file. It's portable, small, fast and very well suited to simple solutions.
http://www.gibby.net.au
Been searching flat files with this open source VB5 program since 1999. Searches flat files at 20,000,000 cps displaying hi-lited hits in matching lines only or full context mode. Results can be exported to a results file. Append txt files together. Search and replace option. Run multiple versions with various default settings. I have been yapping and yapping about flat files, only to get great amounts of resistance from the Geeks. The search uses lots of defaults and quick keys (for typists). From startup to shutdown nothing is faster. Channel9 and thedailyWTF.com have the best threads. Check it out. It's the only program you'll ever need
Challenge: I have better access to my Video, Music, Pics and Text than anyone on Earth.
(off-topic) blippo, that's the best description of XML's human readability I've ever heard. You've gained a fan! (and you are now quoted on our bugzilla motd)
1) My example was done to show how quick and easy it would be to set up and populate a table in Python.
2) He's explicitly mentioned several times that this is for his own private non-web use.
3) Your fly is open.
- Despite popular opinion, I am not perfect.
Actually, the Python SQLite adapter supports the standard Python DB API (PEP 249), and so you can just follow its recommendations (which includes using placeholders).
Not your fault, I'm sure I was being unclear; you can monkey with class dbtext database files with a text editor for data mods, deletes, undeletes, record additions and so on. They're specifically designed to make that easy and less error-prone. It's just that you can use the database engine for delete and undelete as well.
I've fallen off your lawn, and I can't get up.
Why not just use something like a CSV file. It can be opened in Excel/Open Office/Many other programs, if you want to sort it etc, or can be opened in any other text editor, so it is very portable. Seems like a good "Flat File Database" solution to me....
I know of a couple of database engines that use tab delimited ascii files to store data: qddb - http://www.hsdi.com/qddb/ and nosql - http://www.strozzi.it/cgi-bin/CSA/tw7/I/en_US/NoSQL/Home%20Page (mentioned in another comment)
"Find a copy of Q&A for DOS ver 4, it runs under WINE...Another great product is Paradox Dos 3.5 or 4.0, published by Borland..."
Why does one need to run WINE to run a DOS app? Dosemu is a low overhead DOS emulator that is far better suited to running DOS apps on Linux with minimal resource/configuration overhead.
http://dosemu.sourceforge.net/
http://tobu.lightbird.net/overview.html
A database app written on top of sqllite, with tag support for each of the entries. The interface is a little kludgy at this point but looks like it could become a very useful app.
If you're on Windows anyway, then go with the .NET framework for a custom solution. It contains data access classes (ADO.NET) that can query a database (which is overkill for your needs) but the same data queries can also be applied to CSV text files., Excel spreadsheets, etc. as if they were database tables. ADO.NET will abstract away the true data source for the most part. Therefore effectively, you can use SQL query syntax to manipulate your text file, Excel spreadsheet, or many other formats that you might choose or already have on your disk. It's very powerful and elegant, but may require some research.
I've used this successfully on several occasions. It's available from CPAN and requires only Perl. Here's a write about extending it a bit to do joins. http://www.builderau.com.au/architect/database/soa/No-fuss-SQL-joins-without-a-database/0,339024547,320268666,00.htm
dbm files are available in several languages, e.g., Perl and Python.
Or use the Prolog language, which allows easy storage and retrieval of records (Prolog calls them "facts").
Emacs is your friend.
Available cross-platform, is the poster child for Open Source, is extensible to add whatever specialized field searching you want (if basic C-s isn't good enough), lets you use your own format, heaps better than Notepad, smaller than OpenOffice, etc.
Or, if you want something smaller, use nano, jove, or one of the small Emacs work-alikes. Notepad, while available on every Windows system, isn't a very good editor, and it sounds like you just need a better editor.
Put my fist through my alarm clock with its ding-dong death inside my ear. - The Blackjacks.
echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
Yes and no. While it's DBAPI 2.0 compliant, it only supports the "qmark" paramstyle:
(Python 2.5.2 on FreeBSD 7.) The "named" paramstyle looks an awful lot like dkf's Tcl example. My favorite is "pyformat", such as in my PostgreSQL example, but alas sqlite3 was telling the truth above when it claimed not to support it:
"qmark"-style parameters are just fine, though:
Dewey, what part of this looks like authorities should be involved?
just install Cygwin (http://cygwin.com) and learn the command line tools (awk, sed, etc.)? I realize it may require working at the command line (in the Bash shell), but it sounds more like you're doing text manipulation more than anything else. The 'built-ins' provide a very powerful set of tools that can be strung together with pipes, used in shell scripts, etc. Also look into regular expressions as they will most likely provide most of the punch that you'll need.
And for those who don't know Cygwin, it's a Unix-like environment that supports a number of standard 'Unix' tools.
actually my take was he was using notepad to generate and maintain the "database" and wanted to graduate to something a little more sophisticated like a entry form to put records into a flat file like one that is pretty trivial to program in Perl, python or php
Apocalypse Cancelled, Sorry, No Ticket Refunds
I have to agree. You're worried about outgrowning Notepad? Just use WordPad. Problem solved.
For the benefit of the poster, and 90% of the computer using public that seems to make this mistake: SQL is Structured Query Language. It is a way of accessing data in relational databases. It is not a database in itself. Yet Microsoft re-branded an old version of Sybase into SQL Server, and now everything is "SQL." Never mind that EVERY database out there uses SQL to access data, now, whenever you talk to many people, they run "SQL and Oracle!"
A statically linked app written in C++, with Trolltech's Qt4.4 + SQLite3 support will do the trick. If the statically linked binary results too big, just shrink it with UPX, or install the dynamically linked dlls with the app.
Spreadsheet
File Express is relational, works under WINE.
File Express is great, and I've used it for a lot of things (mailing lists, among others, it's got great dupe checking capability). OP, however, was looking for free software, and FE is not free.
Neither, I think, are the other programs you mention, though some are abandonware.
... because I haven't seen a similar product on Windows yet. It's not even free (as in beer, or as in freedom) but it gets the job done.
XTabulator
Why not just use xml? You can put it in an archive if you want or use object de/serialization for convenient programming. If Access is overkill you could even just use a damned flat file. God! Why is this even something that you'd put on Slashdot?
I know that feeling.
It's called LGPL, and it should be used more often, IMHO.
You can create the file using the command line utility which comes free with SQLite.
No Python needed.
Sorry for not having enough Python in my solution, but there you go...
No sig today...
That leads to a followup question, then: can anyone suggest a simple spreadsheet program? OpenOffice and Excel have too much bloat, and take too long to load (well, NeoOffice does; can't speak to the others), when all you need is a way to quickly read and modify tables of data.
Outgrown Notepad?
You should try Notepad++.
It supports column editing and all the fancy works you could need if you're not looking to get your hands dirty with programming.
You could also try storing your data as XML and using any of the readily available XML parsers in virtually every language on every platform; however, you can expect it to hog up lots of memory and be fairly slow, especially if you need DB-like functionality like sorting.
Move all sig!
Which is fine. Adapters don't have to support all the styles, they just have to tell you which one they support. And since the value of the paramstyle constant is a string, a given adapter only gets to pick one. From there it's not too terribly hard to write flexible code that generates queries with the appropriate placeholders.
SQLite might be a "huge" 800k (but then someone advised using OOo ...), but it's used by so many apps (including Firefox), and its API are so clearly defined (C API+SQL) that you can easily consider a quasi perfect black box and drop it in. Simply put, if you roll your own flat file DB managing with file locking and such, you ARE going to have bugs, more of them than SQLite has.
I often prefer using Perl for such easy tasks:
Text::CSV
will handle a CSV file for you quite well (keep in mind that separators need to be distinct from the entered text). It can associate arrays and hashes to the lines.
And even better, if you later feel like your Text::CSV handling is not working well enough, you can switch to or simply add code using:
DBD::CSV
which internally uses Text::CSV anyway. This way you can run SQL queries on your previously created CSV files.
References:
http://search.cpan.org/search?mode=dist&query=Text%3A%3ACSV
http://search.cpan.org/search?mode=dist&query=%3ADBD%3A%3ACSV
TextDB: If you don't have access to a database but you do have access to PHP, and you want your web data stored in a database, then this is what you're looking for! Full sql compatibility is planned as well as a perl port. This is OS-independent and is licensed under GNU GPL, has a web-based interface and is developed using PHP. (check out: http://sourceforge.net/projects/textdb). Berkeley DB (BDB): This is a computer software library that provides a high-performance embedded database, with bindings in C, C++, Java, Perl, Python, Ruby, Tcl, Smalltalk, and many other programming languages. Berkeley DB is redistributed under the Sleepycat Public License, an OSI and FSF approved license and runs on a variety of operating systems including most Unix-like and Windows systems, and real-time operating systems. It comes in three different editions: Berkeley DB(originally written in C), Berkeley DB Java Edition, Berkeley XML DB (check out: http://en.wikipedia.org/wiki/Berkeley_DB)
I release this code under GPL ;) and yes ... it has bugs... use under your own risk...
.CMD FILE on Windows
:START
/P _exCommand=Enter your command (%_cm_exit%,%_cm_list%,%_cm_delete%,%_cm_add%,%_cm_find%):
:LIST
:FIND /P _line=Search for: :DELETE /P _line=Delete all matching: /V "%_line%" TEMPFILE.TXT :ADD
/P _line=Enter the new host entry (location,name,ssid,notes):
:END
--- CUT HERE and ADD THE REST TO A
@ECHO OFF
SET DBFILE=dbfile.txt
SET _cm_exit=exit
SET _cm_list=list
SET _cm_delete=delete
SET _cm_add=add
SET _cm_find=find
IF EXIST %DBFILE% GOTO START
ECHO LOCATION,NAME,SSID,NOTES > %DBFILE%
SET
IF %_exCommand%==%_cm_exit% GOTO END
IF %_exCommand%==%_cm_list% GOTO LIST
IF %_exCommand%==%_cm_delete% GOTO DELETE
IF %_exCommand%==%_cm_find% GOTO FIND
IF %_exCommand%==%_cm_add% GOTO ADD
CLS
@ECHO Invalid Command
GOTO START
more %DBFILE%
GOTO START
SET
FIND "%_line%" %DBFILE%
GOTO START
SET
FIND
MOVE TEMPFILE.TXT %DBFILE%
GOTO START
SET
ECHO %_line% >> %DBFILE%
CLS
ECHO New entry added
GOTO START
ECHO Thank you
I'm not sure of the cost but we used to use C-Tree quite a bit to handle our ERP objects. Its a C based flat file DB which one of its versions supports SQL http://www.faircom.com/products/ctree/
Use the free .Net IDE with an XML-writer and a dataset. Datasets can be filtered is a SQL like manner.
Privacy is terrorism.
Hmmm... Anyone but me note the disparity? Those three things CAN be done. Mod me down if you gotta but I think my question is valid. Why has it got to be all three? Does it really matter if it just "works?"
"So long and thanks for all the fish."
Why don't you just store everything in a textfile, or in several textfiles? You can simply use "grep" to do searches, and another advantage is that you can define the format yourself. No worries that the file format is not being supported in the future. Ok, you've outgrown notepad, but you can write some very simple scripts to ease the task of data-entry. If you don't know a scripting language already (I would prefer python in this case), then this is a nice motivation to learn one.
If Pandora's box is destined to be opened, *I* want to be the one to open it.
Take a look at Metakit. http://www.equi4.com/metakit/index.html. Its single file, small, fast and it has proven itself over many years. It is written in C++ and bindings for Python and Tcl are available. Instead of tables, rows and fields it uses similar concepts called view, index and property. Interestingly, a property may also be a subview thus allowing a mix of a relational database (flexible) and a hierarchical database (fast).
Have you used Netstumbler? It can store plenty of access points in details.
Didn't read all the posts so far but most folks seem to have forgotten Python's inbuilt "dictionary" type and pickling. If it's a fairly small DB then that (without even Sqlite etc..) would do the job. My bet would be try the inbuilt dictionary and if your data needs get too large (or queries too complex or ad hoc) use sqlite.
Andy
As has been said before, OpenOffice Spreadsheet will do the job,
and specifically, there is a Dataform function built into "Data" on the menu.
1. Type your headings in the first row of the spreadsheet
2. Highlight the row as far as the end of the headings,
3. and down for as many rows as you want to include (for now)
4. Click on "Data" then "DataForm" and it will automatically create a "data entry form"
5. Type info in the fields and TAB, then TAB once more to "New" or hotkey Alt+N
We've created a monster! err, a small flat database file of sorts.
I really miss QA by Symantec *sigh*
Don't blame me, it's usually 2 in the morning when I post
Then replace it with e.g. Notepad++ or EditPad or many of the other programs.
Then just use tab divided 'fields' in your txt (or csv) file.
Or if changes do not happen often, you can put it in html file with tables. That you can look at with your favorite browser. Doubleclick the html file and you have your details in nice columns.
Don't fight for your country, if your country does not fight for you.
It is fast. There are lots of applications, also GUI. My largest single table database is 5500000 records. It returns nice tables in between 7s and 15s.
Pål
Try Kexi (http://kexi-project.org) it has features like strong CSV support, MS Access import, and SQLite is the default engine. It's LGPL, no Java and started long before OO Base.
2) He's explicitly mentioned several times that this is for his own private non-web use. But when doing so using good practices is so amazingly easy, why do even demo examples in a different way? It's not like you're using Java or C#; the right way doesn't need to be bureaucratic...
BTW, I checked the python docs and I'm happy to report that they strongly promote doing it the right way too. Thus you can do it right for almost the same amount of code:
mydb.execute("insert into contacts values(?,?,?)", ("Spooky", "Monster", "spook@spammity.spam"))
(I could comment on the merits of named versus positional parameters, but that's getting too close to language flaming. Off topic for here, and uninformative in any case.) 3) Your fly is open. Now you're just making yourself look like a silly kid.
"Little does he know, but there is no 'I' in 'Idiot'!"
There were also other problems which perhaps it could have solved if I knew it better, such as chopping some of the fields from 11 characters to 8 (which of course made the entries non-unique), but it was easier to install vim and do an ed-style edit than try to find how, since there was enough other manipulation I wanted to do that was easier that way. If I'd been working in Linux instead of Windows I would have checked out recent versions of the old v7 join, but I wasn't going to install Cygwin just to do that; perhaps I should have.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
When writing examples, do it right. You're doing the questioner and the community a service.
"Little does he know, but there is no 'I' in 'Idiot'!"
The problem wasn't the example per se. The problem was that it didn't promote good practices to a noob. That's cruel, and not just to the noob either. I hate being drafted in to clean up after someone else's mess that they made through cluelessness, and I bet you do too. Consistently teach people to do stuff the right way, and when it comes to somewhere where it matters, they'll get it right and the rest of us won't catch it in the neck down the line. Self-preservation really.
"Little does he know, but there is no 'I' in 'Idiot'!"
it's not really a database but perfect for structured local storage, in fact i use it for exactly what you describe.
http://www.tiddlywiki.com/
from the front page:
"It's written in HTML, CSS and JavaScript to run on any modern browser without needing any ServerSide logic. It allows anyone to create personal SelfContained hypertext documents that can be posted to a WebServer, sent by email or kept on a USB thumb drive to make a WikiOnAStick"
I think the guy is interested in a database program like Base or Access *shudders*, not a database engine like SQLite or MySQL.
Programming is the art that actually fights back!
Concatenting strings with user values into SQL... bad. Yes. Generalizing to the point where you suggest that quotes are bad any time any place... well kind of makes you sound like a niave noob yourself and invites these sort of comments. Not every SQL statement has user-derived values. Not every SQL statement needs to be parameterized (not every API in the world supports it, either). And not every application is a web application, or deals with hostile input. And implementing security through "coding standards" and "best practices" is a cookie-cutter approach and no substitute for deep groking. If somebody on my team is learning how to code from Slashdot... I'm leaving that team.
ON THE OTHER HAND you are absolutely correct in that BRINGING SQL INJECTION ATTACKS UP in whatever context is a service to the community and a good thing.
And pointing out the underlying issues involved... and that things are not always so black and white... is also a service to the community.
There are quite a few of the usual ideas listed here (Spreadsheet, Buzhug, get a better Editor, etc.). However I second those that suggestes Kexi, as it also has a flatfile DB built in (I just checked). You can build your own forms with it and have your own, custom single table Database App up and running in minutes.
Kexi is the way to go, and you'll even know your way around a neat DB frontend when you need to connect to some SQL persistance later some time.
We suffer more in our imagination than in reality. - Seneca
well, if you have put up with enough abuse to read this far you deserve some sort of answer. Take a look at Treepad (www.treepad.com), it is available for both linux (native or via WINE) and windows and can export a subtree to html.
Do you always ignore requirements when designing a solution, or are you making a special exception in this case?
If you were blocking sigs, you wouldn't have to read this.
Does SHSQL fit your needs ?
WYSIWYG Editor ? VI ! I see text, I get text.
Yes, one can create work-arounds with GPL'd library code, like compiling the library separately and building a socket or messaging interface to it (which interface of course has to be GPL'd as well), but this can be a PITA as you point out. And, as I'm sure you'd agree, workarounds like that are not always possible for performance and other reasons.
Yes, authors who use the GPL have decided that we can use their code in a certain way, and that's their privilege. However, it can be a dumb choice, because as the grandparent poster points out, those of us who do rely on proprietary licensing models for our revenue (and that would be the majority of us) are often unable to give anything back to the community as a result. I'd love to be able to use some GPL'd libraries in our stuff, and I'd love to be able to piss in those libraries to contribute bug fixes and help out in general. But the GPL forces me in another direction, usually to reinvent the wheel or to use something that's closed source and potentially less capable.
If you author a library, think hard about the licensing model. The GPL may not be the best choice -- unless, of course, you have a religious belief that all software should be GPL, which is your decision to make. I don't get into religious arguments, so that would be the end of the discussion.
Bulletproof and almost as fast as DOS
ZGSpro
http://zgspro.com/
E-mail: zgspro@zgspro.com
I can't be certain it runs on vista but if it doesn't I know what I would do.
If debugging is the process of removing bugs, I suppose programming must be the process of putting them in then.
As long as you only use one table, it's a flat file.
Should you decide to go hog wild and split your table into a relational set, it's all there and available.
Can't believe the solutions offered here tonight folks.
.csv and .gg is the native format so you can read it with anything. exports to damn near anything.
Try Table pro, I know. I have run through dozens and dozens.
http://zgspro.com/
E-mail: zgspro@zgspro.com
ZGSpro
About as fast as DOS. WMCYPAF?
Uses no custom database components or controls, and does not change the Windows Register or anything else outside its own directory
It has a small footprint and makes few demands on computer resources, so TablePro can be left running on the desktop.
koonaone
If debugging is the process of removing bugs, then programming must be the process of putting them in I suppose?
Yeah, you *can* nuke the file by using the '>' operator rather than the proper one in the context, '>>'.
What's life without risk?
Technology -- No Place For Wimps! Grateful Dead and Jerry Garcia Chatroom -- http://www.wemissjerry.org
use hamsterdb! it's a file-based database for C with wrappers for C++, Python, Java and .NET.
;)
it supports database cursors, multiple databases per file, can run as in-memory database, and the next release will support logging/recovery.
license is GPL 2 or later.
oh, and i'm the author
http://hamsterdb.com/
http://freshmeat.net/projects/hamsterdb/
The original poster is asking for a solution without providing all of the requirements. The original request is for a database program to replace a Notepad based text file, and yet most of the responses on the 6+ pages of comments have been around which programming language has bindings to which database, or discussions on SQL syntax. It would be helpful if the OP could elaborate a little on whether (s)he even has programming skills.
I read the request as looking for an application to replace the Notepad system. If the OP was just using a text editor, then it doesn't really sound as though (s)he needs SQL. A flat file of records with the ability to do straight text search sounds like an option. The question then is what type of tool can be used to manipulate that system?
Following the text editor/file model, one option, though larger than the 800kb discussion around SQLite could be Emacs with Forms Mode. This is also a (very big) text editor, but allows you to view a file of records in a nice editable form, but you can still open the record file directly and use all the searching capability of an editor. Of course this is probably very much overkill for someone used to using Notepad, but I suggest it merely to point out a type of option.
Another potential option might be to use one of the wiki-on-a-stick systems. Tiddlywiki and Stickwiki would allow the OP to use a browser with a wiki interface to record the data into a single transportable file. (S)He could organize how ever desired, though if the OP wants to be able to extract the data for loading elsewhere, that could be a little problematic as these tools tend to store the text in the generated HTML page in little division blocks. Some may offer export capability though.
So, the question is what are the full requirements, what skills does the OP have, how much configuration and/or system administration is (s)he wanting to do to have this simple system, and how do you wish to manipulate the data that is captured?
If you want to hand edit it, then some tool that saves records in a flat structure (one per row or one per file) that can be manipulated by a text editor is needed. If all you want is some way to record information and search it, then there may be other options. Many PIM tools also offer limited record management systems that you might look into.
Did I other imply otherwise? :-) I just wish the sqlite3 module maintainer would update it to use pyformat params, although it's still perfectly functional.
Dewey, what part of this looks like authorities should be involved?
If you want, give me the specs. I will write it in C with the MinGW compiler.
I will read the file into a binary tree, have the program do what ever changes, input/output you want, then write back to the flat file any changes.
This should only take about an hour to produce. You can have the source when I am done.
The entire executable will probably be about 40kB when compiled.
- I live the greatest adventure anyone could possibly desire. - Tosk the Hunted
If all you're wanting is an easy to use flat-file -- just use excel and learn how to use their VLOOKUP / HLOOKUP functions. It's about as lightweight as you can get (assuming you own excel already).
Unix admins have been using some very basic parts of the unix toolkit for decades when they just need to toss around smallish flat files. You may have outgrown notepad, but I'm sure you haven't outgrown vi & awk, or libdb. Ask some people what they did before there were dozens of oss relational databases (actually, I bet their still doing what they used to do, when the size of the job requires it).
KLAATU, BORADA, NIh*ahem*
I use hsql. It has many options: flat file, memory just to name 2.
I have also heard good things about H2 and apache derby. These have been especially valuable for embedded apps
Foxpro is old, but is very powerfull as a flat database.
I like notetab a lot, wish he'd port it over to Linux...
waiting for ad.doubleclick.net
yeah, I think he really does. GP is right: to request a solution that's both open source _and_ runs on Vista at least entitles us to know why in the world those two mutually-exclusive philosophies are both requirements.
... nah, couldn't be.
Seriously, didn't this strike anybody else as bizarre when you read the article summary? If F/OSS is a requirement in the database portion, why not in the OS? Better yet, if it's a requirement in the database portion, why not just pick an OS best-suited for that task (bdb on *nix is the canonical answer, and has been for 10+ years).
It's almost as if the article summary were a thinly-disguised troll
illum oportet crescere me autem minui
It seems that you're prepared to run Windows Vista.
... ).
... when it comes to something to record information about "Wi-Fi access points that I come across", well, that's different. That must be open source.
You probably use many devices daily that you don't have the source code for their microprocessors, (microwave, alarm clock, TV, telephone, car
But
Why?
People you are suggesting things like python, Java, OpenOffice, various SQL. What would Dennis M. Ritchie or W. Richard Stevens think?
To add a record:
cat >>db.txt
man cut
man paste
Then SELECT becomes grep or awk 1-liners piped into a cut command.
If you want a flat-file API there's FlatDB.Net. It has no front end, but is really easy to use from code. Works with Mono too.
Swoogan
sigs are for losers...and ppl who can think of one.
"...because he wants to hack his own system..."
No, he doesn't need to worry about he himself hacking his own system. But he does need to worry about using bad coding habits.
It often happens that some code that someone threw together is being used 20 years later on a big system.
I very much appreciated the grandparent comment.
Huh? What if he has Vista already because it came with the comp and he doesn't want to pay for any software and he wants to be able to modify it and get the source?
How about Kexi? It is not clear that you want database for programming, so something like MS Access might help. http://www.kexi-project.org/
http://www.sqlite.org/
Probably one of the more advanced flat-file serverless SQL database systems with bindings for many languages, and source code to poke at if you feel the need. They say it weighs in at about 250KiB with everything enabled.
From the site:
SQLite is an embedded SQL database engine. Unlike most other SQL databases, SQLite does not have a separate server process. SQLite reads and writes directly to ordinary disk files. A complete SQL database with multiple tables, indices, triggers, and views, is contained in a single disk file. The database file format is cross-platform - you can freely copy a database between 32-bit and 64-bit systems or between big-endian and little-endian architectures. These features make SQLite a popular choice as an Application File Format. Think of SQLite not as a replacement for Oracle but as a replacement for fopen().
BeauHD. Worst editor since kdawson.
if access to the source mattered, he wouldn't be running Vista. conversely, if he's comfortable running Vista, running a closed-source app alongside all his other closed-source apps on his proprietary, DRM-ridden OS shouldn't be an issue.
I can see reasonable people going both ways, myself - I just find the particular combination of requirements by the OP to be unusual enough to warrant further explanation.
illum oportet crescere me autem minui
> There are very few cases where real-world information is not most accurately
> represented as relational data (in comparison to flat data)
I have a question about this. All you database types (and I do not necessarily mean this in the pejorative sense) never seem to mention how you actually get the data out of the queries. No, I don't know much about databases, as you can tell. However, there does seem to be a strong bias in all db discussions toward directly placing the query output, in text form, onto a web page or some report. By contrast, in my code, I always define my data structures first (C++), and then wonder how to get data from the database into them for manipulation. SQL is entirely text based and all the examples I have seen extract data from queries by looking up a field by name and converting it into a variable (from text?), which seems to be not only inefficient, but highly cumbersome. I certainly don't want to copy data into my structures field by field, and the performance hit (which may become pretty significant after you go through a few million records) appears to be a showstopper. None of the tutorials I've seen demonstrate this particular operation in any other manner. The SQL books never go into code examples at all, and concentrate on forming queries. So tell me, oh db guru, is there some way to directly read a db record into a C++ class, or a query into a C++ vector of such classes, without doing a strtol on every field, like most examples seem to imply?
That doesn't follow. First of all, access to the source might "matter" for this application, but not the O/S (some people care about OSS for practical reasons that vary by the particular application, not out of pure ideology.) Further, access to the source could matter in both cases, but other concerns could apply to the O/S that outweighed it, and those concerns might not apply (or might not apply as strongly) to the application in question. Finally, the O/S may not be within the control of the poster at all.
http://www.firebirdsql.org/
Accurate and very full SQL implementation, fully transactional and ACID compliant, stores everything on disk in a single file, and can be archived in a transportable format for restoring on different OS/Architectures.
I think any scripting language that supports lists as first class variables should do the trick. You can just keep the source file as a flat-file db by itself. Python comes to mind first as its simple and easy to understand, and you can easily manipulate/query the contents of the list (and even refactor it since string functions are simple in python) using a few lines of additional code.
all of that may be true (although there's an awful lot of unrealistic "maybes" involved, particularly as to whether or not the OP controls what OS is in use - go read the article summary again, specifically the portability requirement). I still think the particular combination of requirements was unusual enough to warrant further explanation - or at the very least, to make Brysanix' flame of somebody who responded reasonably enough to "just use Excel or Calc" _un_warranted.
Think about it: if you care enough about the app being F/OSS to make it a requirement, but at the same time make Vista support a requirement, would a reasonable person (in this crowd, a reasonable geek) not be justified in asking what prompted such a half-assed^Wunusual set of requirements? If you're competent enough to be running your own DB and specifying requirements like "must be F/OSS", running it on *nix somewhere is clearly not an obstacle.
(I think this particular horse has been beaten to death now, so I'm done on this thread - a number of solutions that match the original requirements have already been proposed, and although I remain curious as to the rationale, that curiosity will probably remain unsatisfied.)
illum oportet crescere me autem minui
I came into this story wayyyyy too late, hope you can find me amongst all the SQLite arguing...
I use FlatFileSQLDB in many of my production projects. It is written in PHP and is very easy and lightweight. Much is built from PEAR libraries. Send me a message or email if you are interested. The production websites are in my website portfolio. FYI, I have made updates to this DB since its sourceforge release a few years ago, if you are interested.
If I had a sig, this is where it would be.
There's like what, 600 replies to this and almost every one of them talks about writing something yourself (python), or pointing to some type of library (sqlite). You people are total ignoramuses.
An APPLICATION that can do this, which USES the sqlite format, is "tkSQLite". http://reddog.s35.xrea.com/wiki/TkSQLite.html
It's an all-in-one GUI frontend to sqlite files, and gives you a decent GUI to work within.
Seriously, the more I see tech people try to offer their help, the more I realize why tech people get paid mostly crap and get no respect from managers, businesspeople, or anyone else.
What about RRDtool? It's more designed to be used for graphing time based metrics but you could probably use it for other stuff. It's very lightweight and uses a flat file.
I think you can still find it out there. Worked great, one of easiest to use DB applicaitons and should work fine under dosemu. Good luck.
I'm looking for a simple - comes with examples in the help file .BAT script file
free - its IS free
F/OSS flat-file database program - that's its purpose
MS SQL, Access (MS), and Oracle, all of which would be overkill for my purposes - runs with simple
must work in Windows Vista - it does
preferably use a portable format - it uses CSV, TSV, TEXTLINE, TEXTWORD, XML (and 15 other formats)
must not be an online app - its run from the command line
must not require Java - requires nothing but windows
Does such a beast exist? - yes, dozens as you have seen by this thread
simply
logparser "SELECT field1, field2, field3 FROM filename.csv" -i:CSV
And it has all the power of aggregation, sorting, filtering, and many output formats (chart, datagrid,CSV,IIS,NAT,SQL,SYSLOG,TPL,TSV,W3C,XMl) or even directly into a database
Are you nuts? If you'd been on some kind of unix, I could have shown you half a dozen lines of shell combined with sed, grep and sort that would do the job.
Stick Men
Sometime ago I was working with some CSV files and found this program "Commaworks" useful. You can download it from jansfreeware.com. It is a self contained CSV DB & editor.
Your requirementsâ(TM) as you have described them would leave me to believe that you do not need a database. You do not need multi level locking, you do not need centralized access from multi users or machines, you do not need atomic operations on data, and the amount of data in question is not significant so no indexing is required. Just to name a few good reasons. In addition the over head with any resources used by any relational database would not be necessary. Simple flat files should do you fine. And as far as parsing them or formatting them a widely implemented interpreted language like PERL is perfect. If you make the data comma, âoe,â, delimited then open office can make use of the file for other types of formatting needs.
All from the viewpoint of web development, but applicable to any software: RDBMS Alternatives for Web Developers
Hax-fu?
Looking at the large majority of replies to this story (mostly database blinkered) it reminds me why I don't ask slashdot for advice. You asked for a portable format. Databases are not really portable! You want a platform-independent, self-describing, single file-based, storage layer that you can carry on a thumb drive, that won't be locked into an API or persistence layer. www.pytables.org We use it. It'll scale to gigs and gigs in size also because of it's HDF heritage.
Thanks so much for all the replies and suggestions. Even the negative ones, which I could glean some info off. Special thanks to those who emailed me.
;-)
/. comments. This has certainly spurred me to update my knowledge about databases!
My last skirmish with databases was with Filemaker Pro, about 10 years ago. Prior to that, I have vague memories of Foxpro and dBase (!). While I've cobbled together a form or three before, my DB experience has mostly been as an end user.
So it appears my idea of a database is outdated (skewed?), and I have to explore relational DB territory.
Basically, all I need is a better way to organise data about Wifi APs. Something better than just notepad, but at the same time, would not cost me big bucks, need a server app, or require me to learn SQL/PHP.
I'm thinking of some kind of form for data entry, with presets for certain fields (e.g. location, encryption), the ability to view records as a table, and the capability to sort by column.
"Data entry" usually just means adding new APs as I come across them. Other than that, APs can get renamed or un-named (i.e. "hide" SSID), passwords are changed (though not very often, from what I've observed), or encryption is changed (usually, they just turn on WEP).
Being able to do queries would be a plus, something like "Show me all the APs in this location which use WPA2, and which I have used in the last 3 weeks."
I'm evaluating the various solutions posted here, and here are my initial impressions:
1. Various posters suggested that I could actually use a spreadsheet. However, I don't have Excel, and don't intend to buy it. So OO Calc might look like the best bet. However, its need for JRE gives me pause. As Indy might say, "Java... Why it did have to be Java?"
Anyway, using a spreadsheet for this seems kinda overkill, though I suppose it can handle the "sort by columns" task. This seems a bit like using Excel just to store phone numbers, which I've seen quite a few people do
2. OpenOffice Base. Again, Java. And some posters say it is still half-baked. I might just hold my nose and live with Java to test this.
3. Kexi (http://kexi-project.org). This seemed to be the the most promising candidate. So I actually downloaded and installed the Windows offering. Unfortunately the Windows ver is only a demo. It comes with a blinking "buy now" button and flickering text, which cannot be disabled. Hmph... I guess the LGPL allows that. So I dumped Kexi. May try it again if I install Linux.
Plusses: no Java (I think). KDE appearance, which I am partial to.
Minuses: no free version for Windows. Demo is feature-limited, nagware.
4. TextDB also looked promising, but it requires PHP.
5. Treepad Lite, free, but not open source. Might test it later.
6. SQLite Database Browser - Didn't look promising at first. But despite its name this does allow data entry. Gave this a whirl, but data entry is too rudimentary. THAT'S why it has "browser" in the name.
Plusses: no Java. Qt (KDE) appearance. FOSS. Runs in vista.
Minuses: Data entry is rudimentary. Can't sort columns.
Anyway, those are my DB adventures so far. Will continue the search and also plough thru the 600+
Once again, thanks to all who responded.
Regards,
Lee Meng.
So? You got modded insightful, but I don't see why. No offense to you, but I don't think any of your points are valid.
``1) My example was done to show how quick and easy it would be to set up and populate a table in Python.''
Writing code without injection vulnerabilities doesn't have to be any less quick and easy. Given that, why would you choose the quick, easy, and vulnerable way over the quick, easy, and secure way?
``2) He's explicitly mentioned several times that this is for his own private non-web use.''
Just because you don't see any threats to your software at present doesn't mean you should write it in an insecure way. And if you're posting your code in a public forum, I feel you have some responsibility to present good code or at the very least warn about its weaknesses.
``3) Your fly is open.''
Thanks for pointing that out to me. *closes zip* Now, I hope you will react similarly to people pointing out flaws in your code.
As a final note, many programming languages lamentably come with APIs that make it easy to do things insecurely and no easy way to do things securely. I feel this is something we should be working on, because if the secure way is as easy or easier to code than the vulnerable way, I am sure software security will improve.
Please correct me if I got my facts wrong.
What happens when you import:
"Smith","John","703-555-5555"
Does it behave nicer?