Slashdot Mirror

To provide a more accurate comparison, JET is probably more akin to SQLite, which, BTW, is frickin' awesome.

You want SQLite. It's fast, it's simple, and it's Free. You can embed it in your software (regardless of license) and it'll just work.

> Isn't SQLite a bit of a toy though?

No it isn't, take a closer look

SQLite is faster than Postgres for many types of workloads.

SQLite usually beats out MySQL and PostgreSQL for many things. However, it really depends on what you are using it for. SQLite was not made to be a full-blown RDMS. Here is a speed test.

.....Ron

SQLite has this name because it is... lite! It has a very, very small footprint. By your comment, I can tell you have never used it. I have. It is designed to be small and easy to embed into an app with out requiring a bunch of resources.

Ok someone give me details on this "SQLite database engine" please.

I don't know anything about SQLLite but will this add any bloat/performance issues/etc the firefox 3.0?

It's a very lightweight C library which basically enables "on the ground" SQL queries and such. No client/server mechanisms to worry about, no middleware (other than sqlite.dll, and even then you can just take the source and compile it in), and the security of the database is handled by security permissions on the file. That's right, "the" file. A single file contains the schema and data.

It fully supports transactions and is appropriately ACID. For someone who's had his Firefox bookmarks hosed before, this is very welcome for me.

The benefit of this will [hopefully] be fully searchable bookmarks and easy to move the bookmarks around to other computers.

I've used it in the past and it's been great for me. Check it out: http://www.sqlite.org/

Basically, yes. Public domain means "zero restrictions". You can take code available as public domain, make no changes, market it, sell it, redistribute it as if it was your own code. Of course, the coder next door can do the exact same thing. Very few people release code under this "license", although there are examples.

SQLite comes to mind.

or better yet, get a usb thumbdrive and stick SQLite on it so you can ship your product without relying on them having a DBA to set it up on the other side. I still don't see why so many people use MySQL or PGSQL for simple webapps like blogs and CMS systems. You arn't going to have multiple accesses from seperate machines, simultaneous updates, or any other possibly reason for justifying a remote DB. The only reason MySQL is so popular is it was shipped default in so many distros. Switching to PGSQL is just using a different brand of the wrong tool (90% of the time, of course)

How long does it take you to move your webapp+db to a clean machine? I can just rsync.

People who want decent formatting on an 80 column term... pretty much everyone apart from GUI weenies :-o

There's still no excuse for a parser choking on a space when flex and lemon are free.

As a developer on an internal project, I am reluctant to use even BSD licensed code. I have no problem with giving credit where credit is due, I just can't personally guarantee that it will happen. At the moment we don't have formal user documentation or any other good place to put the attribution required such that it will stick around. There's no possible way I can guarantee, when I put something in, that it will remain in the future. The code I'm writing, though it is internal for now, could end up anywhere in a few years. If anyone forgets to copy all the correct attributions when moving the code around, all of a sudden bam! You're illegal.

That's why I only use public domain code, of which a surprising amount exists. At the moment, I'm using SQLite and ANTLR (though v3 is moving to BSD...).

I've actually *sold* a few of licenses to the public domain SQLite library. Companies call me up and say they want to license the product. I carefully explain that no license is necessary and that they can use it forever for free for anything they want. But they still want a license. So I sell them one. So far, I've sold them cheap. Maybe I should charge more....

This appears to be more of an issue in Europe where, apparently, the concept of "public domain" is less well defined than in the US.

I think what you're looking for is SQLite. It's pretty powerful, very fast, and just links to the program as a library.

http://www.sqlite.org/whentouse.html

I haven't worked with it up close (browse around the website regularly, but don't run it), but all the docs I have seen say SQLite uses a B-tree, not a column store. Do you have an alternate reference to such?

SQLlite says not to use it for more than a few gb to tens of gb of data. Sybase IQ, for example, is routinely run with TB plus quantities of data. It's been tested to a trillion plus rows of data and 155 TB of input data (which autocompressed down to 55 TB of diskspace required to store it).

Vertica's headed thataways, I think, but also to be lighterweight and more general purpose than IQ.

We use CVSTrac, and it serves our purposes very well. It's not PHP/MySQL, it's SQLite-based, and is written in C, but to me, that just makes it easier to support -- there is one native process, so just one point of failure, no DB servers to monitor (yet, the data is available in a SQL DB), no web servers to configure (though you can set it up with an external webserver if you wish).

It is free, well-documented and is actively developed. It was written by the same guy who wrote SQLite.

It is also self-hosting, which means that the CVSTrac project uses CVSTrac for issue tracking, so you can go to http://www.cvstrac.org/cvstrac/ and get a good idea of how the system works without ever installing it.

We use CVSTrac, and it serves our purposes very well. It's not PHP/MySQL, it's SQLite-based, and is written in C, but to me, that just makes it easier to support -- there is one native process, so just one point of failure, no DB servers to monitor (yet, the data is available in a SQL DB), no web servers to configure (though you can set it up with an external webserver if you wish).

It is free, well-documented and is actively developed. It was written by the same guy who wrote SQLite.

It is also self-hosting, which means that the CVSTrac project uses CVSTrac for issue tracking, so you can go to http://www.cvstrac.org/cvstrac/ and get a good idea of how the system works without ever installing it.

...personally I'd rather declare any code I produce to belong to the Public Domain or just keep it entirely closed and private. The way law is makes anything else a headache. Seriously, if even lawyers can't agree on anything why does anyone in the Open Source community even bother? Personally I care more about other people being able to benefit from my code than preventing corporations from using it for profit.

SQLite is released to the public domain and it's some damn fine code.

SQL and text searching? Check out the FTS1 module for SQLite...

SQL and text searching? Check out the FTS1 module for SQLite...

I just wish they are wise enough this time to ditch that Sleepycat
(Oracle) db stuff and replace it with decent embedded SQL solution, like
SQLite. That would solve
many issues, dumping rpmdb would be a breeze as would be
creating some nice things like inventory/correlation/verification
service (daemon) top of it etc.

Another issue, actually with yum, would be to split updates in
smaller chunks like Mandriva urpmi does. It's riddiculous that
I need to extend /usr from 4G to 8G just for time of upgrade
and once you add it to LV you really can't take it back ... no
zfs type pools yet :(

It depends on your task. If you are building small to medium-sized web-applications, I would recommend Seaside. For larger projects, there are things like GNUstepWeb and Struts. If you want something slow that doesn't scale well, but is 100% buzzword-compliant, then there's Ruby on Rails. If you want to re-use existing ASP.NET code then you could try Mono.

For many needs, Apache is not a good choice. I personally prefer Lighttpd, which is lighter, faster, and easier to configure. It has nice FastCGI integration, so you can use it with most frameworks.

As for databases, I still haven't found a good reason to use MySQL. If you need a real database, I'd go with PostgreSQL, which is more standards compliant than MySQL, and faster for complex queries. If you want something slightly more structured than a flat file, then try SQLite, which is simple, lightweight, and faster than MySQL for simple queries.

While it wouldn't be suitable for something like slashdot with a lot of concurrent reads and writes, a lot of little things for which mysql is really overkill could use sqlite. It's built into php5.

SQLite is actually very fast and capable of handling large amounts of data, but you do have to be careful to optimize your queries. SQLite doesn't do much query optimization for you, and can make a huge difference in execution. Fortunately, it's not hard to optimize SQLite queries. The pitfalls are documented (if you bother to RTFM), and tools exist to help find the dogs.

I personally built a custom ELT solution using SQLite, where the database grew up to 45gb in size. I also ran benchmarks using other database engines, including MySQL and PostgreSQL. SQLite was 2x faster than MySQL for my use.

If SQLite is ever anything but screaming fast, you either need to a) add appropriate indexes, or b) re-order your queries.

A seemingly unbiased speed comparison (well at least not biased towards Firebird, anyway) can be found here: http://www.sqlite.org/cvstrac/wiki?p=SpeedComparis on

As for features: it has them all. ACID, triggers, stored procedures, will maintain identical copies of the one database on two drives for you, etc, etc. Possibly because of this when I went looking for a database to replace Oracle it seemed to be the one most recommended. At least one commercial vendor has an add on that provides the one thing missing in this role: a stored procedure language compatible with Oracle's PL/SQL.

And yes, I am a happy Firebird user. But not for any of the reasons mentioned so far. I use it because it is dammed easy to bolt onto your current project. No configuration. Small footprint. Ports to anything. Zero ongoing maintenance. That is its heritage you see - it always was a bolt on library for applications that don't even mention the word SQL in their description. So Firebird is doing its job well if the end users and sysadmins aren't aware of its existence. Think about that when you are next tearing your hair out trying to set up some MYSQL database when all you wanted to do is install some tiny web app someone else in the office asked for.

And that leads us to what turns most people off. There are no flashy front ends out of the box. Is comes with three utilities of note: backup, restore and isql: all very simple command line tools. Its an embedded database - you are meant to provide the front end yourself. And the doco, while present, is patchwork of old stuff and separate "changes since ..." files.

But if you are need a backend for a application that doesn't parade its "SQL" credentials Firebird is one possibility. The others are sleepycat (for speed) and SQLLite (for simplicity). You'd be nuts to use anything else, and I wish a lot of projects out there hadn't.

Absolutely correct

"If you think that anyone who uses any open source app is also a software developer... and a good one at that... well, no wonder Linux isn't more popular."

In the 90's I was working for IBM, the CEO made a speech and said "all software has been written, it just needs to be managed". All of the developers snickered, but the longer I stay in the bussiness the more it appears he was right.

I have a BSc in computer science and have been contracting as a C/C++ developer since 1991, I "specialize" in Windows (ie: I know the tools better than *nix tools). I don't use GPL code in commercial software but I often use "free" stuff, particularly in multi-platform source code. eg: zlib and sqlite (interesting wording on the sqlite license).

There are plenty more examples of high quality "free stuff", all of which have been mentioned elsewhere on slashdot.

Firefox 2 includes a critical new underlying database engine--SQLite--which enables new kinds of extensions, such as the free, open-source citation manager and digital research tool Zotero.

I used this recently for running test cases against Python code, and it worked great! I placed some DB population code in my setUp() method so you can run the test from any dir and it works -- no DB server needed!

It works like MS Access (file-based) but supports most of the SQL92 standard.

http://sqlite.org/

Should the developer of an application that uses flat files, possibly because the app is designed to run on inexpensive shared web hosting that may not provide any DBMS, switch to SQL and have the users eat the higher price of hosting?

I'd recommend SQL without an external DBMS.

The main point for speed is to not constantly create tasks and switch between them to get some piece of work done, that will never make something go faster (basically compare doing work X, vs. work X plus switch/create time Y ... the best thing isn't to have Y be low, but for Y to not exist at all).

You mean the technique called "preforking", where the master process starts a dozen worker processes in advance, right?

No, preforking is just moving around when Y happens. Think of it this way:

say you have 16 cores and get 32 "bits of work" a second to process ... common "threaded designs" have you accept all 32 "bits of work" a second from one task, and put them on a queue. Then you have 16 other tasks, taking a "bit of work" doing it and then going back for another (or blocking, if there is none).

Here Y consists of, moving the cache for the data from the "accepting" task to the worker tasks (on a different core), all the syncronization for the queue and the task switch time (you have 17 tasks on 16 cores, so you have to switch). The average time for a bit of work to be processed is X (time to do work) + Y. There are many techniques people come up with for reducing Y, like saying you should use threads, an in memory queue and locks (instead of, say, processes and a pipe with a lock).

An explicit process design might well just have 16 processes, each "accepts a bit of work" (or block waiting) and then does it. Y is non-existant. Average time for a bit of work to be processed is just X.

Now, to be fair, you could implement the "process design" with threads. Although you'd still need locking on any global resources that the threads share, and you probably have random other locks that aren't required in this design but aren't easy to get rid of (think FILE *). And you don't have sixteen compartmentalized tasks, just one.

And, IMNSHO, you are just much less likely to get that result. The temptation to use threading almost implies not doing the design work upfront, on what needs to be shared and what doesn't, so you share everything and as you write code you add blocks where the sharing happens.

If you want unrestricted freedom you want public domain, not just free. There are some great public domain software, but I can only remember one project right now.

http://sqlite.org/omitted.html

http://sqlite.org/ is starting to gain acceptance as a decent database for standalone applications and medium-size static web applications. SQLite3 beat MySQL5 to the punch of having triggers and views. It also has transactions and is atomic. And it won an award from google *and* oreilly. Though a lot of the extraneous features are not present in sqlite (due to its size) but you can add your own functions (and aggregate functions). I don't think it would be much for OReilly to include sqlite in the handbook.

Hell, don't use Access if it's going to be used by more than five users simultaneously. It's designed for occasional, personal, direct usage. Using it as a backend to a single-user application would be OK, as long as that application isn't used over the network, though there are probably some bettter solutions out there.

NEVER, EVER, EVER use an Access database as a backend to a server. It's just asking for trouble.

Unfortunately I get the feeling that a lot of these Access overusage problems stem from managers with just enough knowledge to be dangerous saying to themselves "hey, I've got a great working database right here," and storing their mission-critical data in it without knowing the consequences of that action down the road. I kinda wish that Microsoft would put a fairly prominent warning on Access that that's simply not what it's designed for... maybe when it's been used by more than a few users simultaneously it could pop up a warning to the next one.

Just throwing this out, but maybe there should be a very basic question asked instead? Since these already presume literacy, maybe something like:

Which of these is a number: A 2 R P?

It also has an excellent Tcl interface.

I know that many developers, especially on embedded or hand-held systems, are starting to use SQLite these days instead of BDB. SQLite is small enough, and efficient enough, to be used suitably in such environments. It offers SQL support, which BDB does not. And its library-based architecture is a real plus in such environments. The fact that it's public domain makes it even more appealing.

SQLite is even starting to take over from MySQL for many smaller websites. The overhead (both in terms of system resources and administration) can't be justified when using a server-based system, especially when SQLite does the job just fine.

What we may even see is MySQL squeezed out of the market. I know many database developers who are starting to use PostgreSQL for their higher-end databases, and SQLite for the lower-end. MySQL could be considered the best contender for mid-range DBs, but SQLite and PostgreSQL keep converging, thus pushing MySQL out to a point of irrelevance. Of course, there has been a decade and more of development using MySQL, so for legacy reasons alone it will continue to be used for ages.

i respectfully disagree. SQLite is:

• simple - IMNSHO it's simpler to use an embedded database to manage a huge list of items and metadata instead of a plain XML files
• machine-and-human parseable - while not as easy as vi-ing a textfile this tool helps a lot. it's very elegant and you can always dump and edit the text.
• portable - "Database files can be freely shared between machines with different byte orders."
• reliable - ACID compliance

all that and an elegant API to access your data. some scripting languages have support for it too and you can jump straight to coding right away (PHP for example)

Aside to the moderators: my "flamebait" comment is essentially the same [...]

i don't consider your comment to be flamebait. you just seem to have a serious aversion to databases that get `misused', but you also fail to provide solid reasoning.

and i'm probably biased too, because i use lots of different databases (embedded or not) to store lots of shit, as it's really great to have a nice interface to your data stores.

hell, i might be wrong but in this case it seems a good idea, and i gotta admit i'm pretty dissapointed that they bumped it.

just my 0.02RON

i respectfully disagree. SQLite is:

• simple - IMNSHO it's simpler to use an embedded database to manage a huge list of items and metadata instead of a plain XML files
• machine-and-human parseable - while not as easy as vi-ing a textfile this tool helps a lot. it's very elegant and you can always dump and edit the text.
• portable - "Database files can be freely shared between machines with different byte orders."
• reliable - ACID compliance

all that and an elegant API to access your data. some scripting languages have support for it too and you can jump straight to coding right away (PHP for example)

Aside to the moderators: my "flamebait" comment is essentially the same [...]

i don't consider your comment to be flamebait. you just seem to have a serious aversion to databases that get `misused', but you also fail to provide solid reasoning.

and i'm probably biased too, because i use lots of different databases (embedded or not) to store lots of shit, as it's really great to have a nice interface to your data stores.

hell, i might be wrong but in this case it seems a good idea, and i gotta admit i'm pretty dissapointed that they bumped it.

just my 0.02RON

FYI SQLite is small, fast and stores everything in a file. it's not like they want to store your bookmarks in a fucking Oracle installation. SQLite is embedded and fits the purpose quite well. perhaps not very boner-inducing from a user standpoint, but a programmer can clearly see the benefits of such a thing: easy access, searching, management, etc. even for third party tools.

a vast improvement over the current XMLish kludge (i won't even mention the MORKiness, plenty of other slashdotters did). if you ever tried to do anything else with an XML file (besides transforming it) you know how *fun* that is.

You think setting up a local database is a security risk, but setting up a local web server isn't? Why? You are aware that databases don't have to be servers listening on public ports don't use? You could use something like SQLite.

The important thing is not the implementation itself. It's the data format and/or API. Make the data available, and plenty of people will be willing to write web interfaces, Qt interfaces, GTK interfaces, etc. Expose the API as plain C, and make the data easily importable/exportable, and it really doesn't matter if you produce the crappiest proprietary implementation imaginable, because both the backend and the frontend can be replaced individually.

SQL Server Express IS a dedicated database server. You need to run it as a service. If you want to bundle a database to your app try to use an Access or sqlite file. http://www.sqlite.org/

http://www.sqlite.org/

The website says 250KiB fully configured. That is tolerable, i think.

Hey, I wouldn't call SQLite a "toy" database. It's SQLite; it's geared to light-weight uses. Do you need a full SQL2003 DB in your mobile app? There are many instances where SQLite does well. I'm on a team using it in an app that needs a simple DB, and it's great for the job.

Hey, I wouldn't call SQLite a "toy" database. It's SQLite; it's geared to light-weight uses. Do you need a full SQL2003 DB in your mobile app? There are many instances where SQLite does well. I'm on a team using it in an app that needs a simple DB, and it's great for the job.

Speed isn't everything but some of these are insane.

Indeed. I wonder why people are not using SQLite where they need a fast and not _very, very_ large database (that's the case with most websites). And if there's a need for a big and reliable db -- PostgreSQL is the answer.

> I can't even find what the the license cost for MySQL I have vague recollections of something about $250.00 for a commercial
> license free for non profit. On the web site there is a $595.00 survice contract, but no mention of a commercial use license.

MySQL Network Basic is $600/server/year: https://shop.mysql.com/
You *may* not have to pay anything however. But that depends on GPL/LGPL licensing complexities, and you will probably need a lawyer to know for sure (MySQL AB recommends just buying a license if you're confused).

In comparison, Oracle/DB2/SQL Server are free for small databases, regardless of how you use them. Then if you need to license it can still be cheaper than mysql - since they typically have an initial license (say, $750) and then just 18%/year afterwards. All three of these databases can be *far* more expensive, but db2 in particular can also be cheap in many configurations. I'm running a terabyte-sized data warehouse on about $28k in total licensing. Compared to the cost of the hardware this is nothing.

And of course, postgresql & sql
    - http://www.postgresql.org/
    - http://www.sqlite.org/
are completely free.

> So I can only assume that those of us that need a database and not even a feature rich database will simply move on to something
> else or pony up the $5000 to $40000 to have a DB to keep track of the CD collection.

No, you're fine for several reasons:
    - you can probably get by with the free mysql version for now
    - postgresql is completely free
    - sqlite is completely free - and is probably the best choice for a tiny project like this
    - oracle is completely free for something of this size
    - db2 is completely free for something of this size
    - sql server is completely free for something of this size

Unless of course you're talking about building a competitor to cdnow, in which case you will face some costs ;-)