Yup, that's a problem. If the disk (or the operating system) lied and the data doesn't exist anywhere but in fleeting memories, you are what's known in technical circles as screwed.
There is nothing, absolutely nothing, any database system can to to recover data that was never written to a persistent medium. Sorry, but those are the laws of physics.
Some disks, particularly cheap ones, do lie. If you really care about your bits, I suggest that you take a hard look at your hard disk specifications. Or, maybe, we could develop a program that could detect lying disks.
Falcon, unlike Interbase/Firebird and Postgres, is multi-version in memory but single version on disk, so there is nothing like the Postgres vacuum process. Sweep in Interbase/Firebird is quite different from the Postgres vacuum and is used to reduce the amount of transaction state information that needs to be lugged around. I got quite tired of hearing about sweep problems and designed Netfrastructure (of which Falcon is a small subset) to have neither sweep nor vacuum issues.
Falcon does have a record scavenge thread that kicks off when the size of the record cache reaches a pre-determined limit to free up memory consumed by boring records, but that's a different issue.
Falcon does not have clustered indexes. Falcon performs indexed retrievals with a two phase schema. The first phase screams through the index setting record numbers in a sparse bit map. After the index pass, Falcon fetches record (unless they're already in the record cache) in record order, which is reasonably close to physical order on disk. So, other things being equal, there is no performance difference between primary and secondary indexes. (Actually, it's a little more complicated than this because Falcon keeps partial indexes in memory until a transaction is really quite committed.)
Falcon is also capable of and-ing and or-ing sparse bit maps from scans of multiple indexes before fetching records, but this isn't accessible through MySQL (yet).
Yes, you can control the amount of memory Falcon uses. The record cache is typically set to scavenge at a half gig to about a quarter gig (a record cache is a much more cpu and memory efficient that a simple disk cache), the page cache at about 1K 4,096 pages, and a serial log cache of about 8 MB. Most applications don't have anything like a half gig of "hot" data. Netfrastructure typically runs two or three municipal web sites on $389 BestBuy Sunday specials (nothing like a good supply of free printers).
Seriously, the Falcon model is a partial in-memory database using the disk for backfill. The whole thing doesn't have to fit to scream, just the part in heavy use.
Falcon keeps a serial log that used post-commit to move data to disk and, if necessary, to recover after a server crash. At commit time, all updated records and index nodes are written to the serial log, the serial log is flushed to disk with a non-buffered write, and the transaction commits. The data might not hit the actual database file for another few minutes (or more if the update rate has outstripped the disk bandwidth), but rest assured, your bits have been lovingly written to stable oxide before we consider the transaction committed.
Falcon is designed from the ground up to support large binary blobs (which, frankly, shouldn't be a surprise). In specific, blobs have different data flow path than records avoid double writing blobs to the serial and then to the disk. Blobs are not re-written during record update unless the blob itself has changed. Finally, Falcon goes through some trouble to avoid materializing unreferenced blobs during queries.
And although MySQL currently does provide access, Falcon supports multi-volume blob repositories that do automatic volume rollover based on file size, time period, or both.
What else would you like?
Slashdot Mirror
Yup, that's a problem. If the disk (or the operating system) lied and the data doesn't exist anywhere but in fleeting memories, you are what's known in technical circles as screwed. There is nothing, absolutely nothing, any database system can to to recover data that was never written to a persistent medium. Sorry, but those are the laws of physics. Some disks, particularly cheap ones, do lie. If you really care about your bits, I suggest that you take a hard look at your hard disk specifications. Or, maybe, we could develop a program that could detect lying disks.
Falcon, unlike Interbase/Firebird and Postgres, is multi-version in memory but single version on disk, so there is nothing like the Postgres vacuum process. Sweep in Interbase/Firebird is quite different from the Postgres vacuum and is used to reduce the amount of transaction state information that needs to be lugged around. I got quite tired of hearing about sweep problems and designed Netfrastructure (of which Falcon is a small subset) to have neither sweep nor vacuum issues. Falcon does have a record scavenge thread that kicks off when the size of the record cache reaches a pre-determined limit to free up memory consumed by boring records, but that's a different issue.
Falcon does not have clustered indexes. Falcon performs indexed retrievals with a two phase schema. The first phase screams through the index setting record numbers in a sparse bit map. After the index pass, Falcon fetches record (unless they're already in the record cache) in record order, which is reasonably close to physical order on disk. So, other things being equal, there is no performance difference between primary and secondary indexes. (Actually, it's a little more complicated than this because Falcon keeps partial indexes in memory until a transaction is really quite committed.) Falcon is also capable of and-ing and or-ing sparse bit maps from scans of multiple indexes before fetching records, but this isn't accessible through MySQL (yet).
Yes, you can control the amount of memory Falcon uses. The record cache is typically set to scavenge at a half gig to about a quarter gig (a record cache is a much more cpu and memory efficient that a simple disk cache), the page cache at about 1K 4,096 pages, and a serial log cache of about 8 MB. Most applications don't have anything like a half gig of "hot" data. Netfrastructure typically runs two or three municipal web sites on $389 BestBuy Sunday specials (nothing like a good supply of free printers). Seriously, the Falcon model is a partial in-memory database using the disk for backfill. The whole thing doesn't have to fit to scream, just the part in heavy use.
Falcon keeps a serial log that used post-commit to move data to disk and, if necessary, to recover after a server crash. At commit time, all updated records and index nodes are written to the serial log, the serial log is flushed to disk with a non-buffered write, and the transaction commits. The data might not hit the actual database file for another few minutes (or more if the update rate has outstripped the disk bandwidth), but rest assured, your bits have been lovingly written to stable oxide before we consider the transaction committed.
Falcon is designed from the ground up to support large binary blobs (which, frankly, shouldn't be a surprise). In specific, blobs have different data flow path than records avoid double writing blobs to the serial and then to the disk. Blobs are not re-written during record update unless the blob itself has changed. Finally, Falcon goes through some trouble to avoid materializing unreferenced blobs during queries. And although MySQL currently does provide access, Falcon supports multi-volume blob repositories that do automatic volume rollover based on file size, time period, or both. What else would you like?
OK, I'll bite. Which is the Bug and which is the tank?