For me, it's '###' in a comment, rather than 'XXX'; that's because '###' should never occur in regular C/C++ code. But I do concur; I've got '###' scattered pretty liberally throughout my code as well. They include things ranging from "I really need to complete this subroutine" to "I wonder if there is a better way to do this."
So just because there are about 20,000 BUGBUG lines doesn't really mean much.
FYI, Microsoft uses a specification-driven testing plan which is developed by a head of testing.
If there is one thing that Microsoft does right, by the way, it's this: Microsoft has separate career paths for programmers, testers, management, and content authors. Unlike other companies, which after 5 years of programming, you're expected to be bumped into management and never program again, Microsoft gives you the option to continue up the pay scale as a programmer instead.
What that means is that the head of the testing team for a large and important product such as Win2000 probably has a hell of a lot of Q/A experience.
Microsoft will also use a formal, specifications driven testing plan that included both a bunch of testers who do more "free-form exploration" of the product, and who do a formal testing which is driven by a script. They'll also be using testing automation.
The only problem with the Windows 2000 product is that it's an integrated operating system--that is, unlike Linux development which uses a Unix-like development philosophy of keeping each of the pieces small and managable, Windows 2000 attempts to put a hell of a lot into one large, monolithic black box. This plays hell on any formal testing plan in that you now have to test every possible interaction of every component of that black box. (Linux, on the other hand, can be tested more easily by testing each component separately, and making sure each component interacts with the standard Unix API correctly.)
The monolithic nature of Windows 2000 means that it will be near impossible to create a complete and full test plan, simply because of the huge number of component interactions within the black box. There's probably just too many conditions to test adequately.
In fairness, keep in mind that "defect", or even "bug" for that matter is a very broad term.
I have to concur about this. Realize that any company, such as Microsoft, who uses a formal Q/A program which uses a formal bug tracking system is going to eventually fill that bug log with a lot of niggly little things which frankly don't matter.
For example, one project I worked (a children's game) contained about 50 bugs from one tester who didn't like the color of the eyes of one of the characters. (He claimed they were too brown, and should be bluer.) Each bug report corrisponded to each scene in which the character appeared.
Eventually we scanned all of the art to make sure the eye color was the same, and closed out all of the bugs over the tester's protests.
I still have bug reports in my own bug tracking system covering similar non-issues, "helpful" suggestions, and things like "on-line help manual contains run-on sentence in 3rd paragraph." It's quite possible that the large bulk of the bugs relate to similar on-line content.
What's especially frustrating about such Q/A processes such as the one Microsoft uses is often they will report the easy to spot items, such as run-on sentences in on-line help. But at the same time, fatal, hard to detect crashing bugs will go by unnoticed because the Q/A test plan didn't include a situation which would stress the software in unexpected ways. And with the huge complexity of Microsoft's Windows 2000 product, that's the one that worries me the most: not that the product will ship with 65,000 content bugs, but that the crashing fatal bug which wipes out my hard disk wasn't even detected by their test plan.
My impression was not that his son was "weak", but when he died, his sons returned from doing battle to attend the funeral, and that stopped the westward momentum.
On the other hand, it was said about the Mongol empire that a well-dressed young woman could ride on horseback with two large sacks of gold from the Black Sea to the Pacific Ocean without once being accosted or robbed. Quite an impressive accomplishment for *any* empire or nation-state, even in today's world.
I believe that this is sometimes the case--I have translated acounts of some of the very same roman sooldiers and gladiators you speak of who rather enjoyed killing (or claimed to.)
Well, the predominate religion of the soldiers at that time, Mithriasm (sp?), put a high value on acting like a soldier: that is, it put a high value on killing and being killed with honor. So in a sense, killing folks and dying on your own shield were both considered holy, religious acts.
When Christianity started sweeping the Roman empire with it's acceptance by Constantine, Christianity adopted many of these "millitaristic" aspects. Thus things like "onward Christian Soldiers, marching as to war" has a long, ingraned history in Christian thought. Of course Christianity borrowed heavily from earlier Roman military cults, including things like an afterlife and a "heavenly reward for a life lifed well"--well being redefined to include non-soldierly acts.
In fact, as far as I can find, in Western civilization the notion that killing someone during battle is a bad thing is a recent invention. In large part because of people's disgust at the very cold blooded and calculated way we can now kill thousands or millions with the push of a button.
So I would argue that in fact, the depersonalization of war has made war unthinkable, rather than the other way around. That is, war and killing people was more acceptable to people when we were doing it hand to hand and face to face.
(1) When all you can pick on is spelling, it means you have nothing meaningful to say.
(2) I specifically picked the Khmer Rouge because I am aware the US has done nothing to stop them. I was engaging in something called "irony", a subtle practice which apparently some people just don't get.
if you beleive that killing people can ever be 'appropriate', then you have been brainwashed. the most basic tenet of any real moral philosophy is the sanctity of human life.
When a burglar breaks into your house and puts a gun to your throat, do you believe it is immoral to defend yourself, even if you have to resort to lethal force? Or do you believe you should simply yield, and perhaps die?
The analogy does extend to the international arena. I'm sorry if you don't see that, but unfortunately it does.
. it makes the US look foolish on the world stage when they say to milosevic et al. "don't kill people, killing people is a bad bad thing! hell, we think it's such a bad thing, we are going to send OUR army in and kill YOUR people. basically, they are saying "we will teach you not to kill by killing."... tha's like 'fucking for virginity'... it just makes no sense.
*shrug* I'm sorry if it doesn't make any sense to you, but it's about the use and/or the threat of use of lethal force. Sometimes you can stop someone by talking them down. Sometimes you can stop someone by threatening to blow them away. And sometimes you can't stop them--and wind up having to blow them away before they kill more people themselves.
We tried talking in Kosovo. We tried for *years*. Did no good; in fact, it got to the point where the folks there believed they could do whatever the hell they felt like because President Clinton didn't have the balls to intervene. Are you suggesting that in light of massacres and rape and pillaging and death we should send over yet another polite letter asking if they could stop?
Of course we could always just ignore things in Kosovo--quite a few people here in the United States advocated just that. But if you recall your world history, you'd remember that World War I started because of interlocking relationships between various factions that were (and are now still) fighting over there.
And if you think an isolated massacre sucks, try a world war in a world full of smaller powers with weapons of mass destruction.
how is training your army to be more effective at killing going to reduce the number of casualties, you nitwit! it might reduce the financial cost of war for you, but tha's about it.
Simple: the threat of force causes most people to think twice about starting a war themselves. In fact, it's working with you, so don't think it doesn't work with world leaders as well.
Again, if you recalled your history, you'd note the reason why the United States dropped two nuclear warheads on Japan at the end of World War II was because Japan refused to surrender and refused to stop making war on it's neighbors, and the estimated casualty count for invading Japan was in the multiple million range.
What happend in Nagasaki and Heroshima sucked. What could have happend if we had choosen to invade Japan would have been far worse.
why only two? you mean only two that you can think of... there's things like diplomacy, sanctions, the UN war crimes tribunal, etc. now, i don't have a problem if some highly highly skilled commandos capture these murderers so that they can stand trial in a court of law (what a new-age thought!) but, again, fighting to end some fighting makes you look like an idiot.
Several points.
1) Diplomacy only works when the agressors are willing to sit down and discuss reasonable measures. In the case of Kosovo, diplomacy was tried for *years*, and because the people involved had the honor of small hungry children who promise not to steal cookies from the cookie jar, diplomacy was a total and complete failure.
2) Sanctions only work when the country who is having sanctions used against it cares one whit, and when the parties imposing a sanction actually follow it. Or didn't you read about the Russian oil tankers who were attempting to run the UN imposed blockade against Iraqi oil?
By and large, with a willing leader who doesn't care about his people, sanctions don't work at all. Or did you forget about Cuba, who is moving into a quarter century of sanctions without changing it's government one millimeter?
3) A UN war tribunal only works when the governments who form the tribunal under UN authority are willing to send troups in in order to settle the situation--that is, if the UN is willing to send in troups who are prepared to kill people. Without such forces who are willing to impose the will of the tribunal, such a court of law is little more than a legal masturbation exercise--it may feel good, but ultimately produces nothing but a small mess.
4) As to the commandos: the United States has tried that on rare occassion, and has gotten into quite a bit of trouble. Frankly, sending in commandos is a violation of international law, while sending in troups is not, and for good reason: in theory, nations are not supposed to resort to undermining other nations--their complaints are supposed to be brought out in the open so the international community can know what the hell is going on.
Furthermore, the United States has made sending in commandos as you suggest illegal unless the President signs a special order. So it's not something we can "routinely" do.
well, this is just wrong again. it is not your army that protects you from domestic deathsquads (i didn't know that was a problem in the states, i guess i'll have to look for them next time i'm down there), it would be your local police dept, as far as i can tell.
There are clear lines of jurisdiction. And in the United States, it is the army who is responsible for external invaders, and the states (and by extension the local police) who are responsible for keeping local law and order.
Of course there is a lot of 'bleed over' in sharing technology: the city where I live are using technology developed for the Army to catch speeders at night. And the same training techniques are used, more or less, by the local police force to train cadets.
But what I was talking about was the fact that unlike many areas of the world, having a powerful army who is directly controlled by a civilian government, and who is embedded in a culture who prizes diversity and law and order above chaos and "kill your neighbor if he's not your brother" allows me to sleep better at night.
Now if someone could figure out how to arrange a ring of atoms to create 8 or 16 quantuum states, and arrange them so you could build a three or four bit adder with carry circuit out of two or three of these rings, then I'd be impressed.
I suspect scientists are working on it. 'Cause eventually, we're going to have to start using quantuum states and tweeking the fundamental nature of the universe to build processors fast enough keep up with the computational requirements of Windows 2020...:-)
Now that the Internet (and in the future, the Internet II) are no longer playthings of academics and researchers but are seen as central to a developing dot.com economy, what will be interesting to see is how the Internet develops.
Personally I'm not very concerned with the privacy issues raised by Katz.
Why?
Because by the death of DIVX and by the current popular attitudes towards the DVD lawsuits and the rejection by the general public of technologies which are not "easy to use" (translation: which technologically blocks fair use), I suspect the technology controls that corporations are trying to put in place to block the general public from using intellectual properties they paid for will wither and die on the vine.
Personally if given the choice of buying copy-protected music on the net which (a) only can be played on my computer and not in my car, and (b) could be lost if my hard disk crashes, or buying a CD, I'll buy a CD. That's because a CD is more convenient. MP3s are only taking off in the way they are because they are more convenient than a CD--in particular you can burn a CD-ROM with your music and protect them in case of a hard disk crash, and you can copy them into a portable player or archive them in one location.
But this only outlines a larger battle: the battle between security and ease-of-use. Already you can read newspaper articles about various CIA employees who circumvent their secure home workstations in order to copy highly secret data to insecure home PCs because they want to get their work done. Popular are technologies which circumvent security precautions, such as cookies in lew of password-protected security on shopping sites, and the Macintosh's Key Chain and similar Windows technologies which remember passwords for you.
People want ease of use as much as they want security. And when the security is not in their own personal best interest (such as protecting the intellectual property of the music conglomerates), they tend to say "screw security, I want to play my Madonna music files in my car."
I think the corporations have an uphill battle on their hands. And I think eventually they'll lose--unless they can come up with a technology that is easy to use as a CD and doesn't invade their lives, people will simply resort to older technologies.
For most, freedom is the freedom to be left alone. Forcing people to remember arcane passwords, techniques and telling them "you can't play your music in the car unless you pay me twice" isn't leaving them alone: it's invading their lives in ways that most folks could do without.
The Internet, the successor to the ARPA-net, was developed using funding from DARPA, a DoD group who funds advanced projects. The ARPA-net was designed and developed as a distributed communications system which, amongst other things, could survive a nuclear war. (That was one of the driving design goals of TCP/IP, and that's why IP is so good at surviving things like a router going down.)
I first got started using the 'net way back in '83 when I was a student at Caltech. And at that time, you were only permitted to use the 'net if you had a valid research grant which required use of the ARPA-net account. Of course by about '85, that rule was largely ignored, and the whole thing turned over to total civilian control by about the same time, as I recall.
(Of course at the time I was drinking more beer than paying attention to whose research dime I was e-mailing on; silly me. So take my cronology with a grain of salt.)
But in point of fact, the academic world designed the thing in much the same way the academic world designs things like spy satelites and better ballistic missles and radar jammers and all those other nifty high-tech death machines...
Political science theory going over your head? Here's a brief primer.
In the United States, we've studied these sorts of things to death. Things like the appropriateness and inappropriateness of war. And what most of the better thinkers in the US have come up with is that you only fight a war when another country (a) starts it, (b) that this war will affect citizens abroad in a negative way, (c) you can go in and stop it, and (d) the cost of stopping the war (by engaging the other side) is less than the cost of allowing the beligerant party to continue.
So in the United States, as we are the defacto policeman of the world (and have been to one degree or another since the Monroe Doctrine), we've been trying to figure out how to reduce the cost of war (in terms of casulties on both our side and theirs) in an attempt to promote a degree of "piece" at least for US citizens traveling abroad.
It sucks.
But until you can figure out a pieceful way to solve things like the Balkanization of Yugoslavia (yes, I appreciate the irony) and the murder and/or displacement of millions of innocent civilians in places like Chechnia, Kosovo, or Cambodia in a way which doesn't require some form of force, unfortunately you are going to have two choices.
1) Use force. And that means having very well armed people who are very well trained at killing people, or
2) Allow madmen like the Khamer Rouge run wild, murdering whoever they like.
The world sucks. But having the biggest and baddest army in the world maximizes the chance that when I go to sleep tonight, death squads won't break into my house and murder my wife as she sleeps next to me.
Let's see. We have the Mongolians who, in the process of spreading their might, wiped out millions, one at a time, using nothing more sophisticated than an axe. They were deadly enough that the name Khan just rang all sorts of bells with that Star Trek movie...
Then we have the Romans, whose spread of civilization was done at the cost of all of those insignificant, worthless "gauls" who weren't considered important enough by the Romans to do much more with than slaughter and enslave.
Of course we have the crusades, the various civil wars, holy causes, and of course the Inquisition. All fought with low tech. All fought more or less hand to hand. And many weapons were designed to kill someone only at very close range--such as a double-handed long sword, whose primary purpose was to dismember a knight in shiny armor in much the same way you pull apart a cooked lobster.
People have been killing people in very large numbers at very close range for thousands of years. It wasn't like the invention of a programmable missle controller chip (the predecessor to the Intel 4004) caused people to suddenly realize "hey, I don't have to take moral responsibility for who I kill, so let's fight even bloodier and bigger wars than ever before."
Nah; technology found its way to the private sector where it was used, ah, um, for stuff like this web site.
What it boils down to is a rather common principle in US law where *intent* is 9/10ths of the law.
That is, it is up to the court to determine the *intent* of the developer based on the developer's activities if the reverse engineering effort was done in order to properly achieve interoperability, verses the developer using the reverse engineering efforts to promote piracy.
Personally I find these sorts of distinction a little irritating, in that the court is left attempting to read people's minds to determine if a crime was committed or not. But it's not all that uncommon: most rape cases (for example) basically boil down to a "he said/she said", and factors such as what she was wearing and what he was drinking are used to assess the state of mind of the folks involved to determine if it was a rape or a misunderstanding. Or here in California, the difference between someone getting the gas chamber and someone getting 15 to 25 years is the "state of mind" of the murderer, often inferred by the type of magazines he read or the music he listened to. (That's the difference between premeditated homocide and the non-premeditated "heat of passion" variety.)
Unfortunately for the DeCSS folks, things like the logo "Masters of Reverse Engineering" at the bottom of the screen, or the web site www.dvd-copy.com proclaiming "How to find/trade FREE DVD Movies online" or the less skillful comments by folks here on/. don't exactly help convince the judge that the original intent by the developers of DeCSS was "cross-platform compatability."
What is interesting in all of this, however, is that while the DeCSS code itself may go down the drain, it's quite likely that offshoots of the software, such as LiViD may wind up being found quite legal. That's because while the DeCSS program circulating around does little more than copy the VOD data to a hard disk, the LiViD software actually plays the movies. Thus, one could argue that even though LiViD makes use of the ill-gotten reverse engineering efforts of DeCSS, the intent of the developers of LiViD was to create a movie playback system which does fit within statue.
Further, even if all of the DeCSS stuff goes down the drain, we now have a roadmap on how to repeat the DeCSS effort in a way which would be construed as legal: by making sure that the intent of the development process is clearly outlined from the start to do nothing more than provide interoperability between Linux and DVD hardware on Linux boxes to play DVD movies.
(Yeah, yeah, yeah, I know the mantra: copying DVD movies is impossible because DVD burners don't hold enough data to copy a movie and because blank DVD-R disks have the CSS track prewiped to zero. But this is a week argument at best in that it is possible to play a naked.VOB file, so while it may be difficult to duplicate a DVD Movie Disk, it's not difficult at all to duplicate the.VOB files so a friend can play them back. From there, it's not a huge leap for someone to define an informal standard for a DVD playback engine which plays naked.VOB files as well as DVD Movie disks, a'la the plurality of MP3 players out there. And just wait for the day when they're stringing 10Mb internet wire to your house...)
... except that Microsoft still hasn't learned the art of being subtle in playing politics--probably because they have such contept for anyone outside the Redmond campus.
I get these sorts of surveys all the time. They're used to manipulate politicians--or at least the stupid ones, as any politician with the IQ of a house plant knows these sorts of surveys are always loaded.
But typical questions on the surveys I get are things like "should Americans keep their God given right to bare arms as was granted to us by our Founding Fathers, or should the government take their cue from athiest God-hating communist countries and take our rights away." (Gun Control) Or "should Women have control over their reproductive selves or should government be able to inprison women for attempting to control their fate in light of an unwanted pregnancy?" (Abortion).
These sorts of questionares are always loaded, and it's not a supprise that Microsoft is trying to play the same game. Too bad they're too arrogant to play it well.
Actually the US is based on common law, which means in a sense that the judges get to make it up as they go along depending on the current thinking in legal circles and the current sentiment of the american public. Thus, the current wishy-washy state of things like abortion law or death penalty law.
We don't have slavery, Jim Crow laws and the civil rights movement because a strong willed minority in our government imposed their ethics on an unruly mob; we have these things because the unruly mob by and large wanted them, dispite a vocal minority. And we only have gotten these things recently--while the american Civil War settled the notion of slavery by turning it into a succession question, these other things have only been around with any meaning since the 60's, dispite the efforts of a few for at least a hundred and fifty years.
Public opinion matters when new law is settled on; that's why public opinion is asked for in this case. And in the case of Microsoft, this is one of those rare occassions where we are charting untested waters. (This will be the first time a non-government sanctioned technology monopoly will be dismantled.) So the judges want to know what informed people think.
Actually think "cell phone technology." That is, you rig it so that each transmitter only works within a limit area, with the different transmitters working to "trade off" as a device moves from cell to cell. Broadband only helps make things less prone to noise.
Of course the thing people don't tell you is that this is predicated on cell phone pricing as well. So the real question will be are people willing to surf the web on a 120x120 pixel display for $.10/minute?
Probably not, unless they're a "gizmo freek" or a masochist...
The theory is that the PC era is ending because wireless dumb terminals will slowly replace personal computers in the future. Of course this is predicated on infinitely powerful servers and infinitely large wireless bandwidth, which makes me think that this is all pie in the sky bull.
What's fascinating is that anyone who has half a brain should be noticing the trend towards web-enabled applications which supplant the browser, rather than having the browser replace applications in general. Of course the pundets are usually the last idio^H^H^H^Hfolks to notice these things...
By the way, keep in mind that TI has an interest in having the PC go away, so anything that TI announces about the "end of the PC era" should be taken with a grain of salt.
I think the thing you're missing here is that the term "database" doesn't necessarly apply to just a SQL server performing transactions across a wire. Granted, in this case, the person doing this project should probably invest in Oracle--after all, that's what he's doing. But I did suggest that at the top of my post.
Thing is, a "database" is not just SQL running remotely. Did you ever develop an application that loaded and saved data in a file? Well, that could also be considered a "database". Transaction processing doesn't just apply to traditional databases, either--they're a strategy to prevent your user from losing data just because he hit "save" at the wrong time.
Further, what ever happened to curiosity? What ever happened to the hacker mentality of wanting to know how it works and tinkering with something that's interesting? It does supprise me the number of people who instead of saying "I reinvented the wheel because I thought it would be cool" (hacker mentality) are saying "why the hell did you waste your time doing 'x'?"
Besides, once the guy realizes all that's involved, perhaps he can be the one who makes the mindful decision as to how to perform his employer's wishes?
Actually, and this is so totally off topic I don't know why I'm posting this here, is that if you get enough positive karma, your posts automagically show up as a +2 by default.
Go figure.
Now I have this little button down below that allows me to set it to +1 instead of +2; normally I select that when I submit a random bit of blathering like this. But I forgot to moderate myself down when I posted the above.
Did anyone catch the section of the suit where they quote a random (and probably hand-picked for maximum effect) collection of/. articles? I quote:
For example, postings on slashdot.org as early as July 1999 clearly establish the state of mind of the hacker community. The following is a sample of posts made on July 15, 1999:
I like the "state of mind" of the hacker community--like we have only one mind, and we all agree. Further, some of the quotes are offered to show that we all knew certain aspects of the law which frankly I, as a drone in the hive mind, was not made aware of by the hierarchy.
Further, the declaration makes a bunch of assumptions about how individuals must have known certain things, because they were posted by anonymous cowards here on/. Now am I missing something, but part of the give and take here is that you wind up taking what goes on here with a large grain of salt, especially from anonymous cowards. So saying that "it was discussed on/. must necessarly make it true, and thus making individuals criminally liable" strikes me as a stretch.
Ah, well. It's stupidities like this which make me a little, ah, itchy around some lawyers...
I only outline the above because it's interesting, because sometimes you don't need Oracle-SQL to store a few records, and because frankly, a hell of a lot of people who write applications which use a primitive "database" that simply consists of an array of records. And it wouldn't hurt to add a couple of layers to the low level record I/O routines in that application to make it a touch more bullet proof.
Me, I'm not a big believer in using a cannon to kill a fly. Using a full-on DB engine like Oracle or Transact-SQL just to store an array of records is a wee bit overkill...
Okay, there are two things you can do to solve this problem. First, you could use an "off-the-shelf" database engine. There are quite a few very good open source database engines out there, as well as a few good low cost and high cost engines.
OTOH, I assume you are probably interested in rolling your own engine. So here's a few strategies that you may wish to consider if you are going to do it yourself.
Overall, one of the most powerful metaphores for breaking down a problem into managable pieces is the notion of "layers" or of a program "stack". That is, you break a large problem into a collection of software "layers", like the TCP/IP protocol stack--at the lowest level you have the piece of software that talks to the hardware. Above that, you have the thing which encapsulates the packets. Above that, you have the thing which handles routing. And so forth.
It's a powerful notion because it allows you to break a complex problem such as "develop a database engine which provides transaction rollback and RAM caching" into a bunch of easy to understand units.
For a database, eventually you are going to have to write the records to a file, and take the performance hit that entails. That's just life. And worse--if you want to have proper transaction rollbacks (meaning if the power crashes while you are in the middle of writing a record, you need to have a mechanism to recover from this), that means you are going to have to do *two* file writes--one to write some sort of "recover" record, and one to write the record you are trying to save.
A common way to handle this sort of transaction rollback is to write a record or group of records in three steps. Step (1), read the records you are replacing and write them into a "backup" or "rollback" file. The idea is that this "rollback" file contains a record of everything you changed in the database file, so that if the power fails while you are updating your database, you can "fix" the database by reversing the steps in your rollback file. (You should also checksum the rollback file so you can detect if the power failure occured while creating the rollback file.) Step (2), you update the database by replacing records and by appending new records. Step (3), you delete the rollback file.
Now there are two common database formats that I've encountered out there. The first common database format is a simple file array of fixed-sized records. If each patient record (for example) is a fixed-sized record, you simply create an array of these records in your database.
Of course this has a couple of disadvantages. First, each record is fixed--that means you can potentially waste a lot of space. Second, it's inflexable. And third, there really is no good way to handle adding "out of band" records--that is, records which store information that is not part of this simple array table. Things like hash indexes or B-Tree indexes wind up being stored in separate auxiliary files.
The second common file format I've encountered is documented in "Transaction Processing: Concepts and Techniques" by Jim Gray and Andreas Reuter. (Very heavy, but excellent book.) The idea is that you break your file into a bunch of fixed-sized records whose size is roughly 4K to 32K. Each record contains in it's header a sort of "page table of contents" which indicate the contents of this page. The body contains the data. The idea is that you fit oddly sized blocks of data within the fixed-size record, and mark where those are in the page table of contents so you can find each odd sized block of data using the tuple (page_index, toc_index).
What's good about this is that you can do all sorts of interesting tricks: if you need to replace a block of data with a larger block that no longer fits in the page, you can simply mark in the table of contents a "forwarding address"--that way, the (page_index, toc_index) refers to the same block--just forwarded on another page in the database. You can also cache these records in memory as you read them--creating a higher-level cache which stores a handfull of these pages can speed up access. And it does solve the problem of "how do I handle varchar records" and "where do I store my B-tree index records."
So this is the bottom level--creating code which handles your underlying database, and handles data recovery in case of a system failure.
Now it sounds like you want to keep stuff in memory as long as possible in order to minimize I/O to the file. While that's nice, it has the problem that in a system failure, stuff in RAM can be lost. That's why most commercial databases write transactions to the database as soon as you indicate "END TRANS"--because the idea is that when you end a transaction, the data had better be recoverable in case of a system failure.
I suspect you'll find that most people are looking up names and reading their status a lot more often than they are updating or changing patient information. So on top of your database file I/O engine, I would suggest two things. First, I would suggest a "write-through cache" of data records. The idea is that you keep around a collection of records in RAM, marking how old each record is, and updating that mark when a record is read--that way, records that are no longer being actively used eventually drop out of memory. When the user updates a record, you immediately write that record to the database, update it's timestamp, and move on.
Second, I would recommend that you find the fields that are commonly accessed to look up records (such as name, hospital patient ID, etc), and create either a hash table or a B-tree structure which speeds up looking up by those fields. The reason why is that when your database hits a few thousand records, you need a way to find records that is faster than reading the entire database into memory. A well-managed B-tree object will do the trick--it will permit you to find a patient, yet minimize the number of I/O reads you have to perform to O(log(n))--a big win when n is a few thousand.
If you insist on living a risky life with your data, you can implement a "lazy write" scheme where you write the updated record after it "expires" in memory--and then, optimize your writes by scanning the other records you have in memory that need to be written with this expired record, and write them all together. (This can potentially be a win, especially if you have multiple records living on the same database page--you potentially reduce several page read/page write operations into one page read/page write op.)
However, in my personal tinkering with these techniques in my own database engine, I find that these methods don't necessarly do the trick. That is, if you have 50 updated records, sooner or later you are going to have to do 50 database writes--and if these records are scattered through your database file, at best you will be able to collapse one or two of these records into a single write. So in a sense, you are risking losing a lot of data (losing 50 patients sounds like a lot to me) in the hope that you may get a 2%-4% performance gain. Better to spend a few hundred more on a faster hard disk for your server computer.
In short, I'd avoid any sort of lazy write scheme if possible.
Notice that I haven't put in a word about the multi-user aspect of this. That's because I'm assuming that your writing the database engine as a single process--that is, that the database engine is written as a single process which controls the database, which then communicates with the database clients through a pipe or socket. This has the advantage that it simplifies the whole "record locking" and "race condition" aspect by having one process own the file. And that means on top of your "file I/O", "record cache", and "record search" layers you have a "client access" layer and you're done. Another approach is to implement record locking and create a database engine which allows multiple processes to access the same file. This requires that you be able to implement some form of record locking on the file--that is, that you can (either in the operating system, or simulate through a "record lock file") lock segments of your database file; that way, you can update one segment of the database while another process access other records in the same file. (File locking, by the way, is really only used AFAIK to determine if the database page one process is attempting to read is currently being written over by another process.)
This method isn't much harder to implement than "one database file/one process". It has the advantage that if you are on a box with multiple processors, multiple client access can nicely divide across those processors. However, it's my experience in tinkering with such things that this sort of technique can be harder to code and get right at the record I/O level. (Though it does eliminate all of that "keep track of which client I'm talking to" code.)
Anyways, that's my thoughts on the matter. I do highly recommend the book "Transaction Processing", by the way, as it covers a lot of this (and more!) in incredible detail. The three lessons I've come away from the book is to "layer your code"--that way, error checking can be handled in the same way that error checking is handled on TCP/IP stacks. And second, always have a way to "rollback" or "undo" the last operation (or "transaction"); that way, you can guarentee database consistency in the event of a catestrophic failure, as well as implement a good recovery mechanism in the case of a minor error that doesn't even affect higher level code (such as waiting for a record to become unlocked). And three, you can speed a lot of things up by using a good tree structure to index your records.
Buy a T-shirt! Not only does it donate a few bucks towards The Cause, but the T-shirt also comes with the source code on the back. I got mine, in XXL.
Now if we could just get everyone who reads/. to get a T-shirt, then we could create the first "inverse class-action lawsuit", where an entire class of people are named as the defendant in a lawsuit, instead of the plantiff...:-)
Sorry, my bad. I thought I did my research correctly on Sony and MCA's web sites. (I confused the Sony/Columbia merger with the MCA/Matsushita merger.) However, the point still stands that at one time, Sony was poised to be the entertainment ruler of the universe. And yet they went through a fairly dark time, and have yet to come out on top.
Slashdot Mirror
For me, it's '###' in a comment, rather than 'XXX'; that's because '###' should never occur in regular C/C++ code. But I do concur; I've got '###' scattered pretty liberally throughout my code as well. They include things ranging from "I really need to complete this subroutine" to "I wonder if there is a better way to do this."
So just because there are about 20,000 BUGBUG lines doesn't really mean much.
FYI, Microsoft uses a specification-driven testing plan which is developed by a head of testing.
If there is one thing that Microsoft does right, by the way, it's this: Microsoft has separate career paths for programmers, testers, management, and content authors. Unlike other companies, which after 5 years of programming, you're expected to be bumped into management and never program again, Microsoft gives you the option to continue up the pay scale as a programmer instead.
What that means is that the head of the testing team for a large and important product such as Win2000 probably has a hell of a lot of Q/A experience.
Microsoft will also use a formal, specifications driven testing plan that included both a bunch of testers who do more "free-form exploration" of the product, and who do a formal testing which is driven by a script. They'll also be using testing automation.
The only problem with the Windows 2000 product is that it's an integrated operating system--that is, unlike Linux development which uses a Unix-like development philosophy of keeping each of the pieces small and managable, Windows 2000 attempts to put a hell of a lot into one large, monolithic black box. This plays hell on any formal testing plan in that you now have to test every possible interaction of every component of that black box. (Linux, on the other hand, can be tested more easily by testing each component separately, and making sure each component interacts with the standard Unix API correctly.)
The monolithic nature of Windows 2000 means that it will be near impossible to create a complete and full test plan, simply because of the huge number of component interactions within the black box. There's probably just too many conditions to test adequately.
In fairness, keep in mind that "defect", or even "bug" for that matter is a very broad term.
I have to concur about this. Realize that any company, such as Microsoft, who uses a formal Q/A program which uses a formal bug tracking system is going to eventually fill that bug log with a lot of niggly little things which frankly don't matter.
For example, one project I worked (a children's game) contained about 50 bugs from one tester who didn't like the color of the eyes of one of the characters. (He claimed they were too brown, and should be bluer.) Each bug report corrisponded to each scene in which the character appeared.
Eventually we scanned all of the art to make sure the eye color was the same, and closed out all of the bugs over the tester's protests.
I still have bug reports in my own bug tracking system covering similar non-issues, "helpful" suggestions, and things like "on-line help manual contains run-on sentence in 3rd paragraph." It's quite possible that the large bulk of the bugs relate to similar on-line content.
What's especially frustrating about such Q/A processes such as the one Microsoft uses is often they will report the easy to spot items, such as run-on sentences in on-line help. But at the same time, fatal, hard to detect crashing bugs will go by unnoticed because the Q/A test plan didn't include a situation which would stress the software in unexpected ways. And with the huge complexity of Microsoft's Windows 2000 product, that's the one that worries me the most: not that the product will ship with 65,000 content bugs, but that the crashing fatal bug which wipes out my hard disk wasn't even detected by their test plan.
My impression was not that his son was "weak", but when he died, his sons returned from doing battle to attend the funeral, and that stopped the westward momentum.
On the other hand, it was said about the Mongol empire that a well-dressed young woman could ride on horseback with two large sacks of gold from the Black Sea to the Pacific Ocean without once being accosted or robbed. Quite an impressive accomplishment for *any* empire or nation-state, even in today's world.
I believe that this is sometimes the case--I have translated acounts of some of the very same roman sooldiers and gladiators you speak of who rather enjoyed killing (or claimed to.)
Well, the predominate religion of the soldiers at that time, Mithriasm (sp?), put a high value on acting like a soldier: that is, it put a high value on killing and being killed with honor. So in a sense, killing folks and dying on your own shield were both considered holy, religious acts.
When Christianity started sweeping the Roman empire with it's acceptance by Constantine, Christianity adopted many of these "millitaristic" aspects. Thus things like "onward Christian Soldiers, marching as to war" has a long, ingraned history in Christian thought. Of course Christianity borrowed heavily from earlier Roman military cults, including things like an afterlife and a "heavenly reward for a life lifed well"--well being redefined to include non-soldierly acts.
In fact, as far as I can find, in Western civilization the notion that killing someone during battle is a bad thing is a recent invention. In large part because of people's disgust at the very cold blooded and calculated way we can now kill thousands or millions with the push of a button.
So I would argue that in fact, the depersonalization of war has made war unthinkable, rather than the other way around. That is, war and killing people was more acceptable to people when we were doing it hand to hand and face to face.
(1) When all you can pick on is spelling, it means you have nothing meaningful to say.
(2) I specifically picked the Khmer Rouge because I am aware the US has done nothing to stop them. I was engaging in something called "irony", a subtle practice which apparently some people just don't get.
if you beleive that killing people can ever be 'appropriate', then you have been brainwashed. the most basic tenet of any real moral philosophy is the sanctity of human life.
When a burglar breaks into your house and puts a gun to your throat, do you believe it is immoral to defend yourself, even if you have to resort to lethal force? Or do you believe you should simply yield, and perhaps die?
The analogy does extend to the international arena. I'm sorry if you don't see that, but unfortunately it does.
. it makes the US look foolish on the world stage when they say to milosevic et al. "don't kill people, killing people is a bad bad thing! hell, we think it's such a bad thing, we are going to send OUR army in and kill YOUR people. basically, they are saying "we will teach you not to kill by killing."... tha's like 'fucking for virginity'... it just makes no sense.
*shrug* I'm sorry if it doesn't make any sense to you, but it's about the use and/or the threat of use of lethal force. Sometimes you can stop someone by talking them down. Sometimes you can stop someone by threatening to blow them away. And sometimes you can't stop them--and wind up having to blow them away before they kill more people themselves.
We tried talking in Kosovo. We tried for *years*. Did no good; in fact, it got to the point where the folks there believed they could do whatever the hell they felt like because President Clinton didn't have the balls to intervene. Are you suggesting that in light of massacres and rape and pillaging and death we should send over yet another polite letter asking if they could stop?
Of course we could always just ignore things in Kosovo--quite a few people here in the United States advocated just that. But if you recall your world history, you'd remember that World War I started because of interlocking relationships between various factions that were (and are now still) fighting over there.
And if you think an isolated massacre sucks, try a world war in a world full of smaller powers with weapons of mass destruction.
how is training your army to be more effective at killing going to reduce the number of casualties, you nitwit! it might reduce the financial cost of war for you, but tha's about it.
Simple: the threat of force causes most people to think twice about starting a war themselves. In fact, it's working with you, so don't think it doesn't work with world leaders as well.
Again, if you recalled your history, you'd note the reason why the United States dropped two nuclear warheads on Japan at the end of World War II was because Japan refused to surrender and refused to stop making war on it's neighbors, and the estimated casualty count for invading Japan was in the multiple million range.
What happend in Nagasaki and Heroshima sucked. What could have happend if we had choosen to invade Japan would have been far worse.
why only two? you mean only two that you can think of... there's things like diplomacy, sanctions, the UN war crimes tribunal, etc. now, i don't have a problem if some highly highly skilled commandos capture these murderers so that they can stand trial in a court of law (what a new-age thought!) but, again, fighting to end some fighting makes you look like an idiot.
Several points.
1) Diplomacy only works when the agressors are willing to sit down and discuss reasonable measures. In the case of Kosovo, diplomacy was tried for *years*, and because the people involved had the honor of small hungry children who promise not to steal cookies from the cookie jar, diplomacy was a total and complete failure.
2) Sanctions only work when the country who is having sanctions used against it cares one whit, and when the parties imposing a sanction actually follow it. Or didn't you read about the Russian oil tankers who were attempting to run the UN imposed blockade against Iraqi oil?
By and large, with a willing leader who doesn't care about his people, sanctions don't work at all. Or did you forget about Cuba, who is moving into a quarter century of sanctions without changing it's government one millimeter?
3) A UN war tribunal only works when the governments who form the tribunal under UN authority are willing to send troups in in order to settle the situation--that is, if the UN is willing to send in troups who are prepared to kill people. Without such forces who are willing to impose the will of the tribunal, such a court of law is little more than a legal masturbation exercise--it may feel good, but ultimately produces nothing but a small mess.
4) As to the commandos: the United States has tried that on rare occassion, and has gotten into quite a bit of trouble. Frankly, sending in commandos is a violation of international law, while sending in troups is not, and for good reason: in theory, nations are not supposed to resort to undermining other nations--their complaints are supposed to be brought out in the open so the international community can know what the hell is going on.
Furthermore, the United States has made sending in commandos as you suggest illegal unless the President signs a special order. So it's not something we can "routinely" do.
well, this is just wrong again. it is not your army that protects you from domestic deathsquads (i didn't know that was a problem in the states, i guess i'll have to look for them next time i'm down there), it would be your local police dept, as far as i can tell.
There are clear lines of jurisdiction. And in the United States, it is the army who is responsible for external invaders, and the states (and by extension the local police) who are responsible for keeping local law and order.
Of course there is a lot of 'bleed over' in sharing technology: the city where I live are using technology developed for the Army to catch speeders at night. And the same training techniques are used, more or less, by the local police force to train cadets.
But what I was talking about was the fact that unlike many areas of the world, having a powerful army who is directly controlled by a civilian government, and who is embedded in a culture who prizes diversity and law and order above chaos and "kill your neighbor if he's not your brother" allows me to sleep better at night.
Now if someone could figure out how to arrange a ring of atoms to create 8 or 16 quantuum states, and arrange them so you could build a three or four bit adder with carry circuit out of two or three of these rings, then I'd be impressed.
:-)
I suspect scientists are working on it. 'Cause eventually, we're going to have to start using quantuum states and tweeking the fundamental nature of the universe to build processors fast enough keep up with the computational requirements of Windows 2020...
Now that the Internet (and in the future, the Internet II) are no longer playthings of academics and researchers but are seen as central to a developing dot.com economy, what will be interesting to see is how the Internet develops.
Personally I'm not very concerned with the privacy issues raised by Katz.
Why?
Because by the death of DIVX and by the current popular attitudes towards the DVD lawsuits and the rejection by the general public of technologies which are not "easy to use" (translation: which technologically blocks fair use), I suspect the technology controls that corporations are trying to put in place to block the general public from using intellectual properties they paid for will wither and die on the vine.
Personally if given the choice of buying copy-protected music on the net which (a) only can be played on my computer and not in my car, and (b) could be lost if my hard disk crashes, or buying a CD, I'll buy a CD. That's because a CD is more convenient. MP3s are only taking off in the way they are because they are more convenient than a CD--in particular you can burn a CD-ROM with your music and protect them in case of a hard disk crash, and you can copy them into a portable player or archive them in one location.
But this only outlines a larger battle: the battle between security and ease-of-use. Already you can read newspaper articles about various CIA employees who circumvent their secure home workstations in order to copy highly secret data to insecure home PCs because they want to get their work done. Popular are technologies which circumvent security precautions, such as cookies in lew of password-protected security on shopping sites, and the Macintosh's Key Chain and similar Windows technologies which remember passwords for you.
People want ease of use as much as they want security. And when the security is not in their own personal best interest (such as protecting the intellectual property of the music conglomerates), they tend to say "screw security, I want to play my Madonna music files in my car."
I think the corporations have an uphill battle on their hands. And I think eventually they'll lose--unless they can come up with a technology that is easy to use as a CD and doesn't invade their lives, people will simply resort to older technologies.
For most, freedom is the freedom to be left alone. Forcing people to remember arcane passwords, techniques and telling them "you can't play your music in the car unless you pay me twice" isn't leaving them alone: it's invading their lives in ways that most folks could do without.
The Internet, the successor to the ARPA-net, was developed using funding from DARPA, a DoD group who funds advanced projects. The ARPA-net was designed and developed as a distributed communications system which, amongst other things, could survive a nuclear war. (That was one of the driving design goals of TCP/IP, and that's why IP is so good at surviving things like a router going down.)
I first got started using the 'net way back in '83 when I was a student at Caltech. And at that time, you were only permitted to use the 'net if you had a valid research grant which required use of the ARPA-net account. Of course by about '85, that rule was largely ignored, and the whole thing turned over to total civilian control by about the same time, as I recall.
(Of course at the time I was drinking more beer than paying attention to whose research dime I was e-mailing on; silly me. So take my cronology with a grain of salt.)
But in point of fact, the academic world designed the thing in much the same way the academic world designs things like spy satelites and better ballistic missles and radar jammers and all those other nifty high-tech death machines...
On the Department of Defense's dime.
Political science theory going over your head? Here's a brief primer.
In the United States, we've studied these sorts of things to death. Things like the appropriateness and inappropriateness of war. And what most of the better thinkers in the US have come up with is that you only fight a war when another country (a) starts it, (b) that this war will affect citizens abroad in a negative way, (c) you can go in and stop it, and (d) the cost of stopping the war (by engaging the other side) is less than the cost of allowing the beligerant party to continue.
So in the United States, as we are the defacto policeman of the world (and have been to one degree or another since the Monroe Doctrine), we've been trying to figure out how to reduce the cost of war (in terms of casulties on both our side and theirs) in an attempt to promote a degree of "piece" at least for US citizens traveling abroad.
It sucks.
But until you can figure out a pieceful way to solve things like the Balkanization of Yugoslavia (yes, I appreciate the irony) and the murder and/or displacement of millions of innocent civilians in places like Chechnia, Kosovo, or Cambodia in a way which doesn't require some form of force, unfortunately you are going to have two choices.
1) Use force. And that means having very well armed people who are very well trained at killing people, or
2) Allow madmen like the Khamer Rouge run wild, murdering whoever they like.
The world sucks. But having the biggest and baddest army in the world maximizes the chance that when I go to sleep tonight, death squads won't break into my house and murder my wife as she sleeps next to me.
I see someone's not up on their history.
Let's see. We have the Mongolians who, in the process of spreading their might, wiped out millions, one at a time, using nothing more sophisticated than an axe. They were deadly enough that the name Khan just rang all sorts of bells with that Star Trek movie...
Then we have the Romans, whose spread of civilization was done at the cost of all of those insignificant, worthless "gauls" who weren't considered important enough by the Romans to do much more with than slaughter and enslave.
Of course we have the crusades, the various civil wars, holy causes, and of course the Inquisition. All fought with low tech. All fought more or less hand to hand. And many weapons were designed to kill someone only at very close range--such as a double-handed long sword, whose primary purpose was to dismember a knight in shiny armor in much the same way you pull apart a cooked lobster.
People have been killing people in very large numbers at very close range for thousands of years. It wasn't like the invention of a programmable missle controller chip (the predecessor to the Intel 4004) caused people to suddenly realize "hey, I don't have to take moral responsibility for who I kill, so let's fight even bloodier and bigger wars than ever before."
Nah; technology found its way to the private sector where it was used, ah, um, for stuff like this web site.
What it boils down to is a rather common principle in US law where *intent* is 9/10ths of the law.
/. don't exactly help convince the judge that the original intent by the developers of DeCSS was "cross-platform compatability."
.VOB file, so while it may be difficult to duplicate a DVD Movie Disk, it's not difficult at all to duplicate the .VOB files so a friend can play them back. From there, it's not a huge leap for someone to define an informal standard for a DVD playback engine which plays naked .VOB files as well as DVD Movie disks, a'la the plurality of MP3 players out there. And just wait for the day when they're stringing 10Mb internet wire to your house...)
That is, it is up to the court to determine the *intent* of the developer based on the developer's activities if the reverse engineering effort was done in order to properly achieve interoperability, verses the developer using the reverse engineering efforts to promote piracy.
Personally I find these sorts of distinction a little irritating, in that the court is left attempting to read people's minds to determine if a crime was committed or not. But it's not all that uncommon: most rape cases (for example) basically boil down to a "he said/she said", and factors such as what she was wearing and what he was drinking are used to assess the state of mind of the folks involved to determine if it was a rape or a misunderstanding. Or here in California, the difference between someone getting the gas chamber and someone getting 15 to 25 years is the "state of mind" of the murderer, often inferred by the type of magazines he read or the music he listened to. (That's the difference between premeditated homocide and the non-premeditated "heat of passion" variety.)
Unfortunately for the DeCSS folks, things like the logo "Masters of Reverse Engineering" at the bottom of the screen, or the web site www.dvd-copy.com proclaiming "How to find/trade FREE DVD Movies online" or the less skillful comments by folks here on
What is interesting in all of this, however, is that while the DeCSS code itself may go down the drain, it's quite likely that offshoots of the software, such as LiViD may wind up being found quite legal. That's because while the DeCSS program circulating around does little more than copy the VOD data to a hard disk, the LiViD software actually plays the movies. Thus, one could argue that even though LiViD makes use of the ill-gotten reverse engineering efforts of DeCSS, the intent of the developers of LiViD was to create a movie playback system which does fit within statue.
Further, even if all of the DeCSS stuff goes down the drain, we now have a roadmap on how to repeat the DeCSS effort in a way which would be construed as legal: by making sure that the intent of the development process is clearly outlined from the start to do nothing more than provide interoperability between Linux and DVD hardware on Linux boxes to play DVD movies.
(Yeah, yeah, yeah, I know the mantra: copying DVD movies is impossible because DVD burners don't hold enough data to copy a movie and because blank DVD-R disks have the CSS track prewiped to zero. But this is a week argument at best in that it is possible to play a naked
... except that Microsoft still hasn't learned the art of being subtle in playing politics--probably because they have such contept for anyone outside the Redmond campus.
I get these sorts of surveys all the time. They're used to manipulate politicians--or at least the stupid ones, as any politician with the IQ of a house plant knows these sorts of surveys are always loaded.
But typical questions on the surveys I get are things like "should Americans keep their God given right to bare arms as was granted to us by our Founding Fathers, or should the government take their cue from athiest God-hating communist countries and take our rights away." (Gun Control) Or "should Women have control over their reproductive selves or should government be able to inprison women for attempting to control their fate in light of an unwanted pregnancy?" (Abortion).
These sorts of questionares are always loaded, and it's not a supprise that Microsoft is trying to play the same game. Too bad they're too arrogant to play it well.
Actually the US is based on common law, which means in a sense that the judges get to make it up as they go along depending on the current thinking in legal circles and the current sentiment of the american public. Thus, the current wishy-washy state of things like abortion law or death penalty law.
We don't have slavery, Jim Crow laws and the civil rights movement because a strong willed minority in our government imposed their ethics on an unruly mob; we have these things because the unruly mob by and large wanted them, dispite a vocal minority. And we only have gotten these things recently--while the american Civil War settled the notion of slavery by turning it into a succession question, these other things have only been around with any meaning since the 60's, dispite the efforts of a few for at least a hundred and fifty years.
Public opinion matters when new law is settled on; that's why public opinion is asked for in this case. And in the case of Microsoft, this is one of those rare occassions where we are charting untested waters. (This will be the first time a non-government sanctioned technology monopoly will be dismantled.) So the judges want to know what informed people think.
Actually think "cell phone technology." That is, you rig it so that each transmitter only works within a limit area, with the different transmitters working to "trade off" as a device moves from cell to cell. Broadband only helps make things less prone to noise.
Of course the thing people don't tell you is that this is predicated on cell phone pricing as well. So the real question will be are people willing to surf the web on a 120x120 pixel display for $.10/minute?
Probably not, unless they're a "gizmo freek" or a masochist...
The theory is that the PC era is ending because wireless dumb terminals will slowly replace personal computers in the future. Of course this is predicated on infinitely powerful servers and infinitely large wireless bandwidth, which makes me think that this is all pie in the sky bull.
What's fascinating is that anyone who has half a brain should be noticing the trend towards web-enabled applications which supplant the browser, rather than having the browser replace applications in general. Of course the pundets are usually the last idio^H^H^H^Hfolks to notice these things...
By the way, keep in mind that TI has an interest in having the PC go away, so anything that TI announces about the "end of the PC era" should be taken with a grain of salt.
I think the thing you're missing here is that the term "database" doesn't necessarly apply to just a SQL server performing transactions across a wire. Granted, in this case, the person doing this project should probably invest in Oracle--after all, that's what he's doing. But I did suggest that at the top of my post.
Thing is, a "database" is not just SQL running remotely. Did you ever develop an application that loaded and saved data in a file? Well, that could also be considered a "database". Transaction processing doesn't just apply to traditional databases, either--they're a strategy to prevent your user from losing data just because he hit "save" at the wrong time.
Further, what ever happened to curiosity? What ever happened to the hacker mentality of wanting to know how it works and tinkering with something that's interesting? It does supprise me the number of people who instead of saying "I reinvented the wheel because I thought it would be cool" (hacker mentality) are saying "why the hell did you waste your time doing 'x'?"
Besides, once the guy realizes all that's involved, perhaps he can be the one who makes the mindful decision as to how to perform his employer's wishes?
Actually, and this is so totally off topic I don't know why I'm posting this here, is that if you get enough positive karma, your posts automagically show up as a +2 by default.
Go figure.
Now I have this little button down below that allows me to set it to +1 instead of +2; normally I select that when I submit a random bit of blathering like this. But I forgot to moderate myself down when I posted the above.
Sorry 'bout that.
Microsoft. They're always good for a swift kick or two. Not to mention a lawsuit or three...
I like the "state of mind" of the hacker community--like we have only one mind, and we all agree. Further, some of the quotes are offered to show that we all knew certain aspects of the law which frankly I, as a drone in the hive mind, was not made aware of by the hierarchy.
Further, the declaration makes a bunch of assumptions about how individuals must have known certain things, because they were posted by anonymous cowards here on
Ah, well. It's stupidities like this which make me a little, ah, itchy around some lawyers...
*shrug*
I only outline the above because it's interesting, because sometimes you don't need Oracle-SQL to store a few records, and because frankly, a hell of a lot of people who write applications which use a primitive "database" that simply consists of an array of records. And it wouldn't hurt to add a couple of layers to the low level record I/O routines in that application to make it a touch more bullet proof.
Me, I'm not a big believer in using a cannon to kill a fly. Using a full-on DB engine like Oracle or Transact-SQL just to store an array of records is a wee bit overkill...
Okay, there are two things you can do to solve this problem. First, you could use an "off-the-shelf" database engine. There are quite a few very good open source database engines out there, as well as a few good low cost and high cost engines.
OTOH, I assume you are probably interested in rolling your own engine. So here's a few strategies that you may wish to consider if you are going to do it yourself.
Overall, one of the most powerful metaphores for breaking down a problem into managable pieces is the notion of "layers" or of a program "stack". That is, you break a large problem into a collection of software "layers", like the TCP/IP protocol stack--at the lowest level you have the piece of software that talks to the hardware. Above that, you have the thing which encapsulates the packets. Above that, you have the thing which handles routing. And so forth.
It's a powerful notion because it allows you to break a complex problem such as "develop a database engine which provides transaction rollback and RAM caching" into a bunch of easy to understand units.
For a database, eventually you are going to have to write the records to a file, and take the performance hit that entails. That's just life. And worse--if you want to have proper transaction rollbacks (meaning if the power crashes while you are in the middle of writing a record, you need to have a mechanism to recover from this), that means you are going to have to do *two* file writes--one to write some sort of "recover" record, and one to write the record you are trying to save.
A common way to handle this sort of transaction rollback is to write a record or group of records in three steps. Step (1), read the records you are replacing and write them into a "backup" or "rollback" file. The idea is that this "rollback" file contains a record of everything you changed in the database file, so that if the power fails while you are updating your database, you can "fix" the database by reversing the steps in your rollback file. (You should also checksum the rollback file so you can detect if the power failure occured while creating the rollback file.) Step (2), you update the database by replacing records and by appending new records. Step (3), you delete the rollback file.
Now there are two common database formats that I've encountered out there. The first common database format is a simple file array of fixed-sized records. If each patient record (for example) is a fixed-sized record, you simply create an array of these records in your database.
Of course this has a couple of disadvantages. First, each record is fixed--that means you can potentially waste a lot of space. Second, it's inflexable. And third, there really is no good way to handle adding "out of band" records--that is, records which store information that is not part of this simple array table. Things like hash indexes or B-Tree indexes wind up being stored in separate auxiliary files.
The second common file format I've encountered is documented in "Transaction Processing: Concepts and Techniques" by Jim Gray and Andreas Reuter. (Very heavy, but excellent book.) The idea is that you break your file into a bunch of fixed-sized records whose size is roughly 4K to 32K. Each record contains in it's header a sort of "page table of contents" which indicate the contents of this page. The body contains the data. The idea is that you fit oddly sized blocks of data within the fixed-size record, and mark where those are in the page table of contents so you can find each odd sized block of data using the tuple (page_index, toc_index).
What's good about this is that you can do all sorts of interesting tricks: if you need to replace a block of data with a larger block that no longer fits in the page, you can simply mark in the table of contents a "forwarding address"--that way, the (page_index, toc_index) refers to the same block--just forwarded on another page in the database. You can also cache these records in memory as you read them--creating a higher-level cache which stores a handfull of these pages can speed up access. And it does solve the problem of "how do I handle varchar records" and "where do I store my B-tree index records."
So this is the bottom level--creating code which handles your underlying database, and handles data recovery in case of a system failure.
Now it sounds like you want to keep stuff in memory as long as possible in order to minimize I/O to the file. While that's nice, it has the problem that in a system failure, stuff in RAM can be lost. That's why most commercial databases write transactions to the database as soon as you indicate "END TRANS"--because the idea is that when you end a transaction, the data had better be recoverable in case of a system failure.
I suspect you'll find that most people are looking up names and reading their status a lot more often than they are updating or changing patient information. So on top of your database file I/O engine, I would suggest two things. First, I would suggest a "write-through cache" of data records. The idea is that you keep around a collection of records in RAM, marking how old each record is, and updating that mark when a record is read--that way, records that are no longer being actively used eventually drop out of memory. When the user updates a record, you immediately write that record to the database, update it's timestamp, and move on.
Second, I would recommend that you find the fields that are commonly accessed to look up records (such as name, hospital patient ID, etc), and create either a hash table or a B-tree structure which speeds up looking up by those fields. The reason why is that when your database hits a few thousand records, you need a way to find records that is faster than reading the entire database into memory. A well-managed B-tree object will do the trick--it will permit you to find a patient, yet minimize the number of I/O reads you have to perform to O(log(n))--a big win when n is a few thousand.
If you insist on living a risky life with your data, you can implement a "lazy write" scheme where you write the updated record after it "expires" in memory--and then, optimize your writes by scanning the other records you have in memory that need to be written with this expired record, and write them all together. (This can potentially be a win, especially if you have multiple records living on the same database page--you potentially reduce several page read/page write operations into one page read/page write op.)
However, in my personal tinkering with these techniques in my own database engine, I find that these methods don't necessarly do the trick. That is, if you have 50 updated records, sooner or later you are going to have to do 50 database writes--and if these records are scattered through your database file, at best you will be able to collapse one or two of these records into a single write. So in a sense, you are risking losing a lot of data (losing 50 patients sounds like a lot to me) in the hope that you may get a 2%-4% performance gain. Better to spend a few hundred more on a faster hard disk for your server computer.
In short, I'd avoid any sort of lazy write scheme if possible.
Notice that I haven't put in a word about the multi-user aspect of this. That's because I'm assuming that your writing the database engine as a single process--that is, that the database engine is written as a single process which controls the database, which then communicates with the database clients through a pipe or socket. This has the advantage that it simplifies the whole "record locking" and "race condition" aspect by having one process own the file. And that means on top of your "file I/O", "record cache", and "record search" layers you have a "client access" layer and you're done. Another approach is to implement record locking and create a database engine which allows multiple processes to access the same file. This requires that you be able to implement some form of record locking on the file--that is, that you can (either in the operating system, or simulate through a "record lock file") lock segments of your database file; that way, you can update one segment of the database while another process access other records in the same file. (File locking, by the way, is really only used AFAIK to determine if the database page one process is attempting to read is currently being written over by another process.)
This method isn't much harder to implement than "one database file/one process". It has the advantage that if you are on a box with multiple processors, multiple client access can nicely divide across those processors. However, it's my experience in tinkering with such things that this sort of technique can be harder to code and get right at the record I/O level. (Though it does eliminate all of that "keep track of which client I'm talking to" code.)
Anyways, that's my thoughts on the matter. I do highly recommend the book "Transaction Processing", by the way, as it covers a lot of this (and more!) in incredible detail. The three lessons I've come away from the book is to "layer your code"--that way, error checking can be handled in the same way that error checking is handled on TCP/IP stacks. And second, always have a way to "rollback" or "undo" the last operation (or "transaction"); that way, you can guarentee database consistency in the event of a catestrophic failure, as well as implement a good recovery mechanism in the case of a minor error that doesn't even affect higher level code (such as waiting for a record to become unlocked). And three, you can speed a lot of things up by using a good tree structure to index your records.
Now, what do we do about it?
/. to get a T-shirt, then we could create the first "inverse class-action lawsuit", where an entire class of people are named as the defendant in a lawsuit, instead of the plantiff... :-)
Buy a T-shirt! Not only does it donate a few bucks towards The Cause, but the T-shirt also comes with the source code on the back. I got mine, in XXL.
Now if we could just get everyone who reads
Sorry, my bad. I thought I did my research correctly on Sony and MCA's web sites. (I confused the Sony/Columbia merger with the MCA/Matsushita merger.) However, the point still stands that at one time, Sony was poised to be the entertainment ruler of the universe. And yet they went through a fairly dark time, and have yet to come out on top.