InnoDB Hot Backup is a tool that costs a bit of money. The equivalent tool in other packages (including PostgreSQL) is part of the overall package and free (in both senses of the word for PostgreSQL).
InnoDB Hot Backup is also the ideal method of setting up new slaves if you use the MySQL replication on InnoDB tables.
And this needs to be repeated far and loud: If you aren't using 100% InnoDB tables, you are royally screwed. Folks still running any MyISAM tables take note!
Ever heard of InnoDB? MySQL lets you choose -- on a table-by-table basis -- exactly what parts of your application need to support transactions, foreign keys, etc.
Want to do hot backups? Then all of your tables must be transaction-aware InnoDB tables (and you have to pay for the priviledge of course). Ever perform multi-row inserts? All of the affected tables should be transaction-aware InnoDB tables. If any of your data relates to each other (*cough* relational database *cough*), you need foreign keys. Here come those InnoDB tables again. Got multiple clients modifying the data at the same time? InnoDB again.
So basically what you are saying is that InnoDB tables should be used unless the table in question has no data interaction whatsoever with any other table, should only ever have one-row-at-a-time inserts, single-user access, etc. So...ummm...why would anyone use non-InnoDB tables?
Odd, neither Slashdot nor Yahoo! Finance seem to be having corruption problems...
I don't browse Yahoo Finance so I can't comment on that site or the scope of MySQL's use. As far as Slashdot...ummmm... Are you kidding!?! You are using Slashdot as a metric for the reliability of MySQL? Slashdot is your example of 24/7 uptime and consistency? The site that's regularly down and relies on static renderings of their site to avoid advertising the outages: this is one of your great examples?
I must say that I greatly enjoyed InnoDB's benchmark page. The comparison in E-Week is conveniently listed next to the PostgreSQL comparison giving the impression that E-Week also prefers MySQL over PostgreSQL even though PostgreSQL wasn't in the review. Nice marketing indeed. That aside, I also like the PostgreSQL comparison section: only two years old after all. Neither package has changed substantially in two years have they? <sarcasm>And sure, I could be persuaded that the one test query is indicative of all normal queries one would encounter in normal database operation.</sarcasm> Could it be that perhaps a certain group went hunting for items that InnoDB was markedly better at and avoided the items in which it was weaker. Naw. Couldn't be.
And this part I love the best: not only was the hardware the same, but the tuning techniques were identical. 24MB shared memory buffers. That's it on a 512MB test server with two tables of 100,000 rows. Hunh? No, that doesn't sound like they tuned MySQL overall to the detriment of PostgreSQL. How well do you think MySQL+InnoDB would fare if the app tunings were done with PostgreSQL in mind and just mapped the same settings to MySQL+InnoDB?
Note: The shared buffer setting has little to do with the overall memory usage in a working system (and even less in this benchmark). Why? After shared buffers are filled up, the OS will start to aggressively cache filesystem access (like for example, the ever-accessed database tables). MySQL is written with this situation in mind. PostgreSQL depends more heavily on the shared buffers setting (among others). If both DBs are using all available memory -- which is likely when querying a couple of 100K-row tables -- which database do you think will perform much better with the artificial constraint of a 24MB shared buffer?
And don't start with the "harder to configure" crap. This wasn't an out-of-the-box install of MySQL+InnoDB here. Someone specially configured the 24MB buffer. It took as much effort as it would to properly configure PostgreSQL. There is a wealth of information about tuning PostgreSQL for hardware, for good database organization techniques that work for more than just PostgreSQL, and of course the main PostgreSQL technical document site. (All of this was found in less than two minutes
Because in areas that have BPL, speeds are generally much higher, prices for customers are lower, and latencies are much lower than satellite (which by the way are horrible).
Note: I'm not necessarily advocating that HAM operators be left in the cold. While I am not one myself, I know too many to believe they don't exist in sufficient numbers.
On the other hand -- devil's advocate -- if HAM is indeed targetted toward emergencies, when an emergency strikes, the BPL will probably be down and no longer interfering. Let's be clear here: HAM operators are asserting their rights to broadcast during non-emergency situations.
Your "see their thousands of dollars of radio equipment" argument doesn't hold much water though. Let's say a particular region is known for a stupendous radio station. It's so great, residents over a huge area buy specialized, amplified antennas and supporting equipment to grab the signal clearly over long distances. Now supposed the radio station changes its content to something dramatically different. Is the radio station now liable for the thousands of dollars people spent to get a clear signal? I myself have a DSL modem from a previous carrier that in my current locale is basically a paperweight. You put down the money, you take your chances.
The situation does suck though. No doubt about it.
Do you see the work being done by Microsoft with regard to the "databasefying" of the filesystem as a boon or a hindrance to the adoption of programs that allow complexity to fall to the filesystem?
In other words, do you see a Windows developer who has worked with Longhorn (or whatever the name is now) being more inclined to develop for/port to Linux because of the parity in functionality? And would the converse be true in your opinion?
How much do you cherry-pick (and get cherry-picked) ideas from other modern filesystems? I understand you had some debates with the XFS developers over the "best way" to handle certain storage/algorithm cases. Was this the norm or are things far more insular than that with folks keeping their noses only in their own projects?
Do you have any updated performance metrics for reiser4 in comparison to ext3, xfs, jfs, and reiser3?
I'd be curious to see both small file test, large file tests, and a mixture -- as is the case for websites with movies where the HTML and graphics are 3K but the media files are 50MB and up.
Having seen the BeOS development APIs, Reiser4 strikes a familiar cord with its metadata handling and querying. However, while BeOS had a captive audience (there were no BeOS clones/work-alikes), you are making ReiserFS a part of an established operating system and all of the inertia that "establishment" brings with it.
Where BeOS failed (among other reasons) because of the perceived lack of applications, do you have any thoughts on how to seed development for ReiserFS.
Case in point, where in other filesystem setups you would have a mail spool and/or mailbox as a custom file format (and indexes) on top of the filesystem, Resier4 allows for custom queryable file attributes (metadata) and the ability to efficiently store each message in the queue as a separate file. The mail handling program becomes proportionally less complex as that complexity has been shifted toward the common filesystem layer.
That said -- mostly for the sake of folks that haven't been following Reiser4 development -- do you see a way of encouraging Reiser4-aware programs? After all, it's a chicken and egg issue. Without common usage of Reiser4, application developers will be less inclined to commit their time to it (perhaps?). Without applications that take advantage of its unique features, what reason does the user have to put it on their workstation or server?
Are we looking at custom Linux distributions with this in mind or just piecemeal applications? Personally, I'm looking forward to dropping Ogg Vorbis files into a directory and, through the use of plugins, having their ID3 info automatically searchable. Hmmm... Where was that Fugees song again?
It used to be late 2002. Then around late 2002, I noticed that the homepage had been changed to say the end of this month.
Was it an unforeseen debugging issue? Was there an algorithm change in the mid-to-lower levels? Was it a lack of coding resources due to excessive meetings? Or was it simply an underestimation of the total time required to get to the first public milestone?
I'm sure I'm not the only one who would like to know the type of things that delay releases in the free software world. When you have more info about what slows a project, you can see better how to avoid it in the future.
I would assume only if the Linux kernel is delayed for a very long time. Once Reiser4 is released, I'm sure there will be a shakedown period as more and more people use it and find bugs. This is the case with any complex piece of software. Reiser4 is quite different from Reiser3. It won't be a simple drop-in replacement. I wouldn't bet on it for 2.6, but maybe 2.6.4? Of course this is assuming that the release date for Reiser4 isn't pushed back again (it used to be Dec. 2002).
I would imagine that the filesystem plugins in Reiser4 are enough to warrant some extra head scratching with regard to its utility for kernel developers as well as application developers.
For an exercise, try sending a 900 K XML file off to a server and wait till it's done. Then look at the XML and see how you could make it smaller. It's kinda obvious and sad that it wasn't done in the first place.
Yes, you're right. Your extremely simple dataset would be more compact in that non-XML format.
Now, let's say that someone wants to include notes with those entries. For example, on day 1, "The Pepsi deal finalized." On day 2, "Lightning took out the business park power transformer so that work was suspended for four hours." Seems like a legitimate request, right?
In your format, you must make absolutely certain that no \r or \n charaters make it into the notes. "Easy enough," you say. "That's a short Perl one-liner."
Now someone wants not just the number of sales but the daily revenues to be included (Picture a red-faced boss screaming about how she doesn't care how many nickle and dime jobs were sold but the amount of money coming in). Hmmm... We can't put the revenues after the notes because we have no effective delimiter. We could put the notes in quotes, but then we must escape out all of the quotation marks in previous entries *and* retool our parser to look for quotes. Then again, you could put the revenues before the notes. Once again, you head back and fix the parser.
In XML, you simply add <note> and <revenue> tags. As a bonus, XML would allow for multiple notes per day. Make a DTD, XML Schema, or (better yet) RelaxNG document for your format, and you have an easy method of finding any errors without touching code and without tying your self to any particular programming language.
Yes, there's always BNF. But BNF is harder to write than RelaxNG. There's also the issue of tying the BNF to the document. XML handles this through the use of namespaces. I'm sure you could find a way to tie your documents to your BNF file. You've reinvented the wheel everywhere else, why stop here?
Finally, how does your format handle i18n? Is your parser UTF8-aware? You know that in some languages (those crazy Asians), UTF8 is more bulky than UTF16 or UCS2 right? Big5 and Shift-JIS are in heavy use as well. Make sure not to forget your chartype transcoder.
By writing your own file format, your own parser, and your own validator (you did write a validator, right?), you are spending extra time only to have to spend extra time later. Chances are, your task isn't writing a parser. Let others handle the job of parsing and get on with doing what you really need to do: get the job done.
You're telling me that your users cannot handle the added complexity? I'm sorry, but if someone is technically astute enough to use a text editor and modify a config file by hand, they can handle the trivial extra complexity associated with XML.
The users you describe that can't handle it are using the GUI anyway because *all* configuration files scare them.
That's a bit of a straw man argument don't you think? You're putting up a fragile target and then tearing it down. If I had a bunch of key/value pairs like that, why wouldn't I instead use:
...and so on for hundreds of entries. AND(!) I can add hierarchy at any point if necessary whereas the key/value pair file must be completely reworked.
As for coding difficulty, using the DOM in C++, the code looks something like this:
DomParser doc("myconfig.xml"); Element* config = doc.get_root_node(); if (config->getAttribute("fullscreen")->get_value( ) == "true") { // do something useful }
Assuming you are using libxml++ of course. Personally I prefer using other programming languages, but the code is very similar no matter the language. Let me say that again because it's important: "The code is very similar no matter the language."
And with XML, you get i18n for free. Can your program read Cyrillic characters? What about Chinese characters? Does it handle accent marks correctly? Or are you using ASCII with no contingency for i18n? With XML, it's a no-brainer. If you don't need it, don't use it. If you do need it, it makes recoding much much faster. UTF-8 you say? That works in most cases, it's true, but it doesn't address the other issues of a config file.
With your own config file format, you must handle comments, multi-line values, conversions from this format to the next, incompatible version. All of these are easier when using an XML parser. But you're a 1337 coder. Don't let me stop you. If you like spending time on config files, go right ahead. I prefer to spend time on the rest of the application.
...and the fact that you forgot to pass dst to enc_uint32be...
What happens when you want to have an Alpha box, a Pentium box, a handheld device, and an UltraSPARC box talk to one another? Simple, right? After all, an int is always 32 bits...err...umm...and everything is big endian...err...ummm...and all architectures use the same data structure padding... Well, at least your program took care of the padding issue...for that one data structure.
Wups! We've got a core dump waiting to happen. Okay, so we'll just make sure that everyone is using the same sizes and padding all around for any data structure I may need to pass over the network. Of course, this requires a mapping layer so I don't have to do this for every app and data structure that I write. I know: it's for interfaces and defines the general structure. I'll call it Interface Definition Language or IDL for short. Now I'll make sure that all of this information serializes to the network correctly and decodes on the other end without errors. This will be kind of like a stock broker it that I tell it what I want, and it translates it into something usable but more complex than I need to deal with for each app. I think I'll call it a broker too...an object broker...wait...missing something...messages going back and forth...asking for resources...aha! Object Request Broker! Yeah! Oh wait, but people may have different implementations and I want to be able to work with others. Let's agree on this. We'll call it the Common Object Request Broker...ummm...Architecture! Yeah, that's it!
Hmmm...now I need to make a configuration file for my program. I'll make it plain text. Hmmm...but it needs some kind of structure. I'll make it key/value pairs -- just put in a few equal signs and I'm done. Uh oh. My program is fairly modular, but I want to keep all of the settings in one place. If it's just key/value pairs, everything will get jumbled together. I know! I'll use an INI file. Microsoft used to use those to group items together. Now I can just use those nifty GetPrivateProfileString calls, specify the group and the key, and away I go. But uh oh! I have this subcomponent that requires a group within a group. Let me hack something together... Argh! This data file is getting tougher and tougher to parse. I want to finish writing my program that does something useful, not fiddle away at a dumb configuration file parser. What I need is a standardized, hierarchical format that is still plain text and human readable. Hmmm...what's this "XML" thing? I can have the configuration all in memory or read it in piecemeal? Parsers are already written? If I don't like the parser I'm using, I can just plug in another one? I can read the file from any programming language out there? Sign me up!
FYI: This binary vs. "plain text" tripe needs to go away. All text files are binary files. What is the letter 'B' but a 0x42 (66 in decimal, 01000010 in binary)? It's a piece of translation software that turns that 0x42 into the character 'B' on our screens. I just so happens that <foo/> is clearer to the human eye -- after the preliminary software translation step -- than a serialized C data structure. Clearer to the human eye means that the human fixing bugs can see the error faster. CPUs are hovering aroung the 3GHz range now, but the human mind seems to be falling further and further behind Moore's Law. Perhaps we should help the human mind out a bit and give a bit more work to the CPU.
Yes...I know...I'm a dick. I'm comfortable with that.
I still wonder whether or not a transport based on XML (or even ASN.1 representations of XML) would ever be made available to CORBA. All of the advantages of IDL with the serialization and interoperability of XML.
I agree with you on entities
on
DTD vs. XML Schema
·
· Score: 2, Informative
but I think you are totally off base with regard to CDATA sections. If anything, they make life easier for the parser, not harder -- at least when I was writing a parser, CDATA made things faster and easier. In cases where you are including a great deal of symbolic data -- for example, when you want to include a source code segment or ASCII art -- it is both easier to read, faster to parse, and *less* bloated.
'<' takes up less space than '<'. Assuming you have more than three or four of these in your text node, a CDATA section reduces the size of your document. For the parser, after the CDATA section is begun, only the character sequence ']]>' can end it. This means the parser only has to check for ']]>' and not '<', '&', '<?', '<!', etc.
And yes, there is such a beast called XInclude, but it's currently only a candidate release. It's used like this:
<foo>
<xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="bar.xml" parse="xml">
<xi:fallback>
<para>This text goes in if bar.xml cannot be found or has an error</para>
</xi:fallback>
</xi:include> </foo>
Hopefully most entities can go the way of the dodo.
Re:All this hype about XML
on
DTD vs. XML Schema
·
· Score: 4, Insightful
Yup, I remember CORBA. It's still used. In fact, it can be used with XML. CORBA provides the interfaces for programmatic logic. There is nothing really required to know about how the ORBs communicate with one another. If an ORB decides that its transport mechanism is XML, so be it.
As for SOAP and XML-RPC, what's so hard about compressing it before sending the message? The whole point about XML is that you don't need to write a new parser. You don't need to write a new broadcaster. Your project is about getting a task done, not micromanaging implementation details.
If (and only if) your higher level API/transport is insufficient for the task do you roll your sleeves up and dig in. Do you write everything in assembly? Why not? It would be faster than whatever language you are using now. The reason you don't is that you have better things to do with your time. The goal is important, not the tool. Everyone has standardized on and is optimizing this one particular tool and it works well. So many people have done work so that you don't have to.
Will it change the world? Of course not. It's just a markup language. Will any other computing tool change the world? Of course not. The end users have never cared how you got to the solution. They cared only if you got to the solution faster than the other guy.
Trimming bloat like namespaces and comments? Are you nuts?
How do you embed MathML in another document (like XHTML)? Currently it's with namespaces. How do you propose to do that without namespaces? Just the prefixes? What happens when two different markups use the same prefix? Wups! You're screwed!
No comments? This is supposed to make a better alternative to XML? It won't help readability, and it certainly isn't a major bottleneck during parsing.
Don't want the "bloat" of namespaces and comments? Wait for it... Wait for it... Don't use namespaces and comments in your documents! Wow! What a concept!
Maybe no Unicode in PXML hunh? So much for interoperability for any kind of data. You don't ever want your pet project used in East Asia (or Russia or Greece or most other places in the world) do you? Unicode too bloated? Why not just use ISO-8859-15 (basically ASCII w/ a Euro character -- which incidentally a Euro character isn't available in ASCII)? Oh wait! That's right. You don't want to allow processing instructions, which in XML tell you what encoding is used.
What happens if you want to change some of the basic syntax of PXML? Because you've nuked processing instructions, you can't specify a markup version like you can in XML.
Yes, yes. We've all seen your little pet project. I hope it was just a class assignment.
Try writing (let alone porting) an app that uses reflection in C or C++.
The usable, native compilation subsets of Java from TowerJ, Visual Age, Visual Cafe, gcj, et al. are still more complete than just about any other language out there.
I forget, where's the standard TCP/IP socket interface in C and C++ that works across all platforms?
For that matter, how many C compilers out there are 100% C99 compliant? How many C++ compilers out there are 100% ISO C++ 98 compliant.
I can't use reflection portably in TowerJ, but I can use threads, sockets, database connections, graphics, user interface widgets, XML, file I/O, Unicode character processing, distributed method calls, data structures, general algorithms, logging, regular expressions, compression, i18n, l10n, cryptography, image manipulation, and centralized directories.
Show me another language in common usage that has standardized interfaces for all of these things and also compiles to native code. Personally, I'm quite happy with Hotspot and/or JITs, but we were talking about native (ahead or time) compilation.
A question regarding reflections: have you tried using TowerJ and linking in the JIT only for the spots that need reflection?
Slashdot Mirror
And this needs to be repeated far and loud: If you aren't using 100% InnoDB tables, you are royally screwed. Folks still running any MyISAM tables take note!
Want to do hot backups? Then all of your tables must be transaction-aware InnoDB tables (and you have to pay for the priviledge of course). Ever perform multi-row inserts? All of the affected tables should be transaction-aware InnoDB tables. If any of your data relates to each other (*cough* relational database *cough*), you need foreign keys. Here come those InnoDB tables again. Got multiple clients modifying the data at the same time? InnoDB again.
So basically what you are saying is that InnoDB tables should be used unless the table in question has no data interaction whatsoever with any other table, should only ever have one-row-at-a-time inserts, single-user access, etc. So...ummm...why would anyone use non-InnoDB tables?
I don't browse Yahoo Finance so I can't comment on that site or the scope of MySQL's use. As far as Slashdot...ummmm... Are you kidding!?! You are using Slashdot as a metric for the reliability of MySQL? Slashdot is your example of 24/7 uptime and consistency? The site that's regularly down and relies on static renderings of their site to avoid advertising the outages: this is one of your great examples?
I must say that I greatly enjoyed InnoDB's benchmark page. The comparison in E-Week is conveniently listed next to the PostgreSQL comparison giving the impression that E-Week also prefers MySQL over PostgreSQL even though PostgreSQL wasn't in the review. Nice marketing indeed. That aside, I also like the PostgreSQL comparison section: only two years old after all. Neither package has changed substantially in two years have they? <sarcasm>And sure, I could be persuaded that the one test query is indicative of all normal queries one would encounter in normal database operation.</sarcasm> Could it be that perhaps a certain group went hunting for items that InnoDB was markedly better at and avoided the items in which it was weaker. Naw. Couldn't be.
And this part I love the best: not only was the hardware the same, but the tuning techniques were identical. 24MB shared memory buffers. That's it on a 512MB test server with two tables of 100,000 rows. Hunh? No, that doesn't sound like they tuned MySQL overall to the detriment of PostgreSQL. How well do you think MySQL+InnoDB would fare if the app tunings were done with PostgreSQL in mind and just mapped the same settings to MySQL+InnoDB?
Note: The shared buffer setting has little to do with the overall memory usage in a working system (and even less in this benchmark). Why? After shared buffers are filled up, the OS will start to aggressively cache filesystem access (like for example, the ever-accessed database tables). MySQL is written with this situation in mind. PostgreSQL depends more heavily on the shared buffers setting (among others). If both DBs are using all available memory -- which is likely when querying a couple of 100K-row tables -- which database do you think will perform much better with the artificial constraint of a 24MB shared buffer?
And don't start with the "harder to configure" crap. This wasn't an out-of-the-box install of MySQL+InnoDB here. Someone specially configured the 24MB buffer. It took as much effort as it would to properly configure PostgreSQL. There is a wealth of information about tuning PostgreSQL for hardware, for good database organization techniques that work for more than just PostgreSQL, and of course the main PostgreSQL technical document site. (All of this was found in less than two minutes
The standard library and templates have nothing to do with OOP.
Welcome to generic programming.
If it's an emergency and existing utilities are down, there won't be any BPL interference will there?
Because in areas that have BPL, speeds are generally much higher, prices for customers are lower, and latencies are much lower than satellite (which by the way are horrible).
Note: I'm not necessarily advocating that HAM operators be left in the cold. While I am not one myself, I know too many to believe they don't exist in sufficient numbers.
On the other hand -- devil's advocate -- if HAM is indeed targetted toward emergencies, when an emergency strikes, the BPL will probably be down and no longer interfering. Let's be clear here: HAM operators are asserting their rights to broadcast during non-emergency situations.
Your "see their thousands of dollars of radio equipment" argument doesn't hold much water though. Let's say a particular region is known for a stupendous radio station. It's so great, residents over a huge area buy specialized, amplified antennas and supporting equipment to grab the signal clearly over long distances. Now supposed the radio station changes its content to something dramatically different. Is the radio station now liable for the thousands of dollars people spent to get a clear signal? I myself have a DSL modem from a previous carrier that in my current locale is basically a paperweight. You put down the money, you take your chances.
The situation does suck though. No doubt about it.
Do you see the work being done by Microsoft with regard to the "databasefying" of the filesystem as a boon or a hindrance to the adoption of programs that allow complexity to fall to the filesystem?
In other words, do you see a Windows developer who has worked with Longhorn (or whatever the name is now) being more inclined to develop for/port to Linux because of the parity in functionality? And would the converse be true in your opinion?
How much do you cherry-pick (and get cherry-picked) ideas from other modern filesystems? I understand you had some debates with the XFS developers over the "best way" to handle certain storage/algorithm cases. Was this the norm or are things far more insular than that with folks keeping their noses only in their own projects?
Do you have any updated performance metrics for reiser4 in comparison to ext3, xfs, jfs, and reiser3?
I'd be curious to see both small file test, large file tests, and a mixture -- as is the case for websites with movies where the HTML and graphics are 3K but the media files are 50MB and up.
Having seen the BeOS development APIs, Reiser4 strikes a familiar cord with its metadata handling and querying. However, while BeOS had a captive audience (there were no BeOS clones/work-alikes), you are making ReiserFS a part of an established operating system and all of the inertia that "establishment" brings with it.
Where BeOS failed (among other reasons) because of the perceived lack of applications, do you have any thoughts on how to seed development for ReiserFS.
Case in point, where in other filesystem setups you would have a mail spool and/or mailbox as a custom file format (and indexes) on top of the filesystem, Resier4 allows for custom queryable file attributes (metadata) and the ability to efficiently store each message in the queue as a separate file. The mail handling program becomes proportionally less complex as that complexity has been shifted toward the common filesystem layer.
That said -- mostly for the sake of folks that haven't been following Reiser4 development -- do you see a way of encouraging Reiser4-aware programs? After all, it's a chicken and egg issue. Without common usage of Reiser4, application developers will be less inclined to commit their time to it (perhaps?). Without applications that take advantage of its unique features, what reason does the user have to put it on their workstation or server?
Are we looking at custom Linux distributions with this in mind or just piecemeal applications? Personally, I'm looking forward to dropping Ogg Vorbis files into a directory and, through the use of plugins, having their ID3 info automatically searchable. Hmmm... Where was that Fugees song again?
It used to be late 2002. Then around late 2002, I noticed that the homepage had been changed to say the end of this month.
Was it an unforeseen debugging issue? Was there an algorithm change in the mid-to-lower levels? Was it a lack of coding resources due to excessive meetings? Or was it simply an underestimation of the total time required to get to the first public milestone?
I'm sure I'm not the only one who would like to know the type of things that delay releases in the free software world. When you have more info about what slows a project, you can see better how to avoid it in the future.
The filesystem isn't a relational database. There are efforts to include more metadata and custom file handling, but it's not one-to-one.
A database that uses (requires?) the new features of Reiser4 as a base would definitely be interesting though.
I would assume only if the Linux kernel is delayed for a very long time. Once Reiser4 is released, I'm sure there will be a shakedown period as more and more people use it and find bugs. This is the case with any complex piece of software. Reiser4 is quite different from Reiser3. It won't be a simple drop-in replacement. I wouldn't bet on it for 2.6, but maybe 2.6.4? Of course this is assuming that the release date for Reiser4 isn't pushed back again (it used to be Dec. 2002).
I would imagine that the filesystem plugins in Reiser4 are enough to warrant some extra head scratching with regard to its utility for kernel developers as well as application developers.
Yeah. It's called gzip.
Yes, you're right. Your extremely simple dataset would be more compact in that non-XML format.
Now, let's say that someone wants to include notes with those entries. For example, on day 1, "The Pepsi deal finalized." On day 2, "Lightning took out the business park power transformer so that work was suspended for four hours." Seems like a legitimate request, right?
In your format, you must make absolutely certain that no \r or \n charaters make it into the notes. "Easy enough," you say. "That's a short Perl one-liner."
Now someone wants not just the number of sales but the daily revenues to be included (Picture a red-faced boss screaming about how she doesn't care how many nickle and dime jobs were sold but the amount of money coming in). Hmmm... We can't put the revenues after the notes because we have no effective delimiter. We could put the notes in quotes, but then we must escape out all of the quotation marks in previous entries *and* retool our parser to look for quotes. Then again, you could put the revenues before the notes. Once again, you head back and fix the parser.
In XML, you simply add <note> and <revenue> tags. As a bonus, XML would allow for multiple notes per day. Make a DTD, XML Schema, or (better yet) RelaxNG document for your format, and you have an easy method of finding any errors without touching code and without tying your self to any particular programming language.
Yes, there's always BNF. But BNF is harder to write than RelaxNG. There's also the issue of tying the BNF to the document. XML handles this through the use of namespaces. I'm sure you could find a way to tie your documents to your BNF file. You've reinvented the wheel everywhere else, why stop here?
Finally, how does your format handle i18n? Is your parser UTF8-aware? You know that in some languages (those crazy Asians), UTF8 is more bulky than UTF16 or UCS2 right? Big5 and Shift-JIS are in heavy use as well. Make sure not to forget your chartype transcoder.
By writing your own file format, your own parser, and your own validator (you did write a validator, right?), you are spending extra time only to have to spend extra time later. Chances are, your task isn't writing a parser. Let others handle the job of parsing and get on with doing what you really need to do: get the job done.
keyvalue.confconf.xmlYou're telling me that your users cannot handle the added complexity? I'm sorry, but if someone is technically astute enough to use a text editor and modify a config file by hand, they can handle the trivial extra complexity associated with XML.
The users you describe that can't handle it are using the GUI anyway because *all* configuration files scare them.
if (config->getAttribute("fullscreen")->get_value( ) == "true")
should be
if (config->get_attribute("FullScreen")->get_value ( ) == "true")
That's a bit of a straw man argument don't you think? You're putting up a fragile target and then tearing it down. If I had a bunch of key/value pairs like that, why wouldn't I instead use:
...and so on for hundreds of entries. AND(!) I can add hierarchy at any point if necessary whereas the key/value pair file must be completely reworked.
As for coding difficulty, using the DOM in C++, the code looks something like this:
Assuming you are using libxml++ of course. Personally I prefer using other programming languages, but the code is very similar no matter the language. Let me say that again because it's important: "The code is very similar no matter the language."
And with XML, you get i18n for free. Can your program read Cyrillic characters? What about Chinese characters? Does it handle accent marks correctly? Or are you using ASCII with no contingency for i18n? With XML, it's a no-brainer. If you don't need it, don't use it. If you do need it, it makes recoding much much faster. UTF-8 you say? That works in most cases, it's true, but it doesn't address the other issues of a config file.
With your own config file format, you must handle comments, multi-line values, conversions from this format to the next, incompatible version. All of these are easier when using an XML parser. But you're a 1337 coder. Don't let me stop you. If you like spending time on config files, go right ahead. I prefer to spend time on the rest of the application.
Class.forName(/*Database driver class name*/);
If you haven't done this explicitly, your frameworks certainly have.
What happens when you want to have an Alpha box, a Pentium box, a handheld device, and an UltraSPARC box talk to one another? Simple, right? After all, an int is always 32 bits...err...umm...and everything is big endian...err...ummm...and all architectures use the same data structure padding... Well, at least your program took care of the padding issue...for that one data structure.
Wups! We've got a core dump waiting to happen. Okay, so we'll just make sure that everyone is using the same sizes and padding all around for any data structure I may need to pass over the network. Of course, this requires a mapping layer so I don't have to do this for every app and data structure that I write. I know: it's for interfaces and defines the general structure. I'll call it Interface Definition Language or IDL for short. Now I'll make sure that all of this information serializes to the network correctly and decodes on the other end without errors. This will be kind of like a stock broker it that I tell it what I want, and it translates it into something usable but more complex than I need to deal with for each app. I think I'll call it a broker too...an object broker...wait...missing something...messages going back and forth...asking for resources...aha! Object Request Broker! Yeah! Oh wait, but people may have different implementations and I want to be able to work with others. Let's agree on this. We'll call it the Common Object Request Broker...ummm...Architecture! Yeah, that's it!
Hmmm...now I need to make a configuration file for my program. I'll make it plain text. Hmmm...but it needs some kind of structure. I'll make it key/value pairs -- just put in a few equal signs and I'm done. Uh oh. My program is fairly modular, but I want to keep all of the settings in one place. If it's just key/value pairs, everything will get jumbled together. I know! I'll use an INI file. Microsoft used to use those to group items together. Now I can just use those nifty GetPrivateProfileString calls, specify the group and the key, and away I go. But uh oh! I have this subcomponent that requires a group within a group. Let me hack something together... Argh! This data file is getting tougher and tougher to parse. I want to finish writing my program that does something useful, not fiddle away at a dumb configuration file parser. What I need is a standardized, hierarchical format that is still plain text and human readable. Hmmm...what's this "XML" thing? I can have the configuration all in memory or read it in piecemeal? Parsers are already written? If I don't like the parser I'm using, I can just plug in another one? I can read the file from any programming language out there? Sign me up!
FYI: This binary vs. "plain text" tripe needs to go away. All text files are binary files. What is the letter 'B' but a 0x42 (66 in decimal, 01000010 in binary)? It's a piece of translation software that turns that 0x42 into the character 'B' on our screens. I just so happens that <foo/> is clearer to the human eye -- after the preliminary software translation step -- than a serialized C data structure. Clearer to the human eye means that the human fixing bugs can see the error faster. CPUs are hovering aroung the 3GHz range now, but the human mind seems to be falling further and further behind Moore's Law. Perhaps we should help the human mind out a bit and give a bit more work to the CPU.
Yes...I know...I'm a dick. I'm comfortable with that.
I still wonder whether or not a transport based on XML (or even ASN.1 representations of XML) would ever be made available to CORBA. All of the advantages of IDL with the serialization and interoperability of XML.
but I think you are totally off base with regard to CDATA sections. If anything, they make life easier for the parser, not harder -- at least when I was writing a parser, CDATA made things faster and easier. In cases where you are including a great deal of symbolic data -- for example, when you want to include a source code segment or ASCII art -- it is both easier to read, faster to parse, and *less* bloated.
'<' takes up less space than '<'. Assuming you have more than three or four of these in your text node, a CDATA section reduces the size of your document. For the parser, after the CDATA section is begun, only the character sequence ']]>' can end it. This means the parser only has to check for ']]>' and not '<', '&', '<?', '<!', etc.
And yes, there is such a beast called XInclude, but it's currently only a candidate release. It's used like this:
<foo>
<xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="bar.xml" parse="xml">
<xi:fallback>
<para>This text goes in if bar.xml cannot be found or has an error</para>
</xi:fallback>
</xi:include>
</foo>
Hopefully most entities can go the way of the dodo.
Yup, I remember CORBA. It's still used. In fact, it can be used with XML. CORBA provides the interfaces for programmatic logic. There is nothing really required to know about how the ORBs communicate with one another. If an ORB decides that its transport mechanism is XML, so be it.
As for SOAP and XML-RPC, what's so hard about compressing it before sending the message? The whole point about XML is that you don't need to write a new parser. You don't need to write a new broadcaster. Your project is about getting a task done, not micromanaging implementation details.
If (and only if) your higher level API/transport is insufficient for the task do you roll your sleeves up and dig in. Do you write everything in assembly? Why not? It would be faster than whatever language you are using now. The reason you don't is that you have better things to do with your time. The goal is important, not the tool. Everyone has standardized on and is optimizing this one particular tool and it works well. So many people have done work so that you don't have to.
Will it change the world? Of course not. It's just a markup language. Will any other computing tool change the world? Of course not. The end users have never cared how you got to the solution. They cared only if you got to the solution faster than the other guy.
Trimming bloat like namespaces and comments? Are you nuts?
How do you embed MathML in another document (like XHTML)? Currently it's with namespaces. How do you propose to do that without namespaces? Just the prefixes? What happens when two different markups use the same prefix? Wups! You're screwed!
No comments? This is supposed to make a better alternative to XML? It won't help readability, and it certainly isn't a major bottleneck during parsing.
Don't want the "bloat" of namespaces and comments? Wait for it... Wait for it... Don't use namespaces and comments in your documents! Wow! What a concept!
Maybe no Unicode in PXML hunh? So much for interoperability for any kind of data. You don't ever want your pet project used in East Asia (or Russia or Greece or most other places in the world) do you? Unicode too bloated? Why not just use ISO-8859-15 (basically ASCII w/ a Euro character -- which incidentally a Euro character isn't available in ASCII)? Oh wait! That's right. You don't want to allow processing instructions, which in XML tell you what encoding is used.
What happens if you want to change some of the basic syntax of PXML? Because you've nuked processing instructions, you can't specify a markup version like you can in XML.
Yes, yes. We've all seen your little pet project. I hope it was just a class assignment.
Try writing (let alone porting) an app that uses reflection in C or C++.
The usable, native compilation subsets of Java from TowerJ, Visual Age, Visual Cafe, gcj, et al. are still more complete than just about any other language out there.
I forget, where's the standard TCP/IP socket interface in C and C++ that works across all platforms?
For that matter, how many C compilers out there are 100% C99 compliant? How many C++ compilers out there are 100% ISO C++ 98 compliant.
I can't use reflection portably in TowerJ, but I can use threads, sockets, database connections, graphics, user interface widgets, XML, file I/O, Unicode character processing, distributed method calls, data structures, general algorithms, logging, regular expressions, compression, i18n, l10n, cryptography, image manipulation, and centralized directories.
Show me another language in common usage that has standardized interfaces for all of these things and also compiles to native code. Personally, I'm quite happy with Hotspot and/or JITs, but we were talking about native (ahead or time) compilation.
A question regarding reflections: have you tried using TowerJ and linking in the JIT only for the spots that need reflection?
It doesn't taste any different and it costs more, but people buy it because they think the issue is important.