Apple's resource fork, with it's associated Resource Manager API, way back in 1984, is also another example of a very extensible binary file format.
I guess I would have liked to see something along those lines rather than XML. Perhaps an extensible binary format with a really good open source API for it.
Do you even know what XML stands for?
Hint; take a look at the X.
Good luck "X-ing" your JPEG format.
Yes, I know "X" stands for extensible and that many binary formats are not actually extensible. However, binary files can be extensible as well. TIFF for instance does a very good job of allowing extra blocks of data to be inserted into the stream without throwing off older decoders.
Anyhow, dvdeug made some very good points and I have conceded that XML can be an storage efficient format. That was the major drive of my original post, so I am no longer arguing against XML. See my reply here:
Dvdeug, you made some very good arguments in favor of XML when replying to my thread. I may have to rethink some of my views about it.
I liked it when you mention that KWord uses XML inside a gz file. I hadn't known that and it may invalidate my claim that XML files are less efficient in storage than binary files. All the redundancy in XML should compress very well, and you may even be able to approach a maximum with arithmetic coding.
The only counter-argument I could make is that you are using CPU cycles to find the redundancy, rather than writing out a space-efficient file in one swoop. But that's not a good argument since CPU cycles are very cheap now a days! I guess the trade-off is between CPU cycles and the man hours required to implement and debug a space efficient format from scratch. Of course, the former is always best. Computers are meant to do grunt work.
So I see it now. If you just look at XML itself, it's not very efficient or particularly useful. But the idea is that the result of XML + gz *is* a very efficient binary file. I've never thought of it that way.
Ok, apparently (and not unexpectedly), most of the posts on slashdot are about LISP being a better XML. In the context of the parent article, I concur. But my original post was not even about languages.
Let me sum it up: We seem to have a situation where everyone wants to solve three different problems with one technology. That is the crux of the problem as I see it. I make a modest proposal:
HTML
The ideal format for documents which have mostly text interspersed with markup.
LISP
The ideal format for encoding data structures in a human readable format.
Binary Files
The ideal format for encoding files which have a lot of data and are written by programs (as opposed to humans).
It seems to me that XML tries to do all three and does not do a particularly good job with any one purpose. My original post discusses why XML is a bad replacement for binary files. Most of the posts regarding this article touch on why XML is a bad replacement for LISP. Nobody has said HTML is bad for web pages, because in fact it is perfect for that.
What I haven't seen is a good discussion of what XML is good for (and why it would be superior to any of the three technologies I've mentioned for the respective three uses!)
"The advantage of XML is that you can use an off
the shelf parser for every language instead of
writing a new parser for each language. Let
someone else handle the parsing and you handle
only what you have to."
The assumption you make is that a parser is necessary for reading files. I did not say that we are better off having each developer write a whole new parser for an XML-like grammar for their documents, I am saying that most apps are better off with a binary file format which requires no parser of any kind (technically any file I/O requires a "parser", but I do not call reading a trivial sequences of bytes and offsets a parser). I think grammars tend to be an overkill for many types of data. Most binary files are simply fairly straight-forward dumps of in-memory data structures and the code for reading and writing them is not very complicated.
"As opposed to computer languages now, where
most modern languages (LISP-family excepted)
have context-dependent grammars that are
incredibly hard to parse correctly and each
language has to have a parser written specially
for it."
Ok. The parent article was about programming languages. I made a rather off-topic rant about XML data files in general. My bad.
However, if I confine myself to simply programming languages, I would have to say XML is a very poor substitute for LISP and again seems to be missing the mark. Consider this example:
So even if we are talking about programming languages, XML seems to provide no more than LISP already did, with a much more unwieldly syntax.
LISP also had a remarkably simple parser, so the argument that most other languages use very hard to parse syntaxes does not hold.
"Look at how many file formats the average
graphical viewer has to support. Each one has
its own library, its own bugs, its own security holes..."
Still, even with XML you need to write code that takes the output of the XML parser and transforms that into whatever data structures you need. Those routines are still need to be written once for each document type. So you still end up with multiple libraries and potential for bugs. Having a common XML parser seems to add very little, since it's just adding complexity to the data file without really solving that much.
I'm not saying XML is an entirely bad idea. For some types of data it might work (for example HTML is a good solution for web pages). But of late I've seen too many people pushing XML files for things like word processor documents, databases and other things which are better off with a binary format (so long as the code which generated them is itself open).
"Even though it costs about $5,200 US/$6,500, many
artists are starting to buy it to help them write succesfull songs"
Based on my work with AI this smells like a logical fallacy to me. It is likely that this system is picking out a very small subset of the patterns which make a song a hit. So this system may be able to accurately pick out songs which are in fact hits from those that aren't. However, that does not necessarily mean that any song with a high score will be a hit.
If an artist were to use this system in a tight feedback loop, tweaking his music until it got a high score, the music likely would lack many of the other features of good music, and it would probably sound terrible!
A good research experiment (for those of you who are grad students;) would be to take the artist out of the loop completely and use genetic algorithms to build a MIDI song which would score high according to this rating system. Would that song be a hit? I doubt it. Would it even sound like music? I doubt that too.
I personally do not understand what the entire hype about XML is, or even specifically what problem it is supposed to solve. My understanding is that there was a big push for XML because of a perceived need for open document formats. The idea being that binary formats were proprietary, closed and non-portable.
If this is the problem XML intends to solve, then I feel it is a miguided effort. Binary formats are "closed" only in so far as we do not have access to the source of the program that created them. Once that source is available, binary file formats are open, portable, and a hell of a lot more space efficient than XML. JPEG is a binary file format, yet we have open standards and the committee who designed it released open source reference implementations of the decoder and encoder. Hence, JPEG is an open format and nobody goes around trying to stuff pixels in XML files.
I really think XML is a solution to the wrong problem. The problem is closed source software, not binary files.
What makes this more confusing is that the name "The Great Train Robbery" often indicates the 1963 robbery of the Glasgow-to-London mail train by Biggs:
However, the supposed Train Robbery in Crichton's book happened much earlier. Everything I've seen so far seem to indicate that it never actually took place.
Don't get me wrong, I thought this was a very interesting paper, however I find it amusing that most of the discussion about how to crack a safe is totally irrelevant to the paper's topic. His major point related to computer security is that computer scientists should learn from safe-markers and employ more effective security metrics that define security in terms of the skills and time required to defeat the system. He could have accomplished this simply by describing how safes are rated, and skipped the entire discussion on how to manipulate a safe.
Good writing style (the old hamburger essay, IIRC!) says that details that do not directly support the main point of a paper should be edited out, and by those standards, this should have been a very short paper! (although admitedly much less interesting to Slashdotters and much less fun to read)
I almost get the impression that this paper was written primarily because the author finds safe-cracking to be a cool hobby to write about, and then he retrofitted it into a computer science context in order to present it at a computer security conference. This may also help to justify it and to keep people from criticising it for being a how-to guide to safe-cracking (like the infamous MIT guide).
I read this and really liked it, although I wasn't sure after I read it whether it was indeed true. Do you have any references that affirm this was a true story? If so, I'ld love to know more.
I know the novel itself claims the events are true, but so does Crichton's "Andromeda Strain". Just because a novel/movie claims to be true, it does not mean it actually is (e.g. The Blair Witch Project is a good example of a movie that claims to be true but isn't!).
-- Marcio
Don't CDs have to be shiny and reflective? What is shown in the picture sure looks dull and matte. Has blu-ray eliminated the need for the reflective backing?
-- Marcio
This post piqued my interest. I don't own a Mac so now I'm curious about how this thing works. I wasn't even aware that there was such a thing as a summarizing algorithm. How does it work? I did a search for "Summary" "OS X" on google and I got no interesting leads. Can anyone give me some pointers to places where I could either play with a summarizing program (maybe a web based one) or learn more about how it works?
Your page is about conspiracy theories. Why would this be a conspiracy?
There's a sociology book I read once which argues essentially what you wrote. In fact, it goes further, arguing that reality itself is just a social construction and a set of non-contradicting beliefs that are built up over time. It's an incredible read:
"The Social Construction of Reality: A Treatise in the Sociology of Knowledge", Peter L. Berger & Thomas Luckman
I guess you can this a conspiracy theory if you want, but this book seems to be well respected in its field and isn't the work of some crazy lunatic. I suggest you check it out if you have any interest in this topic.
Won't this be a tad bit dangerous? As the article points out, bitterness helps us avoid noxious foods. Once this additive gets put everywhere, won't there be people getting sick because they happily ingested a whole gallon of spoiled milk or gulped down moldy pizza?
> Isn't it possible that Computer Manufacturers > could use this so these machines would only > boot Windows?... This gives them the power to > do such a thing....
Yes. But you also have the power not to purchase such a machine. And while it is true that you may be compelled to buy these machines in order to access certain DRM-protected content, you also have a choice not to purchase that content (just as today you can choose not to buy copy protected CDs).
Clearly, everyone is concerned about what happens if all content becomes DRM-protected and unavailable to people that are using open/free systems. I wish people would realize that the best way to kill Palladium is to beat Microsoft to it. I wish there was some efforts under way to implement an Open Source DRM system under Linux, perhaps based on the Ogg Vorbis format. If we had a "trusted" Linux platform, signed by the FSF, for example, I'm sure content creators would take advantage of it to sell music or media under Linux. If this catches on before Palladium does, Microsoft would have a hard time pushing their own closed standard and Linux would have gained the upper-hand in a very big way
For occasional/emergency use, I suggest eCallPlus prepaid:
http://www.ecallplus.com
By purchasing a $20 card every three months you can get by with as low as $6.99/mo, which is a lot less than with full service plans. Of course, you only get fifty minutes, so this isn't for heavy cell phone users.
Ok. I think I see what you're saying now. By "interesting" you mean music which would be appealing to someone who is already an "expert" (a connossieur) in a particular genre, not one which would be interesting to someone unfamiliar with that genre.
I can relate to that. To return to the Moby example, I have a friend who is really into alternative music. For many years, I thought her music was horrible, because I couldn't make sense of it. It was only when she showed me a particular Moby CD that I began to find that particular genre acceptable and right now I'm less likely to consider her other music to be "horrible." So I think this is what you meant by Moby being "safe" and "inoffensive" -- it's the type of music you would show someone who is unfamilar with a particular genre, and which by such reason might seem like "cheap crap" to someone who was an expert in a particular type of music.
So I think the crux of the problem is that amazon.com's system is tuned to find such music which is appealing to novices of a particular style while you're looking for recommendations for someone who is already an expert in that particular genre. I see where amazon.com would fail.
It seems to me like it would be entirely possible to build such a recommendation system, but the system would be tuned in a different way than the ranking system which amazon.com employs. I'm sure this is a potentially interesting research area, but since amazon.com's data bank is proprietary, it isn't possible for people to play with it.
Sadly, I don't think we'll have a good recommendation system until people get together and come up with an "open" data set related to music preferences. Once this becomes available, I think we could have some definite progress in this area and different forms of AI could be evaluated.
You make an interesting observation, but I want to point out that most people being idiots does not imply that recommendation systems will necessarily fail. Consider a related example: economics is based on the premise that even though individuals act irrationally (as you say, most people are idiots) a large group of people will on average behave as if they were rational. Here's what David Friedman writes:
"Economics is based on the assumptions that people have reasonably simple objectives and choose the correct means to achieve them. Both assumptions are false--but useful. One reason to assume rationality is that... when predicting a market or a mob, what matters is not the behavior of a single individual but the summed behavior of many. If irrational behavior is random, its effects may average out"
-- David Friedman, Hidden Order: The Economics of Everyday Life
So, if a recommendation system such as amazon's fails it is probably because there isn't enough people using the system and the random effects due to the idiots have not yet averaged out.
BTW, I know Moby and like some of his songs, so I decided to listen to the clips for "Tangent 2002: Disco Nouveau" and I have to agree the style of music isn't anywhere near the same, but yet I sort of liked it. Perhaps the problem is that "amazon.com" learns to associate music that people own in common, but not necessarily whether it is the same style or kind of music.
There are people like myself that may confuse the system by liking music from all sorts of totally unrelated genres (so next time amazon.com suggests some country album to you after you've purchased a new age album, it's probably my fault:)
Slashdot Mirror
I guess I would have liked to see something along those lines rather than XML. Perhaps an extensible binary format with a really good open source API for it.
Yes, I know "X" stands for extensible and that many binary formats are not actually extensible. However, binary files can be extensible as well. TIFF for instance does a very good job of allowing extra blocks of data to be inserted into the stream without throwing off older decoders.
Anyhow, dvdeug made some very good points and I have conceded that XML can be an storage efficient format. That was the major drive of my original post, so I am no longer arguing against XML. See my reply here:
http://slashdot.org/comments.pl?sid=136513&cid=114 05771
So, I actually learned something from Slashdot today.
-- Marcio
I liked it when you mention that KWord uses XML inside a gz file. I hadn't known that and it may invalidate my claim that XML files are less efficient in storage than binary files. All the redundancy in XML should compress very well, and you may even be able to approach a maximum with arithmetic coding.
The only counter-argument I could make is that you are using CPU cycles to find the redundancy, rather than writing out a space-efficient file in one swoop. But that's not a good argument since CPU cycles are very cheap now a days! I guess the trade-off is between CPU cycles and the man hours required to implement and debug a space efficient format from scratch. Of course, the former is always best. Computers are meant to do grunt work.
So I see it now. If you just look at XML itself, it's not very efficient or particularly useful. But the idea is that the result of XML + gz *is* a very efficient binary file. I've never thought of it that way.
-- Marcio
Let me sum it up: We seem to have a situation where everyone wants to solve three different problems with one technology. That is the crux of the problem as I see it. I make a modest proposal:
HTML The ideal format for documents which have mostly text interspersed with markup. LISP The ideal format for encoding data structures in a human readable format. Binary Files The ideal format for encoding files which have a lot of data and are written by programs (as opposed to humans). It seems to me that XML tries to do all three and does not do a particularly good job with any one purpose. My original post discusses why XML is a bad replacement for binary files. Most of the posts regarding this article touch on why XML is a bad replacement for LISP. Nobody has said HTML is bad for web pages, because in fact it is perfect for that.What I haven't seen is a good discussion of what XML is good for (and why it would be superior to any of the three technologies I've mentioned for the respective three uses!)
-- Marcio
The assumption you make is that a parser is necessary for reading files. I did not say that we are better off having each developer write a whole new parser for an XML-like grammar for their documents, I am saying that most apps are better off with a binary file format which requires no parser of any kind (technically any file I/O requires a "parser", but I do not call reading a trivial sequences of bytes and offsets a parser). I think grammars tend to be an overkill for many types of data. Most binary files are simply fairly straight-forward dumps of in-memory data structures and the code for reading and writing them is not very complicated.
"As opposed to computer languages now, where most modern languages (LISP-family excepted) have context-dependent grammars that are incredibly hard to parse correctly and each language has to have a parser written specially for it."
Ok. The parent article was about programming languages. I made a rather off-topic rant about XML data files in general. My bad. However, if I confine myself to simply programming languages, I would have to say XML is a very poor substitute for LISP and again seems to be missing the mark. Consider this example:
(+ 1 1)
Versus:<ADD>
So even if we are talking about programming languages, XML seems to provide no more than LISP already did, with a much more unwieldly syntax.<NUMBER>1</NUMBER>
<NUMBER>1</NUMBER>
</ADD>
LISP also had a remarkably simple parser, so the argument that most other languages use very hard to parse syntaxes does not hold.
"Look at how many file formats the average graphical viewer has to support. Each one has its own library, its own bugs, its own security holes..."
Still, even with XML you need to write code that takes the output of the XML parser and transforms that into whatever data structures you need. Those routines are still need to be written once for each document type. So you still end up with multiple libraries and potential for bugs. Having a common XML parser seems to add very little, since it's just adding complexity to the data file without really solving that much.
I'm not saying XML is an entirely bad idea. For some types of data it might work (for example HTML is a good solution for web pages). But of late I've seen too many people pushing XML files for things like word processor documents, databases and other things which are better off with a binary format (so long as the code which generated them is itself open).
-- Marcio
Based on my work with AI this smells like a logical fallacy to me. It is likely that this system is picking out a very small subset of the patterns which make a song a hit. So this system may be able to accurately pick out songs which are in fact hits from those that aren't. However, that does not necessarily mean that any song with a high score will be a hit.
If an artist were to use this system in a tight feedback loop, tweaking his music until it got a high score, the music likely would lack many of the other features of good music, and it would probably sound terrible!
A good research experiment (for those of you who are grad students ;) would be to take the artist out of the loop completely and use genetic algorithms to build a MIDI song which would score high according to this rating system. Would that song be a hit? I doubt it. Would it even sound like music? I doubt that too.
-- Marcio
I personally do not understand what the entire hype about XML is, or even specifically what problem it is supposed to solve. My understanding is that there was a big push for XML because of a perceived need for open document formats. The idea being that binary formats were proprietary, closed and non-portable.
If this is the problem XML intends to solve, then I feel it is a miguided effort. Binary formats are "closed" only in so far as we do not have access to the source of the program that created them. Once that source is available, binary file formats are open, portable, and a hell of a lot more space efficient than XML. JPEG is a binary file format, yet we have open standards and the committee who designed it released open source reference implementations of the decoder and encoder. Hence, JPEG is an open format and nobody goes around trying to stuff pixels in XML files.
I really think XML is a solution to the wrong problem. The problem is closed source software, not binary files.
-- Marcio
http://www.crimelibrary.com/gangsters_outlaws/cops _others/biggs/index.html?sect=18/
However, the supposed Train Robbery in Crichton's book happened much earlier. Everything I've seen so far seem to indicate that it never actually took place.
-- Marcio
Good writing style (the old hamburger essay, IIRC!) says that details that do not directly support the main point of a paper should be edited out, and by those standards, this should have been a very short paper! (although admitedly much less interesting to Slashdotters and much less fun to read)
I almost get the impression that this paper was written primarily because the author finds safe-cracking to be a cool hobby to write about, and then he retrofitted it into a computer science context in order to present it at a computer security conference. This may also help to justify it and to keep people from criticising it for being a how-to guide to safe-cracking (like the infamous MIT guide).
-- Marcio
I read this and really liked it, although I wasn't sure after I read it whether it was indeed true. Do you have any references that affirm this was a true story? If so, I'ld love to know more. I know the novel itself claims the events are true, but so does Crichton's "Andromeda Strain". Just because a novel/movie claims to be true, it does not mean it actually is (e.g. The Blair Witch Project is a good example of a movie that claims to be true but isn't!). -- Marcio
This one is my favorite:
m
http://isorecorder.alexfeinman.com/isorecorder.ht
Lets you create ISOs, burn ISOs, and copy CDs just by right clicking on the icon!
-- Marcio
Well, at least grandpa will still be able to pirate movies using his good ol' Super 8.
Don't CDs have to be shiny and reflective? What is shown in the picture sure looks dull and matte. Has blu-ray eliminated the need for the reflective backing? -- Marcio
I *love* my BT500. Long live Bluetooth!
This post piqued my interest. I don't own a Mac so now I'm curious about how this thing works. I wasn't even aware that there was such a thing as a summarizing algorithm. How does it work? I did a search for "Summary" "OS X" on google and I got no interesting leads. Can anyone give me some pointers to places where I could either play with a summarizing program (maybe a web based one) or learn more about how it works?
Your page is about conspiracy theories. Why would this be a conspiracy?
There's a sociology book I read once which argues essentially what you wrote. In fact, it goes further, arguing that reality itself is just a social construction and a set of non-contradicting beliefs that are built up over time. It's an incredible read:
"The Social Construction of Reality: A Treatise in the Sociology of Knowledge", Peter L. Berger & Thomas Luckman
I guess you can this a conspiracy theory if you want, but this book seems to be well respected in its field and isn't the work of some crazy lunatic. I suggest you check it out if you have any interest in this topic.
My favorite quote:
Research has been shown to cause cancer in rats.
Won't this be a tad bit dangerous? As the article points out, bitterness helps us avoid noxious foods. Once this additive gets put everywhere, won't there be people getting sick because they happily ingested a whole gallon of spoiled milk or gulped down moldy pizza?
> boot Windows?
> do such a thing....
Yes. But you also have the power not to purchase such a machine. And while it is true that you may be compelled to buy these machines in order to access certain DRM-protected content, you also have a choice not to purchase that content (just as today you can choose not to buy copy protected CDs).
Clearly, everyone is concerned about what happens if all content becomes DRM-protected and unavailable to people that are using open/free systems. I wish people would realize that the best way to kill Palladium is to beat Microsoft to it. I wish there was some efforts under way to implement an Open Source DRM system under Linux, perhaps based on the Ogg Vorbis format. If we had a "trusted" Linux platform, signed by the FSF, for example, I'm sure content creators would take advantage of it to sell music or media under Linux. If this catches on before Palladium does, Microsoft would have a hard time pushing their own closed standard and Linux would have gained the upper-hand in a very big way
...will be called D flat? No, wait, they already have that since C# is the same as Db.
For occasional/emergency use, I suggest eCallPlus prepaid:
http://www.ecallplus.com
By purchasing a $20 card every three months
you can get by with as low as $6.99/mo, which is a lot less than with full service plans. Of course, you only get fifty minutes, so this isn't for heavy cell phone users.
Ok. I think I see what you're saying now. By "interesting" you mean music which would be appealing to someone who is already an "expert" (a connossieur) in a particular genre, not one which would be interesting to someone unfamiliar with that genre.
I can relate to that. To return to the Moby example, I have a friend who is really into alternative music. For many years, I thought her music was horrible, because I couldn't make sense of it. It was only when she showed me a particular Moby CD that I began to find that particular genre acceptable and right now I'm less likely to consider her other music to be "horrible." So I think this is what you meant by Moby being "safe" and "inoffensive" -- it's the type of music you would show someone who is unfamilar with a particular genre, and which by such reason might seem like "cheap crap" to someone who was an expert in a particular type of music.
So I think the crux of the problem is that amazon.com's system is tuned to find such music which is appealing to novices of a particular style while you're looking for recommendations for someone who is already an expert in that particular genre. I see where amazon.com would fail.
It seems to me like it would be entirely possible to build such a recommendation system, but the system would be tuned in a different way than the ranking system which amazon.com employs. I'm sure this is a potentially interesting research area, but since amazon.com's data bank is proprietary, it isn't possible for people to play with it.
Sadly, I don't think we'll have a good recommendation system until people get together and come up with an "open" data set related to music preferences. Once this becomes available, I think we could have some definite progress in this area and different forms of AI could be evaluated.
You make an interesting observation, but I want to point out that most people being idiots does not imply that recommendation systems will necessarily fail. Consider a related example: economics is based on the premise that even though individuals act irrationally (as you say, most people are idiots) a large group of people will on average behave as if they were rational. Here's what David Friedman writes:
... when predicting a market or a mob, what matters is not the behavior of a single individual but the summed behavior of many. If irrational behavior is random, its effects may average out"
:)
"Economics is based on the assumptions that people have reasonably simple objectives and choose the correct means to achieve them. Both assumptions are false--but useful. One reason to assume rationality is that
-- David Friedman, Hidden Order: The Economics of Everyday Life
So, if a recommendation system such as amazon's fails it is probably because there isn't enough people using the system and the random effects due to the idiots have not yet averaged out.
BTW, I know Moby and like some of his songs, so I decided to listen to the clips for "Tangent 2002: Disco Nouveau" and I have to agree the style of music isn't anywhere near the same, but yet I sort of liked it. Perhaps the problem is that "amazon.com" learns to associate music that people own in common, but not necessarily whether it is the same style or kind of music.
There are people like myself that may confuse the system by liking music from all sorts of totally unrelated genres (so next time amazon.com suggests some country album to you after you've purchased a new age album, it's probably my fault
Anyone care for a NoseBlaster card?
Marcio Luis Teixeira