Advertising is even more pervasive than you say: even magpies prefer shiny coins to dull ones. Guess that proves that advertising if for the birds.
Okay, seriously, you are overstating your point. You are ignoring our innate sense of taste, which affects our opinion of design and utility. You are also ignoring the fact that it is our culture, not advertising, that informs most of our purchasing choices. Yes, advertising influences culture, but it is just one input in the massive machine that is society.
I find this business model of yours intriguing. A fully self-sufficient business sounds like a great idea, but I am not sure my business can afford the capital outlay. How did you manage to buy the oil reserves to power your company's vehicle? Or am I assuming too much? Since your company designed and built all your motor vehicle in house, they could run on anything, right?
georgewilliamherbert does not have any good points as far as I can see. His latter comments are offtopic, complaining about non-Intel flash-based drives in general, when this thread is about Intel changing from SLC to MLC. (Ignoring that, I find his lack of references telling.)
To address his former point, the Intel drives use static wear leveling (as far as I know) so even if the drive is full, the flash is fairly evenly worn. That means you can get more space for the same price which mitigates the shorter lifespan of MLC.
Nowhere do I advocate cheaping out on your components, nor are my remarks aimed at home businesses. I think the rest of your comments about operating costs are a bit misplaced.
What you forget is MLC is about 2x cheaper than SLC, so you can get 2x the space for the same price. With wear leveling, extra space is extra lifespan, so MLC dies 5x faster than SLC.
What does that mean for you? I put my money (job) where my mouth is. Our reasonably high traffic OLTP database server uses Intel SSDs as filesystem-level write cache. We get an average write level of 10MB/sec. The minimum expected lifespan of the drive is 2 petabytes. That means we likely have SIX YEARS before the cells start to become unwritable. At that point, no data will be lost: the drive will report the write failures to the OS and store to cells that haven't become unwritable yet, and you will be able to continue operating for the next few months while you get a replacement drive.
Star Trek was pitched as a Space Western thing quite a while before Firefly. Them referring to space as the 'final frontier' was a bit of a give away. Even DS9 was intended as a frontier-town type western.
So pointing out that a legal issue is GPL specific and does not affect open source is general is ideological
No. However, supporting BSD licences over GPL because the latter are 'viral' is an ideological choice. Choosing a BSD license because one believes it to be more 'free' is an ideological choice. My real point here is that the BSD license is at least as 'ideological' as the GPL, and according to your logic, that 'its adoption will be impeded to some degree', just like the GPL.
BSD offers a mature code base.
I completely agree. But that wasn't my point.
For example while Google chose GPL based code Apple, Sun, etc chose BSD based code.
Absolutely! As a result, I can download Google's Android patches. Where can I get Apple's code? Now that OpenSolaris has been canned, where can I get Oracle's code?
How did the BSD-style licenses help Apple and Oracle develop open-source projects?
I see. And would you characterise your support for BSD licenses as... ideological?
Perhaps you should reread what you replied to:
And every time issues like this come out, it drives people further away from developing Open Source alternatives.
Anything that drives people away from GPL sources also drives people away from Open Source licenses, if you agree that the GPL is an open source license.
I can understand that you might be merely trying to add specificity to that comment, but please read what you are replying to one more time:
And every time issues like this come out, it drives people further away from developing Open Source alternatives.
If you can concede that being able to use a mature existing codebase is one of the main reason people use open source code in their projects, and if you concede that a large amount of open source code is only available under the GPL, then it follows that anyone who wants to develop something related to said GPL-only code, would be dissuaded from developing an open source project at all. If the GPL is FUD-encumbered and there is no non-GPL open source code, the people would likely not develop an open source project.
Okay, please read what you replied to just once more:
And every time issues like this come out, it drives people further away from developing Open Source alternatives.
Pay attention to that part that says 'developing Open Source.' Note how OP DIDN'T say 'using open source code'. For all their benefits, BSD-style licenses do not have INTRINSIC reason to develop derivative projects as open source. In fact, if the code WAS BDS'd, this would not be an issue at all, as Google would have been free to do whatever they wanted with those damnable header files.
That argument is idiotic. Drawing on the grandparent post, do you believe that someone who makes their fiancee sign a pre-nup thinks that their spouse plans to cheat on them or divorce them? Of course not.
If someone believes that an item is stolen, then the only recourse is not to accept it. No piece of paper can offer indemnity from that.
However, if you believe the item is not stolen, but recognise that you might be wrong, as is the case for the pawn shop, then you ask for indemnity. Similarly, if you believe in your spouse's fidelity, but recognise that you may have misjudged them, you get a pre-nup.
Unless I am misinformed, most antibodies are transferred to the young mammal from the mother during pregnancy. The mammoth will have many of the same immunities that that mother elephant did.
"As we've gotten more successful, there's a gap between the speed of our publishing pipeline and the speed of our receiving submissions pipeline. Our pipeline of leaks has been increasing exponentially as our profile rises, and our ability to publish is increasing linearly."
I understand your need to retreat to a more defensible position, but I ask you, once again, to actually read the my posts, and possibly your posts too.
I never claim that compression is a cure-all for anyone's database woes. You, on the other hand, *do* make an absolute claim:
Don't do this for any files that regularly get random writes (like, say, database files).
You go so far as to say ALL databases will suffer decreased performance. My thesis is that using well-implemented filesystem-level compression can improve performance for most databases that heavily use random writes, for the reasons in my earlier posts. I cannot, as I stated, comment on the internals of SQL Server, Oracle or Windows, but I do explain theoretical reasons why they should see improved performance. I am still waiting for you to address those claims.
Given that this article is about ZFS, and this thread was originally about ZFS filesystem compression, I assumed that the mention of NTFS was incidental.
As I don't have access to the source code for SQL Server or Oracle, I find it difficult to comment on its internals. By default I would expect Windows to keep the data it writes in the page cache and perform read ahead caching, like it does for all the other IO, ever since MS SmartDrive for DOS. I would also have suspected that the DBMSs you mentioned kept some kind of cache for themselves, but I will defer to you on this as well.
You may want to get hold of the Oracle people, because they are also misinformed.
Regarding your claim of doubling the IO, I suggest you read my post again. You did not actually account for, consider, or address any point I made. You just repeated yourself. That's not how a discussion works. As an example, I will now address your point:
In the most trivial cases, raw data blocks being written into already allocated space in file, you would be almost correct: only the bytes being written would have to be sent to the drive. That case does not apply to any database I have worked with (except maybe MS Access or SQLite, I am not sure). Any real world database has behaviour that is far more sophisticated than that, as I explained in my previous comment. Even in your hypothetical trivial database that directly modifies the data in place, an 'unnecessary' read is unlikely: Conceptually, an INSERT would likely be placed at the end of the file, where all the other recent inserts would have been, thus the data is likely to have been cached. An UPDATE would almost certainly have had to read the rows anyway to determine their eligibility for the update, so the read would also have been necessary.
As for why the feature was not included in the database software in the past, remember that the 'cheapness' of this transparent compression is a result of the uneven evolution of storage devices and processors. Processors have become more powerful a lot faster than the decrease in storage access performance (the explosion of multi-core chips helped a lot, too). We are now able to trade processing power we didn't have before to get increased IO performance.
That notwithstanding, your assumption was also wrong. Databases have had compression for a long time. For example, IBM's DB2 has had support for tablespace-level compression since at least 1993.
In fact, now that I think about it, your even more basic assumption is wrong: What's the most widely-used database in office environments? Here's a hint: It's made by Microsoft, and it's not SQL Server. Or Access.
Answer: NTFS (or possibly FAT, all those USB sticks and SD cards...)
A filesystem *IS* a (buzzword alert) no-sql database. To talk about filesystem compression as separate from database compression is misleading. Personal filesystem compression has been around since at least 1990. Didn't you use Stacker in DOS? With it you could see big benefits in system performance for exactly the same reasons as have been outlined in this topic; the difference there was since there was so much less processing power available, that would quickly become the application bottleneck. That is not true for modern database server.
I apologise, I wasn't clear at that point. Allow me to restate: In that point, I was highlighting that an 8k read will often result in more that 8k being read from the disk. The drive can read more, the OS IO subsystems could predict that the adjacent blocks be needed and also request them, or your database can look at the structure and ask for surrounding data, too.
As a user of ZFS for a database, I disagree with your post on several points.
Firstly, ZFS compression works on 128k (uncompressed) blocks. By default. If you wish to try and game the system, you can adjust the block size to match that of your database (Postgres, as an example, uses 8k blocks: no matter how small a row is, it will result in a read of at least 8k.)
Secondly, on a modern system there are so many levels of caching and IO-optimization, it is almost impossible to get a hard drive that only reads the 8k in question. The OS, your DBMS, or even the drive itself will probably read and cache the surrounding bytes, so there will probably be no additional IO latency.
Thirdly, on rotational media such as hard drives, the rotational time required to read 8k or 128k is trivial in the purest sense. The seek time is by far and away the determining factor on these drives, *especially* if your load is mostly random-access.
Fourthly, and on a related tack, random access writes are not as random as you think. Database internals, and database design best practices make this so. The often-changed parts of your database tend to end up clustered together. Further, chances are if your load is really that high, there will be multiple rows in a cached 128k block that need updating. If your load/isn't/ all that high, why are we having this conversation?
Finally, to cover the bases, remember that in almost every database server, there is always processing time to spare. Once again random access databases illustrate this best, with access times almost exclusively being the determining factor in throughput.
YHBT etc, but that is an interesting point. Your two examples are unrelated, but in a way Mr Narayen is right about crashes. If any application is able to 'crash' a whole computer, then the operating system has a problem. The OS should remain stable, regardless of what programs are executed. (Of course, the fact that an application is buggy means that it too is broken.)
Slashdot Mirror
Advertising is even more pervasive than you say: even magpies prefer shiny coins to dull ones. Guess that proves that advertising if for the birds.
Okay, seriously, you are overstating your point. You are ignoring our innate sense of taste, which affects our opinion of design and utility. You are also ignoring the fact that it is our culture, not advertising, that informs most of our purchasing choices. Yes, advertising influences culture, but it is just one input in the massive machine that is society.
Let's hope he soon goes the way of Jack Thompson and Robert Cringely.
I find this business model of yours intriguing. A fully self-sufficient business sounds like a great idea, but I am not sure my business can afford the capital outlay. How did you manage to buy the oil reserves to power your company's vehicle? Or am I assuming too much? Since your company designed and built all your motor vehicle in house, they could run on anything, right?
If keeping the floor clean means that we can't walk on it, then you're not performing your function.
Bad analogy. You wanting to use crazy hardware at work is closer to you wanting to drive your Harley over the Janitor's clean floor.
I thought that cat macros were catnip to trolls...
georgewilliamherbert does not have any good points as far as I can see. His latter comments are offtopic, complaining about non-Intel flash-based drives in general, when this thread is about Intel changing from SLC to MLC. (Ignoring that, I find his lack of references telling.)
To address his former point, the Intel drives use static wear leveling (as far as I know) so even if the drive is full, the flash is fairly evenly worn. That means you can get more space for the same price which mitigates the shorter lifespan of MLC.
Nowhere do I advocate cheaping out on your components, nor are my remarks aimed at home businesses. I think the rest of your comments about operating costs are a bit misplaced.
No. Two orders of magnitude is 100x. Good SLC vs good MLC is 10x, only a single order of magnitute longer lasting.
http://www.anandtech.com/show/2614/4
What you forget is MLC is about 2x cheaper than SLC, so you can get 2x the space for the same price. With wear leveling, extra space is extra lifespan, so MLC dies 5x faster than SLC.
What does that mean for you? I put my money (job) where my mouth is. Our reasonably high traffic OLTP database server uses Intel SSDs as filesystem-level write cache. We get an average write level of 10MB/sec. The minimum expected lifespan of the drive is 2 petabytes. That means we likely have SIX YEARS before the cells start to become unwritable. At that point, no data will be lost: the drive will report the write failures to the OS and store to cells that haven't become unwritable yet, and you will be able to continue operating for the next few months while you get a replacement drive.
Once jets have landed, you can shoot them, but you can't shoot them down.
http://en.wikipedia.org/wiki/Space_Western
Star Trek was pitched as a Space Western thing quite a while before Firefly. Them referring to space as the 'final frontier' was a bit of a give away. Even DS9 was intended as a frontier-town type western.
So pointing out that a legal issue is GPL specific and does not affect open source is general is ideological
No. However, supporting BSD licences over GPL because the latter are 'viral' is an ideological choice. Choosing a BSD license because one believes it to be more 'free' is an ideological choice. My real point here is that the BSD license is at least as 'ideological' as the GPL, and according to your logic, that 'its adoption will be impeded to some degree', just like the GPL.
BSD offers a mature code base.
I completely agree. But that wasn't my point.
For example while Google chose GPL based code Apple, Sun, etc chose BSD based code.
Absolutely! As a result, I can download Google's Android patches. Where can I get Apple's code? Now that OpenSolaris has been canned, where can I get Oracle's code?
How did the BSD-style licenses help Apple and Oracle develop open-source projects?
I see. And would you characterise your support for BSD licenses as... ideological?
Perhaps you should reread what you replied to:
And every time issues like this come out, it drives people further away from developing Open Source alternatives.
Anything that drives people away from GPL sources also drives people away from Open Source licenses, if you agree that the GPL is an open source license.
I can understand that you might be merely trying to add specificity to that comment, but please read what you are replying to one more time:
And every time issues like this come out, it drives people further away from developing Open Source alternatives.
If you can concede that being able to use a mature existing codebase is one of the main reason people use open source code in their projects, and if you concede that a large amount of open source code is only available under the GPL, then it follows that anyone who wants to develop something related to said GPL-only code, would be dissuaded from developing an open source project at all. If the GPL is FUD-encumbered and there is no non-GPL open source code, the people would likely not develop an open source project.
Okay, please read what you replied to just once more:
And every time issues like this come out, it drives people further away from developing Open Source alternatives.
Pay attention to that part that says 'developing Open Source.' Note how OP DIDN'T say 'using open source code'. For all their benefits, BSD-style licenses do not have INTRINSIC reason to develop derivative projects as open source. In fact, if the code WAS BDS'd, this would not be an issue at all, as Google would have been free to do whatever they wanted with those damnable header files.
That argument is idiotic. Drawing on the grandparent post, do you believe that someone who makes their fiancee sign a pre-nup thinks that their spouse plans to cheat on them or divorce them? Of course not.
If someone believes that an item is stolen, then the only recourse is not to accept it. No piece of paper can offer indemnity from that.
However, if you believe the item is not stolen, but recognise that you might be wrong, as is the case for the pawn shop, then you ask for indemnity. Similarly, if you believe in your spouse's fidelity, but recognise that you may have misjudged them, you get a pre-nup.
The 'Super Bowl' of soccer? Since when can Germany compete in the US 'Major League Soccer' tournament?
(end troll :)
Woman beats baby? What kind of sick game is that?
97 comments so far, and no "I hope you paid your $400 licensing fees, you cocksmoking teabaggers!" troll nostalgia yet?
For shame, Slashdot, for shame.
Unless I am misinformed, most antibodies are transferred to the young mammal from the mother during pregnancy. The mammoth will have many of the same immunities that that mother elephant did.
Could anyone confirm if they are near their processing limit, thus releasing it piece meal?
They claim to be:
http://blogs.forbes.com/andygreenberg/2010/11/29/an-interview-with-wikileaks-julian-assange/
"As we've gotten more successful, there's a gap between the speed of our publishing pipeline and the speed of our receiving submissions pipeline. Our pipeline of leaks has been increasing exponentially as our profile rises, and our ability to publish is increasing linearly."
My knee-jerk reaction is to say, "if you can't find something newly released and fun to play on the Wii, you aren't trying hard enough."
Still, I will bite: Why didn't you buy Monster Hunter Tri, Trauma Team or Kirby's Epic Yarn?
I understand your need to retreat to a more defensible position, but I ask you, once again, to actually read the my posts, and possibly your posts too.
I never claim that compression is a cure-all for anyone's database woes. You, on the other hand, *do* make an absolute claim:
Don't do this for any files that regularly get random writes (like, say, database files).
You go so far as to say ALL databases will suffer decreased performance. My thesis is that using well-implemented filesystem-level compression can improve performance for most databases that heavily use random writes, for the reasons in my earlier posts. I cannot, as I stated, comment on the internals of SQL Server, Oracle or Windows, but I do explain theoretical reasons why they should see improved performance. I am still waiting for you to address those claims.
Given that this article is about ZFS, and this thread was originally about ZFS filesystem compression, I assumed that the mention of NTFS was incidental.
As I don't have access to the source code for SQL Server or Oracle, I find it difficult to comment on its internals. By default I would expect Windows to keep the data it writes in the page cache and perform read ahead caching, like it does for all the other IO, ever since MS SmartDrive for DOS. I would also have suspected that the DBMSs you mentioned kept some kind of cache for themselves, but I will defer to you on this as well.
You may want to get hold of the Oracle people, because they are also misinformed.
Regarding your claim of doubling the IO, I suggest you read my post again. You did not actually account for, consider, or address any point I made. You just repeated yourself. That's not how a discussion works. As an example, I will now address your point:
In the most trivial cases, raw data blocks being written into already allocated space in file, you would be almost correct: only the bytes being written would have to be sent to the drive. That case does not apply to any database I have worked with (except maybe MS Access or SQLite, I am not sure). Any real world database has behaviour that is far more sophisticated than that, as I explained in my previous comment. Even in your hypothetical trivial database that directly modifies the data in place, an 'unnecessary' read is unlikely: Conceptually, an INSERT would likely be placed at the end of the file, where all the other recent inserts would have been, thus the data is likely to have been cached. An UPDATE would almost certainly have had to read the rows anyway to determine their eligibility for the update, so the read would also have been necessary.
As for why the feature was not included in the database software in the past, remember that the 'cheapness' of this transparent compression is a result of the uneven evolution of storage devices and processors. Processors have become more powerful a lot faster than the decrease in storage access performance (the explosion of multi-core chips helped a lot, too). We are now able to trade processing power we didn't have before to get increased IO performance.
That notwithstanding, your assumption was also wrong. Databases have had compression for a long time. For example, IBM's DB2 has had support for tablespace-level compression since at least 1993.
In fact, now that I think about it, your even more basic assumption is wrong: What's the most widely-used database in office environments? Here's a hint: It's made by Microsoft, and it's not SQL Server. Or Access.
Answer: NTFS (or possibly FAT, all those USB sticks and SD cards...)
A filesystem *IS* a (buzzword alert) no-sql database. To talk about filesystem compression as separate from database compression is misleading. Personal filesystem compression has been around since at least 1990. Didn't you use Stacker in DOS? With it you could see big benefits in system performance for exactly the same reasons as have been outlined in this topic; the difference there was since there was so much less processing power available, that would quickly become the application bottleneck. That is not true for modern database server.
I apologise, I wasn't clear at that point. Allow me to restate: In that point, I was highlighting that an 8k read will often result in more that 8k being read from the disk. The drive can read more, the OS IO subsystems could predict that the adjacent blocks be needed and also request them, or your database can look at the structure and ask for surrounding data, too.
As a user of ZFS for a database, I disagree with your post on several points.
Firstly, ZFS compression works on 128k (uncompressed) blocks. By default. If you wish to try and game the system, you can adjust the block size to match that of your database (Postgres, as an example, uses 8k blocks: no matter how small a row is, it will result in a read of at least 8k.)
Secondly, on a modern system there are so many levels of caching and IO-optimization, it is almost impossible to get a hard drive that only reads the 8k in question. The OS, your DBMS, or even the drive itself will probably read and cache the surrounding bytes, so there will probably be no additional IO latency.
Thirdly, on rotational media such as hard drives, the rotational time required to read 8k or 128k is trivial in the purest sense. The seek time is by far and away the determining factor on these drives, *especially* if your load is mostly random-access.
Fourthly, and on a related tack, random access writes are not as random as you think. Database internals, and database design best practices make this so. The often-changed parts of your database tend to end up clustered together. Further, chances are if your load is really that high, there will be multiple rows in a cached 128k block that need updating. If your load /isn't/ all that high, why are we having this conversation?
Finally, to cover the bases, remember that in almost every database server, there is always processing time to spare. Once again random access databases illustrate this best, with access times almost exclusively being the determining factor in throughput.
Sat things up so that the sail will be closed or parallel to the wind when on top of the wheel, and perpendicular to it when on the bottom
Kind of like... this?
http://en.wikipedia.org/wiki/File:Perpetuum1.png
YHBT etc, but that is an interesting point. Your two examples are unrelated, but in a way Mr Narayen is right about crashes. If any application is able to 'crash' a whole computer, then the operating system has a problem. The OS should remain stable, regardless of what programs are executed. (Of course, the fact that an application is buggy means that it too is broken.)
Your sig:
Browsing with classic discussion, noscript, at -1 and nested no hidden comments and I only mod UP
Your post:
"why is a boring muggle like you even doing here?"
-1 for HP reference, -1 for grammatical failure
So in this case you wanted to mod UP by negative 2?