Re:Maybe its just the best tool for the job?
on
IT and Health Care
·
· Score: 1
It's not about security. It's about gathering information and using that data. Computers are better at that than paper forms. That should have been pretty clear by now...
...btrfs is starting from the ground up rather than try to fight those camped on their domains and won't play ball,.... So why don't you stop talking shit, or come up with specific cases to back up your claims.
Didn't you just do that for me?
Things like XFS or JFS are badly maintained and supported because they are too complex and were lumped in from other systems. This is a problem if, for example, XFS is the only serious option for really big volumes.
Reiser3 receives no more improvements, Reiser4 is dead. That doesn't leave much besides ext3. Funnily, ext3 has been catching up in performance just because the other FS are dead. Ok, maybe funny isn't the right word...
Unlike other OSes, Linux has several filesystems to chose for whatever the users' needs are, and new ones will appear from other proprietary systems at a later date. You think NTFS or HFS+ is any better?
Choice is fine when all choices are good. When all choices have serious and different issues, that just means effort has been wasted.
As for NTFS: At least from the application side you know which problems will hit you and which ones not.
fsync() (sync all pending driver buffers to disk) certainly has a major performance cost, but sometimes you do want to know that your data actually made it to disk - that's an entirely different issue from journalling and data/meta-data order of writes which is about making sure the file system is recoverable to some consistent state in the event of a crash.
The two issues are very closely related, not "an entirely different issue". What the apps want is not "put this data on the disk, NOW", but "put this data on the disk sometime, but do NOT kill the old data until that is done".
Applications don't want to be sure that the new version is on disk. They want to be sure that SOME version is on disk after a crash. This is exactly what some people can't seem to understand.
fsync() ensures the first at a huge performance cost. rename() + ext3 ordered gives you the latter. The problem is that ext4 breaks this BECAUSE of the journal ordering. The "consistent state" is broken for application data.
I'll also bet that if the file systems supported transactions (all-or-nothing journalling of a sequence of writes to disk), maybe via an ioctl(), that many people would be using that instead.
Yes. But they are assuming this exists and the API is called rename():)
You specifically have to choose writeback mode in the full knowledge that the datablocks will almost certainly be written after the metadata journal.
I think Ted Tso etc are probably perfectly aware of how it works.
Except that ext4 loses data in ordered mode for exactly the same reason, and we had a big fuss about that the last few weeks, because *someone* (cough) said that it's the application developers fault for not fsync()-ing.
I agree that the who-dun-it part is irrelevant. I disagree on the "SO f***d" part. We have three filesystems that write the journal prior to the data. Basically, we know the issue, and a similar fix can be shared amongst the three affected filesystems.
I would be very surprised if the fix can be shared between the filesystems. At least the most serious among those involved, XFS, sits on a complete intermediate compatibility layer that makes Linux looks like IRIX.
Linux filesytems are seriously in a bad state. You simply cannot pick a good one. Either you get one that does not actively kill your data (ext3 ordered/journal) or you pick one which actually gives decent performance (anything besides ext3).
Obviously, we should have both. It's not like that is impossible. But it's surprising how long those problems lasted. It's not like filesystems are a MINOR part of the entire OS.
Probably part of the reason is that we have JFS, XFS, ext3/4, reiser3/4, tux3, btrfs... Filesytem developers suffer very heavily from NIH syndrome. Instead of one good we have 8 that "almost" work.
But almost is not good for something so essential. This is not the kind of choice that is good. It's time one filesystem wins, gets fixed, and the rest is left dead.
Well, some Linux filesytem developers (and some fanboys) have been chastising other (higher-performance) filesytems for not providing the guarantees that ext3 ordered move provides.
Application developers hence were indirectly educated to not use fsync(), because apparently a filesystem giving anything other than the ext3 ordered mode guarantees is just unreasonable, and ext3 fsync() performance really sucks. (The reason why you don't actually *want* what fsync implies has been explained in the previous ext4 data-loss posts).
Some of those developers are now complaining that their "new" filesystem (designed to do away with the bad performance of the old one) is disliked by users who are losing data due to applications being encouraged to be written in a bad way, and telling the developers that they now should add fsync() anyway (instead of fixing the actual problem with the filesystem).
Moreover, they are complaining that the application developers are "weird" because of expecting to be able to write many files to the filesystem and not having them *needlessly* corrupted. IMAGINE THAT!
As an aside joke, the "next generation" btrfs which was supposed to solve all problems has ordered mode by default, but its an ordered mode that will erase your data in exactly the same way as ext4 does.
Honestly, the state of filesystems in Linux is SO f***d that just blaming whoever added writeback mode is irrelevant.
You are confusing writeback caching with ext3/4's writeback option, which is simply something different.
The problem with all the ext3/ext4 discussions has been the ORDER in which things get written, not whether they are cached or not. (Hence the existance of an "ordered" mode)
You want new data written first, and the references to that new data updated later, and most definitely NOT the other way around.
Linus seems to understand this much better than the people writing the filesystems, which is quite ironic.
As someone who has spent large amounts of time working on improving free and proprietary codecs, I am shocked! Was it all for nut? Have we made our codecs worse by making them better (more true to the original)?
Seriously, from reading the article, there is absolutely no indication this test was in any way scientific. (One of the involved having a title of "Professor" does not make a claim scientific, even though I'm sure we are made to believe that a lot)
So I'll take this claim with about much belief as I take a marketing claim: minus zero!
>By driving the cost of software to zero, OSS >developers have made it difficult for many people >to act creatively due to the high cost of >development. While OSS developers may make some >money developing an open source software package, >they have essentially forever undercut anyone who >might have also developed something similar. > >This, I think, is the reason why OSS is generally >of poor quality (generally speaking) compared to >closed source competition.
There's at least 2 fallacies here:
1) If the OSS software is truly worse, either the system using it will be worse, or someone needs to do the work for fix it up or hammer it into shape -> jobs.
2) Companies who make products made entirely out of off-the-shelf OSS are at big risk of having someone undercut their price. The more value you add, the less risk of this happening.
Open Source software makes it possible to build more advanced stacks because there is a base to begin with. You don't need to reinvent the basics.
Now, it IS true that if your value added part needs to be open sourced due to the GPL, life gets quite a bit more difficult. Would be hard to build a business model on.
>He complains the distribution differences make >life hard for people selling software. Well, >tough, if they want money maybe they should work >for it?
The problem is that they can make a single Windows installer and cover 95% of the market. Some fine tuning may be needed for example to cover old things like Windows 95 (no Unicode) or Vista annoyances, but those are minimal tweaks, and usually you don't have to bother.
On Linux, you'd have to cover the current versions of some major distributions (already a larger effort) just to cover maybe 3% more customers. And every other distro/version is still left out in the cold.
Supporting Linux can in many case simply be a losing proposition for a commercial vendor (particularly small ones). Why do all that effort for a minimal amount of customers? Lets just add more features for Windows users instead, and increase sales much more.
...remove anything at the bottom end of the spectrum. There is simply no point as the entire low frequency range can be represented by just a few coefficients.
The authors have no idea what they are talking about and are probably a combination of prejudiced and stone deaf.
Info about contenders and results of common compression programs on the testset. (All the "just use gzip/rar/winrk/..." fools can stop jabbering now...)
Slashdot Mirror
It's not about security. It's about gathering information and using that data. Computers are better at that than paper forms. That should have been pretty clear by now...
They restricted the hardware to 8 cores. As a result, the best programs, like those who run over clusters, are not playing.
Supposedly, this was to make money not a factor. In reality, some very nice expensive dual Nehalems are in action.
I'm saying that when you fsync() a 1k file on ext3 ordered, you can have 10G of data written out before the call returns.
- Disable the on-drive write-cache with 'hdparm'
Even better: use barriers (not enabled by default on ext3, not sure about ext4).
...btrfs is starting from the ground up rather than try to fight those camped on their domains and won't play ball,.... So why don't you stop talking shit, or come up with specific cases to back up your claims.
Didn't you just do that for me?
Things like XFS or JFS are badly maintained and supported because they are too complex and were lumped in from other systems. This is a problem if, for example, XFS is the only serious option for really big volumes.
Reiser3 receives no more improvements, Reiser4 is dead. That doesn't leave much besides ext3. Funnily, ext3 has been catching up in performance just because the other FS are dead. Ok, maybe funny isn't the right word...
Unlike other OSes, Linux has several filesystems to chose for whatever the users' needs are, and new ones will appear from other proprietary systems at a later date. You think NTFS or HFS+ is any better?
Choice is fine when all choices are good. When all choices have serious and different issues, that just means effort has been wasted.
As for NTFS: At least from the application side you know which problems will hit you and which ones not.
fsync() (sync all pending driver buffers to disk) certainly has a major performance cost, but sometimes you do want to know that your data actually made it to disk - that's an entirely different issue from journalling and data/meta-data order of writes which is about making sure the file system is recoverable to some consistent state in the event of a crash.
The two issues are very closely related, not "an entirely different issue". What the apps want is not "put this data on the disk, NOW", but "put this data on the disk sometime, but do NOT kill the old data until that is done".
Applications don't want to be sure that the new version is on disk. They want to be sure that SOME version is on disk after a crash. This is exactly what some people can't seem to understand.
fsync() ensures the first at a huge performance cost. rename() + ext3 ordered gives you the latter. The problem is that ext4 breaks this BECAUSE of the journal ordering. The "consistent state" is broken for application data.
I'll also bet that if the file systems supported transactions (all-or-nothing journalling of a sequence of writes to disk), maybe via an ioctl(), that many people would be using that instead.
Yes. But they are assuming this exists and the API is called rename() :)
You specifically have to choose writeback mode in the full knowledge that the datablocks will almost certainly be written after the metadata journal.
I think Ted Tso etc are probably perfectly aware of how it works.
Except that ext4 loses data in ordered mode for exactly the same reason, and we had a big fuss about that the last few weeks, because *someone* (cough) said that it's the application developers fault for not fsync()-ing.
So, switching writeback mode to write the data first would simply be using ordered data mode, which is the default...
The thread starts with someone having serious performance problems exactly because ext3 ordered mode is so slow in some circumstances...
Like when you fsync().
I agree that the who-dun-it part is irrelevant. I disagree on the "SO f***d" part. We have three filesystems that write the journal prior to the data. Basically, we know the issue, and a similar fix can be shared amongst the three affected filesystems.
I would be very surprised if the fix can be shared between the filesystems. At least the most serious among those involved, XFS, sits on a complete intermediate compatibility layer that makes Linux looks like IRIX.
Linux filesytems are seriously in a bad state. You simply cannot pick a good one. Either you get one that does not actively kill your data (ext3 ordered/journal) or you pick one which actually gives decent performance (anything besides ext3).
Obviously, we should have both. It's not like that is impossible. But it's surprising how long those problems lasted. It's not like filesystems are a MINOR part of the entire OS.
Probably part of the reason is that we have JFS, XFS, ext3/4, reiser3/4, tux3, btrfs... Filesytem developers suffer very heavily from NIH syndrome. Instead of one good we have 8 that "almost" work.
But almost is not good for something so essential. This is not the kind of choice that is good. It's time one filesystem wins, gets fixed, and the rest is left dead.
Well, some Linux filesytem developers (and some fanboys) have been chastising other (higher-performance) filesytems for not providing the guarantees that ext3 ordered move provides.
Application developers hence were indirectly educated to not use fsync(), because apparently a filesystem giving anything other than the ext3 ordered mode guarantees is just unreasonable, and ext3 fsync() performance really sucks. (The reason why you don't actually *want* what fsync implies has been explained in the previous ext4 data-loss posts).
Some of those developers are now complaining that their "new" filesystem (designed to do away with the bad performance of the old one) is disliked by users who are losing data due to applications being encouraged to be written in a bad way, and telling the developers that they now should add fsync() anyway (instead of fixing the actual problem with the filesystem).
Moreover, they are complaining that the application developers are "weird" because of expecting to be able to write many files to the filesystem and not having them *needlessly* corrupted. IMAGINE THAT!
As an aside joke, the "next generation" btrfs which was supposed to solve all problems has ordered mode by default, but its an ordered mode that will erase your data in exactly the same way as ext4 does.
Honestly, the state of filesystems in Linux is SO f***d that just blaming whoever added writeback mode is irrelevant.
You are confusing writeback caching with ext3/4's writeback option, which is simply something different.
The problem with all the ext3/ext4 discussions has been the ORDER in which things get written, not whether they are cached or not. (Hence the existance of an "ordered" mode)
You want new data written first, and the references to that new data updated later, and most definitely NOT the other way around.
Linus seems to understand this much better than the people writing the filesystems, which is quite ironic.
Parent post sounds scientific but is complete bull.
Does this seem pointless to anyone else? An audio compression technology that takes more storage space than the original, uncompressed format--FTW!
The compressed format will take less space than the uncompressed original. I have no idea where you came up with what you wrote.
This is just an MP3 re-release of a technology that was standardized as MPEG-4 SLS, where it is combined with AAC instead of MP3.
Thomson owns more patents on MP3, so they earn more if people don't use SLS. Not to mention they have to buy encoders and decoder from Thomson.
It's like MP3pro, which was a proprietary version of MPEG-4 HE-AAC. HE-AAC is now standard on most phones, MP3pro is dead.
As someone who has spent large amounts of time working on improving free and proprietary codecs, I am shocked! Was it all for nut? Have we made our codecs worse by making them better (more true to the original)?
Seriously, from reading the article, there is absolutely no indication this test was in any way scientific. (One of the involved having a title of "Professor" does not make a claim scientific, even though I'm sure we are made to believe that a lot)
So I'll take this claim with about much belief as I take a marketing claim: minus zero!
>By driving the cost of software to zero, OSS
>developers have made it difficult for many people
>to act creatively due to the high cost of
>development. While OSS developers may make some
>money developing an open source software package,
>they have essentially forever undercut anyone who
>might have also developed something similar.
>
>This, I think, is the reason why OSS is generally
>of poor quality (generally speaking) compared to
>closed source competition.
There's at least 2 fallacies here:
1) If the OSS software is truly worse, either the system using it will be worse, or someone needs to do the work for fix it up or hammer it into shape -> jobs.
2) Companies who make products made entirely out of off-the-shelf OSS are at big risk of having someone undercut their price. The more value you add, the less risk of this happening.
Open Source software makes it possible to build more advanced stacks because there is a base to begin with. You don't need to reinvent the basics.
Now, it IS true that if your value added part needs to be open sourced due to the GPL, life gets quite a bit more difficult. Would be hard to build a business model on.
That community is also small enough that you'll usually know whom you'd want.
8.10 does not include OpenOffice 3, you have to get it from external sources. And the latest versions is completely broken on KDE.
On 8.04 it's the same situation except it actually works :)
109-bit ECC keys give about 2^54 security
232-bit ECC keys give about 2^116 security
It's only a difference of 2^62!
>He complains the distribution differences make
>life hard for people selling software. Well,
>tough, if they want money maybe they should work
>for it?
The problem is that they can make a single Windows installer and cover 95% of the market. Some fine tuning may be needed for example to cover old things like Windows 95 (no Unicode) or Vista annoyances, but those are minimal tweaks, and usually you don't have to bother.
On Linux, you'd have to cover the current versions of some major distributions (already a larger effort) just to cover maybe 3% more customers. And every other distro/version is still left out in the cold.
Supporting Linux can in many case simply be a losing proposition for a commercial vendor (particularly small ones). Why do all that effort for a minimal amount of customers? Lets just add more features for Windows users instead, and increase sales much more.
>it means each tab runs in a separate thread (although for some reason they called them processes)
Processes and threads are vastly different things.
For example, one thread crashing means all other threads in the same process go down with it.
This is probably exactly why they use processes instead of threads.
...remove anything at the bottom end of the spectrum. There is simply no point as the entire low frequency range can be represented by just a few coefficients.
The authors have no idea what they are talking about and are probably a combination of prejudiced and stone deaf.
Tetration
http://en.wikipedia.org/wiki/Tetration
http://www.cs.fit.edu/~mmahoney/compression/text.h tml
Info about contenders and results of common compression programs on the testset. (All the "just use gzip/rar/winrk/..." fools can stop jabbering now...)
They were certainly not, and they are right.
The contest produces a hard, verifiable result with a hard restriction on resources that you can use to attain it.
If you look at the state of AI research, then you will understand that introducing some cold hard numbers wont hurt.