Given we have just had our best year for SciFi movies that I can remember with Star Trek, District 9, and Moon all extremely good I would have thought SciFi was in good hands with some new benchmarks for future movies to aspire to.
How are filesystems that were designed from the ground up to be journaled filesystems "essentially bolt journaling to the traditional file system UNIX layout"?
Did you actually bother to look at the layout of an XFS filesystem?
You refer to a Debian article on XFS CPU usage, but fail to mention that the conclusion of the aritcle is "Based on all testing done for this benchmark essay, XFS appears to be the most appropriate filesystem to install on a file server for home or small-business needs" and "Personally, I still choose XFS for filesystem performance and scalability".
You then go on to continue the myth about XFS corrupting your data, a myth that has been debunked countless times. Changes were made to XFS some time ago so that it behaves in a way that user's were expecting.
It's a somewhat half hearted blog in my opinion and I'm surprised that slashdot picked it up.
XFS has at least been partly ported to most operating systems in SGI's CXFS. I have serious doubts that a complete port will ever see the light of day since a fair amount of work would be required to complete the ports, and I can't imagine a business plan to do this that would suit SGI.
It was a dark day for Linux when Linus said he didn't believe in kernel debuggers.
Last time I had a good look at what was on offer while developing a filesystem for multiple operating systems, I compared the tools for Linux to Windows, MacOSX, IRIX and Solaris.
Windows was clearly the best for remote kernel debugging with windbg and I don't see that changing anytime soon. A fully fledged debugger, automatic download of kernel symbols from Microsoft and your own repository, reliable capture of dumps etc. That and the API documentation made the black box nature of the O/S a much smaller issue than the open source community would like us to believe. I was always so much more confident if we had a crash due to our software at a customer site that with Windows we had the best chance of capturing and hopefully identifying the problem.
For live debugging IRIX was the best with icrash. Its port to Linux, lcrash, was not as good and reliable at the time, maybe its better now but the lack of symbol files it need on Linux always made it frustrating. And now IRIX is dead.
I've heard some really good things about dtrace, so Solaris and Mac look to have the best live debugging and tracing tool there is today...I see some progress on Linux ports which is great to see.
But when it comes to kernel debugging on Linux, the picture is still bleak.
Can you please point me to a single instance of a complete filesystem being filled with zeros?
Files that have been grown and written into cache may contain zeros because the linux vm did not ask XFS to write that data out yet. The application (and user) always has the option of sync'ing the file to push it to disk straight away.
Re:My take on current filesystems
on
EXT4 Is Coming
·
· Score: 1
This just seems crazy, why invest in 2K when 4K cameras, projectors and storage capable of supporting those rates are coming? Of course I'm assuming that 4K will be good enough for the next 10 years, but since that is better than what we have now...
I worked for an Australian company that with a bit of government funding and support from a major university setup a software engineering course in a rural city (100,000 people?), where they would complete their degree part time while working on real contracts.
No where else other than the major cities could they hope to get a degree like that in Australia. And having the work experience behind them would have made them highly employable.
I still believe the idea was good, but starting this just as the bubble burst meant that there was little work and after a couple of years it was closed down.
There was a lot of difficulty in attracting work to the centre since there were always about their ability being junior engineers. So we had to attract some senior engineers there as well to lead the teams. That was harder than attracting contracts, since we were the only employer in the area looking for those skills. But fundamentally the inability of the company to attract work for itself let alone the training center was its downfall.
What happened to the people that were there? Many have now moved to the cities to complete their degrees and get work.
Many commercial operating systems put an enormous amount of effort into trying to maintain binary compatibility between releases, including Windows. It can be a significant constraint on drammatic innovation, but it does not mean that innovation is not taking place. Just because marketing decide to latch onto a feature and promote it, does not mean engineering are not innovating behind the scenes.
The linux community are unable to maintain binary compatibility in libc between 2.4.X releases, let alone any sort of stable kernel APIs.
New kernel sercurity patch? I'm sorry, you will have to recompile your kernel and all device drivers. Imagine if Microsoft required all the third party device drivers to be rebuilt with every kernel because the APIs _may_ have changed.
I find it interesting that American airports have all these security checks now, some are polite and try to preserve individual's privacy, others like to "strip search" you in front of everyone else, yet when you catch a flight out of the country the level of security is pathetic.
Yet a 747 with >14 hours of fuel to fly to Australia has got to be more dangerous than any internal flight.
If they don't ship the 970-based power macs they are dead, the disapointment will destroy them after all the rumors. Since they haven't dispelled the rumors...
The NSAs distribution does _NOT_ meet "their standards" (Common Criteria), it does not provide auditing (or at least it didn't last time I looked) which is required for most protection profiles. It certainly has not been evaluated.
Which you need to know before interpreting the results...
I have to disagree, I thought Figure 3 illustrated how important it is to baseline to ensure that you are heading in the right direction with each change you make (although they did not have a uni-processor baseline result).
It also showed that with the changes in June they are able to get a 4 times performance with another 7 cpus. Maybe next time they will show how it scales over the number of cpus you have.
That's probably a bit harsh, both IBM and SGI have worked pretty hard to get scalability improvements into the linux kernel. The article does mention some of these things:
Some of the issues we have addressed that have resulted in improvements include adding O(1) scheduler, SMP scalable timer, tunable priority preemption and soft affinity kernel patches.
I think it is reasonable to do benchmarks against likely realworld applications. It seemed clear to me that they understand that the benchmark may not represent a load anyone may actually encounter, but that is outweighed by the ability of someone else to come along and use the same benchmarks.
Some scientific/mathematical benchmarks would also be good to see.
It would be great to see a follow up/some examples on how these tools are used to actually track down a performance problem. I have and I have seen many others take some performance data and make completely the wrong judgement about what is the expected behaviour, what is the bottleneck, and what to do to fix it.
I was also suprised to see that they still use some of the old performance monitoring tools like looking at/proc, and other ascii tools, rather than something like PCP that collects all these statistics together so that you can look at any combination of subsystems on the same time line. Then they could have graphs showing the interraction and load on the disk, cpus, vm, network etc.
As much as I hate to support the US war effort, and wonder if we will ever know if the declaration was factual, if the US and UK are to be believed then a true declaration must be considered vaporware.
Trusted IRIX was recently re-evaluated B1 and IRIX C2 for version 6.5.13 (which was released only about 9 months ago) on currently available hardware. So it is possible with the common criteria to be evaluated within a reasonable timeframe (unlike TCSEC).
It is also worth noting that Microsoft have had Windows 2000 going through a C2 evaluation for over 18 months with a proper hardware configuration unlike the previous NT 4.0 evaluation.
No, as some other replies have indicated, the only reason for 64-bit applications is to access 64-bit addresses. If you don't need to address that amount of memory, then your applications might as well be 32-bit. Any good processor will still give you 64-bit registers etc to work with.
Several attempts were made during the 90's to develop UNIX standards so that independant software vendors (ISVs) could develop and support software for multiple UNIX platforms. To a large extent these failed.
Several attempts have been made by UNIX vendors to make it easier for ISVs to support their products on new releases of their O/Ss, like ensuring shared library compatibility in minor releases. Some vendors have had some success, although probably too late.
At the moment I don't see the Linux community addressing these issues which some would argue have contributed to the slow downfall in UNIX in the battle with M$ and each other.
There are many (although the number maybe decreasing) distributions and there are cracks
growing in the kernel group which is struggling to keep the kernel to a single source tree, once touted as the differentiator for Linux. This along with incompatibility between shared libraries in releases makes it very difficult and expensive to support a software (and even hardware) product under Linux.
Given that IBM were there with AIX and have been through this already, how do you see these issues being addressed, or do you believe they are not major issues?
Is IBM content to just distribute RedHat or some other distribution on their platforms, or are you planning to add your own value-added software on your platforms? If so, are you planning your own distribution?
Slashdot Mirror
Given we have just had our best year for SciFi movies that I can remember with Star Trek, District 9, and Moon all extremely good I would have thought SciFi was in good hands with some new benchmarks for future movies to aspire to.
How are filesystems that were designed from the ground up to be journaled filesystems "essentially bolt journaling to the traditional file system UNIX layout"?
Did you actually bother to look at the layout of an XFS filesystem?
You refer to a Debian article on XFS CPU usage, but fail to mention that the conclusion of the aritcle is "Based on all testing done for this benchmark essay, XFS appears to be the most appropriate filesystem to install on a file server for home or small-business needs" and "Personally, I still choose XFS for filesystem performance and scalability".
You then go on to continue the myth about XFS corrupting your data, a myth that has been debunked countless times. Changes were made to XFS some time ago so that it behaves in a way that user's were expecting.
It's a somewhat half hearted blog in my opinion and I'm surprised that slashdot picked it up.
XFS has at least been partly ported to most operating systems in SGI's CXFS. I have serious doubts that a complete port will ever see the light of day since a fair amount of work would be required to complete the ports, and I can't imagine a business plan to do this that would suit SGI.
It was a dark day for Linux when Linus said he didn't believe in kernel debuggers.
Last time I had a good look at what was on offer while developing a filesystem for multiple operating systems, I compared the tools for Linux to Windows, MacOSX, IRIX and Solaris.
Windows was clearly the best for remote kernel debugging with windbg and I don't see that changing anytime soon. A fully fledged debugger, automatic download of kernel symbols from Microsoft and your own repository, reliable capture of dumps etc. That and the API documentation made the black box nature of the O/S a much smaller issue than the open source community would like us to believe. I was always so much more confident if we had a crash due to our software at a customer site that with Windows we had the best chance of capturing and hopefully identifying the problem.
For live debugging IRIX was the best with icrash. Its port to Linux, lcrash, was not as good and reliable at the time, maybe its better now but the lack of symbol files it need on Linux always made it frustrating. And now IRIX is dead.
I've heard some really good things about dtrace, so Solaris and Mac look to have the best live debugging and tracing tool there is today...I see some progress on Linux ports which is great to see.
But when it comes to kernel debugging on Linux, the picture is still bleak.
Can you please point me to a single instance of a complete filesystem being filled with zeros?
Files that have been grown and written into cache may contain zeros because the linux vm did not ask XFS to write that data out yet. The application (and user) always has the option of sync'ing the file to push it to disk straight away.
For a better understanding of what you are referring to as XFS zeroing:
http://oss.sgi.com/projects/xfs/faq.html#nulls
To see how XFS can now be configured to reduce corruption due to power failure, see:
http://oss.sgi.com/projects/xfs/faq.html#wcache
And no UNIX vendor has ever done that? How do you think IBM got #1?
This just seems crazy, why invest in 2K when 4K cameras, projectors and storage capable of supporting those rates are coming? Of course I'm assuming that 4K will be good enough for the next 10 years, but since that is better than what we have now...
Linux scaling to 512 processors:/ columbia/
http://www.sgi.com/features/2004/oct
The story should be HP has finally caught up to where SGI were 2 years ago.\
I worked for an Australian company that with a bit of government funding and support from a major university setup a software engineering course in a rural city (100,000 people?), where they would complete their degree part time while working on real contracts.
No where else other than the major cities could they hope to get a degree like that in Australia. And having the work experience behind them would have made them highly employable.
I still believe the idea was good, but starting this just as the bubble burst meant that there was little work and after a couple of years it was closed down.
There was a lot of difficulty in attracting work to the centre since there were always about their ability being junior engineers. So we had to attract some senior engineers there as well to lead the teams. That was harder than attracting contracts, since we were the only employer in the area looking for those skills. But fundamentally the inability of the company to attract work for itself let alone the training center was its downfall.
What happened to the people that were there? Many have now moved to the cities to complete their degrees and get work.
I disagree.
Many commercial operating systems put an enormous amount of effort into trying to maintain binary compatibility between releases, including Windows. It can be a significant constraint on drammatic innovation, but it does not mean that innovation is not taking place. Just because marketing decide to latch onto a feature and promote it, does not mean engineering are not innovating behind the scenes.
The linux community are unable to maintain binary compatibility in libc between 2.4.X releases, let alone any sort of stable kernel APIs.
New kernel sercurity patch? I'm sorry, you will have to recompile your kernel and all device drivers. Imagine if Microsoft required all the third party device drivers to be rebuilt with every kernel because the APIs _may_ have changed.
Its not a cluster!
Its not a cluster!
Its not a cluster!
Its a single system.
And I'm going to stop using the internet for the same reason...
I find it interesting that American airports have all these security checks now, some are polite and try to preserve individual's privacy, others like to "strip search" you in front of everyone else, yet when you catch a flight out of the country the level of security is pathetic.
Yet a 747 with >14 hours of fuel to fly to Australia has got to be more dangerous than any internal flight.
If they don't ship the 970-based power macs they are dead, the disapointment will destroy them after all the rumors. Since they haven't dispelled the rumors...
The NSAs distribution does _NOT_ meet "their standards" (Common Criteria), it does not provide auditing (or at least it didn't last time I looked) which is required for most protection profiles. It certainly has not been evaluated.
I have to disagree, I thought Figure 3 illustrated how important it is to baseline to ensure that you are heading in the right direction with each change you make (although they did not have a uni-processor baseline result).
It also showed that with the changes in June they are able to get a 4 times performance with another 7 cpus. Maybe next time they will show how it scales over the number of cpus you have.
Some of the issues we have addressed that have resulted in improvements include adding O(1) scheduler, SMP scalable timer, tunable priority preemption and soft affinity kernel patches.
Some scientific/mathematical benchmarks would also be good to see.
I was also suprised to see that they still use some of the old performance monitoring tools like looking at /proc, and other ascii tools, rather than something like PCP that collects all these statistics together so that you can look at any combination of subsystems on the same time line. Then they could have graphs showing the interraction and load on the disk, cpus, vm, network etc.
As much as I hate to support the US war effort, and wonder if we will ever know if the declaration was factual, if the US and UK are to be believed then a true declaration must be considered vaporware.
It is also worth noting that Microsoft have had Windows 2000 going through a C2 evaluation for over 18 months with a proper hardware configuration unlike the previous NT 4.0 evaluation.
No, as some other replies have indicated, the only reason for 64-bit applications is to access 64-bit addresses. If you don't need to address that amount of memory, then your applications might as well be 32-bit. Any good processor will still give you 64-bit registers etc to work with.
Anyone remember the day SGI sold their soul to M$ if an attempt to ensure the survival of OpenGL with Fahrenheit.
Several attempts were made during the 90's to develop UNIX standards so that independant software vendors (ISVs) could develop and support software for multiple UNIX platforms. To a large extent these failed.
Several attempts have been made by UNIX vendors to make it easier for ISVs to support their products on new releases of their O/Ss, like ensuring shared library compatibility in minor releases. Some vendors have had some success, although probably too late.
At the moment I don't see the Linux community addressing these issues which some would argue have contributed to the slow downfall in UNIX in the battle with M$ and each other.
There are many (although the number maybe decreasing) distributions and there are cracks
growing in the kernel group which is struggling to keep the kernel to a single source tree, once touted as the differentiator for Linux. This along with incompatibility between shared libraries in releases makes it very difficult and expensive to support a software (and even hardware) product under Linux.
Given that IBM were there with AIX and have been through this already, how do you see these issues being addressed, or do you believe they are not major issues?
Is IBM content to just distribute RedHat or some other distribution on their platforms, or are you planning to add your own value-added software on your platforms? If so, are you planning your own distribution?