Thanks for your informative posts. Sadly we can't afford a SAN anyway, though it might be a nice idea in future. What I don't understand is, how can a SAN improve the time for fdatasync() - i.e. for the data to be flushed to physical disk, and then control to return to the application? This is essential for database stuff.
As to my disinclination to trust battery backed cache - if the power goes out, it means we have about 4 hours to get it back. If that also fails, we have dataloss.
Could you be more specific about what actually gave the improvement? Was it just something simple, eg RAID 5 -> RAID6.
My main point though was that for supercomputer simulations (but not email or warehouse management), it's OK to risk data, and then just re-start the simulation from an hour ago. So why not just enable the disk write-caches or put the database on a RAMdisk? Without the safety requirements (such as no write-cache), the benefits of SSDs aren't needed.
BTW, I am very happy with the Intel SSDs for reliability - though just to be safe, we're using a RAID 1 array of two of them per server, and a RAIS array of two servers for the system (DRBD keeps all the RAID pairs in sync between primary and hot-spare server).
I've just gone through the process of setting up a pair of servers (HP DL380s) for Linux/Postgres. Our measurements show that the Intel X25-E SSDs beat regular 10k rpm SAS drives by a factor of about 12 for fdatasync() speed. This is important for a database system, as a transaction cannot COMMIT until the data has really, really hit permanent storage. [It's unsafe to use the regular disk's write cache, and personally, I don't trust a battery-backed write cache on the RAID controller much either. So not having to wait for a mechanical seek is really useful. Read speeds are also better (10x less latency), and the sustained throughput is about 2x as good.
So, yes, SSDs are a good idea for database loads, where the interaction is with the real world, and where once a transaction has completed, some other real-world process has happened. BUT, most supercomputer workloads are, in principle, re-startable (i.e. if you lose an hour's work due to a hardware failure, you can just re-run the simulation code, and throw away the intermediate state).
So, for simulations, the cost of dataloss is an hour of re-work, not irretrievable information. Given that, we can get much better performance by storing everything in RAM, enabling all the write-caches, and sticking with standard SATA, provided that, every so often, the data is flushed out to disk. If something goes wrong, just revert to the last savepoint, which could be an hour ago, rather than having to be 10ms ago.
[BTW, HP "don't support" SSDs in their servers, but the Intel SSD X25-E disks do work just fine. Though I did, unfortunately, have to buy some of HP's cheapest SAS drives ($250 each) just to obtain the mounting kits for the SSDs.]
I totally agree. The DHS should be eliminated, together with all it stands for.
As a British citizen, I won't travel to the USA again - I have fond memories of the times when visitors were welcomed, rather than fingerprinted like criminals. This "security" does have a cost to the USA - our family alone has taken at least 10 vacations elsewhere as a result of the fingerprinting and guantanamo policies. That means at least $100,000 that we have spent in Europe rather than the USA.
> Even when we finally see the year of Linux on the desktop, Linux on the laptop is still a long ways off.
Not so - it just depends on the machine. I recommend you get a few 2nd-hand X31/X32 thinkpads on eBay. They're small, light, do everything (except optical drive), and are made of titanium. They beat the current netbook generation, on both specs and price. Put Linux on them, as you *will* find yourself needing some of the open-source tools.
Sorry - was having a tired moment. I meant to point out that it's wide-band as opposed to narrow-band. The fact that in this case, the broadband signal happens to be in the special case of baseband isn't really relevant here. Incidentally, 100m is abut 0.5us (given that the relative permittivity for plastic is slightly less than 1)
But ethernet is a broadband signal with a huge range of frequencies. You might optimise out one class of standing wave, only to bring in another. Also, the 100m length restriction isn't really to do with loss increasing wrt frequency; it's a restriction on the time of flight of a pulse. If you're trying to prevent collisions, then the time-of-flight cannot be too long, otherwise the response speed of the switches at either end is limited. (USB has a 5m restriction for the same reason).
If we limited commercial copyright to 5-10 years, then it would hugely help new artists. By reducing the value of the back-catalogues, it would mean a strong incentive for publishers and music-labels to support new music.
The point you missed is that the 105W halogen-incandescent emits almost twice as much light as the "115-watt-equivalent" CF bulb.
CF bulbs are more energy-efficient, but not nearly as good as they claim to be. The claim used to be "6-7 times less energy". I'd venture that a factor of 2 is more like it.
Of course, part of the problem is that the CFL manufacturers over-promise and under-deliver. A "100 Watt equivalent" really isn't.
The best-of-breed incandescent is currently a Philips "-30%" halogen bulb called the "Eco Classic". This looks like a regular bulb, emits an exceptionally good light, and contains a halogen capsule internally. Power in: 105 Watts. Light out: 2100 lumen. Actually, Philips are under-promising: by comparison a good 100 W classic bulb gives out 1300 lumens.
The best of breed CFL that I've seen is 19 Watts for 1250 lumen. (This claims to be the equivalent of a 115 Watt bulb.) Derate this for the power-factor error, and you get 38 Watts.
That makes the CFL "better" in terms of energy consumption by a factor of 1.6 . On the other hand, you have to balance the fact that the light-quality is much worse - CFLs appear to emit "grey" light!
So we should stick to incandescent lights, be good about turning them off when we leave the room, and fix our energy crisis supply-side (which means Nuclear power).
>> That's all very well, but in most cases, the machine is a single-user desktop, and it's sitting *idle* during the 5-40 seconds in >> which the filesystem writes are being batched up. It seems to me that the writes should be sent to disk immediately, unless the disk >> is already busy, in which case they may be delayed for (at most) 40 seconds, while other reads take place ahead of them.
> Mount your filesystem with the "sync" option, that should do what you want I guess. Performance will be bad though. > There are only two ways to do this: either you do it completely synchronously, and get a guarantee of the write being done when the > application is done writing, or you have a delay of arbitrary length. If you have a delay, even if it's 1ms, and you care about the > possibility of something going wrong at that moment, the application has to deal with the possibility. Reducing the delay only makes > it less likely, but given enough time it'll happen.
Thanks for your reply - I stand enlightened about NCQ. What I still don't get is, why not begin a write immediately? Batching makes sense if the system is loaded, but what is the advantage in waiting for a long interval when nothing else is happening?
Also, if we are going to wait for long times, why not apply some kind of performance-training heuristic (like TCP does with retransmission intervals)?
That's all very well, but in most cases, the machine is a single-user desktop, and it's sitting *idle* during the 5-40 seconds in which the filesystem writes are being batched up. It seems to me that the writes should be sent to disk immediately, unless the disk is already busy, in which case they may be delayed for (at most) 40 seconds, while other reads take place ahead of them.
There is still noflushd, which allows the disk to spin down for up to 5 minutes - if you really want the power-saving at the expense of risking your data. Also, with a modern SATA disk supporting Native Command Queuing, the OS should immediately write the data to the disk's buffer, and the disk's firmware gets to decide about re-ordering.
As for the argument about using sqlite - why have yet another abstraction? After all, the filesystem is already a sort of database!
Surely a helicopter's design is very much like that of cryptography. The algorithm/source-code/blueprints/manufacturing designs can be widely known, provided the secret key isn't. So maybe the helicopter's security will even be enhanced by the "many-eyes" effect.
1. One of the nice features of a wiki is that it's very hard to create a new article without FIRST linking to it. So you cannot avoid indexing documents. That forces a sane structure.
2. Make your wiki the primary place to search. If you have other documents (pdf, word, etc), you can store them in the wiki itself, or somewhere totally different), but make sure the wiki contains at least a pointer.
3. With sensitive info (passwords, etc), I obviously don't put these in the wiki. However, the wiki says where they are kept.
May I congratulate the nameless civil servant who threw this spanner into the wrench. I'd like to hope that this particular "error" can be attributed to competence, rather than oversight.
When I do this (and for outlook), I delete the desktop icon, remove the taskbar shortcut, and move the main menu entry into Accessories. Anyone who *really* wants it can go Start->Run->iexplore. Some users have to be helped! You could possibly take a copy of the IE icon, and point it to firefox?
Also:
* It's still really, really, really ugly (far too much unbroken grey, widgets have no contrast)
* The replacement for kcontrol makes it harder, not easier to find settings, since the sidebar has gone
* konsole is much slower - I like some of the new features, but maximising a konsole window or swapping tab
now takes about 1 second, compared to "near instant".
* It totally violates the Unix principle of lots of small programs, each doing 1 thing.
* Whose dumb idea is it to have widgets run in the screensaver? Can you say "security hole" ?
The worst thing is that KDE3 bugs are now getting marked Wontfix, whereas KDE4 won't be usable for probably another year. In the meantime, all the distros are shipping KDE4 and dropping KDE3. I've been a fan of KDE since 2001, and I'll keep trying to like KDE4, but so far, I expect that when Mandriva drop KDE3, I shall end up with Gnome.
The fuse is there for another reason: the price of copper. After WWII, cooper was so expensive than we developed the ring main: a 32A circuit looping around each floor of the house, rather than having a spur direct to the fuse box for every socket. Since the 32A fuse in the fuse box couldn't protect appliances, the plug must have an internal fuse. But the UK plug is definitely the safest in the world. It's also impossible to insert a finger/pencil/fork/etc into the live pin of the socket by accident: the shutter only opens when the ground pin is inserted.
Yes, I had one of these. It's a real pain, because when something is broken, you expect to hear beep codes on the PC speaker. If the speakers are unplugged, as a result of disassembly before diagnostics, then no error messages are presented. The error would probably have been "[cp]U Fail", not "You fool" - though the latter interpretation isn't unreasonable in the context!
Indeed. I, and my family have voted with our feet and our wallets. We used to come from the UK to the USA for 2 holidays a year, but decided to go elsewhere (mainly Europe or Canada) as a result of the fingerprint requirement. When the USA decides it wants to be hospitable, we'll come back; as long as they want fingerprints, we'll stay away. [I should add that we have "nothing to fear"; we just value our privacy.]
As we are financially fortunate, I estimate that the change in our vacation habits alone has diverted about $100k from the US economy.
I use the nvidia driver quite a lot, but I find that its non-free nature tends to bite. Notably, because it isn't part of the kernel, it's too easy to get the nvidia driver and the kernel version out of sync. Then *bad* things happen to the users. As a result, I dare not do remote updates of computers that I support.
It wouldn't be quite so bad, excepting that the nv driver doesn't support twinview, and the vesa driver won't work with many widescreens. So, if I could buy a dual-head DVI card with an intel chip, I would. 3d acceleration isn't something I really need except for xscreensaver!
Slashdot Mirror
Thanks for your informative posts. Sadly we can't afford a SAN anyway, though it might be a nice idea in future. What I don't understand is, how can a SAN improve the time for fdatasync() - i.e. for the data to be flushed to physical disk, and then control to return to the application? This is essential for database stuff.
As to my disinclination to trust battery backed cache - if the power goes out, it means we have about 4 hours to get it back. If that also fails, we have dataloss.
Could you be more specific about what actually gave the improvement? Was it just something simple, eg RAID 5 -> RAID6.
My main point though was that for supercomputer simulations (but not email or warehouse management), it's OK to risk data, and then just re-start the simulation from an hour ago. So why not just enable the disk write-caches or put the database on a RAMdisk? Without the safety requirements (such as no write-cache), the benefits of SSDs aren't needed.
BTW, I am very happy with the Intel SSDs for reliability - though just to be safe, we're using a RAID 1 array of two of them per server, and a RAIS array of two servers for the system (DRBD keeps all the RAID pairs in sync between primary and hot-spare server).
I've just gone through the process of setting up a pair of servers (HP DL380s) for Linux/Postgres. Our measurements show that the Intel X25-E SSDs beat regular 10k rpm SAS drives by a factor of about 12 for fdatasync() speed. This is important for a database system, as a transaction cannot COMMIT until the data has really, really hit permanent storage. [It's unsafe to use the regular disk's write cache, and personally, I don't trust a battery-backed write cache on the RAID controller much either. So not having to wait for a mechanical seek is really useful. Read speeds are also better (10x less latency), and the sustained throughput is about 2x as good.
So, yes, SSDs are a good idea for database loads, where the interaction is with the real world, and where once a transaction has completed, some other real-world process has happened. BUT, most supercomputer workloads are, in principle, re-startable (i.e. if you lose an hour's work due to a hardware failure, you can just re-run the simulation code, and throw away the intermediate state).
So, for simulations, the cost of dataloss is an hour of re-work, not irretrievable information. Given that, we can get much better performance by storing everything in RAM, enabling all the write-caches, and sticking with standard SATA, provided that, every so often, the data is flushed out to disk. If something goes wrong, just revert to the last savepoint, which could be an hour ago, rather than having to be 10ms ago.
[BTW, HP "don't support" SSDs in their servers, but the Intel SSD X25-E disks do work just fine. Though I did, unfortunately, have to buy some of HP's cheapest SAS drives ($250 each) just to obtain the mounting kits for the SSDs.]
I totally agree. The DHS should be eliminated, together with all it stands for.
As a British citizen, I won't travel to the USA again - I have fond memories of the times when visitors were welcomed, rather than fingerprinted like criminals. This "security" does have a cost to the USA - our family alone has taken at least 10 vacations elsewhere as a result of the fingerprinting and guantanamo policies. That means at least $100,000 that we have spent in Europe rather than the USA.
> Even when we finally see the year of Linux on the desktop, Linux on the laptop is still a long ways off.
Not so - it just depends on the machine. I recommend you get a few 2nd-hand X31/X32 thinkpads on eBay. They're small, light, do everything (except optical drive), and are made of titanium. They beat the current netbook generation, on both specs and price. Put Linux on them, as you *will* find yourself needing some of the open-source tools.
Sorry - was having a tired moment. I meant to point out that it's wide-band as opposed to narrow-band. The fact that in this case, the broadband signal happens to be in the special case of baseband isn't really relevant here. Incidentally, 100m is abut 0.5us (given that the relative permittivity for plastic is slightly less than 1)
But ethernet is a broadband signal with a huge range of frequencies. You might optimise out one class of standing wave, only to bring in another. Also, the 100m length restriction isn't really to do with loss increasing wrt frequency; it's a restriction on the time of flight of a pulse. If you're trying to prevent collisions, then the time-of-flight cannot be too long, otherwise the response speed of the switches at either end is limited. (USB has a 5m restriction for the same reason).
If we limited commercial copyright to 5-10 years, then it would hugely help new artists. By reducing the value of the back-catalogues, it would mean a strong incentive for publishers and music-labels to support new music.
The point you missed is that the 105W halogen-incandescent emits almost twice as much light as the "115-watt-equivalent" CF bulb.
CF bulbs are more energy-efficient, but not nearly as good as they claim to be. The claim used to be "6-7 times less energy". I'd venture that a factor of 2 is more like it.
Of course, part of the problem is that the CFL manufacturers over-promise and under-deliver. A "100 Watt equivalent" really isn't.
The best-of-breed incandescent is currently a Philips "-30%" halogen bulb called the "Eco Classic".
This looks like a regular bulb, emits an exceptionally good light, and contains a halogen capsule internally.
Power in: 105 Watts. Light out: 2100 lumen.
Actually, Philips are under-promising: by comparison a good 100 W classic bulb gives out 1300 lumens.
The best of breed CFL that I've seen is 19 Watts for 1250 lumen. (This claims to be the equivalent of a 115 Watt bulb.)
Derate this for the power-factor error, and you get 38 Watts.
That makes the CFL "better" in terms of energy consumption by a factor of 1.6 .
On the other hand, you have to balance the fact that the light-quality is much worse - CFLs appear to emit "grey" light!
So we should stick to incandescent lights, be good about turning them off when we leave the room, and fix our energy crisis supply-side
(which means Nuclear power).
>> That's all very well, but in most cases, the machine is a single-user desktop, and it's sitting *idle* during the 5-40 seconds in
>> which the filesystem writes are being batched up. It seems to me that the writes should be sent to disk immediately, unless the disk
>> is already busy, in which case they may be delayed for (at most) 40 seconds, while other reads take place ahead of them.
> Mount your filesystem with the "sync" option, that should do what you want I guess. Performance will be bad though.
> There are only two ways to do this: either you do it completely synchronously, and get a guarantee of the write being done when the
> application is done writing, or you have a delay of arbitrary length. If you have a delay, even if it's 1ms, and you care about the
> possibility of something going wrong at that moment, the application has to deal with the possibility. Reducing the delay only makes
> it less likely, but given enough time it'll happen.
Thanks for your reply - I stand enlightened about NCQ. What I still don't get is, why not begin a write immediately? Batching makes sense if the system is loaded, but what is the advantage in waiting for a long interval when nothing else is happening?
Also, if we are going to wait for long times, why not apply some kind of performance-training heuristic (like TCP does with retransmission intervals)?
That's all very well, but in most cases, the machine is a single-user desktop, and it's sitting *idle* during the 5-40 seconds in which the filesystem writes are being batched up. It seems to me that the writes should be sent to disk immediately, unless the disk is already busy, in which case they may be delayed for (at most) 40 seconds, while other reads take place ahead of them.
There is still noflushd, which allows the disk to spin down for up to 5 minutes - if you really want the power-saving at the expense of risking your data. Also, with a modern SATA disk supporting Native Command Queuing, the OS should immediately write the data to the disk's buffer, and the disk's firmware gets to decide about re-ordering.
As for the argument about using sqlite - why have yet another abstraction? After all, the filesystem is already a sort of database!
Surely a helicopter's design is very much like that of cryptography. The algorithm/source-code/blueprints/manufacturing designs can be widely known, provided the secret key isn't. So maybe the helicopter's security will even be enhanced by the "many-eyes" effect.
A few extra thoughts, from experience.
1. One of the nice features of a wiki is that it's very hard to create a new article without FIRST linking to it. So you cannot avoid indexing documents. That forces a sane structure.
2. Make your wiki the primary place to search. If you have other documents (pdf, word, etc), you can store them in the wiki itself, or somewhere totally different), but make sure the wiki contains at least a pointer.
3. With sensitive info (passwords, etc), I obviously don't put these in the wiki. However, the wiki says where they are kept.
D'Oh. I must have had one of those "put the coffee in the fridge and the milk in the cupboard" moments when I wrote that.
I am British - but I do hope that just occasionally one of our civil servants might put country before government.
May I congratulate the nameless civil servant who threw this spanner into the wrench. I'd like to hope that this particular "error" can be attributed to competence, rather than oversight.
When I do this (and for outlook), I delete the desktop icon, remove the taskbar shortcut, and move the main menu entry into Accessories.
Anyone who *really* wants it can go Start->Run->iexplore. Some users have to be helped!
You could possibly take a copy of the IE icon, and point it to firefox?
Also:
* It's still really, really, really ugly (far too much unbroken grey, widgets have no contrast)
* The replacement for kcontrol makes it harder, not easier to find settings, since the sidebar has gone
* konsole is much slower - I like some of the new features, but maximising a konsole window or swapping tab
now takes about 1 second, compared to "near instant".
* It totally violates the Unix principle of lots of small programs, each doing 1 thing.
* Whose dumb idea is it to have widgets run in the screensaver? Can you say "security hole" ?
The worst thing is that KDE3 bugs are now getting marked Wontfix, whereas KDE4 won't be usable for probably another year. In the meantime, all the distros are shipping KDE4 and dropping KDE3. I've been a fan of KDE since 2001, and I'll keep trying to like KDE4, but so far, I expect that when Mandriva drop KDE3, I shall end up with Gnome.
The fuse is there for another reason: the price of copper. After WWII, cooper was so expensive than we developed the ring main: a 32A circuit looping around each floor of the house, rather than having a spur direct to the fuse box for every socket. Since the 32A fuse in the fuse box couldn't protect appliances, the plug must have an internal fuse. But the UK plug is definitely the safest in the world. It's also impossible to insert a finger/pencil/fork/etc into the live pin of the socket by accident: the shutter only opens when the ground pin is inserted.
Yes, I had one of these. It's a real pain, because when something is broken, you expect to hear beep codes on the PC speaker. If the speakers are unplugged, as a result of disassembly before diagnostics, then no error messages are presented. The error would probably have been "[cp]U Fail", not "You fool" - though the latter interpretation isn't unreasonable in the context!
Indeed. I, and my family have voted with our feet and our wallets. We used to come from the UK to the USA for 2 holidays a year, but decided to go elsewhere (mainly Europe or Canada) as a result of the fingerprint requirement. When the USA decides it wants to be hospitable, we'll come back; as long as they want fingerprints, we'll stay away. [I should add that we have "nothing to fear"; we just value our privacy.]
As we are financially fortunate, I estimate that the change in our vacation habits alone has diverted about $100k from the US economy.
You could always start with GNU Hello World:
http://ftp.gnu.org/gnu/hello/
I use the nvidia driver quite a lot, but I find that its non-free nature tends to bite. Notably, because it isn't part of the kernel, it's too easy to get the nvidia driver and the kernel version out of sync. Then *bad* things happen to the users. As a result, I dare not do remote updates of computers that I support.
It wouldn't be quite so bad, excepting that the nv driver doesn't support twinview, and the vesa driver won't work with many widescreens.
So, if I could buy a dual-head DVI card with an intel chip, I would. 3d acceleration isn't something I really need except for xscreensaver!
Thank you - that looks really useful. I'll give it a go...