iSCSI adds a lot of protocol overhead, and tuning it to work well with a given application and network load becomes quite difficult above gigabit speed. When you're using a fairly reliable transport, such as FC or Ethernet with pause frames, you can dispense with that, and get near-optimal throughput and latency with very little tuning.
The bandwidth is there. I can get 960 Mb/s sustained application-layer throughput out of a gigabit ethernet connection. When you have pause frame support and managed layer 3 switches, you can strip away the protocol overhead of iSCSI, and keep the reliability and flexibility in a typical data center.
The goal of this project is not to replace fibre channel fabrics, but rather to extend them. For every large database server at your High End customer, there are dozens of smaller boxes that would greatly benefit from centralized disk storage, but for which the cost of conventional FC would negate the benefit. As you've noted, iSCSI isn't always a suitable option.
You're probably right that people won't use this a whole lot to connect to super-high-end disk arrays, but once you hook up an FCoE bridge to your network, you have the flexibility to do whatever you want with it. In some cases, the cost benefit of 10Gb ethernet vs. 2x 4Gb FC alone will be enough motivation to use it even for very high-end work.
Why not? Switched fabric topology has no inherent latency benefit over star topology, and the majority of servers in a data center aren't doing anything that need any more sophisticated throughput aggregation than 802.3ad (LACP) bonding will give you. As long as you have pause frame support (a prerequisite for this FCoE implementation) you can create a lossless ethernet network, which eliminates the need for much of the protocol overhead of something like iSCSI, as long as you're staying on the LAN.
FCoE seems to fill a niche somewhere between conventional Fibre Channel and iSCSI. Intel is betting that it's a fairly large niche, and will drive a lot of 10 Gb ethernet sales, and I think they're right.
Re:They might have some scalability issues
on
NYSE Moves to Linux
·
· Score: 1
A lot has changed since the 2.6.5 days. SGI is now running Linux on Itanium systems with up to 4096 CPUs (and high NUMA factors) under a single kernel image.
And that's the point, really. Linux development proceeds far more rapidly than anything in the proprietary world. For a long time, that meant Linux was rapidly catching up to proprietary Unix. Now it means Linux is leaping ahead.
I don't think we can really legislate ethics, but we could at least legislate disclosure. If you're selling subscriptions, I don't think it's an unfair burden to disclose at purchase time whether or not your writers have editorial independence. Extending that to anyone posting their opinion on a forum somewhere would be a tad overboard though.
The actual error isn't terribly worrying, but the process failure that led to the breach of their contract, especially for something that could have been complied with quite easily, is not the sort of thing you want to see going on at a company that makes closed source voting machines.
I've had it. I'm completely sick of hearing about companies throwing good money away trying to build "cool" software that has no business model other than "become popular and then sell ads", and at the end of the day really accomplishes nothing more than marginally improving geeks' chances of getting laid. Have we learned nothing from the dotcom era? These companies are blinded by the promise of harnessing the hype rampant on sites like slashdot. Yes, they're idiots, but we're enabling them.
Google is absolutely expert at converting hype to ad revenue, but that's because they know when to stop throwing good money after bad and cut their losses when the hype fades. It may result in some cool apps, but don't think for a second that OpenSocial will "change everything", unless of course by "change everything" you really mean "make it easier to meet women without leaving the house".
I argue that virtualization provisioning improves security by improving homogeneity in the data center, reducing the potential for human error. Human error is a far greater security risk than architectural defects.
[disclosure text="I work for a company that sells virtualization."]
Theo's expertise, and indeed that of the entire OpenBSD project, is in the realm of provably correct security. Virtualization adds yet another layer where something can go wrong. Sure there are and will be bugs. We're finding them and fixing them, just as we've always done. From an absolute security standpoint, Theo's right.
Of course, most businesses couldn't care less. Businesses don't view security as an absolute thing, because human factors make it generally impossible. Businesses view security as a risk, with associated probabilities and costs, worst-case scenarios, likely scenarios, mitigation strategies, and ultimately, diminishing marginal returns. For businesses using virtualization to consolidate systems, it generally reduces risk because it makes it easier to implement policies that mitigate human factors.
To be precise, virtualization *technology* decreases security, but virtualization *solutions* increase security, at least when done well, which is much more practical than the technical absolute of "done right".
Good plan on your end, but I pity the administrator who just got paged because this broke their kerberos domain and has to spend the whole weekend converting everything to GMT.
Several security protocols mandate close time synchronization to minimize the risk of replay attacks, so failure to deploy this time zone change causes a denial of service. In particular Kerberos is impacted, and increasing the permissible time skew by a few orders of magnitude on every box in the domain, which not all implementations support, creates a substantial risk unless you're set up for ticket pre-authentication, which puts a greater load on the server, is not well supported by all clients, and is thus often not enabled. Admittedly, if you're using a network of Debian stable machines, you should be okay, but god forbid someone should use a Debian stable box in an Active Directory deployment.
Similar problems may exist for SSL (https, ldaps, imaps anyone?) but I'm not sure if a one hour difference would exceed the tolerance in many applications.
I forgot to mention, s/390 mainframes already do this, and they do it online, without any data loss. The chips have two cores that do the same work in parallel. If they disagree on the result, they back up and try again. If they disagree again, they mark the chip bad and move it over to a hot spare. Of course, they're not exactly targeted at the desktop market.
The Barcelona/Phenom architecture allows each core (plus the northbridge) to run on its own power plane, and for cores to be turned off completely. Of course, core 0 is the bootstrap processor, so that core has to always be enabled, or they have to have a way to change which one is core 0 before it leaves the factory. Otherwise the BIOS won't be able to bring the other cores online.
The idea of post-factory error detection isn't so far-fetched. If a chip passes QA, the sorts of defects you'll see later in its life are likely to be thermally induced, and the likelihood that the defect will manifest prior to loading of the BIOS is very low. You're not using the MMU or the FPU at all, you're not using much of the cache, you can be running at your minimum power setting, and you're not doing it long enough to heat up much. If a core gets marked bad due to an excess of MCEs, similar to how many systems can mark DIMMs bad on excessive multi-bit ECC errors, the BIOS simply doesn't need to bring it online at boot time. Even if core 0 is the faulty one, you can probably load just enough of the BIOS to bring a good core online and finish booting, since you're not straining it enough to cause thermal problems, and you're only using a tiny fraction of the instruction set and die transistors. This sort of High Availability feature probably won't make it to the desktop right away, but as core counts keep increasing, it's inevitable.
I don't really care about the security angle either way. Most of the time I use nmap, it's for debugging on test systems that are behind several layers of firewall and NAT. Yeah, it's a debugging tool too.
Then again, in the age of DRM, all debuggers are apparently hacking tools.
a) NCQ really is broken on a whole bunch of drives.
b) You have 4 GB RAM, 12 GB swap, 10 GB swap free, and 1 GB swap cached, with less than 250 MB buffers + cached, and you're setting vm.swappiness=20. You might mitigate the problem by raising vm.swappiness to a higher value, but you really need to close some firefox tabs or buy more RAM. No amount of I/O scheduling work will fix your performance.
Live kernel updating would mean you can never change data structures, and if you had a bug that resulted in incorrect or incomplete data in any of those structures, you still wouldn't be able to fix it without a reboot, unless you have a system with infinite WORM memory that can solve the halting problem and doesn't receive any input from the outside world.
In other words, it would be useless for bug fixes, and only feasible for bolting on entirely new functionality. We can already do that with loadable modules.
Yes, but how much less heat? DC rectifiers waste heat too. 3-phase power supplies on blade chassis are considerably more efficient than typical 1-phase pizza box power supplies. The blade system has an efficient entry cost an order of magnitude more than the pizza box, and 3 orders of magnitude less than the DC data center, but is closer to the DC data center in AC conversion efficiency. This makes it a very good compromise for the vast majority of enterprises that do not buy their data centers by the acre.
Don't get me wrong, I think DC is great, but until the market grows a lot more, it won't really be cost-efficient for the masses.
-48VDC is standard for some kinds of telecom equipment, so there's plenty of server gear that will work in -48VDC data centers, and it's very efficient. Unfortunately, using this niche gear requires very large economies of scale, on the order of tens of millions of dollars to be cost-effective.
For mere mortals, blade servers are a better compromise. When you have 4 power supplies per 10 servers, instead of 20, you can afford to invest in more efficient equipment. It's still not as efficient as the rectifiers used in large telecom data centers, but it's a big improvement, and it takes 1000x to be cost-effective.
In the embedded world, most of the interesting hardware you're controlling is attached through serial or serial-like interfaces. That's precisely why this patchset is worthwhile.
Slashdot Mirror
iSCSI adds a lot of protocol overhead, and tuning it to work well with a given application and network load becomes quite difficult above gigabit speed. When you're using a fairly reliable transport, such as FC or Ethernet with pause frames, you can dispense with that, and get near-optimal throughput and latency with very little tuning.
Bullshit.
The bandwidth is there. I can get 960 Mb/s sustained application-layer throughput out of a gigabit ethernet connection. When you have pause frame support and managed layer 3 switches, you can strip away the protocol overhead of iSCSI, and keep the reliability and flexibility in a typical data center.
The goal of this project is not to replace fibre channel fabrics, but rather to extend them. For every large database server at your High End customer, there are dozens of smaller boxes that would greatly benefit from centralized disk storage, but for which the cost of conventional FC would negate the benefit. As you've noted, iSCSI isn't always a suitable option.
You're probably right that people won't use this a whole lot to connect to super-high-end disk arrays, but once you hook up an FCoE bridge to your network, you have the flexibility to do whatever you want with it. In some cases, the cost benefit of 10Gb ethernet vs. 2x 4Gb FC alone will be enough motivation to use it even for very high-end work.
Why not? Switched fabric topology has no inherent latency benefit over star topology, and the majority of servers in a data center aren't doing anything that need any more sophisticated throughput aggregation than 802.3ad (LACP) bonding will give you. As long as you have pause frame support (a prerequisite for this FCoE implementation) you can create a lossless ethernet network, which eliminates the need for much of the protocol overhead of something like iSCSI, as long as you're staying on the LAN.
FCoE seems to fill a niche somewhere between conventional Fibre Channel and iSCSI. Intel is betting that it's a fairly large niche, and will drive a lot of 10 Gb ethernet sales, and I think they're right.
A lot has changed since the 2.6.5 days. SGI is now running Linux on Itanium systems with up to 4096 CPUs (and high NUMA factors) under a single kernel image.
And that's the point, really. Linux development proceeds far more rapidly than anything in the proprietary world. For a long time, that meant Linux was rapidly catching up to proprietary Unix. Now it means Linux is leaping ahead.
I've had a financial customer quote the figure of $20 million/hour. TCP stalls cost them money.
At least in the graphics world, "faster and usually correct" is acceptable.
http://www.angryfacts.com/
I don't think we can really legislate ethics, but we could at least legislate disclosure. If you're selling subscriptions, I don't think it's an unfair burden to disclose at purchase time whether or not your writers have editorial independence. Extending that to anyone posting their opinion on a forum somewhere would be a tad overboard though.
a) They didn't think it was that big of a deal.
b) They forgot.
The actual error isn't terribly worrying, but the process failure that led to the breach of their contract, especially for something that could have been complied with quite easily, is not the sort of thing you want to see going on at a company that makes closed source voting machines.
I've had it. I'm completely sick of hearing about companies throwing good money away trying to build "cool" software that has no business model other than "become popular and then sell ads", and at the end of the day really accomplishes nothing more than marginally improving geeks' chances of getting laid. Have we learned nothing from the dotcom era? These companies are blinded by the promise of harnessing the hype rampant on sites like slashdot. Yes, they're idiots, but we're enabling them.
Google is absolutely expert at converting hype to ad revenue, but that's because they know when to stop throwing good money after bad and cut their losses when the hype fades. It may result in some cool apps, but don't think for a second that OpenSocial will "change everything", unless of course by "change everything" you really mean "make it easier to meet women without leaving the house".
I argue that virtualization provisioning improves security by improving homogeneity in the data center, reducing the potential for human error. Human error is a far greater security risk than architectural defects.
[disclosure text="I work for a company that sells virtualization."]
Theo's expertise, and indeed that of the entire OpenBSD project, is in the realm of provably correct security. Virtualization adds yet another layer where something can go wrong. Sure there are and will be bugs. We're finding them and fixing them, just as we've always done. From an absolute security standpoint, Theo's right.
Of course, most businesses couldn't care less. Businesses don't view security as an absolute thing, because human factors make it generally impossible. Businesses view security as a risk, with associated probabilities and costs, worst-case scenarios, likely scenarios, mitigation strategies, and ultimately, diminishing marginal returns. For businesses using virtualization to consolidate systems, it generally reduces risk because it makes it easier to implement policies that mitigate human factors.
To be precise, virtualization *technology* decreases security, but virtualization *solutions* increase security, at least when done well, which is much more practical than the technical absolute of "done right".
[/disclosure]
Good plan on your end, but I pity the administrator who just got paged because this broke their kerberos domain and has to spend the whole weekend converting everything to GMT.
Several security protocols mandate close time synchronization to minimize the risk of replay attacks, so failure to deploy this time zone change causes a denial of service. In particular Kerberos is impacted, and increasing the permissible time skew by a few orders of magnitude on every box in the domain, which not all implementations support, creates a substantial risk unless you're set up for ticket pre-authentication, which puts a greater load on the server, is not well supported by all clients, and is thus often not enabled. Admittedly, if you're using a network of Debian stable machines, you should be okay, but god forbid someone should use a Debian stable box in an Active Directory deployment.
Similar problems may exist for SSL (https, ldaps, imaps anyone?) but I'm not sure if a one hour difference would exceed the tolerance in many applications.
Disclaimer: I work for a commercial distributor.
Once adoption of version 3 slows, they'll just release version 3.5 and we'll all have to buy new books anyway.
"Acceptable forms of payment are American Express, Discover, MasterCard and Visa."
I forgot to mention, s/390 mainframes already do this, and they do it online, without any data loss. The chips have two cores that do the same work in parallel. If they disagree on the result, they back up and try again. If they disagree again, they mark the chip bad and move it over to a hot spare. Of course, they're not exactly targeted at the desktop market.
The Barcelona/Phenom architecture allows each core (plus the northbridge) to run on its own power plane, and for cores to be turned off completely. Of course, core 0 is the bootstrap processor, so that core has to always be enabled, or they have to have a way to change which one is core 0 before it leaves the factory. Otherwise the BIOS won't be able to bring the other cores online.
The idea of post-factory error detection isn't so far-fetched. If a chip passes QA, the sorts of defects you'll see later in its life are likely to be thermally induced, and the likelihood that the defect will manifest prior to loading of the BIOS is very low. You're not using the MMU or the FPU at all, you're not using much of the cache, you can be running at your minimum power setting, and you're not doing it long enough to heat up much. If a core gets marked bad due to an excess of MCEs, similar to how many systems can mark DIMMs bad on excessive multi-bit ECC errors, the BIOS simply doesn't need to bring it online at boot time. Even if core 0 is the faulty one, you can probably load just enough of the BIOS to bring a good core online and finish booting, since you're not straining it enough to cause thermal problems, and you're only using a tiny fraction of the instruction set and die transistors. This sort of High Availability feature probably won't make it to the desktop right away, but as core counts keep increasing, it's inevitable.
...you should see what happens to the case when you install Windows ME on it!
I don't really care about the security angle either way. Most of the time I use nmap, it's for debugging on test systems that are behind several layers of firewall and NAT. Yeah, it's a debugging tool too.
Then again, in the age of DRM, all debuggers are apparently hacking tools.
a) NCQ really is broken on a whole bunch of drives.
b) You have 4 GB RAM, 12 GB swap, 10 GB swap free, and 1 GB swap cached, with less than 250 MB buffers + cached, and you're setting vm.swappiness=20. You might mitigate the problem by raising vm.swappiness to a higher value, but you really need to close some firefox tabs or buy more RAM. No amount of I/O scheduling work will fix your performance.
Live kernel updating would mean you can never change data structures, and if you had a bug that resulted in incorrect or incomplete data in any of those structures, you still wouldn't be able to fix it without a reboot, unless you have a system with infinite WORM memory that can solve the halting problem and doesn't receive any input from the outside world.
In other words, it would be useless for bug fixes, and only feasible for bolting on entirely new functionality. We can already do that with loadable modules.
Yes, but how much less heat? DC rectifiers waste heat too. 3-phase power supplies on blade chassis are considerably more efficient than typical 1-phase pizza box power supplies. The blade system has an efficient entry cost an order of magnitude more than the pizza box, and 3 orders of magnitude less than the DC data center, but is closer to the DC data center in AC conversion efficiency. This makes it a very good compromise for the vast majority of enterprises that do not buy their data centers by the acre.
Don't get me wrong, I think DC is great, but until the market grows a lot more, it won't really be cost-efficient for the masses.
-48VDC is standard for some kinds of telecom equipment, so there's plenty of server gear that will work in -48VDC data centers, and it's very efficient. Unfortunately, using this niche gear requires very large economies of scale, on the order of tens of millions of dollars to be cost-effective.
For mere mortals, blade servers are a better compromise. When you have 4 power supplies per 10 servers, instead of 20, you can afford to invest in more efficient equipment. It's still not as efficient as the rectifiers used in large telecom data centers, but it's a big improvement, and it takes 1000x to be cost-effective.
In the embedded world, most of the interesting hardware you're controlling is attached through serial or serial-like interfaces. That's precisely why this patchset is worthwhile.