Even if you set up your htdocs on a file share with a webserver you're talking largely about static files. Unless you seriously screwed up that would all be served out of buffer-cache so it's highly unlikely the share would ever become the bottleneck.
The spreadsheet model did predict severe damage (and in at least one scenario total penetration). They ignored it. Spreadsheets don't kill people. Bad decisions do.
My intent wasn't to provide a thorough cost breakdown of the technology. That's impossible to do without taking individual site requirements and existing infrastructure and capacity into account.
My point was that quoting a wholesale drive cost was far from an accurate represent and that the cost of tape is often highly overestimated.
Is it right for every client? Of course not. The buy-in cost for the drive is far too high if a client has a relatively small data set. It can also be a poor fit for clients who consider nearline access of the data a priority, though in those cases a hybrid approach including snapshots is often the right answer.
On the other hand, the larger the data set the more attractive the lower media cost and higher density becomes. Likewise customers who have a need for deep backup cycles or true archiving (say for auditing puprosed).
In other words - don't trust people who try to sell you on the One True Way.
Fine. Get the cartridges, but what about the capital cost minus depreciation of the drive? What about random access?
Random access is why snapshots also have their place.:) Archival backups and nearline backups solve different sets of problems.
Now weigh those against an inexpensive jbod frame with a 2gb FC backplane.
What kind of capacity are we talking. For a small site you can pick up a little 2U unit that'll store 6.4TB uncompressed for under $5k. Or if you're running a larger site you can snag a 4U unit with two drives for about $15k that'll handle 30.4TB with optional expansion to 60.8TB native.
What's the write speed of LT vs a tasty little GB SAS drive?
120MB/sec per drive without compression. And now that you've talking about SAS drives your per TB cost is hopelessly optimistic. Even OEM packaged terabyte SAS drives are going to run you about a quarter a gigabyte, which is now four times the media cost of an LTO-4 solution.
Rackspace? You can put a dozen into about 4U.
So about 12TB in 4U compared to the 30TB unit I mention above.
Cooling? Although I'll grant you green cost, the random accessibility out-classes the seek time and tape insertion by a human cost dramatically.
Have you never heard of a tape library?
Stable media? Tape? Sometimes.
Properly handled tape is incredibly stable.
Shelf space?
If you're doing off-site storage, that's going to be an issue regardless of what media you're using. And as I pointed out, tape is far more compact and far lighter than disks.
No need to use tape anymore. Get out of the reality distortion field, but do the right thing by testing what you have and doing drills to ensure that whatever you have, works and is a procedure understood by all.
I'm not the one dismissing an entire class of technology while demonstrating ignorance of its costs and benefits.
Even accepting your price that's a cost of about 12.7 cents per gigabyte and you can get 800GB native LTO-4 tapes for about $50, which comes out to about 6.3 cents per gigabyte.
But quoting costs for desktop grade SATA drives severely understates the true cost. For any non-trivial site installation you're talking near-line rated drives, drive caddies, storage shelves and additional SAN fabric. Then price out the additional power, cooling and rack space. Then price offsite shipping and storage for the bulkier, heavier and more delicate disk option.
Mirroring has its place. Snapshotting has its place. And backups to stable media still has its place too.
While I've been a cable and internet customer since they moved into our area three or four years ago. No extended outages*, no equipment problems, no service problems, no billing issues. The only time a specific channel was unavailable was while they had the scuffle with LIN broadcasting over licensing fees.
The service hasn't been perfect of course but the biggest complaint I can come up with is that the free video on demand channels used to be a bit spotty, but I can't remember the last time I had that problem. And I was a bit more forgiving while they were working on upgrading (or outright replacing) Adelphia's antiquated equipment.
(* The one exception was in the wake of a major ice storm in 2006, but my neighbourhood was also without power or phone service for the better part of a week.)
Oh, sure, the "cheap pr**cks* can pay an extra quarter here, an extra dime there, another nickel somewhere else and suddenly my bill is ten or twenty bucks more a month.
Solid tactile response, sturdy as fuck, easy to clean. The loud clicking is a bonus feature. Especially when you want to continue typing when you turn to talk to someone.:)
(Seriously? Thin keys? Yuck. Different (key)strokes for different folks, I guess.)
Right, because programmers are expensive but administrators, managers, rack space, electricity and cooling are all free. Some level of optimization is always worth it for any but the most trivial applications. If you're doing an application that's realistically only going to service a relatively fixed amount of traffic and you're talking the difference between a $1000 server and a $2000 server (maybe an application server deployed in branch offices or an MTA for a small business) - sure, go ahead and go with slow and unoptimized. Any serious enterprise-wide or internet facing application and you're just asking for trouble if you don't keep performance in mind from the get go and at least identify the low-hanging fruit.
OPEC failed to demonstrate the ability to set oil prices even when they controlled a far larger percentage of world production.
Look at the last time oil prices spiked the way they did in the past two years. It took thirty years before prices reach the same (infaltion adjusted) peak. And that was with relatively healthy economic growth during the intervening years.
I'm sure prices will bottom out before long, but they're not going to jump back a hundred dollars a barrel any time soon.
Eh? You don't get a fsck on mount by default with ext3 unless you've exceeded either the check interval or max mount count. If the filesystem has the needs_recovery flag set (i.e. wasn't cleanly unmounted) the journal will be replayed.
Lengthy filesystem checks on large filesystems holding large filesystems are only a problem if you don't create the filesystem properly in the first place. The time to do a fsck is pretty much linear to the number of inodes; specify a reasonable bytes-per-inode ratio and fsck time drops dramatically. Even this is unnecessary under ext4 since they no longer check unused inode groups.
If you RTFA they give brief summary of the additional features of ext4: "Later, others (Lustre, IBM, Bull - see Theodore T'so comment) got involved and added extents, delayed allocation, online defragmentation, and more." and the development wiki goes into further detail.
System administrators who want to something small to carry when they're on call? Students who want something small to take notes on? Parents who want something light and relatively cheap for their kids?
I don't know that I would call it fairly simple myself. The basic picture of what software does what isn't too bad, but some of the configuration is pretty obtuse under the hood, the LDAP schema and SOAP interface are fairly large, and the internal mailbox format is a bit wonky. Anything and everything for a particular user (including folder data, contacts, appointments, mail messages, tags, conversation data) is shoved into a single table (mail_item) with some of the data massaged into columns which marginally make sense and the rest shoved into a bencoded data structure with terse field labels (e.g. 'u', 'v', 'nc', 'st').
That said, yeah, if the concern is the technical ability to keep running even if the parent company abandons the product (or jacks the price unreasonably) they're marginally safer. Lock-in isn't really any worse though.
Eh? There's no courier in there. The mailbox portion is about as nonstandard as something they wrote themselves could be.;) Beyond that the rest of the architecture is actually pretty easy to replace. I'd still go for a commercial license for this sort of license. Too many features culled from the community edition (including their backup/restore, mailbox migration tools, clustering scripts, etc... little that can't be worked around, but it's a pain in the ass if you're not pretty intimately familiar with the innards.)
There's no exim. It's postfix. And the indexing is Apache Lucene, nothing they created.
On the other hand the mailbox format is proprietary (combination of on-disk blobs and database backed metadata) so it really isn't much safer in terms of lock-in. Migrating to another platform will involve exporting to some interim format or doing a protocol sync. User preferences are maybe somewhat more accessible, since they're stored largely as LDAP attributes, but the schema is proprietary as well.
I've never used Cyrus personally, but in the e-mail world that's a fairly small mail cluster. We have a legacy system built with open source components running 50,000 active users hosted on three boxes, and we could drop that to two if we were willing to sacrifice redundancy. Granted the majority of accounts are relatively small quota and the user base is shrinking, but when it was our primary mail system we had hundreds of thousands.
On our current system we push about 50,000 users (with unlimited quota) per backend server, with the main limiter being storage space. The SMTP servers on the front-end (which also handle spam filtering) handle enough traffic for a few hundred thousand users each (pooled, not dedicated to any particular machine).
That isn't to say outsourcing is a bad idea. It's less about the software (many of us rely heavily on the same open source packages freely available) than the expertise to use it effectively.
Slashdot Mirror
A second drive doesn't necessarily mean mirroring.
Erm, Slashdot ate my link. That'll learn me not to pay attention to the preview.
All trace of who disappeared?
Even if you set up your htdocs on a file share with a webserver you're talking largely about static files. Unless you seriously screwed up that would all be served out of buffer-cache so it's highly unlikely the share would ever become the bottleneck.
The spreadsheet model did predict severe damage (and in at least one scenario total penetration). They ignored it. Spreadsheets don't kill people. Bad decisions do.
Modern dimmers aren't resistive.
Or you could have replaced them with dimmer compatible CFLs.
My intent wasn't to provide a thorough cost breakdown of the technology. That's impossible to do without taking individual site requirements and existing infrastructure and capacity into account.
My point was that quoting a wholesale drive cost was far from an accurate represent and that the cost of tape is often highly overestimated.
Is it right for every client? Of course not. The buy-in cost for the drive is far too high if a client has a relatively small data set. It can also be a poor fit for clients who consider nearline access of the data a priority, though in those cases a hybrid approach including snapshots is often the right answer.
On the other hand, the larger the data set the more attractive the lower media cost and higher density becomes. Likewise customers who have a need for deep backup cycles or true archiving (say for auditing puprosed).
In other words - don't trust people who try to sell you on the One True Way.
Fine. Get the cartridges, but what about the capital cost minus depreciation of the drive? What about random access?
Random access is why snapshots also have their place. :) Archival backups and nearline backups solve different sets of problems.
Now weigh those against an inexpensive jbod frame with a 2gb FC backplane.
What kind of capacity are we talking. For a small site you can pick up a little 2U unit that'll store 6.4TB uncompressed for under $5k. Or if you're running a larger site you can snag a 4U unit with two drives for about $15k that'll handle 30.4TB with optional expansion to 60.8TB native.
What's the write speed of LT vs a tasty little GB SAS drive?
120MB/sec per drive without compression. And now that you've talking about SAS drives your per TB cost is hopelessly optimistic. Even OEM packaged terabyte SAS drives are going to run you about a quarter a gigabyte, which is now four times the media cost of an LTO-4 solution.
Rackspace? You can put a dozen into about 4U.
So about 12TB in 4U compared to the 30TB unit I mention above.
Cooling? Although I'll grant you green cost, the random accessibility out-classes the seek time and tape insertion by a human cost dramatically.
Have you never heard of a tape library?
Stable media? Tape? Sometimes.
Properly handled tape is incredibly stable.
Shelf space?
If you're doing off-site storage, that's going to be an issue regardless of what media you're using. And as I pointed out, tape is far more compact and far lighter than disks.
No need to use tape anymore. Get out of the reality distortion field, but do the right thing by testing what you have and doing drills to ensure that whatever you have, works and is a procedure understood by all.
I'm not the one dismissing an entire class of technology while demonstrating ignorance of its costs and benefits.
Even accepting your price that's a cost of about 12.7 cents per gigabyte and you can get 800GB native LTO-4 tapes for about $50, which comes out to about 6.3 cents per gigabyte.
But quoting costs for desktop grade SATA drives severely understates the true cost. For any non-trivial site installation you're talking near-line rated drives, drive caddies, storage shelves and additional SAN fabric. Then price out the additional power, cooling and rack space. Then price offsite shipping and storage for the bulkier, heavier and more delicate disk option.
Mirroring has its place. Snapshotting has its place. And backups to stable media still has its place too.
While I've been a cable and internet customer since they moved into our area three or four years ago. No extended outages*, no equipment problems, no service problems, no billing issues. The only time a specific channel was unavailable was while they had the scuffle with LIN broadcasting over licensing fees.
The service hasn't been perfect of course but the biggest complaint I can come up with is that the free video on demand channels used to be a bit spotty, but I can't remember the last time I had that problem. And I was a bit more forgiving while they were working on upgrading (or outright replacing) Adelphia's antiquated equipment.
(* The one exception was in the wake of a major ice storm in 2006, but my neighbourhood was also without power or phone service for the better part of a week.)
Oh, sure, the "cheap pr**cks* can pay an extra quarter here, an extra dime there, another nickel somewhere else and suddenly my bill is ten or twenty bucks more a month.
There have been closed source drivers for a couple years already.
Solid tactile response, sturdy as fuck, easy to clean. The loud clicking is a bonus feature. Especially when you want to continue typing when you turn to talk to someone. :)
(Seriously? Thin keys? Yuck. Different (key)strokes for different folks, I guess.)
I've bought two so far. Planning on buying a third. Probably a few more for the kids when they get older. :)
Right, because programmers are expensive but administrators, managers, rack space, electricity and cooling are all free. Some level of optimization is always worth it for any but the most trivial applications. If you're doing an application that's realistically only going to service a relatively fixed amount of traffic and you're talking the difference between a $1000 server and a $2000 server (maybe an application server deployed in branch offices or an MTA for a small business) - sure, go ahead and go with slow and unoptimized. Any serious enterprise-wide or internet facing application and you're just asking for trouble if you don't keep performance in mind from the get go and at least identify the low-hanging fruit.
OPEC failed to demonstrate the ability to set oil prices even when they controlled a far larger percentage of world production.
Look at the last time oil prices spiked the way they did in the past two years. It took thirty years before prices reach the same (infaltion adjusted) peak. And that was with relatively healthy economic growth during the intervening years.
I'm sure prices will bottom out before long, but they're not going to jump back a hundred dollars a barrel any time soon.
After the smashing success of Internet Mail 2000 he would be a fool not to!
Eh? You don't get a fsck on mount by default with ext3 unless you've exceeded either the check interval or max mount count. If the filesystem has the needs_recovery flag set (i.e. wasn't cleanly unmounted) the journal will be replayed.
Lengthy filesystem checks on large filesystems holding large filesystems are only a problem if you don't create the filesystem properly in the first place. The time to do a fsck is pretty much linear to the number of inodes; specify a reasonable bytes-per-inode ratio and fsck time drops dramatically. Even this is unnecessary under ext4 since they no longer check unused inode groups.
If you RTFA they give brief summary of the additional features of ext4: "Later, others (Lustre, IBM, Bull - see Theodore T'so comment) got involved and added extents, delayed allocation, online defragmentation, and more." and the development wiki goes into further detail.
System administrators who want to something small to carry when they're on call? Students who want something small to take notes on? Parents who want something light and relatively cheap for their kids?
Well it does support tail packing. Draw your own conclusions.
I don't know that I would call it fairly simple myself. The basic picture of what software does what isn't too bad, but some of the configuration is pretty obtuse under the hood, the LDAP schema and SOAP interface are fairly large, and the internal mailbox format is a bit wonky. Anything and everything for a particular user (including folder data, contacts, appointments, mail messages, tags, conversation data) is shoved into a single table (mail_item) with some of the data massaged into columns which marginally make sense and the rest shoved into a bencoded data structure with terse field labels (e.g. 'u', 'v', 'nc', 'st').
That said, yeah, if the concern is the technical ability to keep running even if the parent company abandons the product (or jacks the price unreasonably) they're marginally safer. Lock-in isn't really any worse though.
Eh? There's no courier in there. The mailbox portion is about as nonstandard as something they wrote themselves could be. ;) Beyond that the rest of the architecture is actually pretty easy to replace. I'd still go for a commercial license for this sort of license. Too many features culled from the community edition (including their backup/restore, mailbox migration tools, clustering scripts, etc ... little that can't be worked around, but it's a pain in the ass if you're not pretty intimately familiar with the innards.)
There's no exim. It's postfix. And the indexing is Apache Lucene, nothing they created.
On the other hand the mailbox format is proprietary (combination of on-disk blobs and database backed metadata) so it really isn't much safer in terms of lock-in. Migrating to another platform will involve exporting to some interim format or doing a protocol sync. User preferences are maybe somewhat more accessible, since they're stored largely as LDAP attributes, but the schema is proprietary as well.
I've never used Cyrus personally, but in the e-mail world that's a fairly small mail cluster. We have a legacy system built with open source components running 50,000 active users hosted on three boxes, and we could drop that to two if we were willing to sacrifice redundancy. Granted the majority of accounts are relatively small quota and the user base is shrinking, but when it was our primary mail system we had hundreds of thousands.
On our current system we push about 50,000 users (with unlimited quota) per backend server, with the main limiter being storage space. The SMTP servers on the front-end (which also handle spam filtering) handle enough traffic for a few hundred thousand users each (pooled, not dedicated to any particular machine).
That isn't to say outsourcing is a bad idea. It's less about the software (many of us rely heavily on the same open source packages freely available) than the expertise to use it effectively.