Slashdot Mirror
Internal Costs Per Gigabyte — What Do You Pay?
CodePwned writes "I recently took over a position at a rather large company where I discovered my group was paying $30 per gigabyte per month! That's $360 per year per gigabyte to our own IT department. While I understand costs are different depending on the scale, redundancy, backup and support methods, there doesn't seem to be any good papers on what range you should expect your costs to be. So far, my research shows an average of $1 per gigabyte or less for internally hosted space. What do you pay?"
Well at my job we pay whatever the current prices are from our vendor. CDW seems to have pretty reasonable prices, and we do a lot of other ordering from Newegg and Border States Electric (they're regional to up north by MN, IIRC.) so we get pretty good prices as far as I can tell. Sounds like the OP needs to talk to some vendors.
you don't want to know what people pay, you want to know what their costs are.
Bandwidth? Storage? Backup? Downloads from a particular site? What the hell are we talking about here?
I suppose it all depends on what, and how, you're measuring. Is that money spent on backup tapes, raid systems, flash drives, or what? Is that for offline storage, frame-relay throughput, ISP bandwidth. Does IP telephony get rolled up in that? The question seems a little vague to me.
When our name is on the back of your car, we're behind you all the way!
Until about two years ago we were charged about CHF10 (CHF=Swiss Franc) per 100MB per month to our internal (now mostly outsourced) IT provider, that is 3 times more than you, roughly. But I don't know what our service charges are now. On top of that we pay a flat rate of CHF600 per person for IT support per month. Doesn't matter how much you call IT. I felt like we were ripped off, but no-one seemed to care.
Performance, reliability, and price, pick any two.
High performance and reliable storage tends to be expensive.
High performance and cheap tends to require a lot of maintenance.
Reliable and cheap tends to be really really slow.
So if they are on a SAN with that one gig spread across 50 drives, there are some applications that need that speed.
Gonzo Granzeau
"Nothing the god of biomechanics wouldn't let you into heaven for.." -Roy Batty
the question
http://www.acetonestudio.com
I thought it was RAM
http://stephan.sugarmotor.org
$3/Gb/Month
I might be mistaken but if I'm getting you correctly "internally hosted space", about $100 for a TB drive that translates to about $0.09 per gb. If it's internal why would you have to pay per year for that?
I am Bennett Haselton! I am Bennett Haselton!
For backed up to tape storage? Storage replicated to another, remote datacenter? Snapshotted at regular intervals?
SAN storage? NAS? Direct attach? On arrays with 10 drives, 100 drives, or 1000 drives?
Fast SAS or FC drives? SATA arrays? 5400 RPM? 7200? 10k? 15k?
If you're paying $360/GB/yr for low end storage that sucks. For very high end, with replication and snapshots and the fastest drives and so forth, that's pretty high, but not an order of magnitude high.
We're paying about $1000/GB on our SAN.
"Depends" is your answer. Though I'm assuming you're talking about disk, not tape nor VTL. Do you buy direct from the manufacturer or through a channel? How big is your company? What's the total installed base so far? General Electric pays much less per GB than some midsize company with 100TB.
Do you mean for SATA disk in a tier2 array or SSD in a tier1 array?
Costs go up when you include snapshots and replication.
Do the editors even ask the submitter to be more specific?
shameless self promotion::: (you asked)
ClearBits offers unlimited bandwidth / distribution of up to 10GB for $45/year, $0.98/GB/month additional, and less for higher usage.
http://www.clearbits.net/about/member_benefits
We pay a one time $40 per gigabyte as the capital cost of acquiring the storage. There is no monthly cost. I think $40 is still way too much.
SpyDock: Scientific Python in a Docker container
there is no way for $1 per month per gb you will get daily backups, off site storage, redundancy, speed, etc.. an 99.99% uptime. and internal storage your not counting all the hardware for the "internal storage" nor are you counting A/C, Electricity, physical space, etc. etc..etc.. Network, Bandwidth.
My boss does.
Stop beeing retards and do internal invoicing. Not like the board of directors is billing "per decision". IT department as a billing self ruling department is so damn stupid. And no one seem to understand it. It is like having a fire department only going to fires that is in line with their mission statement.
And having an it department invocing per GB instead of having a budget for storage and then the company can allocate it as they please. And if someone has a bigger need, it should be a question for the company. Not a matter of giving a profitable it department.
- To understand recursion, we must first understand recursion -
That's what normally happens internally in a company. The company that does not specializes in IT pays its own IT group larger amount of money for different reasons, limited but not included
1. You have your own network.
2. You put money to pay for your people own and support
3. You put your own reliability depending on the money you have
4. Your data is managed by your people and it's backed up at your own responsibility
5. Having your data closer to you probably gives you more bandwidth availability (speed and latency).
Of course you can buy your own NAS and place it behind your desk, and you'll be responsible for your data, but of course if something happens... it's on you.
It depends on what you get for that $30 per Gb/month. If the only thing you get for that is storage and support costs come out of a different pot, then you are paying too much. On the other hand if that $30 represents all of the IT budget for your company, then it might be about right (might not be as well, ther e are too many variables in that case).
The truth is that all men having power ought to be mistrusted. James Madison
Full service IT at it's best.
In the poster's case, he doesn't say exactly how they're being "charged" only that it works out to $30/gigabyte.
That's always a point of contention among mangers: who should pay what part and how much of the overhead costs because it reflects in their numbers.
RIP America
July 4, 1776 - September 11, 2001
Hi,
I am willing to bet that the "gigabyte" usage is simply a cost driver. Accounting simply needs to know how to divide up IT costs and settled on this as a cost driver, possibly one of many, to determine what it takes to support each department.
This is neither new nor entirely bad. Sometimes it is better to go with an easy-to-implement, but only partially accurate number than one that is perfectly accurate but impossible to implement.
See my journal for slashdot ID's by year. Mine created in 2005. http://slashdot.org/journal/289875/slashdot-ids-by-year
It is considered overhead costs an thus part of the IT yearly budget.
$1/gig is barely a mirrored pair array stuck in a PC. Probably not even 15krpm SAS.
What exactly do you need?
Lots of storage?
Lots of I/O?
Redundancy?
Backups?
Disaster Recovery?
People to manage it?
It all costs money. If you don't need any of that then by all means, buy a PC, stick a couple of 1.5Tb drives in it, raid 0 them and call it the department server. Of course, other people in your organisation may have mandated all of the above.
Deleted
One client of mine pays [much] more outrageous prices, one more TB and they have to build another datacenter it seems. A lot of posters have pointed out the factors that go into the storage costs equation, but with internal budgets and make believe money, it is often a political issue.
Also factor in Electricity, Bandwidth to the site, IT support, backups, setup and maintenance fees. Even if you host it locally sure it'll be less than $1 to purchase the drive, but you are going to be racking up quite a bill in the things I just brought up.
Do it yourself. Get almost a petabyte for $7867:
http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/
My answer:
67 terabytes is 67000 Gigabytes.
$7867 / 67000 = 11 cents per gigabyte.
Your mileage may vary.
Including the costs of fileservers, switches, operating systems, all-inclusive vendor extended warranty (3 years), and redundancy, our cost per GB is about 15$ (purchase cost, excluding che cost of people that manage the servers and excluding the cost of the backup system). Your montly fee seems to me way over the industry standard.
Higher ed, 4gb FC EMC SAN, weekly tape backups:
First 2GB is free, after that it's $7/GB/yr
What you may be seeing, especially if you are working for a very large company, could just be a cost allocation scheme, not a real money cost as you are thinking of it. If your department brings in revenue, the organization needs to match expenses to it for purposes of Management Accounting.
For instance, imagine you know it costs $X to run one of your cost centers. That dollar amount includes everything from the manpower, the equipment, the facility...everything. Now, they need to assign these costs to the departments that actually make money in a way that makes sense. They could do this by carefully costing out each service they provide and assigning an overhead rate, blah blah. That tends to be a pain. You do it if you have to...but you try not to have to. Another, easier, way of doing it is determining a usage metric (CPU hours, GB of storage, number of tickets) and using that to determine each profit center's percentage allocation of the overall cost.
So, the $60 per GB may not even be close to a market rate for storage. However, if all the departments used twice as much storage next year, the per GB cost might fall to $31 per GB (slightly more than half to account for the fact that there would obviously be more real costs). Conversely, if you convinced your management to contract externally for storage, everyone else might find their per GB cost rise, since the fixed costs would be static.
I think he is concerned whether $30 / Gb / month is a rip-off - given reasonable standards for reliability & performance.
I recently took over a position at a rather large company where I discovered my group was paying $30 per gigabyte per month! That's $360 per year per gigabyte to our own IT department.
It is incentive to keep your data free of porn, lolcats, pictures of the 2007 Christmas party, CD rips, etc. It also helps to pay for the Ferraris that the IT department drives.
When our name is on the back of your car, we're behind you all the way!
we do .47 per GB with a multiplier of 1.25 for RAID5 and 2.0 for RAID 1+0. This is on EMC clariion arrays.
Don't think of it as $360/GB per year, think of it as $360,000/TB per year.
It's possible the "internal" cost is simply some form of rudimentary metric to measure the relative usage that is made of the IT dept by each of the other depts (so each dept is charged accordingly).
--Ivan
Really just guessing here, but is your internal storage, by any chance, provided by a group company function, while you work for a subservient company? It does seem a common way to claw back cash to the parent company.
We pay approx $75,000 per Tb per year.
Tier 1 enterprise class SAN. fibre, triple redundant, vtl and then tape behind that with HSM etc etc etc then with layers of federation in front of that, both hard- and soft-. Effectively we can loose the entire SAN, and the switch over is quick enough that most apps won't see it, and we won't loose any data or have an issue.
Which is all good and well, but when your looking to handle a couple of Pb of data, the bill quickly adds up.
per year per GB for backedup and replicated to a second datacenter of SAN storage.
The contributor and /. editor need to RTFA.
If this guy has started at a local govt entity or highly formalized private enterprise, I would suggest he squares himself up and forgets this sort of thing.
I have seen this directly. For example the corporate staff (including CEO) gets their wages divided into cost center charges.
Quick and dirty example: $3,000,000(corporate wages and expenses) / 500 cost centers = $6,000 per annum.
Alternatively if he wishes to keep digging, he will find himself pushed to the outer, or worse, managed out of his position.
In post Patriot Act America, the library books scan you.
The main "file server" had only 2 TB of storage, for a ~200 user site. Our department started using high resolution imagery, and we were burning through the remaining 500 GB of free space. They told us our department needed to upgrade the hardware, and offered to set up a 4 TB NAS for us for $10,000 (or maybe $20,000?) I questioned why don't they upgrade the file server and they said it was full. I found out later all the servers on site were 5 year old HP machines.
...it's the people supporting the bits. At my company, storage is also insanely expensive compared to the personal consumer space, but that's because unlike the personal consumer space, our data centers have high reliability, and lots of personnel (along with their 401k plans, insurance, office space, and other expenses) who have to be paid to support the systems.
Towards the Singularity.
Platinum storage at UBS APAC is currently around 100kUSD per gig per per year. Its mainly used for production systems and as such is guaranteed to have 5min snapshots.
Ask them for the equation that they use to calculate this cost. Then, break it down into pieces that you can monitor (total throughput, reads/writes, data storage used, active vs. latent storage, tech support calls, backups/restores, etc.). Use something like Cacti to graph it with $$ as your Y axis. I'm not kidding. With a little work you could build this and then pay precisely what you owe. If it's still cheaper to internalize your storage, then go for it.
Where I work I implemented the SAN so had to quantify all the costs associated with implementing it and worked out an actual pound figure. In the end it was best for our accounting practices to charge a one off fee which could then go back into procuring more storage when required. When you take into account all the hard drives, drive arrays, storage controllers, fibre channel switches, backup solution, tapes etc. it cost the business £15 per gigabyte. So it sounds like where you are they are charging a lot
We get quite often that they can go to the local PC world to buy a terrabyte drive for £50 but when you explain to them that if that drive dies so does their data and the fact that realistically only one persona can use that drive at any time they reluctantly cough up.
ultimately if you dont like the price, dont pay for it. Delete the data and see how much that would cost the business.
Like many others are saying on here - you need to give us a lot more information here. There is a huge difference in the cost per gig between a Netgear and a NetApp. You also didn't mention if your cost analysis includes your OpEx costs (Operational Expenses) - things like hard costs (labor) and soft costs (power, hvac, floorspace, etc).
Tell us more and we'll be able to help you out better.
Ron Gage - Westland, MI
We charge people something like $30-40/GB one time cost for space on the NetApp 2020s. Reason is there's two redundant NetApps in two different buildings, one of which is also backed up to a tape library with tapes rotated out to a vault in yet another building. Also the NEtApps have space set aside to do hourly snapshots, in case you delete something. Costs just a little bit for that kind of performance and reliability. For our lesser storage system, which is basically just a supermicro case filled with 2TB WD RE4-GPs we charge like $0.20/GB. You get it on a RAID-6 system, but no backups, no redundancy, if it dies the data is gone. For desktops, well it costs whatever you can get a drive for probably $0.10/GB or less.
Costs entirely depend on the performance and even more on the reliability you demand. If you say "I can't ever lose my data," fine we can do that. However you need to be willing to drop some serious dollars. You can't have cheap and reliable.
That's the problem we run in to is users want to pay bottom dollar, but then scream and wail that it is our fault if anything gets lost. Just doesn't work that way. You have research you need stored? That's what the expensive ass NetApp storage is for. It would damn near take a nuclear strike to cause data loss there. Not willing to pay? No problem, but no bitching if your drive goes tits up.
This is just storage space, not web pages/applications, or software etc. We're talking digital assets of the company such as documents, images, videos... etc. Basic, run of the mill file storage is being priced at $30 per gig, per month. It's basically just a giant network share. It doesn't need to be co-located just your typical raid array with some method of disaster recovery.
I'm interested in what other companies charge internally for file storage.
it's really not a fair question.
some may be paying more, or yours may be high depending upon what your storage infrastructure looks like.
are you on HP EVA or XP storage? IBM SVC?
or do you just use some basic gig-e SAN, or DAS/NAS?
let's start there, and then let's talk about how your storage works. is it tiered, is it smart, is it redundant? how important is some of the data? what type of contracts do you have in place? what is your storage plugging into, and how?
answer these questions, and give your question some context.
not only is time travel possible, it's irrelevant.
At the universisty where I work. IT charges $3.00 per GB/year to store data on a NetApp SAN. It then costs you another $3.00 GB/year for backups.
NOTE: In case you're wondering the two prices are charged separtely in case you have data that doesn't need to be backed up or have data that needs to be backed up but isn't stored on the SAN.
We pay a flat fee of $110/GB for storage from our IBM DS8300 SAN. This includes cabling and tape backups. I work at a telco in Canada.
It pleases us to make this generous offer of $27 per gig, saving you 10% over your current outrageous costs. We would be happy to make a long term contract with you, guaranteeing no cost increases for the next 20 years. In addition, by paying in advance, we will be happy to discount the price further to reflect the savings on our billing costs, at the rate of $0.02 per gigabyte. Please sign below
As a storage administrator, I don't think you are factoring in protection levels replication, licensing etc. If you want protected data, you can't run down the Best Buy and pick up a 1TB hard drive and hang it off the back of a server. Internally we charge about $25k/TB of Tier3 storage. This data has to be protected by RAID (so you lose disks) then replicated to a backup frame locallly (also RAIDed) then offsited. You also have to keep in mind nightly snapshots of your data consume space, and that there are WAN links that you have to pay for to offsite your data. After the initial buy, the storage team picks up the cost for maintenance out of our budget. We also have to pay license fees for things like NFS, CIFS etc so that you can access your data, and the maintenance on a single frame might be $100k/year from the vendor. All of these things have to be paid for, so I know it *seems* expensive when you compare it to a JBOD from a retail chain, but there are huge benefits to the level of protection and enterprise features that you get with enterprise storage (point in time snapshots, ability to quickly refresh development databases with production data, gold volumes for cloning, deduplication etc)
The big reason for internal IT departments to charge other departments for services rendered is this:
When it comes time for a manager to "earn" his bonus, the first thing he looks at is cutting the budget for less profitable departments.
The IT department rarely has external clients for income, but is absolutely vital to keeping the business running.
Therefore to keep some short sighted pencil pusher from crippling the company with a failing infrastructure, the IT department has to show a "profit" for the services it renders.
Our SAN is keeping 4 copies of the data (2 local and 2 remote) and it costs us about $3.50 per GB/month over 3 years for SAN and bandwidth, but without power or data-center space factored in. I think you're paying a little too much.
----------
If your answer is Microsoft, you obviously didn't understand the question.
Per gigbyte of what? Not certain what you are talking about; in normal Slashdot style I will answer your question anyway.
It is standard practice in many large corporations for departments to pay ‘charges’ for the infrastructure and supplies they are using. Not real money, just numbers so that the bean counters can figure out what stuff costs to get things done, and to juggle the numbers to make things look better or worse (as directed by their managers).
At one place I worked they charged for server disk space. The theory being that it cost money not only to buy the disk or make it redundant, but also to back it up incrementally, offsite forever, transport it for me through the network, process the data on the servers, pay the IT staff to support me, to be trained and to go on vacations. The number usually was much higher than you’d expect, and actually included *all* IT expenses. They used disk space as a *fair* measure of my groups IT needs.
Remember too: some places will backup your PC via the network, so just because the disk is cheap on the PC doesn’t mean that is the whole expense per GB.
I don’t know what it costs these days at a big company. $30/month sounds like an old number for old infrastructure, not your desktop PC. Thank God I work for a small place now and the accounting is almost sane.
I don't think that more than one third of a million dollars can be justified to store one terabyte of data, no matter what the infrastructure involved. At that price, you can afford to colocate one hundred servers in one hundred different datacenters and replicate your data to all of them, including the staff to manage them all. $360k per year to store 1TB is insanity.
How many IOps can you get with that? How many of the disks can go down before you lose data? Can you lose a building or server (electrical surge) and still have the data? How far back can you go in backups (1 month, 3? 6? 1 year?)? Are there any backups? Does it include monitoring, so that—assuming you have redundancy—you're notified of a bad disk? What's the SLA on getting a new drive? Is it dual- or single-parity—if redundant—so that if you hit your UER you can still reconstruct the data during the RAID rebuild?
Yes, we get it, disk is cheap. It's everything else about your data that isn't.
AC to protect my identity.
At a large financial institution I know it's about $12/yr for tier 2 disk (FC, 15K RPM), $2 for SATA via FC. NAS varies depending on a lot of things. Disk replication and snapshots usually double or triple the price depending on how often and what type. Fiber ports costs extra as well as cables. Tier 1? Few groups have justified 73GB 15K RPM FC RAID 10 drives. It costs them A LOT of money.
Backup is about $35/GB. Way more if tapes or archive is needed.
A total of $60/GB/yr isn't too crazy given the services we offer. It's not like our storage group is making a profit, we're unlikely to even cover all of our costs at those levels.
There's a big difference between TCO to the company and whatever price the IT department charges *your* department for service. IT department prices are typically based on historical costs that don't necessarily represent the Moore's Law equivalent precipitous drop in disk space costs, and are often based on gold-brick engineering practices and specialized applications.
For instance, if you're running a high-end database or an Exchange server that's supporting the whole company, it needs to have a blazingly fast SAN array from EMC or somebody, and instead of using $50 1TB SATA drives, it's using $300 15000rpm 300-Gb SAS drives with SSD accelerators and uber-fancy controllers, built into a framework that lets them do maximum IOPS and live no-performance-hit backups. On the other hand, if you're trying to back up desktop data in case of laptop failures, or provide shared file storage where people can retrieve dull bureaucratic standards documents, performance isn't critical, price and volume are, so you want a big slow cheap Network Attached Storage device packed full of TB SATA drives, with a bit of RAID to deal with the occasional drive failure, and still some kind of backup system or maybe a tape-loading robot (if tapes are still even cost-effective.)
And it's not uncommon for IT departments to charge you for the former, even if you'd rather have the latter.
I'm dealing with a variant on this problem right now, for a network management application. The servers and storage in the data center are designed for blazing speed, but the application I'm trying to support is customers who want to archive all their network event data for a couple of years to make Sarbanes and Oxley and their friends happy, so I need fast servers for today's data, maybe something medium-speed for a week's data (but SATA's probably enough), and 98% of my data will never be looked at again but the rules want it online, not in a box of tapes.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
He wants to know what prices other people's IT departments charge their users, because he thinks he's way out of line even for that. Costs are a whole nother game entirely - lots of people aren't clear about what their costs are...
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Another great way to muddy the waters is to misconstrue the question. You don't have to speculate. There's a difference between a vague question and casting a wide net. The transaction value of a single post is maximized in providing a specific answer to a specific question, whereas the transactional value of a discussion forum is maximized by having many situations and specifics put forward, including scenarios not originally envisioned.
There's at least as many undeclared assumptions in your narrow pedanticism as in the original question.
The question could be more evocative of his actual circumstance, but perhaps the poster was afraid of blowing his cover. Is the net cast on this question so broad that this can't be a useful exchange? Probably, if the question generates more complaints than contributions.
For decent storage $1 isn't realistic at all...
Most enterprise drives are at least 500EUR for 500GB, which would mean 1EUR per GB... However, you need to buy storage controllers, set aside some disks as hot and/or cold spares. Then you do RAID6 over say about 8 drives.
And then there's backups... which are commonly done to cheaper consumer grade disks....
And disks do use power you know...
So everything included, it'll easily cost 25EUR per year or so, not including labour...
So 5EUR per gigabyte per month isn't unreasonable including simple backups and some labour...
Our data center provider offers storage on their FC SAN ( > 150mbps I/O) at a cost of $2.50/GB/month and an additional $2.50/GB for backups. This includes 24x7 support, 99.99% uptime, and is hosted in a tier 3+ data center. My guess is that smaller SANs cost more per GB, but you are getting boned at $30/GB.
On the other hand, if you are requiring some sort of high performance DAS with off site replication, then I bet the cost is considerably higher.
ÕÕ
Well, if he is working for a bank then I think the cost per gigabyte could be justified. Lets says that the banking information for a given customer consumes 10 megabytes(this number is subject to a great variance), basically its all just text though right? Now per terabyte we have 100,000 customers. Im taking the cost per gigabyte to be cost per year, so the cost per customer is $36 per year or $3 per month. Now if you have any loans as do most people 3$ out of your monthly payment is pretty much insignificant. I would imagine that the redundancy involved here for a bank is 2 steps above insane, and it works because a bank can afford to do it.
That said, I imagine the number the author is giving us is skewed somehow to include to include the cost of something that should not have been included in the figure. Even if it is just from very expensive IT, the cost of in house software, and crazy good testing you can see that the cost per gigabyte means very little compared to the value of information on that gigabyte.
Jeff is that you?
Is worst case for redundant secure two site storage for volumes over 2tbytes
If the IT department's blatant over charging causes his department to rely on local hard drives to store all their media and a drive goes pfft!, there goes a ton of man-hours. They should charge a flat rate per person, then a minimal amount per gig to encourage users to store critical data on the server, rather than on client hard drives.
Our IT shop charges $35/gig per year for raided and backed-to-tape storage. Still no bargain, but at least within the realm of sanity. If you need a lot of storage, they'll cost out purchasing and maintaining dedicated hardware for your needs.
Never let a lack of data get in the way of a good rant.
The downside of unlimited storage is it wreaks havoc with backups, replication and other intensive tasks that have to move that storage around.
I've worked in more than a couple of places with an "all you can eat" philosophy and you end up with a data monster that's expensive and complicated to backup, replicate and work with.
And usually they grew to a size (a place I work with right now has one just crossing the 2 TB threshold) where they become impossible to deal with -- departments can't/won't shrink them because the pile of crap is so large they'd need a couple of FTEs just to go through it all.
Chargebacks make perfect sense -- business units treat IT resources like they were free and use them EXTREMELY inefficiently (scanning thousands of images at 1200 dpi to use in on-screen PowerPoints as one example). It's perfectly reasonable to allocate those costs to business units so management can really get an idea if a business unit is profitable.
Fixed that for you.
Check out my sci-fi/humor trilogy at PatriotsBooks.
Is this company by any chance multinational? And is the IT department technically/legally a separate entity in another country? Internal price gouging to shift profits to $tax_haven has been done before...
________
Entranced by anime since late summer 2001 and loving it ^_^
Actually our HP XP12000 costs us around $30/GB, not including backups, maintenance etc.
Comment removed based on user account deletion
Certainly at some point in the past whatever you're doing and the costs behind it made sense. Other's have raised the question why or how you'd do this but none of us can answer it without knowing some extremely detailed numbers from your organization.
So, let's make an assumption: the level of complexity of your IT operations, as well as the IT overhead costs in the G&A section of your organizations budget, have remained constant since the $30 /GB rule was enacted. We're also assuming that those were the factors that led to the $30 /GB rule. (The reality is, a past IT director probably threw a dart at the wall and picked a number from thin air that he thought was reasonable.)
Therefore, you can extrapolate from that roughly what the 2010 cost should be based on Kryder's Law, which basically can be summarized as: storage density doubles annually. So, if you enacted the $30 /GB rule 5 years ago, then you should either: give them 32GB for $30 or charge them $.93 /GB.
----- obSig
We had large EMC DMX (high end) arrays, and we paid about $1000 per usable (mirror) TB of FC disk, but that was all in, and there was no reccuring costs. When there was serious investments it got more tricky (backend changes, frames full, etc). The Netapp route was much cheaper. People give me crap now at my current company because to buy a high end shelf on my netapp costs me about $14,000 which is 14 * 300G disks for speed. That comes out to $3 per gig, and that is before RAID-DP. If I go for space buying the 2TB SATA disks are cheap.
Being "cranky" is a prerequisite for posting on slashdot.
We pay... $30/GB one time, $5/GB annually for the "fast" SAS drive space on the SAN. $12/GB one time, $2/GB for the slower SATA drive space on the SAN. Both of these configurations are replicated, although I am not familiar with the exact configuration.
We pay about $12/GB/year for storage on 15k FC disks with RAID-DP and replicated across town. This does not include backups to tape, that's an extra fee. We are also in the process of working out lower cost storage without replication and on SAS (or SATA) disk. It's really silly to compare consumer grade USB storage to enterprise, replicated and professionally supported storage but it happens all the time.
I assume they get data backuped to a different planet for that kind of money...
Their IT department is probably headed by some young, hip, spikey-haired self-starter who wants to turn his department into a profit center. And by that I mean he wants to deprive other departments of their actual profit.
Can we stop treating enterprise IT services like they're luxuries that should be taxed, and instead think of them as something vital to the survival of an organization? Give them the funding they need to operate and provide their essential services to other departments. You're only over-complicating cost management and covering up your IT department's poor performance records by acting like you're bringing in money when in fact you're just moving it from the left pocket to the right.
IT will never be a profit center. Don't try to masquerade it as one.
Excellent answers so far.. but i don't see any to the main question: What do you pay? Assuming he means GB of storage, how much does your dept pay IT per GB?
It really does depend on what's included in that cost. There are a number of factors that may be included, many of which have already been presented, but I'll list here again anyway.
Raw storage cost of a particular type (NAS, SAN, etc.), which is always higher than consumer grade. You're paying for better performance and higher reliability, usually.
Padding for Redundancy within said array
Amortized cost of the processing unit
Network bandwidth allocation (because the network part of IT charges the storage part of IT for their bandwidth)
Amortized salary of the storage administrator(s)
Cost to backup, including possibly replication to another site (and hence doubling the raw storage cost), backup software, etc.
The list can go on and on depending on how granular your IT department is with charge backs.
Our raw storage for top tier NAS storage is probably in the $5-10/GB /year, but that doesn't include any other additional costs.
Hope this helps.
If pricing is supposed to help ensure proper resource allocation amongst different organizations, it seems to follow that it would help allocate resources within an organization as well.
Too-high and too-low costs both cause problems.
I actually work in IT and am currently dealing with storage, redundancy, etc. for some clients. I don't actually charge per GB of usage, but I can see why the $30/GB pricing model might be in place.
You didn't give enough information to really be able to answer your question, but the way I can see this being reasonable is for $30/GB of production space and something lower for archive space (maybe $1/GB.) Why? Simple. It can take a long time to restore large amounts of data, and I've got clients that want their production data NOW if the server goes offline/ RAID controller goes down/ new guy accidentally deletes the entire production store. If you don't have appropriate separation of production and archive files, it takes a very long time to get the important files back because you're restoring both the "important, have to have them now" files and the "I'm only going to look at these files once a year" files. By charging so much for the production space, it would encourage the users to move anything they aren't currently working on into the archive space, making restoration of the production files much faster.
Actually, it's: "Quality, Cost, Time. Pick two." That's the balance every project manager has to walk when they actually "build" something, like a supercomputer and its power and cooling systems in 90 days.
EVERYTHING, gets calculated into the costs for a cost center, say like the one you associated with said supercomputer. Office space, electrical costs, office supplies, salaries with fringes, machines, PDUs, switches, cables, pizzas, sodas, t-shirts, etc., etc., et cetera! Everything that involves money attached to it. Then that amount is divided into a unit, say cpu/hrs, where cpu/hrs = 365.25 days * 24 hours * 2200 cpus = 19,285,200 cpu/hrs. So you take that huge monetary value, divide it by another really ugly number, and get a reasonable idea of what the operation costs in a billable amount folks are familiar with, i.e., $/cpu-hrs.
S3 is, what, $0.15 a gig per month plus the same amount to transfer it. I believe that's with 3n redundancy (has anyone lost data stored in S3?.) Is the throughput/latency you're getting 100 times better than S3? If not you're probably getting ripped off.
I assumed this was for storage but after thinking about it more, I have no idea. Maybe it was per GB transferred for bandwidth usage?
- Blazing fast storage using NetApp (or similar vendors) - $4/GB/year (limited to 16TB)
- The same performance from a 16TB ZFS app - $1.50/GB/year
- High-end HA storage (50-500TB in a rack, SSD caches) - $1/GB/year
- Low-end backup storage (40TB per appliance) - $0.30/GB/year
Custom electronics and digital signage for your business: www.evcircuits.com
We completely lost a server and every scrap of intellectual property on it. The backups were flawed. Nothing worked right. 30/mnth would have saved our company millions if the provider were reliable. We would have saved money if it had cost 300/gb/mnth
I'd go on a Vegan diet but the delivery time from Vega is too long. --brownkitty
when we buy anything we plan to buy another one in 4 years and take support in accordance, except network which we buy for 10 years
san cost us 40k€ for 12tb
manpower to buy/install/maintain is 12k€
backup (license and hw) is at 100k€
40k+12k+100k=150k€
4k€ per month
3.12€ per gb per month
support fee not included but roughly add 15% to this
We have a 1TB RAID 1+0 array (4 disks) of 7200 RPM SATA drives over gigabit LAN. We have 2 external 1TB on a weekly rotation with offsite backup.
Assuming an expected life of ~3 years on each disk, and $150/per (when they were new), that puts us at $900 for the upfront costs of the disks alone. Spread over 36 months, that's $25/mo/TB which works out to ~$.025/mo/GB.
Now add in my support/maintenance [backups, initial configuration, etc] salary (assuming ~8 hours/month @ $28/hr) [I have other duties] = $224
Now add in the bank box for off-site storage which we'll call $6/month for the sake of round numbers.
So we've got 25+224+6 = 255/mo/TB for the storage, or about 25 cents per gig per month.
Sounds like I should start charging more, eh?
Look around. Times have changed. There are online backup companies that do business grade managed online backup for $0.15/GB/mo. This is what we charge at AltDrive.
I manage IT for a small but data intense co. We charge .06 per Gb per day. Approximately $1.80 per Gb per month. This figure was calculated from all costs (capex, labour) including data archiving costs and ammortised over 2 years. It doesn't include any internal profit taking.
I only leech.
Is the per storage unit the only metric used to derive the internal transfer "cost"? Or is it the only metric?
Hardware, backup hardware, floor space, cooling, network, external network, off site backup, transfer costs of media, depreciation, staff, software, software development, etc.
Do you provide separate funding from your cost center for the desktop, portable, phone, fax, printers, etc? Pay for per page printing in a tiered manner (desktop, group printer, black and white at the central facility, color, binding, etc.)?
What labor market are you in? Done any corporate downsizing so that fixed costs have stayed constant where usage has declined?
What I'm trying to understand is just what the per unit of storage cost has to cover.
What starts out as a simple question needs lots more background before you are able to compare apples to apples. Most IT groups don't make a profit as such but try to estimate for budget purposes whet you will need 18 months in advance of actual usage. Try it sometime and you'll understand that plucking a number is probably as good a method if done by the right person with an understanding of the organization and what it will be trying to do over the next year or so.
... for not knowing accounting.
There are fairly well established practices for handling charge-backs between divisions in a company, and well established ways of determining the dollar value of benefit to either the providing division, the receiving division, or the overall company.
And of course there are often reasons other than the immediate bottom line for purchasing services internally rather than outside - reasons that trump the "cheapest is best" attitude.
Assuming the CodePwned is not trained in Accounting, I'd suggest that he or she politely ask the person who is in charge of finances for an explanation for the costs involved.
Three Squirrels
"If the IT department's blatant over charging causes his department to rely on local hard drives to store all their media and a drive goes pfft!, there goes a ton of man-hours."
Uh... as much as making palatable the original bill of 30$ per month per gig? If that's the case, they are only maximizing their profits in quite a free market way. After all, the whole issue of backcharging is in order to bring the marvellousness of free market within the corporation!
"Our IT shop charges $35/gig per year for raided and backed-to-tape storage."
How much does it charges per deployed PC? Per LAN cable yard? Per CPU cycle at the server room? Per port (and does it take into account if it is 10, 100, 1000, 10000MB?)? Per IP? You see the trend: there's an awful lot of things your IT dept. is not charging back to you and usually that means that your IT dept. will have to procure them anyway within a ridiculously low budget, if any at all.
$30/GB/month, now I know you work at Microsoft! ^-^
I think this was about the same outrageous cost of storage in the corporate datacenter farm 2 years ago, although there were also different cheaper options for larger storage needs. And there was also a one-time setup fee before even the monthly bills...
So I made a proposal for our own department for a 8TB storage solution, consolidating several smaller file servers into one (plus another for the backup) with this DAS enclosure: http://www.pc-pitstop.com/sata_enclosures/scsat84xt.asp
Cost was about $6K for the 2 enclosures, 2 RAID SATA controllers and 18 1TB Adaptec drives. The servers were scavenged, the OS and backup software was for free, the set up was also provided by us for free. This was a one-time cost for less than a week of what the IT department would bill us, and zero hassle compared to all the paperwork required to fill in such a request, which basically forces you to overestimate the storage space need, as you don't want to repeat this horrific experience. Maintenance was supposed to be minimal, the disks were configured in RAID 6 to allow for 2 simultaneous drive failures, we had also planned for a couple of spare disks already mounted in spare trays to quickly hot-swap failed ones. Basically maintenance was required only to restore the backups or specific files to an earlier restore point in case of data corruption.
You might get a problem with corporate policies that require the use of the centralized datacenter. I left that department shortly after my proposal, don't think it went anywhere.
I'd make the same proposal today, except that, if the storage need requires it, I'd probably get 16-drive DAS towers or rackmounts with 2TB disks (just 8 disks is a waste for RAID 6), about $6K for 28TB of usable space out of 32TB raw space, plus another 28TB for the backup server set in a separate location.
the other departments are actually buying YOU...AND the fuxking down hardware as a total solution
if u want to calculate it directly, u have to include ur service hrs and HR costs as well
u cannot and should not just divide the whole thing into $/G
becoz u r not inside the harddisk
and ur harddisk is not just bought once and forget, u have the running/operational costs as well....and most of the time, the boss direct decision is to cut YOU rite in the head
...to make managers in profit centers accountable for the costs that go into their profits?
Once you get out of the realm of real businesses and into the funny money realm of large corporation "profit centers" so many of the costs associated with running a real business get lost -- rents, furniture, HR costs (payroll and benefit admin overhead), utilities, and of course, IT costs.
A lot of these profit center managers are prone to treat $Profit = $income - ($payroll + $direct_expenditures) and ignore the other costs, no matter how wastefully they're incurred.
Their attitude is so we ran 200k color copies, big deal. So we've got a file & email servers with 2-3 TB of crap, MP3s, etc. So we all stream TV, audio/video and still want to download a gigabyte in 10 minutes. We insist on expensive laptops, Adobe Creative Suite on every computer, ponies, balloons, etc.
What's funny is when you work with REAL small businesses that have to pay for everything it's amazing the corners they'll cut and the sacrifices they'll make for IT (old, slow PCs, bad software, slow networks, bad/no backups, etc). It goes too far with them sometimes, but in those worlds they "get" it.
I think chargebacks are another way to keep these managers honest and in some ways, not skew upper managements view of how profitable or not the business actually is.
Sarbanes Oxeley
$85 Canadian / gig one time charge - minimum 13 gig.
Running three Hitachi sans. Backing these up with lto drives.
That $85 includes back up tapes but $85/gig does cover all the backup costs.
LTO tapes are a horrendous price.
Posting anon because I'm guessing this is internal company information, but the latest quote is that it costs our company $42,000 per terabyte.
Or how about an index of the data and a cache on SSDs (including indexes of all the old data) and all the real data on bulky SSD?
Does it really make sense to spend the bulk of the cost on maybe a 15% performance improvement, by moving data between disks a lot?
If you store on SATA initially and always, you never have to worry about deciding archiving procedures, only about deciding what to cache, and how to best ensure the cache is efficient and resulting in the best performance..
grf, First line : real data on bulky SATA. (replace bulky SSD with bulky SATA).
because it forces people to use their storage wisely! I'm in serious awe of the bawls that IT department has...
That's nothing. Our corporate IT department (we're a fortune 20 company) charges other departments in the organization 6 hours @ $85/hour to (drumroll......) create a xen guest on an existing server that's "owned" by the department being billed.
Typing "xm create foo" = $510
Then they bill the victim retail price for a RHEL license on top of that. So you'd think we'd get a little support...nope. You want to take a piss around here, you fill out a few tickets and wait until the 71st hour of their 72 hour SLA before anyone can be bothered to get off their ass to read the ticket.
So we've just been ignoring them and rolling out CentOS and staffing our own mini IT group instead. At least they'll get up off their bums when something is broken and fix it.
I was working as a researcher at one of the US top universities a few years back. The internal IT group charged something like $10/GB/month to our personal research account for backup space on the network drive.
We quickly figured out iPods were way cheaper per GB, and we kept trying to convince our deans to let us buy loads of them as an alternative "cost-efficient" medium for external back-ups.
Nice try - no cigar
$0.66 / GB / month, provided by ITS (central IT) at the University where I work, including backups with great historical snapshots. Here's their internal pricing: http://www.uwo.ca/its/services.pdf
I think that it is likely that some idiot has decided that every department has to make a profit instead of the company as a whole making the profit. Cost centres then have to pass on everything plus an artificial markup above cost to indicate "performance" and real decisions are made on that instead of any sort of relevent information.
One striking example of this being misapplied was a group that had a legal section, an estates section and an accounts section and they all had to show a profit or lose staff. The accounts section was purely a cost centre so they charged the other two sections that actually made money and had a nice profit that enabled them to put on extra staff while there were job losses from poor "performance" in the revenue earning sections.
It's really all bullshit that has middle managers ready to kill each other in places where it happens and ultimately gets in the way of anything useful. Simplistic pepsi swilling CEOs that swan around the world like rock stars while their company burns seem to like to have that sort of environment - not that I'm going to name any paticular Mexican bandit.
Plan
Step one - Buy two fully equipped Drobo Pros. (12TB usable each one for redundancy)
Step two - hirer a guy to stare at them all day.
Step three - sell space to other Depts at $15 a month.
Step four - retire in Jamaca with the $180,000 a month - $10,000 for the Drobos and $50,000/year for the watcher. (thats 2.1 Million dollars the first year)
Read that as "Damn thats to much per gigabyte."
All amounts are us US Dollars:
:-)
1 esata 4 drive raid enclosure - $225
4 1.5 TB 7200rpm Seagate Barracuda drives - $320
Total: $545.00 (including shipping)
Storage using RAID 5 - 4.5TB
Cost / GB: $0.1211 (12 cents) / GB
That's about what I pay these days...
Sometimes, real fast is almost as good as real-time.
That's hosted on 146Gb spindles on a HDS USP-V with realtime DR replication and Tape backup.
Supported vendor disks (even sata disks) are not $200 like on high street they're more like $2k and 15000 rpm SAS/FC disks are $5k
Enterprise disk storage is still expensive sadly.
the smallest number of disks you can use is 8 (7D1P)
...and in the ways that are best for those forced to follow the policy, not best for the company as a whole.
Some years ago, I worked for a large public utility with an internal IT department that used a chargeback model to "defray" the costs of having an internal IT department. They charged, for example, $200/month for an office to have a networked printer (yes, that was the actual number) in addition to the lease cost. Since the individual office managers having to pay for IT services out of their own budgets weren't stupid, they did the predictable thing: they went out to the computer store and bought $100 inkjet printers instead, hung them off individual machines, and shared them out. Sure, at the page counts being pushed through them, the printers only lasted 4-6 months, and the ink cost per page was not at all economical compared to a laser printer - but the cost savings to the individual office's budget was huge even if it cost the company far more money.
Further, since the cost of having IT do anything was so high, most business units developed their own "unofficial" IT staffs - people who in theory held other jobs, but actually spent much of their time doing off-the-books IT work - also bad for the company financially but a huge budget bonus for the business units involved, since the cost of having those extra people was far less than the cost IT would have charged them for the services those people provided.
I think you also must think about the virus scanners and the data that is on the disc.
Because data is the most important stuff in big business, just make a very high amount per GB..
explain to your CEO if there is data on it where the company can be sue for, or is stolen (copy past some source code from internet) you also need to have a buget for the legal stuff, and then it can run in to the millions of dollars.
So it is better to ask just for "all the money of all the departments" for your internal ICT department, else you just tell that on the internal disc also the temporary internet files are stored so all the "browsing" will also be stored on it, and that you need to check it if there is illegale stuff in these files .. so you need some extra man power.
also you need to be able to fire people that browse the wrong sites... ; )
I think if you tell than to the management they just give you all the money you need, and just tell you forgot what we asked you ; )
If a company losses it data it can work anymore .. so the price per GB.. is not possible.
Losing one 1 email can bankrupt your business.
so just give ICT all the money ... because they ask this stupid question ..
Internal costs are not real money. If they charge more internal costs you company will not make more profit. The game changes immediately when you considering outsourcing the same tasks.
In any organization cost of each item/service is eventually recovered from cutomer directly or indirectly. So I am sure people high up put lot of time and energy in deriving these costs, as they are the ones responsible for answering to shareholders.
When I go over quota on my current plan, I am charged at the rate of $0.15 a meg for two gig, and then capped. There's $300, or $150 for a gig.
I know you're complaining about what your IT team charge your internal department for the service. Don't whine to slashdot, whine to the CIO
Yay me!
I pay nearly $55/GB/Month for a couple applications, so $30 isn't that bad. Complaining about it based on the fact that a hard drive is $0.05/GB is an idiotic argument at best.
And I do have a chargeback breakdown that covers the entire amount, and it's well justified for a regulated enviroment that's replicated to multiple sites with dual backup and guaranted 5ms average I/O response time.
The original question should be is the value of the application and associated data worth $30/GB/month to the business? If it's a bunch of PST's and MP3's, then obviously not. If it's an order entry system, CRM, DSS, SCM, etc...then it quite possibly may very well be worth that much.
In a government agency we're charged internally $8/GB/yr for online storage. We have a hot replica ready to come online if the primary goes down for any reason, it's incrementally backed up to tape (using Tivoli) each night and the encrypted weekly tapes are transferred to an offsite location each week.
All that for $8/GB/yr. Supposedly that's the cost-neutral point for them including hardware maintenance, salaries, power, cooling, etc.
Is SOX still relevant? I thought that was on the chopping block at the legislative level. PCI is more concerning to me.
FTFBOY.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
We use a similar model, but we only get charged once. We pay somewhere on the order of $100-$150 a GB, once. That covers all costs, forever. $30/GB per MONTH is just absurd. We work in a financial fortune 500 and probably have the worst regulatory/backup burden of any issue that I am aware of (maybe legal or healthcare are worse?). You could hire a few developers for that price. This is how little shadow NAS boxes start showing up on networks and then the admins flip out. Personally, thats what I would do if I were you- just buy a NAS box and throw a few raided terrabytes in there with a backup solution (I assume you have data that is important, but NOT critical, like most orgs). If/when you get caught, it will be a wake up call to management. At least that's been my experience anyway, it always takes a few people outright rebelling against bureaucracy before they loosen the leash a bit.
An aside to this insanity is that in every IT department I have ever worked in, there has been a shortage of space. Development requires lots of space, but in the one size fits all mentality in most large firms, they can't understand why we need so much space when the sales people can store their excel sheets and powerpoints just fine in 1/2 the space. Its insane really- I can't build out of my home directory, so I have to write fairly complicated scripts to copy files over to local disks on the dev boxes and then build. This of course also means that compiling from within an IDE or even vi/emacs is impossible. I don't develop strictly on a local drive of any box because they are not backed up, the boxes frequently go down (forcing me to switch to another box), and those directories are wiped on a monthly basis.
This is why, if I ever become CTO one day, I want to actually have two competing groups within IT, that have to compete for their business from the various groups. I think the competition to bring down costs and deliver better service would far outweigh the reduced overhead of having one single group.
For some of them it's PCI, and that may actually be the bigger driver by now. Either way, it's the perceived threat of auditors rather than actual operational need :-)
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
I worked at a large CG film production company. The IT department serviced several productions with their own budgets and accounting. Their budgets had just two IT line items -- storage gigabytes and CPU hours. Few people in the productions had enough time or inclination to dig into details of support, backup, high bandwidth requirements, transaction costs, etc. They just assumed they would get the best available. Furthermore the IT departments spent considerable time and money on researching and testing “the best available”. The only way for the IT department to manage their internal budget was to take what they spent, and reverse engineer a number to charge productions for storage and CPU (which was of course about 10-20 times the off-the-shelf rate). This wasn’t a problem until budgets got severely squeezed and the storage and CPU market disparity became grossly apparent. This is apparently becoming a common problem. I’ve heard of lots of instances these days individuals and occasionally whole departments at big companies are “going rogue”, by using their own hardware or public cloud (Amazon EC2) resources -- just to avoid internal IT department policies.
I need fast servers for today's data, maybe something medium-speed for a week's data (but SATA's probably enough), and 98% of my data will never be looked at again but the rules want it online, not in a box of tapes.
You probably know this since it's your current work problem , but still:
It's a common problem, and the solution is usually called HSM (for hierarchical storage management). It can be run in the application (have the application move old data to another slower/cheaper volume) or at the SAN/NAS level. I've even heard of a system that archives to tape transparently. You see the file on the disk in whatever application you have, but actually requesting contents from it can take some time since the contents of seldomly-requested files are fetched from tape.
I think you're approaching the problem wrong. It isn't as important how much you pay for storage as it is that you don't overpay for your organization/division. Its ok for you to ask about their calculation and what went into the $30/mo figure.. but, its clear from your post that you don't really understand what that figure is comprised of.
A better approach would be to let the storage dept. know that you're challenged by these figures and you would like a better understanding of how they arrived at them. This discussion may identify services you may be receiving (such as site-to-site replication) that you don't need. They, in turn, may be able to reduce your storage bill by eliminating this overhead from your data.
On the other hand, if you approach them with 'Hey, I read around on the internet and I can get HDD's from Newegg for $1/GB!'... they're going to go on the defensive. First, they're going to be annoyed because you don't know what you're talking about and then they're going to use your ignorance as evidence that they can discount anything you say.
I can assure you... if they've taken the time to distill down a specific number ($30/GB/mo)... then, there's something that went into that calculation. Don't assume they're ignorant of their own technologies and how to deliver them.
What kind of storage are we talking about here? NAS or SAN? iSCSI or FC? I can tell you that just CapEx you can easily spend $30/GB just installing a fiber channel SAN. If you're talking the cost of the directors, fc switches, hbas, controllers and disk arrays, not to mention the SAN software licensing. You can easily spend $100K for less than 10TB of FC SAN space, not including the FC infrastructure, which is massively expensive. You'll pay easily >$300 per FC N_Port (obviously you need two ports on independent switches to feed each host for redundancy - so thats $600/host), which doesn't even consider the core directors to feed them and another $700/port for your 4-8Gb HBAs (we pay >$1300 for dual port 4Gb QLogic HBAs).
We really need more information to determine if the pricing is reasonable. Is he paying $30/GB for NAS or SAN? What kind of IOPS can it deliver? What kind of SLA? If we're talking about CIFS space for home directories, lord yes, he's paying an order of magnitude too much. If he's using it to drive a VDI or server virtualization implementation then no that's not even remotely unreasonable.
Let's not also forget that all those insanely fast disks are front-ended by massive amounts of RAM cache (many many many gigabytes) that SAN vendors charge absolutely obscene prices for. We pay over $2,000 per gigabyte of RAM cache for our controllers. It's enough to make you want to cry.
Apparently, you don't understand my point. Yes, it may very well be cheaper to re-create all the media on the dead drive than to pay $360,000 per terabyte to store it on the server. But the CIO should have his nuts cracked for forcing his customers to make that choice. If IT is making up all their expenses by charging insanely high rates for data storage, their funding model is seriously flawed.
Our shop charges a flat rate that includes basic support and services, if you need more than that you pay a nominal fee, depending on what you need. If they gave everything away except storage and tried to make it all up there, no one would use net storage and we'd have a broke IT shop.
Never let a lack of data get in the way of a good rant.
"Apparently, you don't understand my point."
I think I understand your point, probably I failed at making mine one clear.
"Yes, it may very well be cheaper to re-create all the media on the dead drive than to pay $360,000 per terabyte to store it on the server."
In fact what I said is that probably it would be more expensive to recreate the info on the dead drive than to pay whatever "monopoly money" IT asks for storage, especially considering everything else that might come within the "storage" entry in the bill.
"the CIO should have his nuts cracked for forcing his customers to make that choice"
That's considering the CIO had any option on the charge-back scheme. Maybe he has it, maybe not. On a side note, the same can be said about the other departments: if they find the storage bill to be too high, have they the ability to say "thanks, but no: I'll cover my storage needs anywhere else". In my experience the answer is problably "no" for both questions: neither the CIO nor the Depts. have any saying neither in the backcharge scheme nor in the provider selection.
"If they gave everything away except storage and tried to make it all up there, no one would use net storage and we'd have a broke IT shop."
It's not clear to me if you are talking about an internal IT dept. or an outsourced one but given that my basic premise, here and now (it may change as time and facts change), is that IT backcharging is a bad-bad-bad idea, I have no problem accepting your point: a backcharging scheme disconnected to reality is doomed to fail -no wonder since even a backcharging scheme properly related to reality is doomed to fail.
Try Adapter SAS/SATA RAID controllers with SSD caching, SSD disks are about $7/GB, so the cache are probably in that range too, not $2,000/GB.
Wish I could, SSDs aren't supported by my SAN vendor (yet). What vendors are selling tier0 SSD cache at only $7/GB? That's a great price I would think?
* Sorry, "Adaptec", not "Adapter"
http://www.adaptec.com/en-US/products/Controllers/Hardware
They actually have only 2-3 such boards, so we're not there yet. You don't need SSD disks for the caching, the cache is on board, but I just checked, it's a ridiculous 4GB for an extra $330 compared to the non-SSD cache model. So it's more like $83/GB.
4GB is what you get on hybrid SATA/SSD disks, so I guess it is not adequate for a RAID controller that can handle 256 disks.
I've read some people are starting to mix SATA and SSD: SATA arrays for backups, documents storage, etc., and SSD arrays for high IOpS data such as database read/write transactions.
Since SAS and SSD disks are basically the same price, $1.2/GB for SAS and $1.4/GB for SSD, with SSD prices dropping continuously, no mechanical parts, pure NAND memory speed, mixing SATA and SSD arrays makes sense, and you can drop global storage prices a lot with the SATA arrays.
Still, the lack of SATA III controllers, the small size of the SSD cache, and the few glitches with SSD drives after many write operations would make me wait until these issues are resolved.
SAS is doomed!
I work in one of these "corporate" IT houses and we offer disk in two ways, backed up or raw. Backed up is around $10 per GB per month with a 7 year retention period and around $2.00 per raw GB.
This is all on enterprise equipment on TIER 1 Disk FC and new generation SAS and automated ATL's. The economy comes with scale. The most expensive part is setting up and the first 10 or so TB or disk. Once you start using sensible arrays, you can get 50+ TB per array which vastly reduces your costs as apposed to say some of the older arrays which maxed out at 10 or 20TB.
With the appropriate investment in Management Tools, you can manage 200+TB of primary storage (brown spinny stuff) and 750+TB of secondary storage (tape) with only 2 admins. Again the once of costs of implementing the management software is generally given a 12 month ROI against the reduction in storage staff. (Not having appropriate management software is a common mistake and can significantly drive up the cost per GB and the time to provision).
We had a local department try to use COTS disk arrays rather than using the enterprise environment. 3TB array for $15,000.00 + Management Costs + Maintenance Costs + Hosting Costs (power, network etc) came out quite competitive. However, it didn't integrate well with AD and when they upgraded AD to 2k8 the vendor didn't release a firmware patch for about 8 months rendering the information on the array unusable (20 staff unable to do their core work for 8 months.......). When the firmware was released, it trashed the raid pool and required a rebuild (This took 2 weeks of work to troubleshoot and fix......) This is just one issue I have listed, they have had a multitude. So the net result is over the last 5 years of them using their own "Cost Effective" storage, they have paid roughly twice what they would have if they just used the enterprise storage. If nothing had broken, then yeah sure it would have been better, but lets face it, how much consumer grade stuff is built to last these days?
Here's why. Say an IT department has to meet a target of 20% profit. Consider that if an outsourced contract comes in at cost price for IT plus fifteen percent a department head is going to take that and cost his company fifteen percent. Anybody that is sensible has to act like an idiot due to a stupid policy to best use their own budget.
However a policy setting manager that acts like an adult would see that an IT department sells nothing to customers and simply consider them as a cost of running the place. Otherwise an IT section ends up wasting time on a defacto accounts and billing section and other gross stupidities.
Bill them for what is spent - but an attempt at making an internal "profit" from another portion of your own organisation is a counterproductive exercise as sane as trying to gnaw out your own liver.
Some Accountants and even some IT people love that sort of thing since they can use it to make themselves look more important.
Does it really make sense to spend the bulk of the cost on maybe a 15% performance improvement, by moving data between disks a lot?
It can, for example, if your application is spending 85% of the time working on a small subset of your storage. Move that hot data to SSD and you could revolutionise the performance profile of the app.
FWIW, pick the right I/O profile to migrate to SSD, and you can expect far more than a 15% performance improvement. OLTP style I/O profiles (many, small random I/O) with a highly parallel RAID layout (say, RAID5 15+1 or RAID6 14+2) could net you a significant performance improvement over typical traditional layouts.
It's not a magic bullet, but for the right I/O profile even only a single RAID5 3+1 raidset on SSD can demolish much bigger raidsets on magnetic disk in terms of performance.
"Necessity is the plea for every infringement of human freedom. It is the argument of tyrants; it is the creed of slaves
You're missing my point.. i'm not saying "don't use SSDs", i'm saying, don't use it as an authoritative place for data... Why not use the SSD as a cache device automatically managed by the OS of your storage hardware and always keep the authoritative info on slow storage?
For example, in ZFS world you would use appropriate SSDs as your ZIL and L2ARC, load the server with 64GB of RAM or so, and service most read IOPS directly from cached RAM.
And perform most random writes on SSD initially, to be consolidated soon and batched up as a smaller number of sequential writes to slower block storage.
As long as the cache is bigger than your small subset, you get the data you need hot, and you don't need to take time to come up with application-specific logic or manually move files, and have to keep track of what's where...
Since slow storage is authoritative, as your 'small subset' changes over time, you aren't wasitng I/Os to move files back to slow storage, you're just retiring cache pages that have cooled off, whether those cache pages be on RAM or on SSD is not that important.
Use general-purpose OS tools to keep your hottest data in RAM and on faster storage.
Then you don't need to waste money attempting to devise custom schemes which do not perform significantly better than using SSD as a cache.