Slashdot Mirror
Internal Costs Per Gigabyte — What Do You Pay?
CodePwned writes "I recently took over a position at a rather large company where I discovered my group was paying $30 per gigabyte per month! That's $360 per year per gigabyte to our own IT department. While I understand costs are different depending on the scale, redundancy, backup and support methods, there doesn't seem to be any good papers on what range you should expect your costs to be. So far, my research shows an average of $1 per gigabyte or less for internally hosted space. What do you pay?"
you don't want to know what people pay, you want to know what their costs are.
Bandwidth? Storage? Backup? Downloads from a particular site? What the hell are we talking about here?
I suppose it all depends on what, and how, you're measuring. Is that money spent on backup tapes, raid systems, flash drives, or what? Is that for offline storage, frame-relay throughput, ISP bandwidth. Does IP telephony get rolled up in that? The question seems a little vague to me.
When our name is on the back of your car, we're behind you all the way!
Until about two years ago we were charged about CHF10 (CHF=Swiss Franc) per 100MB per month to our internal (now mostly outsourced) IT provider, that is 3 times more than you, roughly. But I don't know what our service charges are now. On top of that we pay a flat rate of CHF600 per person for IT support per month. Doesn't matter how much you call IT. I felt like we were ripped off, but no-one seemed to care.
Performance, reliability, and price, pick any two.
High performance and reliable storage tends to be expensive.
High performance and cheap tends to require a lot of maintenance.
Reliable and cheap tends to be really really slow.
So if they are on a SAN with that one gig spread across 50 drives, there are some applications that need that speed.
Gonzo Granzeau
"Nothing the god of biomechanics wouldn't let you into heaven for.." -Roy Batty
the question
http://www.acetonestudio.com
I think the OP is talking about total cost of ownership here, not purchase price. TCO is all-inclusive, covering network bandwidth to make use of the space, backup and redundancy, paying someone to keep it running, electricity to keep it spinning, a share of a fileserver box to put it in, etc, etc.
...when you're writing a game...tweak the difficulty of "Easy" to something [your mother] can cope with. -- onion2k
I thought it was RAM
http://stephan.sugarmotor.org
$3/Gb/Month
I might be mistaken but if I'm getting you correctly "internally hosted space", about $100 for a TB drive that translates to about $0.09 per gb. If it's internal why would you have to pay per year for that?
I am Bennett Haselton! I am Bennett Haselton!
For backed up to tape storage? Storage replicated to another, remote datacenter? Snapshotted at regular intervals?
SAN storage? NAS? Direct attach? On arrays with 10 drives, 100 drives, or 1000 drives?
Fast SAS or FC drives? SATA arrays? 5400 RPM? 7200? 10k? 15k?
If you're paying $360/GB/yr for low end storage that sucks. For very high end, with replication and snapshots and the fastest drives and so forth, that's pretty high, but not an order of magnitude high.
"Depends" is your answer. Though I'm assuming you're talking about disk, not tape nor VTL. Do you buy direct from the manufacturer or through a channel? How big is your company? What's the total installed base so far? General Electric pays much less per GB than some midsize company with 100TB.
Do you mean for SATA disk in a tier2 array or SSD in a tier1 array?
Costs go up when you include snapshots and replication.
Do the editors even ask the submitter to be more specific?
We pay a one time $40 per gigabyte as the capital cost of acquiring the storage. There is no monthly cost. I think $40 is still way too much.
SpyDock: Scientific Python in a Docker container
My boss does.
Stop beeing retards and do internal invoicing. Not like the board of directors is billing "per decision". IT department as a billing self ruling department is so damn stupid. And no one seem to understand it. It is like having a fire department only going to fires that is in line with their mission statement.
And having an it department invocing per GB instead of having a budget for storage and then the company can allocate it as they please. And if someone has a bigger need, it should be a question for the company. Not a matter of giving a profitable it department.
- To understand recursion, we must first understand recursion -
It depends on what you get for that $30 per Gb/month. If the only thing you get for that is storage and support costs come out of a different pot, then you are paying too much. On the other hand if that $30 represents all of the IT budget for your company, then it might be about right (might not be as well, ther e are too many variables in that case).
The truth is that all men having power ought to be mistrusted. James Madison
Full service IT at it's best.
In the poster's case, he doesn't say exactly how they're being "charged" only that it works out to $30/gigabyte.
That's always a point of contention among mangers: who should pay what part and how much of the overhead costs because it reflects in their numbers.
RIP America
July 4, 1776 - September 11, 2001
Hi,
I am willing to bet that the "gigabyte" usage is simply a cost driver. Accounting simply needs to know how to divide up IT costs and settled on this as a cost driver, possibly one of many, to determine what it takes to support each department.
This is neither new nor entirely bad. Sometimes it is better to go with an easy-to-implement, but only partially accurate number than one that is perfectly accurate but impossible to implement.
See my journal for slashdot ID's by year. Mine created in 2005. http://slashdot.org/journal/289875/slashdot-ids-by-year
$1/gig is barely a mirrored pair array stuck in a PC. Probably not even 15krpm SAS.
What exactly do you need?
Lots of storage?
Lots of I/O?
Redundancy?
Backups?
Disaster Recovery?
People to manage it?
It all costs money. If you don't need any of that then by all means, buy a PC, stick a couple of 1.5Tb drives in it, raid 0 them and call it the department server. Of course, other people in your organisation may have mandated all of the above.
Deleted
One client of mine pays [much] more outrageous prices, one more TB and they have to build another datacenter it seems. A lot of posters have pointed out the factors that go into the storage costs equation, but with internal budgets and make believe money, it is often a political issue.
Do it yourself. Get almost a petabyte for $7867:
http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/
My answer:
67 terabytes is 67000 Gigabytes.
$7867 / 67000 = 11 cents per gigabyte.
Your mileage may vary.
Higher ed, 4gb FC EMC SAN, weekly tape backups:
First 2GB is free, after that it's $7/GB/yr
What you may be seeing, especially if you are working for a very large company, could just be a cost allocation scheme, not a real money cost as you are thinking of it. If your department brings in revenue, the organization needs to match expenses to it for purposes of Management Accounting.
For instance, imagine you know it costs $X to run one of your cost centers. That dollar amount includes everything from the manpower, the equipment, the facility...everything. Now, they need to assign these costs to the departments that actually make money in a way that makes sense. They could do this by carefully costing out each service they provide and assigning an overhead rate, blah blah. That tends to be a pain. You do it if you have to...but you try not to have to. Another, easier, way of doing it is determining a usage metric (CPU hours, GB of storage, number of tickets) and using that to determine each profit center's percentage allocation of the overall cost.
So, the $60 per GB may not even be close to a market rate for storage. However, if all the departments used twice as much storage next year, the per GB cost might fall to $31 per GB (slightly more than half to account for the fact that there would obviously be more real costs). Conversely, if you convinced your management to contract externally for storage, everyone else might find their per GB cost rise, since the fixed costs would be static.
I recently took over a position at a rather large company where I discovered my group was paying $30 per gigabyte per month! That's $360 per year per gigabyte to our own IT department.
It is incentive to keep your data free of porn, lolcats, pictures of the 2007 Christmas party, CD rips, etc. It also helps to pay for the Ferraris that the IT department drives.
When our name is on the back of your car, we're behind you all the way!
Don't think of it as $360/GB per year, think of it as $360,000/TB per year.
Great!
Except your terms of service don't appear to mention anything along the lines of availability, backups, etc..
I am a viral sig. Please copy me and help me spread. Thank you.
It's possible the "internal" cost is simply some form of rudimentary metric to measure the relative usage that is made of the IT dept by each of the other depts (so each dept is charged accordingly).
--Ivan
If this guy has started at a local govt entity or highly formalized private enterprise, I would suggest he squares himself up and forgets this sort of thing.
I have seen this directly. For example the corporate staff (including CEO) gets their wages divided into cost center charges.
Quick and dirty example: $3,000,000(corporate wages and expenses) / 500 cost centers = $6,000 per annum.
Alternatively if he wishes to keep digging, he will find himself pushed to the outer, or worse, managed out of his position.
In post Patriot Act America, the library books scan you.
The main "file server" had only 2 TB of storage, for a ~200 user site. Our department started using high resolution imagery, and we were burning through the remaining 500 GB of free space. They told us our department needed to upgrade the hardware, and offered to set up a 4 TB NAS for us for $10,000 (or maybe $20,000?) I questioned why don't they upgrade the file server and they said it was full. I found out later all the servers on site were 5 year old HP machines.
...it's the people supporting the bits. At my company, storage is also insanely expensive compared to the personal consumer space, but that's because unlike the personal consumer space, our data centers have high reliability, and lots of personnel (along with their 401k plans, insurance, office space, and other expenses) who have to be paid to support the systems.
Towards the Singularity.
Like many others are saying on here - you need to give us a lot more information here. There is a huge difference in the cost per gig between a Netgear and a NetApp. You also didn't mention if your cost analysis includes your OpEx costs (Operational Expenses) - things like hard costs (labor) and soft costs (power, hvac, floorspace, etc).
Tell us more and we'll be able to help you out better.
Ron Gage - Westland, MI
We charge people something like $30-40/GB one time cost for space on the NetApp 2020s. Reason is there's two redundant NetApps in two different buildings, one of which is also backed up to a tape library with tapes rotated out to a vault in yet another building. Also the NEtApps have space set aside to do hourly snapshots, in case you delete something. Costs just a little bit for that kind of performance and reliability. For our lesser storage system, which is basically just a supermicro case filled with 2TB WD RE4-GPs we charge like $0.20/GB. You get it on a RAID-6 system, but no backups, no redundancy, if it dies the data is gone. For desktops, well it costs whatever you can get a drive for probably $0.10/GB or less.
Costs entirely depend on the performance and even more on the reliability you demand. If you say "I can't ever lose my data," fine we can do that. However you need to be willing to drop some serious dollars. You can't have cheap and reliable.
That's the problem we run in to is users want to pay bottom dollar, but then scream and wail that it is our fault if anything gets lost. Just doesn't work that way. You have research you need stored? That's what the expensive ass NetApp storage is for. It would damn near take a nuclear strike to cause data loss there. Not willing to pay? No problem, but no bitching if your drive goes tits up.
This is just storage space, not web pages/applications, or software etc. We're talking digital assets of the company such as documents, images, videos... etc. Basic, run of the mill file storage is being priced at $30 per gig, per month. It's basically just a giant network share. It doesn't need to be co-located just your typical raid array with some method of disaster recovery.
I'm interested in what other companies charge internally for file storage.
it's really not a fair question.
some may be paying more, or yours may be high depending upon what your storage infrastructure looks like.
are you on HP EVA or XP storage? IBM SVC?
or do you just use some basic gig-e SAN, or DAS/NAS?
let's start there, and then let's talk about how your storage works. is it tiered, is it smart, is it redundant? how important is some of the data? what type of contracts do you have in place? what is your storage plugging into, and how?
answer these questions, and give your question some context.
not only is time travel possible, it's irrelevant.
At the universisty where I work. IT charges $3.00 per GB/year to store data on a NetApp SAN. It then costs you another $3.00 GB/year for backups.
NOTE: In case you're wondering the two prices are charged separtely in case you have data that doesn't need to be backed up or have data that needs to be backed up but isn't stored on the SAN.
Backup and redundancy can get awfully expensive, particularly if an online backup product like Evault is used. I don't know if its worth 30 dollars, but it's a pretty fucking good milkshake.
Hoist Number One and Number Six.
The big reason for internal IT departments to charge other departments for services rendered is this:
When it comes time for a manager to "earn" his bonus, the first thing he looks at is cutting the budget for less profitable departments.
The IT department rarely has external clients for income, but is absolutely vital to keeping the business running.
Therefore to keep some short sighted pencil pusher from crippling the company with a failing infrastructure, the IT department has to show a "profit" for the services it renders.
Let alone a SAS70.
Hoist Number One and Number Six.
Our SAN is keeping 4 copies of the data (2 local and 2 remote) and it costs us about $3.50 per GB/month over 3 years for SAN and bandwidth, but without power or data-center space factored in. I think you're paying a little too much.
----------
If your answer is Microsoft, you obviously didn't understand the question.
Per gigbyte of what? Not certain what you are talking about; in normal Slashdot style I will answer your question anyway.
It is standard practice in many large corporations for departments to pay ‘charges’ for the infrastructure and supplies they are using. Not real money, just numbers so that the bean counters can figure out what stuff costs to get things done, and to juggle the numbers to make things look better or worse (as directed by their managers).
At one place I worked they charged for server disk space. The theory being that it cost money not only to buy the disk or make it redundant, but also to back it up incrementally, offsite forever, transport it for me through the network, process the data on the servers, pay the IT staff to support me, to be trained and to go on vacations. The number usually was much higher than you’d expect, and actually included *all* IT expenses. They used disk space as a *fair* measure of my groups IT needs.
Remember too: some places will backup your PC via the network, so just because the disk is cheap on the PC doesn’t mean that is the whole expense per GB.
I don’t know what it costs these days at a big company. $30/month sounds like an old number for old infrastructure, not your desktop PC. Thank God I work for a small place now and the accounting is almost sane.
There's a big difference between TCO to the company and whatever price the IT department charges *your* department for service. IT department prices are typically based on historical costs that don't necessarily represent the Moore's Law equivalent precipitous drop in disk space costs, and are often based on gold-brick engineering practices and specialized applications.
For instance, if you're running a high-end database or an Exchange server that's supporting the whole company, it needs to have a blazingly fast SAN array from EMC or somebody, and instead of using $50 1TB SATA drives, it's using $300 15000rpm 300-Gb SAS drives with SSD accelerators and uber-fancy controllers, built into a framework that lets them do maximum IOPS and live no-performance-hit backups. On the other hand, if you're trying to back up desktop data in case of laptop failures, or provide shared file storage where people can retrieve dull bureaucratic standards documents, performance isn't critical, price and volume are, so you want a big slow cheap Network Attached Storage device packed full of TB SATA drives, with a bit of RAID to deal with the occasional drive failure, and still some kind of backup system or maybe a tape-loading robot (if tapes are still even cost-effective.)
And it's not uncommon for IT departments to charge you for the former, even if you'd rather have the latter.
I'm dealing with a variant on this problem right now, for a network management application. The servers and storage in the data center are designed for blazing speed, but the application I'm trying to support is customers who want to archive all their network event data for a couple of years to make Sarbanes and Oxley and their friends happy, so I need fast servers for today's data, maybe something medium-speed for a week's data (but SATA's probably enough), and 98% of my data will never be looked at again but the rules want it online, not in a box of tapes.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
He wants to know what prices other people's IT departments charge their users, because he thinks he's way out of line even for that. Costs are a whole nother game entirely - lots of people aren't clear about what their costs are...
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Another great way to muddy the waters is to misconstrue the question. You don't have to speculate. There's a difference between a vague question and casting a wide net. The transaction value of a single post is maximized in providing a specific answer to a specific question, whereas the transactional value of a discussion forum is maximized by having many situations and specifics put forward, including scenarios not originally envisioned.
There's at least as many undeclared assumptions in your narrow pedanticism as in the original question.
The question could be more evocative of his actual circumstance, but perhaps the poster was afraid of blowing his cover. Is the net cast on this question so broad that this can't be a useful exchange? Probably, if the question generates more complaints than contributions.
TCO is very dependent on what you're doing.
If I recall correctly, one of the large sites I worked for had about 10Gb to 15Gb online at any given time for one site. It was served off of cheap servers with IDE drives. But, since there were millions of daily visitors (somewhere between 4 million on a slow day, to over 8 millon on normal high days, and peaks well above that), the bandwidth costs were huge in comparison to anything else.
Some places, the content itself is more expensive than the bandwidth. Sometimes folks go nuts with the hardware, so a $50k server cost more than their content and bandwidth bills.
At one place, the cost of the staff well overwhelmed the cost of everything else. It didn't take a lot of space nor bandwidth, but it took a lot of people to keep it working something resembling properly.
Serious? Seriousness is well above my pay grade.
Our data center provider offers storage on their FC SAN ( > 150mbps I/O) at a cost of $2.50/GB/month and an additional $2.50/GB for backups. This includes 24x7 support, 99.99% uptime, and is hosted in a tier 3+ data center. My guess is that smaller SANs cost more per GB, but you are getting boned at $30/GB.
On the other hand, if you are requiring some sort of high performance DAS with off site replication, then I bet the cost is considerably higher.
ÕÕ
Backup and redundancy can get awfully expensive, particularly if an online backup product like Evault is used. I don't know if its worth 30 dollars, but it's a pretty fucking good milkshake.
Don't forget salary costs in IT to support it all.
That said, if IT is really using $30 per Gig then they aren't necessarily using the right online backups or tools. Redundancy doesn't cost that much. Geographical redundancy doesn't cost that much. Off-site "vaults" can be pricey, but not that expensive. If I were in the OPs shoes, I'd ask the IT department to share with me how they came up with that number. It's simple and not too confrontational. If their math is fuzzy or their numbers don't make sense, I'd follow up with them justifying that cost. It could just be that there is no IT budget, it all comes out of other budgets through this sort of thing. It could also be that someone has built a very good empire.
Well, if he is working for a bank then I think the cost per gigabyte could be justified. Lets says that the banking information for a given customer consumes 10 megabytes(this number is subject to a great variance), basically its all just text though right? Now per terabyte we have 100,000 customers. Im taking the cost per gigabyte to be cost per year, so the cost per customer is $36 per year or $3 per month. Now if you have any loans as do most people 3$ out of your monthly payment is pretty much insignificant. I would imagine that the redundancy involved here for a bank is 2 steps above insane, and it works because a bank can afford to do it.
That said, I imagine the number the author is giving us is skewed somehow to include to include the cost of something that should not have been included in the figure. Even if it is just from very expensive IT, the cost of in house software, and crazy good testing you can see that the cost per gigabyte means very little compared to the value of information on that gigabyte.
Is worst case for redundant secure two site storage for volumes over 2tbytes
If the IT department's blatant over charging causes his department to rely on local hard drives to store all their media and a drive goes pfft!, there goes a ton of man-hours. They should charge a flat rate per person, then a minimal amount per gig to encourage users to store critical data on the server, rather than on client hard drives.
Our IT shop charges $35/gig per year for raided and backed-to-tape storage. Still no bargain, but at least within the realm of sanity. If you need a lot of storage, they'll cost out purchasing and maintaining dedicated hardware for your needs.
Never let a lack of data get in the way of a good rant.
Do you usually scream irrationally angry questions at all the advertisements you see? Those are questions you direct towards sales while you are evaluating the product. Failing to address those in an advertisement is hardly grounds to get cranky.
The downside of unlimited storage is it wreaks havoc with backups, replication and other intensive tasks that have to move that storage around.
I've worked in more than a couple of places with an "all you can eat" philosophy and you end up with a data monster that's expensive and complicated to backup, replicate and work with.
And usually they grew to a size (a place I work with right now has one just crossing the 2 TB threshold) where they become impossible to deal with -- departments can't/won't shrink them because the pile of crap is so large they'd need a couple of FTEs just to go through it all.
Chargebacks make perfect sense -- business units treat IT resources like they were free and use them EXTREMELY inefficiently (scanning thousands of images at 1200 dpi to use in on-screen PowerPoints as one example). It's perfectly reasonable to allocate those costs to business units so management can really get an idea if a business unit is profitable.
He was talking about the cost of storage, not the cost of operating the entire business.
It's kind of a stretch to include every single input cost when computing "per gigabyte".
I can almost see including redundancy, but even then it ought to be on a separate line for "backups". I can see figuring in the cost of electricity for your networked storage, and maybe the cost of a backplane, but personnel costs? I don't think so.
If your comptroller wants you to include personnel in your per-gigabyte storage costs, ask him if he includes that fucking awful cologne he wears in his office supplies costs. Then kick his flabby ass for him. For me.
You are welcome on my lawn.
Fixed that for you.
Check out my sci-fi/humor trilogy at PatriotsBooks.
Is this company by any chance multinational? And is the IT department technically/legally a separate entity in another country? Internal price gouging to shift profits to $tax_haven has been done before...
________
Entranced by anime since late summer 2001 and loving it ^_^
Comment removed based on user account deletion
Certainly at some point in the past whatever you're doing and the costs behind it made sense. Other's have raised the question why or how you'd do this but none of us can answer it without knowing some extremely detailed numbers from your organization.
So, let's make an assumption: the level of complexity of your IT operations, as well as the IT overhead costs in the G&A section of your organizations budget, have remained constant since the $30 /GB rule was enacted. We're also assuming that those were the factors that led to the $30 /GB rule. (The reality is, a past IT director probably threw a dart at the wall and picked a number from thin air that he thought was reasonable.)
Therefore, you can extrapolate from that roughly what the 2010 cost should be based on Kryder's Law, which basically can be summarized as: storage density doubles annually. So, if you enacted the $30 /GB rule 5 years ago, then you should either: give them 32GB for $30 or charge them $.93 /GB.
----- obSig
We had large EMC DMX (high end) arrays, and we paid about $1000 per usable (mirror) TB of FC disk, but that was all in, and there was no reccuring costs. When there was serious investments it got more tricky (backend changes, frames full, etc). The Netapp route was much cheaper. People give me crap now at my current company because to buy a high end shelf on my netapp costs me about $14,000 which is 14 * 300G disks for speed. That comes out to $3 per gig, and that is before RAID-DP. If I go for space buying the 2TB SATA disks are cheap.
No he wasn't He was talking about the cost he pays to his internal IT for storage. In most companies using a chargeback model for IT services, it does in fact include All of the costs the GP mentioned.
He did not indicate what storage included at his company, however it is not uncommon to include redundancy and backup in some offerings.. $30 for a gigabyte of replicated tier I storage with backup is still high.
We pay about $12/GB/year for storage on 15k FC disks with RAID-DP and replicated across town. This does not include backups to tape, that's an extra fee. We are also in the process of working out lower cost storage without replication and on SAS (or SATA) disk. It's really silly to compare consumer grade USB storage to enterprise, replicated and professionally supported storage but it happens all the time.
Their IT department is probably headed by some young, hip, spikey-haired self-starter who wants to turn his department into a profit center. And by that I mean he wants to deprive other departments of their actual profit.
Can we stop treating enterprise IT services like they're luxuries that should be taxed, and instead think of them as something vital to the survival of an organization? Give them the funding they need to operate and provide their essential services to other departments. You're only over-complicating cost management and covering up your IT department's poor performance records by acting like you're bringing in money when in fact you're just moving it from the left pocket to the right.
IT will never be a profit center. Don't try to masquerade it as one.
It really does depend on what's included in that cost. There are a number of factors that may be included, many of which have already been presented, but I'll list here again anyway.
Raw storage cost of a particular type (NAS, SAN, etc.), which is always higher than consumer grade. You're paying for better performance and higher reliability, usually.
Padding for Redundancy within said array
Amortized cost of the processing unit
Network bandwidth allocation (because the network part of IT charges the storage part of IT for their bandwidth)
Amortized salary of the storage administrator(s)
Cost to backup, including possibly replication to another site (and hence doubling the raw storage cost), backup software, etc.
The list can go on and on depending on how granular your IT department is with charge backs.
Our raw storage for top tier NAS storage is probably in the $5-10/GB /year, but that doesn't include any other additional costs.
Hope this helps.
How exactly does someone "prove" to you who actually programmed their software? That's a fairly stupid question AC. Code comments doesn't prove anything. Pay slips doesn't prove anything. You'd actually need a high-def camera looking over their shoulder while they developed it, for the entire development time, and expect them to provide you with all that footage.
Angry and irrational. You should be a wiki editor!
"The true measure of a person is how they act when they know they won't get caught." - DSRilk
Actually, it's: "Quality, Cost, Time. Pick two." That's the balance every project manager has to walk when they actually "build" something, like a supercomputer and its power and cooling systems in 90 days.
EVERYTHING, gets calculated into the costs for a cost center, say like the one you associated with said supercomputer. Office space, electrical costs, office supplies, salaries with fringes, machines, PDUs, switches, cables, pizzas, sodas, t-shirts, etc., etc., et cetera! Everything that involves money attached to it. Then that amount is divided into a unit, say cpu/hrs, where cpu/hrs = 365.25 days * 24 hours * 2200 cpus = 19,285,200 cpu/hrs. So you take that huge monetary value, divide it by another really ugly number, and get a reasonable idea of what the operation costs in a billable amount folks are familiar with, i.e., $/cpu-hrs.
- Blazing fast storage using NetApp (or similar vendors) - $4/GB/year (limited to 16TB)
- The same performance from a 16TB ZFS app - $1.50/GB/year
- High-end HA storage (50-500TB in a rack, SSD caches) - $1/GB/year
- Low-end backup storage (40TB per appliance) - $0.30/GB/year
Custom electronics and digital signage for your business: www.evcircuits.com
We completely lost a server and every scrap of intellectual property on it. The backups were flawed. Nothing worked right. 30/mnth would have saved our company millions if the provider were reliable. We would have saved money if it had cost 300/gb/mnth
I'd go on a Vegan diet but the delivery time from Vega is too long. --brownkitty
I only leech.
Is the per storage unit the only metric used to derive the internal transfer "cost"? Or is it the only metric?
Hardware, backup hardware, floor space, cooling, network, external network, off site backup, transfer costs of media, depreciation, staff, software, software development, etc.
Do you provide separate funding from your cost center for the desktop, portable, phone, fax, printers, etc? Pay for per page printing in a tiered manner (desktop, group printer, black and white at the central facility, color, binding, etc.)?
What labor market are you in? Done any corporate downsizing so that fixed costs have stayed constant where usage has declined?
What I'm trying to understand is just what the per unit of storage cost has to cover.
What starts out as a simple question needs lots more background before you are able to compare apples to apples. Most IT groups don't make a profit as such but try to estimate for budget purposes whet you will need 18 months in advance of actual usage. Try it sometime and you'll understand that plucking a number is probably as good a method if done by the right person with an understanding of the organization and what it will be trying to do over the next year or so.
However, the price of $360/gb/year is still ridiculous.
First of all, a 2TB hard drive costs no more to run than an 80gb hard drive.
The equipment cost of storage space to back it up should be no more than the price of a 'duplicate' HDD to retain a copy of the data on, and the use of bandwidth to transfer that backup.
Also, the costs of a HDD and server to put a HDD in do not recur every month, they recur once every few years, and are less as time goes buy: hardware is damn cheap.
System administration might be more expensive, but reflects only an inefficiency of the IT department, and that cost inefficiency should be the IT department's problem.
... for not knowing accounting.
There are fairly well established practices for handling charge-backs between divisions in a company, and well established ways of determining the dollar value of benefit to either the providing division, the receiving division, or the overall company.
And of course there are often reasons other than the immediate bottom line for purchasing services internally rather than outside - reasons that trump the "cheapest is best" attitude.
Assuming the CodePwned is not trained in Accounting, I'd suggest that he or she politely ask the person who is in charge of finances for an explanation for the costs involved.
Three Squirrels
"If the IT department's blatant over charging causes his department to rely on local hard drives to store all their media and a drive goes pfft!, there goes a ton of man-hours."
Uh... as much as making palatable the original bill of 30$ per month per gig? If that's the case, they are only maximizing their profits in quite a free market way. After all, the whole issue of backcharging is in order to bring the marvellousness of free market within the corporation!
"Our IT shop charges $35/gig per year for raided and backed-to-tape storage."
How much does it charges per deployed PC? Per LAN cable yard? Per CPU cycle at the server room? Per port (and does it take into account if it is 10, 100, 1000, 10000MB?)? Per IP? You see the trend: there's an awful lot of things your IT dept. is not charging back to you and usually that means that your IT dept. will have to procure them anyway within a ridiculously low budget, if any at all.
First of all, a 2TB hard drive costs no more to run than an 80gb hard drive.
I would never use the $80 2TB hard drives sold today for serious storage - that's a ridiculous proposal. SAS or regular SCSI only for me. I've ordered 10 2TB drives, and 2 1.5TB drives in the last 5 months. More than half are on their way back (or already shipped back). of the 7 drives being shipped back, 4 were DOA. One lasted a whopping 4 hours of use (and 26 hours uptime). And none of the other ones lasted more than 4 months (the most "reliable" of the dead drives has been sitting here for a month waiting for me to get around to shipping it back). All from NewEgg, unopened, new in box. Seagates and Western Digitals.
So, now, look up SAS for high end server/storage solutions (or SCSI) and hardware based redundancy, or a true enterprise class hardware/software combination, and you will find that 2TB costs far more than $80.
StarTrekPhase2 - The Five Year Mission Continues!
Did you buy the Enterprise class SATA drives such as the WD RE line? Or the Seagate Barracuda ES. Those are typically 5-year warranty for 24x7x365 use. But they cost more than $80 for a terabyte but are well worth it. And HP will ship those in a lot of small servers such as the Proliant ML110 / 150 / 350. .Net and Java teams have a few handfuls of these as dev boxes and they've had to replace only a few of these over the years and have never lost a RAID.
Our
Right now, with our CDW pricing, a 1 TB Western Digital RE3 is $175; a Seagate Cheetah NS.2 SAS 600GB is $500. The Cheetah is definitely worth more
but not 3x for most of what we do.
Pain is merely failure leaving the body
No... these were just generic replacement drives for regular computers.
StarTrekPhase2 - The Five Year Mission Continues!
$30/GB/month, now I know you work at Microsoft! ^-^
I think this was about the same outrageous cost of storage in the corporate datacenter farm 2 years ago, although there were also different cheaper options for larger storage needs. And there was also a one-time setup fee before even the monthly bills...
So I made a proposal for our own department for a 8TB storage solution, consolidating several smaller file servers into one (plus another for the backup) with this DAS enclosure: http://www.pc-pitstop.com/sata_enclosures/scsat84xt.asp
Cost was about $6K for the 2 enclosures, 2 RAID SATA controllers and 18 1TB Adaptec drives. The servers were scavenged, the OS and backup software was for free, the set up was also provided by us for free. This was a one-time cost for less than a week of what the IT department would bill us, and zero hassle compared to all the paperwork required to fill in such a request, which basically forces you to overestimate the storage space need, as you don't want to repeat this horrific experience. Maintenance was supposed to be minimal, the disks were configured in RAID 6 to allow for 2 simultaneous drive failures, we had also planned for a couple of spare disks already mounted in spare trays to quickly hot-swap failed ones. Basically maintenance was required only to restore the backups or specific files to an earlier restore point in case of data corruption.
You might get a problem with corporate policies that require the use of the centralized datacenter. I left that department shortly after my proposal, don't think it went anywhere.
I'd make the same proposal today, except that, if the storage need requires it, I'd probably get 16-drive DAS towers or rackmounts with 2TB disks (just 8 disks is a waste for RAID 6), about $6K for 28TB of usable space out of 32TB raw space, plus another 28TB for the backup server set in a separate location.
That's a problem... you should be using Enterprise SATA (or SAS) drives for 24/7 storage.
There are a little bit more expensive, but they are not $360 per year per GB more expensive.
Hell, SSDs such as Intel X25-E are only like $800 for 64GB.
That's a far cry from $30/month/GB * 64 * 12 = $23,040 per year.
The electricity and one-time equipment fees to obtain a server to put the SSDs in don't explain that price either.
...to make managers in profit centers accountable for the costs that go into their profits?
Once you get out of the realm of real businesses and into the funny money realm of large corporation "profit centers" so many of the costs associated with running a real business get lost -- rents, furniture, HR costs (payroll and benefit admin overhead), utilities, and of course, IT costs.
A lot of these profit center managers are prone to treat $Profit = $income - ($payroll + $direct_expenditures) and ignore the other costs, no matter how wastefully they're incurred.
Their attitude is so we ran 200k color copies, big deal. So we've got a file & email servers with 2-3 TB of crap, MP3s, etc. So we all stream TV, audio/video and still want to download a gigabyte in 10 minutes. We insist on expensive laptops, Adobe Creative Suite on every computer, ponies, balloons, etc.
What's funny is when you work with REAL small businesses that have to pay for everything it's amazing the corners they'll cut and the sacrifices they'll make for IT (old, slow PCs, bad software, slow networks, bad/no backups, etc). It goes too far with them sometimes, but in those worlds they "get" it.
I think chargebacks are another way to keep these managers honest and in some ways, not skew upper managements view of how profitable or not the business actually is.
Quality is highly variable with new SATA 1.5TB+ drives. But if you drop down to 750 or 500 GB drives the reliability is indistinguishable from SAS/SCSI/FC. Infant mortality is about the same as SCSI/FC, and once you're past that the drives will usually outlast the machine. True SCSI and FC are still stuck at $0.50/GB for a 320GB drive. SAS isn't even worth talking about since it's the same drives with the same controllers with SAS protocol enabled. I'd rather use double the number of drives at half the price and spend the money on decent raid cards that support raid multiple redundancy. More disks = more spindles = more redundancy = better. Replacing a drive per year isn't a big issue and isn't even a panic when you can survive a double or triple failure. The only place you need to spend the money is if you really need speed. Then thirty-two 130 MB drives in a RAID 10 configuration might be warranted. But please, set aside some hot spares.
Support SETI@home
No... no I shouldnt. All of our servers run on enterprise grade SAS or SCSI.
The drives I am discussing were for customer machines. We do PC repair. We are NOT sticking an SAS drive in a piece o crap Dell Dimension.
Please read the post I first responded to. I was responding to someone else who thought that an $80 cheapo SATA drive was a valid choice of hardware to use in an enterprise storage solution, to which I responded (paraphrased) "no way, we buy tons of them and most have failed"
Sorry I didnt clarify that we buy them for regular PC repairs and use only SAS or SCSI for real stuff (or maybe I kinda did).
StarTrekPhase2 - The Five Year Mission Continues!
Yeah, $30/GB/month is kind of crazy. We bought fairly expensive midrange storage with all the maintenance and software bought up front and the cost is still only $.14/GB/month, double that and make the bandwidth as expensive as the array and you're still at only $.42, double that for backups (disk to disk with redundant Avamar's at each end) and you still are at ~$.85/GB/month, how you make something 35x more expensive than that I have no clue, unless you have a dedicated storage admin running like two arrays and his salary is part of the chargeback.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
But please, set aside some hot spares.
Please don't, use guard space like XIV and EVA do, that way rebuilds are wicked fast (for the class of drive) and you aren't wasting IOPS on idle spindles.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
$30 for a gigabyte of replicated tier I storage with backup is still high.
It all depends on what those guys in IT are doing...
If you add in the cost of videos, and look at the drive space taken after ripping, it works out about right.
Or how about an index of the data and a cache on SSDs (including indexes of all the old data) and all the real data on bulky SSD?
Does it really make sense to spend the bulk of the cost on maybe a 15% performance improvement, by moving data between disks a lot?
If you store on SATA initially and always, you never have to worry about deciding archiving procedures, only about deciding what to cache, and how to best ensure the cache is efficient and resulting in the best performance..
grf, First line : real data on bulky SATA. (replace bulky SSD with bulky SATA).
That's nothing. Our corporate IT department (we're a fortune 20 company) charges other departments in the organization 6 hours @ $85/hour to (drumroll......) create a xen guest on an existing server that's "owned" by the department being billed.
Typing "xm create foo" = $510
Then they bill the victim retail price for a RHEL license on top of that. So you'd think we'd get a little support...nope. You want to take a piss around here, you fill out a few tickets and wait until the 71st hour of their 72 hour SLA before anyone can be bothered to get off their ass to read the ticket.
So we've just been ignoring them and rolling out CentOS and staffing our own mini IT group instead. At least they'll get up off their bums when something is broken and fix it.
You're missing the point. There's a reliable, affordable midpoint between the big, cheap but, as per your experience, highly unreliable SATA drives and a server-class SAS or UltraSCSI disk and that's the Enterprise SATA drives I mentioned.
I work for a large Healthcare organisation that has a big investment in an multi-cabinet array for medical images and having Enterprise class SATA disks really hits the sweet spot for price / performance / capacity for them.
And even if the SAS solution would be 5 times faster, the real bottleneck is the network and the money saved with SATA drives means that much more 1Gib or 10Gig fiber can be installed and more spindles can be added to bump up the performance, if necessary.
It would be tough to say if this would work better than SAS or FibreChannel for something like databases or Exchange but since these images are large and static, sequential read access is what delivers the goods here not random I/O so SAS or SSDs aren't presently worth the investment.
And, getting back to your cheapo Dell Dimensions - yes it would be pretty ridiculous to put SAS in those boxes, especially since a reliable add-in disk controller would probably double the price, but, if you care about your customers data, the Enterprise SATA disks are an affordable compromise for their peace of mind.
Pain is merely failure leaving the body
You're missing the point. There's a reliable, affordable midpoint between the big, cheap but, as per your experience, highly unreliable SATA drives and a server-class SAS or UltraSCSI disk and that's the Enterprise SATA drives I mentioned.
Used em, killed em. Star Trek New Voyages: Phase 2, filming Kitumba. 3 of 8. Took two weeks.
StarTrekPhase2 - The Five Year Mission Continues!
Having done the modeling for this, here's some components to consider.
I have one example (AUD) where this works out as follows:
Do the maths above: excluding power, staff, tape libraries and rack space etc, it's already over $25 per usable GB for Tier 1 (and $13.57 in this model for the Tier 3 storage). Redundancy is baked in. Spare disks in the SAN are baked in (so a disk failure means immediate rebuild not "wait 4h until a replacement can be installed by an engineer" rebuild. And for this cost, we also have things like automatic deduplication of data (backup and online), data replication, historical backup to tape and so on and so forth.
It should, as I said elsewhere in this thread:
We bought fairly expensive midrange storage with all the maintenance and software bought up front and the cost is still only $.14/GB/month, double that and make the bandwidth as expensive as the array and you're still at only $.42, double that for backups (disk to disk with redundant Avamar's at each end) and you still are at ~$.85/GB/month.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
I think that it is likely that some idiot has decided that every department has to make a profit instead of the company as a whole making the profit. Cost centres then have to pass on everything plus an artificial markup above cost to indicate "performance" and real decisions are made on that instead of any sort of relevent information.
One striking example of this being misapplied was a group that had a legal section, an estates section and an accounts section and they all had to show a profit or lose staff. The accounts section was purely a cost centre so they charged the other two sections that actually made money and had a nice profit that enabled them to put on extra staff while there were job losses from poor "performance" in the revenue earning sections.
It's really all bullshit that has middle managers ready to kill each other in places where it happens and ultimately gets in the way of anything useful. Simplistic pepsi swilling CEOs that swan around the world like rock stars while their company burns seem to like to have that sort of environment - not that I'm going to name any paticular Mexican bandit.
Do the maths above: excluding power, staff, tape libraries and rack space etc, it's already over $25 per usable GB for Tier 1
Right, so this means that the IT department in the summary could buy a new storage system every month, since they are charging $30/GB per month.
Since an internal IT department should not be trying to make a profit, the charge should be just enough to pay for the total cost over the life of the hardware. For a 3 year life, that means that if the purchase price was $36/GB, the charge should be $1/GB per month.
Now, if you're trying to make a profit, that's a different story...charge whatever the market will bear.
Got you covered here.
http://www.tarsnap.com/
brandelf -t FreeBSD
Plan
Step one - Buy two fully equipped Drobo Pros. (12TB usable each one for redundancy)
Step two - hirer a guy to stare at them all day.
Step three - sell space to other Depts at $15 a month.
Step four - retire in Jamaca with the $180,000 a month - $10,000 for the Drobos and $50,000/year for the watcher. (thats 2.1 Million dollars the first year)
Read that as "Damn thats to much per gigabyte."
All amounts are us US Dollars:
:-)
1 esata 4 drive raid enclosure - $225
4 1.5 TB 7200rpm Seagate Barracuda drives - $320
Total: $545.00 (including shipping)
Storage using RAID 5 - 4.5TB
Cost / GB: $0.1211 (12 cents) / GB
That's about what I pay these days...
Sometimes, real fast is almost as good as real-time.
Right, so this means that the IT department in the summary could buy a new storage system every month, since they are charging $30/GB per month.
Actually, no it doesn't.
They have already paid over $25 dollars for every GB that they offer to the rest of the company.
Now, if you want to reduce these costs, you have to do a risk assessment. Under the model described in AUD above, there are at least 5 copies of a given set of data. Do you NEED the two DR copies ? Depending on the processing model, some intermediate files don't need to have any more than a single copy. In other words, storage space CAN cost $25 - $30 / GB, but thats for the rolls royce version.
Just make sure that if you ask for the Trabant version, that's really all you need.
Disagree -- with video, it becomes cheaper, because you handle more at volume; At a video service provider I worked at, our cost per (unreplicated) GB was usually ~18c to acquire and 12c to maintain. Even with large scale replication, we didn't come near 30$/GB. At another company, Netapp was the name of the game, and there 30$ (replicated) gig was about right. Obviously those aren't directly comparable, but it's more about how you use your storage (and how it handles failure) than how much storage you use.
-- (appended to the end of comments you post, 120 chars)
Do the maths above: excluding power, staff, tape libraries and rack space etc, it's already over $25 per usable GB for Tier 1 (and $13.57 in this model for the Tier 3 storage). Redundancy is baked in. Spare disks in the SAN are baked in (so a disk failure means immediate rebuild not "wait 4h until a replacement can be installed by an engineer" rebuild. And for this cost, we also have things like automatic deduplication of data (backup and online), data replication, historical backup to tape and so on and so forth.
Now amortize it over 3-5 years, account for failures/replacements and the IT staff to maintain it, M&E costs and you might get around $1.40/Month.
D.O.U.O.S.V.A.V.V.M.
...and in the ways that are best for those forced to follow the policy, not best for the company as a whole.
Some years ago, I worked for a large public utility with an internal IT department that used a chargeback model to "defray" the costs of having an internal IT department. They charged, for example, $200/month for an office to have a networked printer (yes, that was the actual number) in addition to the lease cost. Since the individual office managers having to pay for IT services out of their own budgets weren't stupid, they did the predictable thing: they went out to the computer store and bought $100 inkjet printers instead, hung them off individual machines, and shared them out. Sure, at the page counts being pushed through them, the printers only lasted 4-6 months, and the ink cost per page was not at all economical compared to a laser printer - but the cost savings to the individual office's budget was huge even if it cost the company far more money.
Further, since the cost of having IT do anything was so high, most business units developed their own "unofficial" IT staffs - people who in theory held other jobs, but actually spent much of their time doing off-the-books IT work - also bad for the company financially but a huge budget bonus for the business units involved, since the cost of having those extra people was far less than the cost IT would have charged them for the services those people provided.
Internal costs are not real money. If they charge more internal costs you company will not make more profit. The game changes immediately when you considering outsourcing the same tasks.
In any organization cost of each item/service is eventually recovered from cutomer directly or indirectly. So I am sure people high up put lot of time and energy in deriving these costs, as they are the ones responsible for answering to shareholders.
I pay nearly $55/GB/Month for a couple applications, so $30 isn't that bad. Complaining about it based on the fact that a hard drive is $0.05/GB is an idiotic argument at best.
And I do have a chargeback breakdown that covers the entire amount, and it's well justified for a regulated enviroment that's replicated to multiple sites with dual backup and guaranted 5ms average I/O response time.
The original question should be is the value of the application and associated data worth $30/GB/month to the business? If it's a bunch of PST's and MP3's, then obviously not. If it's an order entry system, CRM, DSS, SCM, etc...then it quite possibly may very well be worth that much.
In a government agency we're charged internally $8/GB/yr for online storage. We have a hot replica ready to come online if the primary goes down for any reason, it's incrementally backed up to tape (using Tivoli) each night and the encrypted weekly tapes are transferred to an offsite location each week.
All that for $8/GB/yr. Supposedly that's the cost-neutral point for them including hardware maintenance, salaries, power, cooling, etc.
You'd probably want to include a maintenance cost on the disk shelves as well and not just the controllers. While the controllers themselves may have some disk drives in them (like a ds3400 or a ds4700) the expansion shelves will need maintenance contracts as well (exp3000 and exp810). These maintenance contracts cover defunct drives and other such hardware issues. Typically the older a piece of hardware gets the more expensive the contract is to renew year after year. These typically cost from $2k to $4k per year per device depending upon how old the device is. Then like David said, this is all excluding time value of employees. $40 dollars per gigabyte per year sounds like a fairly real world number if you are in a large corporate environment. Hell, I work at a small to medium size business and we have 17 terabytes of enterprise level storage. We don't pay $680k a year for storage but we don't have the ability to restore data like a normal shop.
Is SOX still relevant? I thought that was on the chopping block at the legislative level. PCI is more concerning to me.
Whoosh! You were thinking $25/GB up-front cost, and trying to reduce it. Summary said they were paying $30/GB/month. Which, if you consider only hardware costs, quite literally means that they could buy all the hardware needed for a complete system every single month.
You still need to figure in salaries, maintenance, electricity, cooling - but all of that should not come to $30/GB/month. And yet according to the summary it does, since it would only take one month's allocation to buy all the hardware.
FTFBOY.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
So far, my research shows an average of $1 per gigabyte or less for internally hosted space. What do ou pay?"
That's the last two lines of the summary, which leads me to believe he's talking local server storage, not backup or external storage nor is he talking drive cost per gig.
One thing I can see pushing the cost per gig that high is a SAN but as he doesn't tell us what the department provides (is it a data wharehouse? Fortune 100 company?) all of that information is missing. Sorry Pal, you need to file a better bug report and ask smarter questions by including information that's required to give you any kind of explanation.
Mod me up/Mod me down: I wont frown as I've no crown
Not if the per gigabyte charge is basically a proxy or a factor chosen to apportion out the IT costs to units and departments.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
I bet that the cost figure is an antique from when disks were much lower in capacity.
Now a days terabyte is a single disk. It wasn't that long ago that at terabyte was a couple of 3U rack slots.
A lot of this depends on scale too. If you are a small division, then you don't one guy who sole job is provisioning disk space. I've found with anything with computers, it's "use it or lose it" If you aren't doing something on a routine basis, then when you do have to do something, you have to crack the docs, and review.
How much speed do you need? At one point for database access the rule was to put in as many spindles as you could afford. YOu wanted to keep the queues for each disk as short as possible
Finally how much data integrity do you need? If the entire database has to be considered a huge binary object with zero tolerance for errors you have a different level than if you are satisfied with 4 nines of your Word docs loading without crashing word.
I'm not a database admin. Just a hasbeen sysadmin. The pros can comment on other circumstances that can drive the costs up.
Third Career: Tree Farmer Second Career: Computer Geek First Career: Teacher, Outdoor Instructor, Photographer.
We use a similar model, but we only get charged once. We pay somewhere on the order of $100-$150 a GB, once. That covers all costs, forever. $30/GB per MONTH is just absurd. We work in a financial fortune 500 and probably have the worst regulatory/backup burden of any issue that I am aware of (maybe legal or healthcare are worse?). You could hire a few developers for that price. This is how little shadow NAS boxes start showing up on networks and then the admins flip out. Personally, thats what I would do if I were you- just buy a NAS box and throw a few raided terrabytes in there with a backup solution (I assume you have data that is important, but NOT critical, like most orgs). If/when you get caught, it will be a wake up call to management. At least that's been my experience anyway, it always takes a few people outright rebelling against bureaucracy before they loosen the leash a bit.
An aside to this insanity is that in every IT department I have ever worked in, there has been a shortage of space. Development requires lots of space, but in the one size fits all mentality in most large firms, they can't understand why we need so much space when the sales people can store their excel sheets and powerpoints just fine in 1/2 the space. Its insane really- I can't build out of my home directory, so I have to write fairly complicated scripts to copy files over to local disks on the dev boxes and then build. This of course also means that compiling from within an IDE or even vi/emacs is impossible. I don't develop strictly on a local drive of any box because they are not backed up, the boxes frequently go down (forcing me to switch to another box), and those directories are wiped on a monthly basis.
This is why, if I ever become CTO one day, I want to actually have two competing groups within IT, that have to compete for their business from the various groups. I think the competition to bring down costs and deliver better service would far outweigh the reduced overhead of having one single group.
How many IOPS do you need? With that kind of budget, you can sprinkle enterprise SSDs all over the place. One terabyte of storage with enterprise Intel SSDs (16x) goes for $8,384. You can have triple redundancy on the drive level and triple redundancy on the server/city level and still have spent a fraction of the $360k. You can do daily non-incremental tape backups, thinning out your backup frequency as they get older (keep daily backups for the first week, then weekly for the first month, then monthlies for the rest of the time, re-using carts as they are thinned out), and still barely scratch the surface of $360k (LTO tapes are cheap). You can keep tons of spare parts on-site. I mean, at some point, your degree of overbuild is so insane that it's excessive. Disks are cheap. Everything else about your data isn't cheap, but it's not exorbitant either.
To answer your first question (about IOPS), the theoretical performance of each 16-drive array would be about 192512 IOPS (8KB random writes). Times three per array, times three per server, and the total theoretical IOPS of the system that I described (assuming the rest of the hardware can deliver it) is 1.7 million IOPS.
For some of them it's PCI, and that may actually be the bigger driver by now. Either way, it's the perceived threat of auditors rather than actual operational need :-)
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
I worked at a large CG film production company. The IT department serviced several productions with their own budgets and accounting. Their budgets had just two IT line items -- storage gigabytes and CPU hours. Few people in the productions had enough time or inclination to dig into details of support, backup, high bandwidth requirements, transaction costs, etc. They just assumed they would get the best available. Furthermore the IT departments spent considerable time and money on researching and testing “the best available”. The only way for the IT department to manage their internal budget was to take what they spent, and reverse engineer a number to charge productions for storage and CPU (which was of course about 10-20 times the off-the-shelf rate). This wasn’t a problem until budgets got severely squeezed and the storage and CPU market disparity became grossly apparent. This is apparently becoming a common problem. I’ve heard of lots of instances these days individuals and occasionally whole departments at big companies are “going rogue”, by using their own hardware or public cloud (Amazon EC2) resources -- just to avoid internal IT department policies.
I need fast servers for today's data, maybe something medium-speed for a week's data (but SATA's probably enough), and 98% of my data will never be looked at again but the rules want it online, not in a box of tapes.
You probably know this since it's your current work problem , but still:
It's a common problem, and the solution is usually called HSM (for hierarchical storage management). It can be run in the application (have the application move old data to another slower/cheaper volume) or at the SAN/NAS level. I've even heard of a system that archives to tape transparently. You see the file on the disk in whatever application you have, but actually requesting contents from it can take some time since the contents of seldomly-requested files are fetched from tape.
I think you're approaching the problem wrong. It isn't as important how much you pay for storage as it is that you don't overpay for your organization/division. Its ok for you to ask about their calculation and what went into the $30/mo figure.. but, its clear from your post that you don't really understand what that figure is comprised of.
A better approach would be to let the storage dept. know that you're challenged by these figures and you would like a better understanding of how they arrived at them. This discussion may identify services you may be receiving (such as site-to-site replication) that you don't need. They, in turn, may be able to reduce your storage bill by eliminating this overhead from your data.
On the other hand, if you approach them with 'Hey, I read around on the internet and I can get HDD's from Newegg for $1/GB!'... they're going to go on the defensive. First, they're going to be annoyed because you don't know what you're talking about and then they're going to use your ignorance as evidence that they can discount anything you say.
I can assure you... if they've taken the time to distill down a specific number ($30/GB/mo)... then, there's something that went into that calculation. Don't assume they're ignorant of their own technologies and how to deliver them.
We're paying about $1000/GB on our SAN.
That looks like a hard number but it isn't. OP says $30/GB/month. You say $1000/GB. The time dimension is missing. Accounting rules vary, but simply dividing $1000/GB by 5 years gives less than $17/GB/month. Then discuss what you get for that, and that's where OP's information is lacking . . . Though he's paying too much if he's getting anything less than "This is where you put the data you need all the time, any time, with milli-second delays, otherwise your clients leave, sue, your company goes under, and your next job is in a fast-food joint".
What kind of storage are we talking about here? NAS or SAN? iSCSI or FC? I can tell you that just CapEx you can easily spend $30/GB just installing a fiber channel SAN. If you're talking the cost of the directors, fc switches, hbas, controllers and disk arrays, not to mention the SAN software licensing. You can easily spend $100K for less than 10TB of FC SAN space, not including the FC infrastructure, which is massively expensive. You'll pay easily >$300 per FC N_Port (obviously you need two ports on independent switches to feed each host for redundancy - so thats $600/host), which doesn't even consider the core directors to feed them and another $700/port for your 4-8Gb HBAs (we pay >$1300 for dual port 4Gb QLogic HBAs).
We really need more information to determine if the pricing is reasonable. Is he paying $30/GB for NAS or SAN? What kind of IOPS can it deliver? What kind of SLA? If we're talking about CIFS space for home directories, lord yes, he's paying an order of magnitude too much. If he's using it to drive a VDI or server virtualization implementation then no that's not even remotely unreasonable.
Can I ask how you came up with a monthly cost? Did you take the CapEx and divide it by the expected life of the storage system?
Let's not also forget that all those insanely fast disks are front-ended by massive amounts of RAM cache (many many many gigabytes) that SAN vendors charge absolutely obscene prices for. We pay over $2,000 per gigabyte of RAM cache for our controllers. It's enough to make you want to cry.
Apparently, you don't understand my point. Yes, it may very well be cheaper to re-create all the media on the dead drive than to pay $360,000 per terabyte to store it on the server. But the CIO should have his nuts cracked for forcing his customers to make that choice. If IT is making up all their expenses by charging insanely high rates for data storage, their funding model is seriously flawed.
Our shop charges a flat rate that includes basic support and services, if you need more than that you pay a nominal fee, depending on what you need. If they gave everything away except storage and tried to make it all up there, no one would use net storage and we'd have a broke IT shop.
Never let a lack of data get in the way of a good rant.
Wait how in the world is the network the bottleneck? Are they just too cheap to upgrade or is this storage accessed via applications across a WAN? Forgive me it's just very rare to hear someone say that the network is the bottleneck in the age of 60Gb/s infiniband, 10Gb Datacenter Ethernet and 8Gb/s FC.
No you didn't. Not unless your definition of "mid-range" is 1Gb ethernet iSCSI and 7200RPM sata disks. Even then I would find it highly unlikely.
"Apparently, you don't understand my point."
I think I understand your point, probably I failed at making mine one clear.
"Yes, it may very well be cheaper to re-create all the media on the dead drive than to pay $360,000 per terabyte to store it on the server."
In fact what I said is that probably it would be more expensive to recreate the info on the dead drive than to pay whatever "monopoly money" IT asks for storage, especially considering everything else that might come within the "storage" entry in the bill.
"the CIO should have his nuts cracked for forcing his customers to make that choice"
That's considering the CIO had any option on the charge-back scheme. Maybe he has it, maybe not. On a side note, the same can be said about the other departments: if they find the storage bill to be too high, have they the ability to say "thanks, but no: I'll cover my storage needs anywhere else". In my experience the answer is problably "no" for both questions: neither the CIO nor the Depts. have any saying neither in the backcharge scheme nor in the provider selection.
"If they gave everything away except storage and tried to make it all up there, no one would use net storage and we'd have a broke IT shop."
It's not clear to me if you are talking about an internal IT dept. or an outsourced one but given that my basic premise, here and now (it may change as time and facts change), is that IT backcharging is a bad-bad-bad idea, I have no problem accepting your point: a backcharging scheme disconnected to reality is doomed to fail -no wonder since even a backcharging scheme properly related to reality is doomed to fail.
3 large hospitals - average age 40 years. Hundreds of closet switches between 4 and 12 years old so the gigabit adapters in the desktop are pretty much wasted for the moment as switches support Gigabit Ethernet access ports weren't affordable until recently.
In many places, the cabling can't support anything above 100 Mbps or needs to be replaced. And we're gov't funded so we don't have infinite cash.
Any money we save not putting the latest,fanciest, most overpriced , rotating-rust coated storage into the data center is money we can spend improving the last-mile ( or last 100 meters )
Pain is merely failure leaving the body
Wow, where do you come up with $8.6/GB?
Hitachi's 300GB SAS disks are about $350, so that's $1.2/GB raw.
If you are talking usable space, say you have 10 disks per shelf combined in RAID 6, that's 7 usable disks, 2 for data parity, 1 as hot spare. Add another shelf for backup, that's still only $3.3/GB with RAID redundancy, backup and hot spares (probably removing another $19.5/GB from the lines for backup disk, DR backup disk, DR disk, DR backup disk, and the tapes, that really don't scale for big datacenter. $3500/TB tape "software"?
SATA III enterprise-level disks have the same 64MB cache as SAS disks and will probably have about the same IOPS once SATA III RAID controllers come along, plus controllers already use SSD caching for even better performance, so the cost of the same configuration as above is only $0.2/GB, for 14TB of usable space in 1 shelf, compared to 2.1TB with the Hitachi SAS disks.
Now that SSD is starting to mature, I think SAS SANs are doomed. SAS will never reach the same performance and reliability as SSD (no mechanical parts) for the same price. Even now, SSD raw disk cost is $1.4/GB ($700/512GB) compared to SAS $1.2/GB.
Try Adapter SAS/SATA RAID controllers with SSD caching, SSD disks are about $7/GB, so the cache are probably in that range too, not $2,000/GB.
Sounds like an architectural problem. Have you considered moving their operating environments closer to the storage using something like Virtual Desktops or Citrix Presentation Server? You'd only need about 30kb/s to each endpoint and you could provide massive throughput easily between app servers and storage in the datacenter.
Wish I could, SSDs aren't supported by my SAN vendor (yet). What vendors are selling tier0 SSD cache at only $7/GB? That's a great price I would think?
HP EVA 6400 replicating to a mix of storage platforms at DR using NSI Doubletake. The primary site array is using 88*450GB 15k FC drives. 30TB usable for just over $250k, comes out to $.14/GB/month over the 60 months we bought warranty for it (included in purchase price).
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Our Citrix farms are already maxed out and we are paying close to $500,000 annually in Citrix licensing. Also a lot of the medical imaging workstations have instruments that need to interact with the images.
And I don't think 30kbps per client is enough anyway for visual examination of these images - they are being used for patient diagnoses.
Also, most of the medical imaging workstations have been replaced in the last 18 months and are pretty beefy.
Also, the Citrix farms are in a datacenter 30 miles away with a 1 gigabit fiber link while the Imaging storage is in an onsite datacenter.
Spending money to buy and build a new Citrix farm and increasing the MetroLAN link would likely benefit only the medical imaging folks.
Fixing and upgrading the local network helps everyone and we can't afford to do both at the same time.
Pain is merely failure leaving the body
Yes, TCO would be higher but since storage admin is only a few percent of a FTE for us it's negligible, power and cooling has some affect but it's dwarfed by CapEx so it doesn't change the order of magnitude problem. We rolled 60 months of maintenance into the CapEx so there's no additional cost there.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
You forgot: fiber infrastructure, administrators, money to audit/secure/acquire, power, hvac, square footage to host it. That's just off the top of my head I'm sure there's far more we're both forgetting.
You're only describing CapEx to acquire a storage system not the total cost of ownership. That's what I'm trying to illustrate.
Wow $500k to Citrix isn't cheap. How many users? What licensing level? Without divulging too much information we pay about $50k/year/1000-users to citrix if I remember correctly. You really need more than 100Mb fast-e for imaging software? What kind of bandwidth requirements does it have? I'm just really curious now (I work in healthcare as well).
What I can tell you is that our largest farm usually has 2700 concurrent users from about 8am-3pm , with, of course, a big dropoff at lunchtime. We have 4 or 5 others but they are much, much smaller.
We've never been given hard numbers on the bandwidth requirements for the imaging programs but they are always complaining about speed; a good deal of it is likely caused by duplex mismatches which we are cleaning up but there are thousands of desktops and switchports to change. Part of the problem is that we have a managed services provider who's just been raping us for years and has let a bunch of stuff go to hell.
But, the contract is almost up, we're giving them the heave-ho and I hope things will start improving soon.
The fly-in-the-ointment is that the offsite datacenter belongs to them and we have to get our stuff out by the end-of-contract date, which is why there are no plans to add any new servers, except locally, until next year.
Pain is merely failure leaving the body
* Sorry, "Adaptec", not "Adapter"
http://www.adaptec.com/en-US/products/Controllers/Hardware
They actually have only 2-3 such boards, so we're not there yet. You don't need SSD disks for the caching, the cache is on board, but I just checked, it's a ridiculous 4GB for an extra $330 compared to the non-SSD cache model. So it's more like $83/GB.
4GB is what you get on hybrid SATA/SSD disks, so I guess it is not adequate for a RAID controller that can handle 256 disks.
I've read some people are starting to mix SATA and SSD: SATA arrays for backups, documents storage, etc., and SSD arrays for high IOpS data such as database read/write transactions.
Since SAS and SSD disks are basically the same price, $1.2/GB for SAS and $1.4/GB for SSD, with SSD prices dropping continuously, no mechanical parts, pure NAND memory speed, mixing SATA and SSD arrays makes sense, and you can drop global storage prices a lot with the SATA arrays.
Still, the lack of SATA III controllers, the small size of the SSD cache, and the few glitches with SSD drives after many write operations would make me wait until these issues are resolved.
SAS is doomed!
I suppose you are referring to the Seagate Barracuda 1.5 and 2TB disks built in China which experience all of DOA, infant and teenage mortality.
The ones built in Taiwan, especially the enterprise-level XT version with 64MB of cache don't have this problem. Interesting customer comments on newegg about this issue. That 2TB drive is twice more expensive at $200, but that's still 12 times cheaper than the $650-700 required for a 500-600GB SAS or SSD drive.
RAID 10? I wouldn't use anything less safe than RAID 6/60 for my precious data. Hot spares are great and work fine for RAID 5/50, I am still trying to find out if Adaptec RAID controllers manage hot spares in RAID 6/60. They are less important since RAID 6 allows 2 simultaneous drive failures, but still, at a cost of only a $200 disk, it saves a lot of technician time.
The infrastructure, but not HBA's was included in that price, as I said administration is a couple hours a month or less on average so not a major cost factor. Power and HVAC are real costs but are dwarfed by capex whenever I've done a detailed TCO calculation to the point where I don't consider them a major factor (IE I would never not do a project that otherwise made sense due to the fractional addition to TCO for them).
But my overall point is that if my capex cost is $.14/GB/month charging more than 200x that is just insane.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Here's why. Say an IT department has to meet a target of 20% profit. Consider that if an outsourced contract comes in at cost price for IT plus fifteen percent a department head is going to take that and cost his company fifteen percent. Anybody that is sensible has to act like an idiot due to a stupid policy to best use their own budget.
However a policy setting manager that acts like an adult would see that an IT department sells nothing to customers and simply consider them as a cost of running the place. Otherwise an IT section ends up wasting time on a defacto accounts and billing section and other gross stupidities.
Bill them for what is spent - but an attempt at making an internal "profit" from another portion of your own organisation is a counterproductive exercise as sane as trying to gnaw out your own liver.
Some Accountants and even some IT people love that sort of thing since they can use it to make themselves look more important.
Does it really make sense to spend the bulk of the cost on maybe a 15% performance improvement, by moving data between disks a lot?
It can, for example, if your application is spending 85% of the time working on a small subset of your storage. Move that hot data to SSD and you could revolutionise the performance profile of the app.
FWIW, pick the right I/O profile to migrate to SSD, and you can expect far more than a 15% performance improvement. OLTP style I/O profiles (many, small random I/O) with a highly parallel RAID layout (say, RAID5 15+1 or RAID6 14+2) could net you a significant performance improvement over typical traditional layouts.
It's not a magic bullet, but for the right I/O profile even only a single RAID5 3+1 raidset on SSD can demolish much bigger raidsets on magnetic disk in terms of performance.
"Necessity is the plea for every infringement of human freedom. It is the argument of tyrants; it is the creed of slaves
You're missing my point.. i'm not saying "don't use SSDs", i'm saying, don't use it as an authoritative place for data... Why not use the SSD as a cache device automatically managed by the OS of your storage hardware and always keep the authoritative info on slow storage?
For example, in ZFS world you would use appropriate SSDs as your ZIL and L2ARC, load the server with 64GB of RAM or so, and service most read IOPS directly from cached RAM.
And perform most random writes on SSD initially, to be consolidated soon and batched up as a smaller number of sequential writes to slower block storage.
As long as the cache is bigger than your small subset, you get the data you need hot, and you don't need to take time to come up with application-specific logic or manually move files, and have to keep track of what's where...
Since slow storage is authoritative, as your 'small subset' changes over time, you aren't wasitng I/Os to move files back to slow storage, you're just retiring cache pages that have cooled off, whether those cache pages be on RAM or on SSD is not that important.
Use general-purpose OS tools to keep your hottest data in RAM and on faster storage.
Then you don't need to waste money attempting to devise custom schemes which do not perform significantly better than using SSD as a cache.