Slashdot Mirror
Ask Slashdot: Personal Tape Drive NAS?
New submitter hey_popey writes "I would like to piggyback on a previous Ask Slashdot question. Do you know of any realistic way to use a tape drive solution at home, not as a backup, but as a regular NAS? I would like, for example, to save the torrents of my Linux distributions on it, and at the same time, play the family videos on a computer. It would seem at a first glance that the transfer rates and capacity of Linear Tape-Open (1.5TB, 280MB/s in 2010) and the functionality of LTFS would allow me to do that, but I don't know the details, or whether this would be economically viable."
The big disadvantage of tapes is that it has long seek times. Not 'long' as in a few times that of a hard disk, but 'long' as in: can take a full minute to do. Access of multiple files on a normal HD is done by reading a meg of the first file, then seeking to the second file and reading a meg, going back to the first file and reading a meg etc. On a tape drive, even when the seek time is only, say, 10 seconds, you'd get a total throughput of 100K/sec that way. And I'm not even talking about the havoc that using it for storage of torrent files wreaks on it: that's a random-access process if I ever saw one, and the seek times on tape would kill your bandwidth very quickly, and probably your tapes too (because of wear&tear).
you're not thinking this through, are you? it's a tape-drive...
No. Just no.
Tape is great for reading or writing sequential data but trying to access random files would suck, which is exectly how it would access files if you are trying to access movies while writing other data to it.
The only way I see it working would be to have a HD or SSD acting as a cache between the tape drive and the network.
These comments are my personal opinions and do not necessarily reflect the opinions of the other voices in my head.
I don't know the details either, but the mechanics of a tape drive kind of suggest to me that maybe the format isn't well-suited for random reads.
No. Your question has no rational purpose other than to attempt to create a corner in a circular room
As a NAS a tape drive has three flaws--
Cost.
Reliability
Software.
Tape Drives are designed as peripherals that were either reading or writing the tape media. Read/Write is not an option--- ever heard of Seek Time?
Why do you even ask this question? You are using Linux, so you are a smart enough guy to know that there are better options.
The only advantage of tape is cost/GB, but to make what you are asking to work (and at acceptable performance) you are going to waste SO MUCH of your time, which I think is more valuable to you than the few bucks you could save.
you'd need a cache drive in front of it.
the mb/s is ok yes, but that's for linear read/write from the tape.
"While specifications vary somewhat between different drives, a typical LTO-3 drive will have a maximum rewind time of about 80 seconds and an average access time (from beginning of tape) of about 50 seconds.[21][dead link] Note that due to the serpentine writing, rewinding often takes less time than the maximum."
the tape is also only good for 260 full passes.
just buy a hd based nas, archive to tape if you really archive that much stuff. but load it on hd first for gods sake.
world was created 5 seconds before this post as it is.
The determining factor is definitely cost. A tape loader or even just a single tape drive is pretty expensive, even when buying used and provided you have the right equipment to house it if it's not in its own enclosure. The price of media is comparable to physical drives of equal space. Honestly, it would be cheaper and less of a hassle to build a disk-based NAS.
Unless you stuff a large harddisk inbetween as cache, I don't see how you can make this perform anywhere near bearable.
Note that frequent write/delete cycles will fragment tape space like you wouldn't believe (perhaps a weekly tape reorganization job would be in order?).
I used to work on z/OS where using tape for normal storage isn't unheard of; typically files not accessed for a while are moved to a tape robot.
When trying accessing one of those files, it did so by writing the file back to harddisk for actual access.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
The overwhelming issues with latency aside, a 1.5TB (native not compressed) LTO drive will set you back ~1800 USD and you'll need an extra ~100-150 for a SAS controller that can drive it. For that price you can by yourself 24TB of HDD storage (12 x 2TB) with enough money left over for a decent SATA/SAS RAID controller. If you setup a RAID 10 array you'll have 12TB exponentially faster access times and better data security (unless you make copies of every tape).
No.
Hard drives are cheaper, easier, more useful.
Tape is an order of magnitude cheaper than disk, and always on hdd has spinning storage which adds up to a bill.
Some seem to think you want to use this as a HDD, though I'm at a loss as to why. Maybe they just want to call someone an idiot.
But the cost of the drive means you dobn't get much of a benefit until you have hundreds of tapes. Before then, your nas box using 200W is a bigger factor than twenty drives spinning. At a hundred tapes, your electric bill could be upped by 1kW continuous for hdd over tape. The drive could pay for itself over a couple of years in that case for your electric bill alone.
I've tried LTFS and it put the tape into such a bad state that I had to return it to maxell for replacement. Don't use LTFS if you value your data.
your best bet : Cache-a
http://www.cache-a.com
I've used it since 2009/2010, works very well and perfect for your need. Don't worry about the look of their website. The hardware is great.
If you invest enough and build your own custom hardware you could get a reel to reel tape drive, turning your living room into a 1960s mainframe.
Add a panel of randomly blinking lights and it's a retro sci-fi extravaganza!
Have you considered punch cards? You can get a vintange IBM 370 for only a few hundred thousand and a warehouse to store all the punch cards for just several million. Put it in China and you can have a few servants ravage up with forklifts and storage boxes with the cards and scramble to put them in the reader and upload it back to your home media server.
I mean who cares about using a cheap $200 external usb drive like everyone else pretending we somehow live in the 21st century ... pfft
http://saveie6.com/
HP do a LTFS which if I remember correctly treats the tape as a normal drive.
The way to do it would be to treat it as much as an archive as possible. Too many read/writes will wear out the tape in no time at all. If all you are thinking of having is a few tapes which have old torrents of films or things you would occasionally access, there shouldn't be much of an issue beyond the obvious seek times. Be prepared to have your tapes wear out and keep a good supply of cartridges and a second drive.
The obvious benefit is cost. Buy second hand. I buy second hand QICs and they are very cheap indeed, and LTO kit isn't much different.
Seriously, you're trying to use an onion when you need a screwdriver.
One word: Nope.
Forget for a moment, the cost of a LTO tape drive (easily over $1500 for a decent one as it needs to be LTO5) plus a SCSI, SAS or Fibre HBA to drive it, but then even looking at the cost of the tapes, it might be, say, $50 for an 800GB tape (with seek times measured in minutes)
Compare this to a hard drive. less than $100 for 1TB, fast seeks, decent throughput and no fancy controller, or specialised software required.
As nice as something like LTFS sounds, it has some major limitations - you simply don't use it for concurrent access to two or more files, or for random access to parts of a file.
What it's really designed for is a platform and software independent way of backing up stuff to a tape using something like good ol' tar, and bypassing the requirement for any specialised software to read your backups back from the tape. What's old is new again.
Specialist Mac support for creative pros, Melbourne
They are not only useless at home. They are completely useless as a backup solution in the first place. They refuse to read in 95% of their intended usage scenarios, including, but not limited to, incompatible/failed tape drives, missing/obsolete/buggy/outright stupid software, degraded/stretched/torn off tape, mislabeled/misordered media and so on. And then again, they cost $$$$$, because PHB's keep on buying them. And they do, because they like solid-looking stacks of backups. Even if no one prescribing them in the backup plans had ever tried to restore a single file in the last 20 years. Or ever.
Hard disks are good. They are also good for backups. They are cheap, they sell them in the shop down the street, they work 99.99% of their intended usage scenarios, do very well in every other usage scenario, and they can be easily connected to any computer, just to see what's in.
... that I consider the editors to have gotten trolled.
Anyone who seriously considers to set up some sort of NAS at home should be aware of the seek-times of tapes (let alone wear).
I might do the OP injustice, but this "Ask slashdot" reeks of the intended creation of a "completely ridiculous" proposal. It's like "I want to take a car on a roadtrip through the country, but can I use square tires instead?"
If it is not that, the OP did not even spend a fraction of a second looking into the matter himself (as in comparing prices for tapes and a HDD solution and in all honesty, what has such a question lost on slashdot? And even beyond that: Why would you even WANT to use tapes instead of a plain old HDD?
This topic here is... no matter what it is. It's silly and ridiculous.
RDX is much more cost-efficient for small setups and claims the same durability. Sure, the RDX disks are more expensive than tape but the drives are available for $60. And they act like HDD because that is basically what they are.
If an experiment works, something has gone wrong.
Obvious troll is obvious. Look at the guys posting history. He isn't this dumb..
Great way to troll slashdot though.
Consider operation. Either hire a tape operator or get yourself a tape robot. Go for IBM as it has the longest track history in saving data to tape while it still remains available as file (OK, dataset if you're picky.)
/.. You're not supposed to know everything but you should be prepared to do some research yourself. Practically disk space is dirt cheap and you could keep all your data on it indefinitely. Tapes you should only use to backup and occasionally to restore (deleted file, disk broke down, etc...) files.
Seriously though. This is
I hadn't the slightest objection to his spending his time planning massacres for the bourgeoisie... (P.G. Wodehouse)
If you've ever used a cassette deck on one of the older VIC-20's or TI-99's, you'd loose this idea real fast. Those sucked because you had to record the counter on an index card for every tape you used.
If though modern tape drives are faster, and don't have to be positioned manually, you've still got the VIC-20 problem of no filesystem to speak of. No matter what you do, it's a sequential medium. It's a stream.
Unless you are a very large business you don't need tape. And even business can make a good case not to use it these days.
Just let tape die. it was fine for it's day. but now it's time to get rid of the obsolete.
... there used to be a product called "Desktape" made by a company called Optima Software.
Basically it kept a cached (on the system drive) directory listing of all the files on the tape, and then made a (virtual) disk using that directory which was mounted on the desktop (hence the name). The user would perform file transfers with this "disk" in much the same way as he would a real disk, he could copy files to and from it by dragging and dropping, similarly erasing or copying over files. Note that I said file transfers; direct random access to this "disk", while possible, were strongly recommended against because the tape would seek to one block, then seek to the next etc. so, for example, launching an application from the tape was ill-advised. Anyway, when the tape was ejected, the directory would be updated on the tape.
Still it was great because it made backing up very simple (no special utility to run) and this disk would behave just like a real disk so that you could run regular disk utilities on it like "Virtual Disk" (which kept searchable online copies of directory listings of offline volumes).
The software was hardware agnostic which means it could work with a variety of tape drives so maybe it would work with LTO. Alas, the software only ran on pre-OS X Macintoshes and the company is long gone. I would dearly love it if someone could revive this software and make it work with a "modern" OS! Can't someone buy the IP of this company?, surely the development (patents?) is worth something. (I wish there was some sort of law saying that abandoned software like this would, after 5 years, be put in the public domain; of course for this to work the source code would have to be continually archived at, say, the Library of Congress in case of sudden bankruptcy. Not too feasible.)
Unless your application will stream in the file as fast as possible, all the time, this won't work. The tape can only go so slow; when you go below that speed, the tape "shoeshines" which rapidly wears out both the tape and the drive.
Tapes simply were not designed for applications like streaming.
My first question is why? They are open source, they tend to exist forever..... and even if somehow you are worried that you wont be able to find version x, just download the source tarballs and whatever binary blobs you need(not many). Takes up way less space and if you absolutely need version x.y.z of your distro, you can just rebuild it.
Did you know you can refrigerate your food by placing it in front of your air-conditioner?
And who needs a stove or oven? Simply wrap your food in your discarded tinfoil hats, and place it on your engine block; by the time you get to the office, breakfast will be ready.
I've also heard you can pound nails with a screwdriver if you adjust your grip...
As a co-worker of mine is fond of saying: "There are no stupid questions. Except for that one..."
He's asking if there is an opensores equivalent of HSM. I think.
All anyone can do is say how stupid he is for asking, but HSM technology has existed for, well, a long fucking time.
Does an open source implementation of HSM exist? Sounds extremely fucking useful to me.
http://en.wikipedia.org/wiki/Hierarchical_storage_management
You put a big storage array in front of the tape robot. Files are spooled off to multiple tapes and deleted from the array with a stub left in place, if they haven't been used in months. The array acts as a big cache to mitigate the latency of the robot, as all caches do.
Only useful for people who can afford robots. Honestly, you don't want lots of please insert tape X again.
What solvent are you using?
Tape is too costly and you'd need a complex set of software to handle the not-on-disk-check-the-tape-and-pull-it-to-disk. That exists for enterprises at an enterprise cost. Tapes that are used too much stretch and break. They are designed for linear access, not random access.
Another alternative would be to use Bluray Data Storage. Build a catalogue of files per disc. Store that catelog on disk and you'd be able to access it. I've been doing this on DVD media for about a decade.
Simply by storing anything on a data DVD with par2 protection to recover from bit-rot that happens on all media, but is extremely likely on optical media. Every DVD is labled with a number - 001, 002, 003 ... there is a corresponding file on every system here, dvd-001, dvd-002, .... you get the idea. A tiny bash or perl script with a web front-end lets my family easily search for recorded TV and movies. It returns the index for the disc .... 345, they go into the collection, the discs are ordered by number, pull out 345, drop it into the server and start watching anything on that disc, including the show/movie they wanted.
If any file on a disc doesn't work or there is any failure whatsoever, they tell me and I use all the tools dd_rescue, par2, etc to recover the files and burn a fresh copy of the data to new media. So far, we haven't lost any files and only 3 DVDs have needed it all this time. I started doing this in 2002 according to disc dvd-001.
The great thing is that this method scales as storage scales. Most of the media is DVD now, but it can be bluray or 3TB HDDs just as easily. I suspect when very large HDDs become $50, I'll start using those. We already have a dual eSATA "dock" to swap the disks into. Having equivalent storage to 600 data DVDs on a 3TB HDD certainly would be nice, but that is a lot of eggs in 1 basket, as Mom would say. After compression, a single layer data DVD stores about 4 or 5 recorded movies. A dual-layer data DVD stores about 8-10 recorded 480p movies. You can do the math. We're talking many thousdsands of movies, TV series, gameshows, family videos, ebooks, websites, anything you'd like to archive. Just be aware that you'd want a backup of all that stuff. HDDs definitely fail from time to time and never when it is convenient.
Today, Bluray optical data is probably the sweet spot for offline storage features and costs. Plan on about 1GB per SD movie and 3 TV show episodes fit in 1GB, so a 25G or 50G Bluray will hold a bunch.
This is not a sensible idea at all. Not just because of the access times, however, but due to the way that many tape drives write to the tape.
Each write operation to a tape moves the End Of Data mark to the position on tape where the last write operation finished. This prevents the drive from reading beyond the end of useful data. Now, if someone were to try to use a tape drive for random writes, the End Of Data mark will be in the wrong place, preventing access to the rest of the data lying beyond that point.
Tapes do still have their uses in certain organisations, but trying to use tapes as large disks is pointless from a technical point of view, unless you are prepared to restore the entire contents to disk cache in order to edit a file, and then re-write the new tape contents back down to tape in linear fashion.
About as useful as a barbecue made of ice.
No colour or religion ever stopped the bullet from a gun
Come on now, if someone is smart enough to look up the specs for the technology they surely, surely know this is a completely ridiculous suggestion for a home environment.
Yes the mammoth automated tape libraries exist(ed) for huge business over the last 50 years, however a home version which doesn't require user interaction for tape changes is madness, plus, depending on the OS - the system may well want to check / index / scan the files frequently. Is this person proposing using multiple drives? Seriously what's the deal here?
I guess if it's a troll, bravo - it's stupid yet they clearly let it on the site and I as well as others are biting.
If it's not a troll and you're legitimately asking,.... I really don't know what to say,... do you actually want a thought out response to such a ridiculous fucking question?
Editors: you're better than this, I thought anyhow,...
As everyone else, that using a collection of tapes with a single or small number of tape drives is impractical.
Instead, I recommend reading up on hierarchical storage:
http://en.wikipedia.org/wiki/Hierarchical_storage_management/
Also, if you really want a solution which does what you ask for, SamFS may be for you:
http://www.c0t0d0s0.org/archives/4240-Less-known-Solaris-features-SamFS.html/
Bye,
os10000
I would prefer not seeing these sort of silly user queries henceforth.
This might work if you have a tape changer as secondary storage and disk as your primary storage, and spool out little-used data to tape and restore on demand. I _think_ this is what the Removable Storage service was for in Windows 2000/2003, but I think it required additional software and may no longer be part of the OS. The idea is that if a file isn't accessed in a long time, it's replaced with a "stub" and moved to tape. If you access it, the file is pulled back from tape to disk. Of course, the tapes need to be available, and if you run something that tries to access every file you're gonna have a bad time.
One problem you'll find is that consumer hard drives are cheap, and there's no such thing as consumer-grade tape devices any more. Tape gear will either be expensive or used. That tips the cost/benefit heavily towards throwing disks at the problem for home use.
What kind of question is this? Seriously
I would rather have a system to emulate tape on a disk (SCSI) I have these old machines and can't get the boss to pay for anything more that DAT... So basically I have no backup at all (for those who have never used DAT it is basically write only storage.... when it is possible to write to it). The OS's/hardware is so old that it only works with tape (and some inapp backup is hard coded for tape and the source (and the company that wrote it) is no where to be found.
A quick google reveals that a 1.5TB LTO tape costs $40. I saw brand new 3TB seagate hard drives at MicroCenter the other day for $99.99. So, you want to store 3TB.
You can either do it with tape, and all the problems that implies, and a rather expensive tape drive, or you can use cheap disks for $7.75 more per TB. I'd go with the disks any day of the week.
"He who would learn astronomy, and other recondite arts, let him go elsewhere. " -- John Calvin, commenting on Genesis 1
Dreamworks used tape for their Hulk VFX. I wouldn't surprised if other studios did too for the purpose of disaster recovery.
There's a commercial software product from EMC called DiskXtender. It does exactly what you want. It's also quite expensive by home use standards. I know of no cheap/free solution that does what DX does.
For all you wannabes hating on tape: you've obviously never actually had to do a cost analysis on tape versus platter storage. LTO is fast, capacious, reliable, and shockingly cheap per TB stored. Only the drives are expensive, and the hardware life cycle on them is 3-5 years, about the same as the design life cycle on server hardware.
For highest criticality storage (Say your ERP system DB), a SAN LUN mirrored offsite is the gold standard. And it costs like gold, too. But the world (and your server room) is probably full of systems that really only need a once/day (or maybe even once/week) snapshot, and if you need data for them, you can stand to wait a bit to get it back. For both sorts of system there's the other problem: how do you handle data archiving? Our business has all kinds of contractual and regulatory requirements for long term or even indefinite data archiving.
Using platters for archiving is just dumb. They suck power, require continuous maintenance (MTBF) and generate heat whenever they're running, and you just don't need low-latency storage for last year's business records. Once you're outside the DR plan recovery window, you're down to archiving, and there's no cheaper and more stable way to archive data than to high density tape. A $20 tape in offsite storage costs about $6/year to retain and stores 1.5 terabyte natively with optional hardware compression and/or encryption. You just can't touch that with a disk drive.
Disk dedup and consolidation is, however, a wonderful thing. It means that tapes can be written at optimum data rates regardless of what goes on in backing up the client. It also means that your onsite file recovery for the last few days can happen from the disk cache, which is very popular with the userbase.
It should probably be re-phrased as "So I stole an LTO-3 tape drive from where I (used to) work, and I'm looking to re-purpose it in a NAS, because I didn't get the opportunity to steal enough hard drives because my bag was full of now de-commisioned LTO-3 tapes since we went to LTO-5 recently"
Just give back the tape drive...no questions asked.
...there's a fatal misunderstanding in this concept. I see a lot of posters saying "no no no, he just wants to backup his Linux ISOs and movies!" and fine, in that case, yes, a tape backup would be great. But to do any kind of regular access of that medium would just be tragic. See my filesystem explanation below to learn why.
Furthermore, the submitter said his intent was to download torrents ON TO the tape. Torrents are NOT sequential downloads - that is, if the beginning of the file is Packet A and the end is Packet Z, you're not going to receive everything nice and neat like A, B, C, D, and so on until you hit the EOF. It means your seeder might send you Packet F first because that's the part he has, and another seeder might send you Packet A later on whenever that becomes available.
And while, yes, tapes make a lovely BACKUP medium, they're just that - backups. Tapes are not designed to be read/write accessed 24x7 for the equivalent life of a hard disk. You write to them once a week for your nightly backup rotation, and hope to god you never have to read them.
This submitter is also completely ignoring the fact that most filesystems are NOT designed to be sequential-access. As many other posters have mentioned, "regular" filesystems access data in small random chunks. It doesn't matter if the actual file is "sequential" or not - the filesystem doesn't give a damn. It treats all files the same way. Have you ever heard the phrase "Disk Thrashing"? Yeah, try to download your torrents onto a tape drive and that's pretty much what you'll get. Forcing a "regular" random-access filesystem to operate on top of an entirely sequential storage medium will burn out your little Hipster NAS before ext3 even finishes writing the inodes.
Remember on an old manual typewriter, how you typed everything out nice and neat and sequentially? If you ever have access to one, try typing a sentence in reverse - from right to left. It's hard enough as is. Now try typing it in a random order. It ain't gonna happen before you get bored and move on to some other terrible idea with no logical foundation.
Have looked at an RDX drive ? Example - http://www.tandbergdata.com/us/index.cfm/products/removable-disk/rdx-quikstor/
"Hay guys, I was wondering if storing my porno on an abacus would be a cheap way to keep it available on the off chance I ever get an erection."
Seriously WTF slashdot...WTF man...
The most glaringly obvious reason why this won't work is highlighted in bold in your question, lets examine for a second:
"save the torrents of my Linux distributions on it, and at the same time, play the family videos on a computer"
This requires both concurrent read & write cycles to the same medium, and as Tape is ,for the most part, a sequential storage medium this would require writing to the end (empty) area of tape while trying to read from the beginning/middle (anywhere in between) data area of the tape.
So to answer your question: NO.
There are a lot of naysayers on here, but since you did not specify the capacities you are handling I'm going to assume that you are working with hundreds of terabytes, the scale at which using a tape library starts to really make economic sense. Any kind of "use tape to complement hard disk storage" scheme will use a robot-driven tape library. If you aren't in this class of solution, then the other posters are right, do not even consider going down this path, the expense ($10K USD entry level) is not worth it.
What you are looking for is called a Hierarchical Storage Management solution. They are all proprietary software (the hardware part of the solution is pretty much functionally interchangeable), there is no production-grade open source offering, which is unfortunate. The proprietary ones I know of don't allow hooks to programmatically customize inject/retrieval policies and operations, the primary reason to want an open source alternative (though Tivoli Storage Manager has an extensive API that someone could use to roll their own HSM with its own API complete with programmatic hooks).
If you find these financial requirements too onerous, then as a middle ground solution I recommend you get commercial-grade hard disks like Western Digital Black with 5-year warranties to hold everything you have, a single tape drive for traditional backups, some software that supports incremental backups, and climate controlled storage for tapes.
Your argument has some merit, but I've consistently found it not to really hold true for the smaller or mid-sized companies I've worked for.
The "high cost of entry but low maintenance cost" of tape just doesn't always pan out, IMO. For starters, you've always got the issue that the tape media isn't readable by anything except a system configured with one of those expensive tape drives designed to work with that generation and type of tape. That means if you're using those tapes for disaster recovery by storing them off-site, you've got to be sure you can read them back in if/when disaster strikes. If that disaster is, say, a fire in your building that destroys the computer equipment? Now you've got to be sure you've got quick access to a machine capable of restoring the data on those tapes so you can set up replacement systems from it. That means you may have to buy a duplicate, costly tape drive, and store it off-site with the media, or pay an outside firm to do the data recovery for you (because they have the needed equipment to read your tapes for you).
When your off-site data is stored on standard-issue hard drives, you don't need anything more fancy than an external USB hard drive enclosure to get to the data.
(And yeah, I realize you could do the accounting on something like this by claiming those extra costs are part of the "initial expense" instead of the "maintenance" portion.... but either way, you're spending the money.)
And that thing about putting a box of tapes in a panel van and bumpy roads not worrying you? Yeah, I suppose .... but any decent hard drive parks its heads when it's not spinning, and really shouldn't be affected by that sort of vibration. We've had nightly backups running here at two locations for years where the backup media is a consumer-grade SATA hard drive in a removable hot-swap drive tray, and people take them home every night by tossing one in the trunk of their car or truck. I think we've only had 1 drive fail out of the entire 2 week rotation used for both sites after 3 years or so of use. That's well within what you'd expect fro drives that never even get removed and carried around.
Back in the before time, I knew a guy who ran a BBS who came about a DAT drive by some method or another. I assume he stole it from work. Anywho, that was a lot of storage back then and he wanted to use it to store files for the BBS. What he came up with was a caching system where people tagged the files they wanted, the files would be copied from tape to temp storage on the hard drive, would be downloaded by the user, then deleted from the hard drive. I had a similar system for the CD-ROM changer on my multi-line BBS. If someone on line 3 wanted file from Disc_2 and someone on line 1 wanted a file from Disc_7, the poor thing would just thrash back and forth between discs until I added the caching system.
But it's just totally impractical today. I've got a 26tb array for my bulk storage. Even with hard drive prices still a bit inflated, it could be built for $2500 with nice drive cages. $3200 for 39tb using 3tb drives.
An LTO 5 library is going to run you $5000 for just the drive/library. You weren't going to stand there swapping tapes by hand, were you? Another $550ish to fill it with tapes. And that's only 1.5tb gigs per cartridge. (Native capacity is what you should be using for this type of data.) 16*1.5=24tb online for around $5500. Nevermind the cost of the caching system that would be needed just to make it work in even the most crude manner with a minimum of 5 minutes between initial request and the file being available for use. More than double if it spans tapes.
So roughly double the cost for a similar amount of storage with horrendous access times. Sounds like a plan. You should totally do it and report back.
What about Datman? http://www.datman.com/ Ha ha!
Buy a Synology NAS and put anywhere between 5 and 36 hard drives (depending no the model and your requirements) in it. You can even add drives ON THE FLY to expand your storage. Perfect solution of home or small office/home office backups.
"You can get a vintange IBM 370 for only a few hundred thousand"
Where? Or are you just talking out your ass?
360's and 370's are extinct.
4300's maybe, but not 370's
Really, you're going with that?
I'm a big retard who forgot to log out of Slashdot on Mike's computer! LOOK AT ME.
when I was ten year old I would see ads for syquest drives in magazines, and dreamt of 44MB or 80MB cartridges when I had a 40MB hard disk drive (indiana jones atlantis was taking 10MB from it). same thing when we had a 486 DX/2 66 with a 120MB drive and we had to choose between doom and descent. I didn't realize at the time that a crappy backup drive wasn't meant for even DOS gaming. in retrospect it sucks that we didn't get an old hard drive too (even an old by then 80MB drive) and especially it's a shame we couldn't get four 1MB sticks of memory to play duke3D on that PC.
Look into Plan 9. While it never really got off the ground commercially, it's the successor to Unix. All resources are distributed over the network & storage servers know how to manage multiple levels of storage, being able to move data from 'fast' to 'slow' as it ages.
my sig's at the bottom of the page.
Using a tape drive for NAS? Um, no:
Zathros: Time is infinite. You are finite. I am finite. This... Is wrong tool. Never use this. No no no nono.
For backups, tape isn't too awful bad. The more data you need to store, the better tape looks. But as online storage? Forget the cost for a moment and look at the technology. Tapes wear out with heavy useage, then you need a new tape and the time to put the data back on it. Where you going to stash the data temporarily til you can put it back on the tape when the old tape is worn out and unreadable? Those of us who grew up using casette-based storage for their Apple II/Color Computer/Atari machines know that tape storage takes awhile to load. There is no random seek on a linear tape. And what if you loaded the wrong tape?
For NAS, get a drive box and set them up for the appropriate RAID configuration, just remember:
RAID is NOT a backup solution. That's where your tapes come in.
Understanding the scope of the problem is the first step on the path to true panic.
Since the tapes read as they pass over, any and all pending reads will complete by the time it has moved to one end of the tape.
Since the sustained speed of a tape drive is actually faster than almost every HDD available, this means that several large files can be retrieved faster off tape than of disk.
And if it's all going over network, that's far far slower.
There are numerous problems with tape. I will list just a few:
* Tapes are sequential, and in order to list a directory, you need to read in the entire tape, THEN you get the listing
* No random access (see above) so as the tape seeks your program/stream/process will stop and wait - and wait, and wait
* Tapes are unreliable - and only become more unreliable with each use because of media dropouts, wear, not to mention tape head wear and gradual magnetization
Want a SAN? Hard disks are very cheap now. Build a cheap server using a hardware SAS RAID controller - or heck, even a RocketRAID hybrid/"fakeRAID" controller will be vastly superior to tape.
The Christian Right is Neither (Christian nor right). See: Matthew 23, Matthew 25, Ezekiel 16:48-50
Just go buy a 3TB HD for $150!
Tapes never make sense for ramdom access storage. They barely make sense for permanent sequential storage anymore.
Maybe you can buy a printer and a scanner and you can print to paper for writes and scan said paper for reads.
Really. Look behind you - there it is.
Not everything that can be measured matters; Not everything that matters can be measured.
Meanwhile my workplace is paying more than $1000/per month for 6Mb/6Mb and some of the people we want to get data to or from are not on links that fast :(
The fabled low cost/GB of tape only comes into play after you've amortized the cost of the drive over many tapes.
The long access time makes tape feasible as an online storage medium only if you have a disk subsystem in front of it and HSM SW automatically keeps frequently accessed files on disk. That SW alone costs more than a decent size NAS subsystem which also means that you have to amortize the SW cost over many drives, i.e. you need a library. It's required for reasonable reliability too.
This is big iron technology that's only used if disk would be too expensive for the amount of data you want to store, say more than a few thousand disk drives, and if the access pattern are suitable.
thegodmovie.com - watch it
Where idiots get serious answers.
test
Yes it's doable. You'd need something like SAM-QFS (recently opensourced by oracle, the parts you need only on opensolaris) to do it or you'll have rotten performance.
You'll also need a tape robot in order to have enough tape storage onhand.
As it happens, I do have a tape robot onhand (Neo4000 with LTO2 drives), but the LTO2 drives mean it's more practical at home to use ZFS and 12 drive RAIDZ2 than a hierarchical storage system (60 slots at 200Gb each=10Tb) running on tape drives which practical experience shows don't like household levels of dust - apart from the storage issues I don't want to have to run a cleaning cycle every week.
Trying to directly access using LTFS would work, but not cope well with multiple file requests. That's not what it's intended for.
As a work server, it's a different matter, especially if you have a big enough library and enough data stored to make it practical (I think the current threshold is about 400Tb, which is just over what I'm handling now on our main NAS setup)
This is like asking if it would be possible to use floppy disk to retrieve a 1MB file, I (and others) don't see the point of your question,hard disk drives are a lot cheaper, substantially faster and a lot more reliable. The time is takes for the tape drive to boot up, get mounted to the OS file system and access the area where the file is located is ridiculously long.
TOP DSLR Cameras Reviews of the top DSLRs
I have a few DTL8000 drives and over 200 used tapes. They store 40/80 GB. I'd love to have something that migrated little used files to tapes like an HSM system.
When I was a college student in the late 80s, files would migrate to "offline", which meant tape. When you requested a file, it might take a week or so. Something kept track of what file got migrated to which tape. When a user requested it, it got manually mounted, restored, and a notice got to the user. They didn't have robots, just cheap student labor.
Is there anything like this adapted to Linux? How could SamFS on a Solaris system work? Without a robot?
"Torrent" and "Tape Drive"
Clients randomly request pieces they don't have. They're specifically coded to not request sequential blocks. Speaking as someone that has written bittorrent clients, I can't think of a more horrible way to serve up a torrent than on a linear-access system such as tape. Peers and seeds seek continuously.
The only way you could possibly make this work would be to advertise you only have block 0. Then when they get that from you, announce you now have block 1 also. etc. Messy, but doable - the only way really for the seed to dictate what order the packets are served. Would probably thoroughly confuse the tracker. Once you had another seeder available in the swarm, your smartest move would be to disconnect until all the seeds were gone again.
I work for the Department of Redundancy Department.