Slashdot Mirror


Ask Slashdot: >2GB Backup Software for Linux?

Fer asks: "Are there backup program for Linux that do not have filesystem or volume size limits? I am trying to make a full backup of a 22 GB FTP server, using 4 GB TR-4 tapes. I have tried tar, dump, afio, taper, and afbackup, and every one of them either did not allow >2GB volumes or had weird problems with >4GB filesystems. Currently I am using dd to do the job, but I think there must be another option. Any suggestions on free programs which I may use?"

173 comments

  1. No backup solution - not enterprise ready by Anonymous Coward · · Score: 0

    Yet another reason why Linux is not ready for bigtime enterprise business. We have around 50 NT workstations, five UltraSparcs for e-mail and firewalling, and around ten boxes running Linux 5.2 for misc stuff. Two DATs for backups for the Sparcs and three DLT for all NT workstations. But how am I supposed to backup Linux? Wait ages for tar? Textmode amanda? I tried kdat but it lacks most functionality I need.

    So I export the Linux boxes' drives through NFS and SMB and back em up like the NT machines with Seagate Backup or via the Sun's DAT drives. Works. But that is ugly.

    Linux is ok for medium size internet servers, but to actually get work done, I choose NT for its applications and Sparcs for its lack of people pulling into too many different directions.

    One plus for Linux though, most stuff is free and easy to get from the web. But it is only free if your time is worth nothing. To quote a famous person.

    MGK

    1. Re:No backup solution - not enterprise ready by HoserHead · · Score: 2
      There are a couple of problems with this post.
      1. ...around ten boxes running Linux 5.2: Red Hat Linux 5.2, I assume - or perhaps SuSE. There is no Linux 5.2 as of yet, though - it's 2.2.
      2. Wait ages for tar?: I haven't noticed tar being slow. What in particular is wrong with it for backup? That's kind of its point - tape archiver.
      3. Textmode amanda?: What's wrong with textmode? If something does the work, why is it bad if it's textmode?
      There are plenty of people talking about how GNU tar is perfect for what they need. I'm not sure exactly how the lack of GUI backup tools somehow makes Linux not 'enterprise-ready' - perhaps you'll enlighten me? After all, if there is a deficiency in this department, it takes people who actually need to use this sort of thing to point it out so it can be fixed.
    2. Re:No backup solution - not enterprise ready by CelestialScum · · Score: 1

      Textmode == No good ?? I got two words: Crontab, Scripts.
      Now, how can that really be beaten ?? GUI only limits your options in this, since you are reliant on what the author decides to put in there (if you don't plan on re-writing it yourself of course), and GUI also has the nasty habit of being resource taxing to keep running on a high-useage server.
      Better to tweak all your compiles and tune your regular commandline programs and deamons..

  2. Nice troll, mister AC by Anonymous Coward · · Score: 0

    "wait for tar?"

    um, they're all slow.

    "textmode amanda?"

    uh, what's wrong with a textmode app?

    sounds more like whining. if you think it's so deplorably non-enterprise-ready, either go look for the professional stuff that's been around since '79 (look at lone-tar's webpage!) or write your own. your whining doesn't help answer the man's question.

    by the way, backing up via SMB is A Bad Idea because you lose the true file attributes (on the native filesystem) in the export.

  3. Re:Linux's 2GB limit by Anonymous Coward · · Score: 0

    That bytes. Here I am complaining that M$ limited AVI files to 2 GB by using just a long signed pointer, when ext2 couldn't even handle a larger file anyways... I'm not gonna have to try another OS on my server now, am I? Maybe I should just switch to BeOS for my video stuff and be done with it... :-(

  4. NT: Not enterprise ready. by Anonymous Coward · · Score: 0

    We could write books about why NT is not ready for the enterprise. And the only reason why NT is cheaper than UNIX is if the time of those who administer it is worth nothing. Wait; NT isn't even cheaper than UNIX in this case. Look at the price of windows terminal server compared to X windows! NT is not even an enterprise OS. Hell, you can't even remotely administer it or attach large numbers of users to it. It's not even a multiuser OS. How the hell could it even pretend to be an enterprise OS?

  5. Re:Why Tape? by Anonymous Coward · · Score: 0

    Yes, but my point was that if your crontab backed up your filesystem to your second hard drive after the file was deleted/modified then it is gone forever while someone making tape backups will still have a tape from a while ago since tape users keep more than one backup of their filesystem while hard drive backup users have only the latest backup.

  6. Hmmmmmmm, you are just probably really stupid. by Anonymous Coward · · Score: 0

    I'm getting the impression that you don't really
    have anything to do with backup. I wrote a very
    elegant perl script to backup about .5 TB of
    customer data using convenient 2 gig logical tape
    size. The logical tapes get written to a 15 tape
    stacker with a DLT 7000 drive in it. All linux.
    It's very very fast. restores are very fast since
    I log where all the data goes. I just have to put
    in the tape, forward to the right logical tape,
    then select the file I need to restore. Of course,
    I've only ever done this in practice. Since we
    run linux, there has never been a crash that wasn't fixed after the raid un-degraded itself.

    ; )

  7. I've used Legato Networker and it sux by Anonymous Coward · · Score: 0

    They used it at my former place of employment. It was a piece of crap.

  8. multi volume through pipe by Anonymous Coward · · Score: 0

    I have not done this, but this should work for piping multi volumes to your cd-writer.

    You want to pipe the output of tar to a program which opens a pipe to your cd-writer and writes data until it's written a CD's worth (650M?) then it closes the pipe and waits for you to replace the CD and press enter. It would be relatively simple to code:

    while(there_is_data_on_stdin)
    {
    open_pipe(cd_writer_program)
    while(less_than_650M_written)
    {
    read_from_stdin
    write_to_pipe
    }
    close_pipe(cd_writer_program)
    if(there_is_data_on_stdin)
    {
    print "CD Written, replace CD and press enter"
    wait_for_enter
    }
    else
    {
    print "Done"
    }
    }

  9. Re:How many files are you backing up? by Anonymous Coward · · Score: 0

    If you're using GNU tar, just go

    tar -cf my.tar /files/parta
    tar -Af my.tar /files/partb

    ie append instead of creating several archives.
    Unless you want to create several archives for another reason.

  10. Re:Why Tape? by Anonymous Coward · · Score: 0

    While others have pointed out some good reasons to use tape for backup, one of my biggest concerns about using a hard disk is the sensitivity of the drive. Tape drives are typically a lot less sensitive to a variety of forces than are hard disk drives. And I would be less concerned about dropping a tape than dropping a disk drive.

    Where I work we just talked a department out of using a Jaz drive for their database backup. We have a contract with a storage service to come pick up our tapes daily and store them offsite. They will guarantee that the tapes will be transported and stored safely. They cannot make the same guarantee with hard disk backup solutions.

    Yeah, tapes are old. But if you have a spiffy DLT drive then backups really don't take too long.

  11. BRU 2000 Personal Edition comes with Redhat... by Anonymous Coward · · Score: 0

    I've used BRU 2000 PE (which is bundled with the "Official" Redhat box) several times and had no troubles with my 4G filesystem (or with spanning across multiple 2G DAT tapes).

  12. Re:2gig backup by Anonymous Coward · · Score: 0

    Software compression is not needed no more.. most tape back up have their own compression.. like the DLT Where is DLT support for Linux anybody?

  13. Re:linux problems with files > 2Gb must end. by Anonymous Coward · · Score: 0

    First off I'm assuming you mean Linux.. and 2nd off I had a 8.4G partition that worked just fine and dandy under linux (along with a 6.4 and a 2)

  14. Re:linux problems with files > 2Gb must end. by Anonymous Coward · · Score: 0

    You have an 8.4 gig partition, but the poster was talking about file sizes not filesystem sizes. You can not (not without some tweaking anyways) have a file larger than 2 gigs with linux, your filesystem can be considerably larger however.

  15. Re:linux problems with files > 2Gb must end. by Anonymous Coward · · Score: 0

    This is real sad. Other Unices (the BSD's for one) allow you to have files larger the 2GB. My place of work regularly deals in files of the 40GB mark (seismic data, if you must know). This is one case where purity has gotten the better of common sense.

  16. No ext3 needed: ext2 does HUGE files by Anonymous Coward · · Score: 0

    Want a 10 GB file on ext2? No problem, but you
    will need 64-bit hardware to do it. Get an Alpha,
    UltraSPARC, or Merced prototype. I'd say your best
    choice for now would be the Alpha.

  17. Intel Limitations by Anonymous Coward · · Score: 0

    It has *nothing* to do with purity and everything to do with the limitations on Intel.
    Blaming the kernel developers for things beyond their control will get you nowhere.

  18. Re:what kind of tar? by Anonymous Coward · · Score: 0

    Was this on a linux system? If so what was the modle of this changer and did you need kernel support for it? I have been trying to find a good changer for linux and would like user input.

  19. Re:tar | dd by Anonymous Coward · · Score: 0

    Here's what I do:

    ls -lR > __Index__
    tar cvvfz /dev/nst0 ./*

    :)

    BTW I wrote a SCSI-Tape-HOWTO and gave it to the metalab.unc.edu/pub/Linux/docs/howto maintainer but it hasn't appeared yet... maybe I'll send it again, some of you might find it useful

  20. Re:ADSM by Anonymous Coward · · Score: 0

    ADSM sucks!

    No really - and not just because it costs shitloads of money...

    ADSM may be useful for a centralized backup of serveral (not too many) heterogenous machines (Unices, WinNT boxes, Novell-servers, IBM Mainframes...).
    But it is not very useful for Database Online-Backup or other non-mainstream Unixes like DG/UX or Irix (ADSM client way too buggy).

    If you want to spent some money on a good Backup solution (centralized/decentralized), look at Veritas Netbackup or Networker. Both do support Linux as clients.

  21. Re:Legato Networker by Anonymous Coward · · Score: 0


    I wish i had such a good experience as yours. In our case we had it for a year or more before i came to my current company. It never worked reliably in a NT machine when i was here, and as far as i know, never did well in the past. We couldnt trust it sometimes to get the files we wanted. Archive tapes sometimes would be lost with strange errors, leaving us sucking our thumbs. We migrated to some Linux servers with 150GB raid sets on them. We tried the unsupported Networker client for Linux, but it didnt work with the latest 5.5 Networker. Then we tested ARKEIA. Installed first time, configured without looking to the manuals, made my first backup imediately, and recovered files at ease. Did that for two weeks. Not a single problem till now. And cheaper too. We are selling our Networker junk (CDs, manuals and licenses (NT server,Unix&NT clients)) cheap now, mail me at pmanuel@iname.com if you want them.

  22. Re:2gig backup by Anonymous Coward · · Score: 0


    What dlt support? It supports a tape device and and a tape control device. The rest is up to the backup application, i guess. At least i have a ADIC VLS DLT400 tape library in Linux and it works flawlessly, you just need to tell Arkeia (our software) the devices (dev/sga & /dev/st0 for instance) and the dlt library tape and you are done.


    Pedro

  23. Re:Backups & dump by Anonymous Coward · · Score: 0

    Yes its reliable until you come to restore then find that there was an error on the tape and the WHOLE of the tar backup is useless! Dont trust
    important backups to tar!
    Mike

  24. Re:Snapshot filesystem backup solution for linux? by Anonymous Coward · · Score: 0
    snapshot backup capabilities for ext2 like Veritas

    Snapshot log filesystems are a lovely idea, but whenever I've tried them (development E4000 box at work running Solaris2.6) they hang the machine when they fill up. This doesn't build confidence of this as a reliable backup mechanism.

    On Sun's Sunsolve site there are a few known issues with snapshot logs on E10K machines (our platform of choice). The listed workaround is "Don't use snapshot log filesystems."

    How helpful. :)

  25. Re:Taper segfaults by Anonymous Coward · · Score: 0

    The taper-6.9 that comes with RedHat 6.0 segfaults
    but I have compiled 6.8.4 and backed 500MB w/o problems. I also have used 6.8.4 on Slackware (libc5) backing 1GB+. The one thing that doesn't work for me is backing the acctual file instead
    of the symlink.
    This is done on 4mm-DAT/Adaptec 2940

  26. Re:Jamie: Sony AIT - 25GB Native by Anonymous Coward · · Score: 0

    Exabyte has a 40Gbyte DAT drive called Mammoth that is pretty fast. It uses the same AME ( Advanced Metal Evaporation ) type tapes as the AIT. It is a bit pricey, but I hear they have a scaled-down version called Mammoth LT that uses 125m AME tapes instead of the 170m tapes.

  27. Patch? (was: Re:Linux's 2GB limit) by Anonymous Coward · · Score: 0

    Patch for the 2 gig limit on i386s? Where?

  28. There is something wrong with your reply by Anonymous Coward · · Score: 0

    1. This reply is counter productive
    2. This reply does not answer any questions
    3. Pointing out flaws in a question instead of answering a question leads me to believe that you have no clue at all.

    1. Re:There is something wrong with your reply by Anonymous Coward · · Score: 0
      3. Pointing out flaws in a question instead of answering a question leads me to believe that you have no clue at all.

      If the question is stupid, then it is perfectly possible that no answer can be done, so the only possible reply is "your question is stupid".
      The question is "But how am I supposed to backup Linux? Wait ages for tar?" ; the reply was "what makes tar so slow (in comparison to other software) ?". This is a legitimate question: for instance transfering a full system with tar via network shown me that the bottleneck was the Ethernet 10 Mbps (2GB transfered at 8 Mbps).
      Plus actually you didn't ask a question but simply shouted "Linux isn't entreprise ready, I can't make a backup".
      If you want to backup specific, or changed files, then '-atime', '-ctime' options of 'find' are your friends. For checking integrity, or not relying on modification times, 'md5sum' (as in: find /home -xdev -type f -print0 | xargs -0 md5sum > /MD5SUMS.home), can be used. Useful also for checking files on burnt cdrom (the MD5SUM file should be burnt).

      So actually the initial statement: So I export the Linux boxes' drives through NFS and SMB and back em up like the NT machines with Seagate Backup or via the Sun's DAT drives. Works. But that is ugly.
      Linux is ok for medium size internet servers, but to actually get work done, I choose NT for its applications and Sparcs for its lack of people pulling into too many different directions.
      sounds really like the statement of a very stupid Unix admin, unable to use command line, write scripts, and cron jobs, and just able to click on some GUI interface.

      If you want to say that the backups may not be safe because of the quality of the tools might not be comparable to those of commercials one, then I would answer that you could write scripts to double-check (based on md5sum). This is the job of the administrator.

      Know if you say that Linux is not entreprise-ready for people with zero-knowledge in system administration then you're right. But for many enterprise, the IT system is vital, so there is little point in not hiring competent professionnals.

  29. Re:Backups & dump by Anonymous Coward · · Score: 0

    Or try the a(utomatic end of tape) flag in my version of dump here: http://perso.cybercable.fr/pop Stelian. pop@cybercable.fr

  30. Re:Travan sucks? by Anonymous Coward · · Score: 0

    Thousands of backup/restore cycles for DLT tapes?
    Yikes!! Not highly recommended. AIC is ok, but the format isn't as fault tolerant as DLT. Poor substitute for DLT if you care about your data being safe.

    Cheers,
    An AC who cares about his client's data.

  31. IMPORTANT: ADSM patch by Anonymous Coward · · Score: 0

    If you are using the unsupport ADSM v3 client for Linux then be sure to watch for lingering dsmstat child processes or rename dsmstat to something else to ensure that dsmc doesn't run it.

  32. Urk! Where's my machete ... by Anonymous Coward · · Score: 0

    I just have to question your experience. I'm sorry, but you have to have had limited/bad experience with ADSM to favorably compare Legato NutJerker to it, or this is a troll. Oh boy, I gotta watch the blood pressure ...

    ADSM is complicated. It is also bulletproof when set up properly and does lots of neat tricks that makes working with huge data sets spanned over tons of tapes far less of a chore. That is the bottom line. I have never heard of anyone who had had large system experience with ADSM suggest going back. tar is an option. tar ceases to become and option when you have single files over 1TB in size.

    Assuming that this isn't a troll, give ADSM another chance and RTFM.

  33. Re:Travan sucks? by Anonymous Coward · · Score: 0

    Does anyone read the data back which has been backed up? One day we had to restore our filesystems from DLT-7000. It failed. Since then we read all our backups back. We see that about 10% of our tapes give unrecovered read errors. We replaced all drives several times, we replaced all our tapes several times, we had ATL replace our tape-library. The problem still exists.
    Our backup is about 750 GB.

  34. Re:Not on a 32-bit system it mustn't. by Anonymous Coward · · Score: 0
    Programs don't even typically load 2-4GB worth of file at a time, but that doesn't mean we can't or shouldn't use the contents of a 80GB file, just because we have a 32-bit processor.

    Agreed, but only if they read/write sequentially. Which for a 80 GB file would take 4 hours. Otherwise you must seek/mmap: thus modify and recompile all your programs (and prepare for file corruption, since you're likely to forget one seek hidden in macros, not to mention that the program probably store offset with 32 bits, and this must be modified too). In contrast, with Linux on 64 bits architectures, the programs are already compiled with 64 bits system calls.

  35. Re:Backup to CD-R? by Anonymous Coward · · Score: 0
    Write the CD, then read it back, checking it's contents.

    It is better to first create a md5sum of the files of the CD, along with the files: for instance:
    find /dirToBurn -type -f -print0 | xargs -0 md5sum > > /dirToBurn/MD5SUMS

    Then after having burnt your cdrom including the MD5SUM, you just have to mount it and do a 'md5sum -c /cdrom/MD5SUM'

    That way you can check your cdrom at any time. In ten years you'll be still able to check it (checking if your outdated, rusty, IDE CD drive is still working in the process).

  36. media fire safes by Anonymous Coward · · Score: 0

    There are different standards for fireboxes designed to store magnetic media vs. paper. These urls gives some good data (some are marketing, some not):

    http://www.safesupplies.com/experts.html
    http://www.safesupplies.com/dataprotection.html
    http://www.mediasafe.com
    http://www.mediasafes.com

    Subject: [5.5] Tape Media Lifetimes (Longevity) {Brief}
    From: Tape

    See http://www.nml.org/resources/misc/commission_repor t/contents.html and
    http://www.phlab.missouri.edu/~ccgreg/tapes.html for some info on this
    topic. Other contributions gladly accepted, this topic comes up
    frequently in the newsgroup. (rdv, 1996/3/20)

    (above from comp.arch.storage FAQ at rtfm.mit.edu)

    A number of safe companies make "media-rated" (there are different definitions) safes. Part of the problem with looking at safes designed to fall three stories is that the tapes would be screwed up (likely) or the MO disks almost certainly (MO is the only real guarantee of long, i.e., 80-100 year, storage, but it is, after all, glass). DAT tapes shatter, DLT has to be rewound, QIC stuff fails in odd ways, and don't get me started on 8mm ... and this has just been my experience from falls of 3m or less. Three stories (15-20m) would undoubtedly do more damage.

    www.mosler.com

    The safes are nice, but at the usual Mosler price. A little like Snap-On or FACOM tools.

    www.nml.org used to have a lot of stuff, but they have a new web site, it is hard to read, and I couldn't find the papers that they had there. If you have more time than me ...

    In any case, get a media safe, load the tapes into foam packing each time they are stored, keep it on the ground floor, and good luck. AND KEEP OFF SITE BACKUPS.

  37. Re:COMMERCIAL: BRU 2000 by Anonymous Coward · · Score: 0

    Please note that the "demo" version that comes with "boxed" Red Hat 6.0 is NOT a fully functioning version.

    For personal backup, the $89 personal edition of BRU has a nice GUI and has no problems with filesystems and media greater than 2GB. I have personally backed up a 20gb filesystem to a 30gb tape drive with no problem with BRU.

    There's a reason why BRU is the #1 commercial backup solution for Linux...

    -- Eric Lee Green ( eric@estinc.com )

  38. Re:ADSM by Anonymous Coward · · Score: 0

    ftp://index.storsys.ibm.com/adsm/nosuppt/adsmv3.li nux.tar.Z

  39. dd is probably the wrong tool - use tar by Anonymous Coward · · Score: 1


    I use gtar + gzip (i.e. tar czvf ...)

    Works fine on my 4 gig tape drive.

    Mark

  40. Re:Why Tape? by Anonymous Coward · · Score: 1

    If all you want is a poor man's RAID 1 mirroring then yes, a second hard drive where you can store a tar-gzip of your filesystem will do fine, you can also purchase a second hard drive of the same capacity as your first and tell the linux kernel to mirror all data to the second hard drive, although this will cost a little extra money since the hard drive will have to be larger since it will be holding uncompressed data, it will save you the cpu time of having to gzip everything not to mention it will make immediate backups of your files rather than whenever your crontab runs.

    This, however, is not what tape users use tape for. Nobody with a tape drive has just one tape to go with that drive, they have many tapes to go with that drive, they have backups from yesterday, last week, and last month.

    A hard drive backup can only be used in the case of disaster recovery, if one drive crashes, you have all the data neatly stored on the second. However, if a file has been deleted/modified and you later realize that you need the old file back, you are SOL if your second hard drive has already been backed up to, while a tape user will have a backup from before the file was deleted/modified.

    Hard drive backups are good for disaster recovery only; tape backups are good not only for recovering your entire filesystem, but for recovering individual files from your filesystem as well.

    Oh, did I forget to mention that tape is cheaper too?

  41. Re:Jamie: Sony AIT - 25GB Native by Anonymous Coward · · Score: 1
    IBM makes 25G IDE hard drives which I'm planning to buy a couple for as backup drives. They would probably stay spinning all the time but only mount a couple times a month to copy any files that needed to be backed up. I might use tar/bzip on one drive and straight find/cp on another.

    The GP series runs at 5400 RPM, slower than their other high-capacity IDE drive, the GXP, although I'd like it to be even lower around 4000 RPM like the Quantum Bigfoot TS drives. For $100 less per drive, I might get a couple Quantum 19.2 Bigfoot TSs instead. For the ``need that file I deleted a year ago'' situation I would burn a CD-R or two every couple months for longer term storage.

    But then Toshiba has that $450 DVD-RAM drive, with double-sided 5.2G media around $40...

  42. Re:Backup to CD-R? by Anonymous Coward · · Score: 1

    Don't tapes cost more, though? I'm not sure of the price of a tape nowadays (I'm thinking I saw some for about $50-$100 the other day), but a $30 1000x rewriteable CD sounds best to me. And a $1 non-rewriteable can be more economic if you like to "rotate" your backups, but still keep some (you'll keep 'em all this way). You won't need to buy 10 tapes, any pay $500-$1000 in tapes.

    Of course, for a large installment (ie. Business, not at home), tapes are the only way to go. You can't fit 10 GB on a CD, and using over 10 CD's is just prohibitively sLooooW!

    Note: I beleieve cdrecord will burn from stdin, but if I remember right, the author strongly reccomends against this... Oh well, I've got a CD-Rewriteable, guess I'll give it a shot later and tell you my luck.

  43. tar | split by Anonymous Coward · · Score: 1

    If your filesystem only supports 2GB files, then just pipe the output of tar into split to break it up into chunks. Then write the chunks to tape.

    BTW: TR-4 tapes hold about 3.8 GB of data, so you might want to use slightly smaller chunks than 2GB, to make sure you can fit them onto the tape without wasting space.

  44. odd moderation by Anonymous Coward · · Score: 1

    You're right. One weird thing I noticed was that while the score of your post is 2, the reason next to it is flamebait. A +2 score should not have a negative reason next to it. But the fact that it has +2 makes it inherently more likely to be read.

    My view is that the question referred to backup solutions, and you offered one. I disagree with the solution, and am preparing to respond to your post. But your post seems to be entirely on topic. It looks like some good moderators came by and corrected the scoring problem.

    anonymous moderator

  45. Re:what kind of tar? by Anonymous Coward · · Score: 1

    At my last job, we used tar with great success to back up one 100GB and one 150GB RAID to a 14-tape DLT changer. There is a bug in GNU tar regarding incremental backups and renaming of files; I don't know if it's been fixed yet.

  46. Re:linux problems with files > 2Gb must end. by Anonymous Coward · · Score: 1

    The Intel 32-bit(ness) or processor / system architecture has nothing to do with >2GB file size problems. For instance, a true 64-bit file system can (and does) exist on a 32-bit machine, which would lead to having >2GB files.

    In fact, if you abstract the filesystem from the OS well, allow for 64-bit references, you could have any kind of large file system, like SGI, WinNT, etc... Maybe he just needs to expand the integer sizes in the kernel to support true(r) 64-bit file systems. Maybe that's what this is all about.

  47. afio >2GB by Anonymous Coward · · Score: 1

    Afio should handle >2GB tapes since version 2.4.5. It has never had a limitation on the size of the filesystem. If there are cases where it fails for >2GB, I would like to get a bug report in e-mail.

    Koen.
    (koen@win.tue.nl, current afio maintainer)

  48. Re:How about good backup software for heterogenous by Anonymous Coward · · Score: 1

    Backup Exec and Legato are good choices for NT. If your needs aren't complex, NTBACKUP.EXE works. You can backup files on Linux boxes using SAMBA (don't ask me about ownership/permissions) and tape drive support under NT is quite good.

    HP just released a cheap (~$250) 7/14GB Colorado IDE tape drive using Travan technology that I use to backup my network. [i'm not an hp rep, i just think hp is a reliable company and that's a good price for that size] For some reason the software and drivers that shipped with it don't work with NT Server (just Workstation) likely due to some "licensing" issues, since the binaries would be identical...

    "If Bill Gates had a nickel for every time Windows crashed...
    Oh, That's right. He does."

  49. Re:Travan sucks? by Anonymous Coward · · Score: 1

    I found that you have to retension travan tapes 2 or 3 times if you want to get any kind of reliability for large files (say the three and a half gigs I tar/gzip onto mine). Travan is the "common man's" tape drive; it's not really meant for serious business operations. At work, we use mostly DLTs with a few drives still using DAT.

    The one thing that bothers me most about Travans is how HOT they get. There's a reason why each tape has a heat sink built in!

  50. Re:Why Tape? Terabyte systems! by Anonymous Coward · · Score: 1

    A ten-gig UDMA hard drive is $175 at the computer store down the street.

    An eight-gig Travan tape is $20 at the same place.

    From Web vendors, a DLT that will compress up to 70 GIGS of binary data is $80.

    I think that pretty much speaks for itself.

  51. Legato Networker by Anonymous Coward · · Score: 2

    Legato Networker works great and can be used in the enterprise network.

  52. Re:Backup to CD-R? by Anonymous Coward · · Score: 2

    Try this command and see if it works

    mkisofs -R /filesystem/ | cdrecord -v fs=6m speed=2 dev=0,0 -eject -dummy -

    and if it does remove the -dummy to make the real thing

    mkisofs -R /filesystem/ | cdrecord -v fs=6m speed=2 dev=0,0 -eject -

    You may have to change some of the flags. If your CDR won't do double speed remove the speed= (or change it to 4 if you are lucky enough to have a 4x drive) and if the drive is at a different SCSI id you'll have to change the dev= flag. I find that the fs= flag is not necessary but that it doesn't hurt either.

    One caveat is if the filesystem is larger that 650MB you will have just have made yourself a coaster. You can check the file system size with

    du -s /filesystem

    or, for more accurate results,

    mkisofs -print-size /filesystem

  53. Re:Not on a 32-bit system it mustn't. by Anonymous Coward · · Score: 4

    The Intel 32-bit(ness) or any kind system architecture has nothing to do with >2GB file size problems. 64-bit+ integers all used all the time and you probably dont even think about it, i.e. 128-bit encription, NTFS, timestamps, version numbers on MS binaries, etc.... It's just a bit more of a bother on non-64bit processors.

    When we had 16-bit DOS, did it mean we can only have 64K files. You might say, "but 32-bit only addresses 2GB (signed) of memory or whatever". Programs don't even typically load 2-4GB worth of file at a time, but that doesn't mean we can't or shouldn't use the contents of a 80GB file, just because we have a 32-bit processor.

    A true 64-bit file system can exist on a 32-bit machine, which would lead to having >2GB files. In fact, if you abstract the filesystem from the OS well, allow for 64-bit references, you could have any kind of large file system, like SGI, WinNT, etc... Maybe he just needs to expand the API / working integer sizes in the kernel to support true(r) 64-bit file systems. Maybe that's what this is all about.

  54. Re:Commercial solution: arkeia by Russ+Steffen · · Score: 1

    I have to second that. Arkeia rocks, especially in a mixed Linux and NT environment since it seems to be the only Unix-based backup system that preserves NT's permissions, ownership and ACLs.

  55. Re:Are you writing straight to tape? by William+Aoki · · Score: 1

    Not in the entire Unix world; only in the Unix-on-32-bit-processor world. For example, AlphaLinux and Digital Unix (now with the odd name Compaq Tru64) shouldn't encounter sort of problem.

  56. Tape - easier on the pocketbook for big backups by William+Aoki · · Score: 1

    A price check on CDW shows 2GB Jaz cartridges selling for $124.95. The Tandberg drive we use at work can put 16GB (uncompressed) on a single $65.71 tape. The server the tape drive is on has a bunch of Seagate ST39140W drives which hold about 8GB apiece (CDW says they hold 9.1 GB) and cost $339.10. The hot swap hardware for the drives adds to the cost, and the disks have to be present when the system is booted.

    Two of those disks cost $678.20 and would have about 2GB more capacity than a $65.71 tape (IIRC a 25GB tape can be had for about $78), and $999.60 (probably a little less, because there's a discount for 3-packs) to buy eight Jaz cartridges which would equal a single tape. The disks and cartridges would probably be faster (although it would be annoying to sit for hours changing cartridges when you can stick in a tape and go home), but imagine paying $8000 to keep twice-a-week backups going back six weeks when you could pay $800 plus the one-time expense of the drive. Imagine if you did daily backups (which we don't), or kept a few months worth of backups... or kept 'em indefinately...

    The drive *is* rather expensive, but because of media prices tape is cheaper than other solutions if you make frequent backups and keep them for a while.

    ...and, the 8 Jaz cartridges take up as much space in your fireproof container (which should hopefully be designed to protect magnetic media, as mentioned elsewhere in this thread) as four tapes - that's four backups in the physical space of one.

    1. Re:Tape - easier on the pocketbook for big backups by tzanger · · Score: 1

      ...and, the 8 Jaz cartridges take up as much space in your fireproof container (which should hopefully be designed to protect magnetic media, as mentioned elsewhere in this thread) as four tapes - that's four backups in the physical space of one.

      ...and in the first fire that eats through the floor and crashes said safe on the floor below, you lose your four backups since JAZ drives can't stand being dropped off a desktop, let alone a story.

      I don't like JAZ. at all. ZIPs are damn near indestructable, as are tapes.

  57. Re:Backup to CD-R? by William+Aoki · · Score: 1

    A big tape is cheaper than a $30 rewritable CD when you consider that (if I did my math right) you'd pay about $738 to buy CD-RWs with the capacity of a $65 tape (assuming $30 dollars per 650 MB rewritable CD and 16GB per $65 dollar tape - I got the tape size&cost from the tapes used on our Linux server), but the cost of the drives is murder for a home user. You're right about CDs being cheaper if you want to keep all the backups forever (about $24 dollars for 16GB of write-once CDs, if I didn't screw up my math).

    But nice big fast tape drives are sooo much more expensive than CD burners... :(

    (gahck, if you read my posts in this article you'd think I'm some crazy tape evangelist... I sound like your steriotypical OS bigot... uh-oh, here come the men in white coats... :)

  58. Re:Travan sucks? by SuperQ · · Score: 2

    Yes, the reason is not the speed, or weather it does backups properly.. it's a cost of operation, and a reliabality factor.. travan tapes cost between 15 and 25 $ each, where dat tapes are around 5-10 $. DLT tapes are more expensive, but have a capacity of over 35 gig un-compressed. It's also a reliabilty thing. Travan tapes are limited to several hundred backup-restore cycles, DLT/DAT are in the thousands.. which means that they produce less errors. DLT tapes are even more cool, they have a shelf storage life of upto 30 years, compared to the 5 years of Travan (I don't know about DAT tapes)

  59. Re:NOOOOoooo..... by colins · · Score: 1

    There is a linux client for Networker. Been unsupported for a while now, but even that just changed recently.

    -Colin

  60. Lone-Tar by Indomitus · · Score: 4

    We use a program called Lone-Tar at the ISP where I work. We have a couple of +2gig partitions and it handles them nicely. It also has internal dialogs for setting up cron jobs, and tons of other stuff. I like it a lot, as much as one can like a backup program I guess. Lone-Tar.com.

    1. Re:Lone-Tar by mikelieman · · Score: 0

      Another recommendation for Lone-Tar. It's beautiful.

      --
      Technology -- No Place For Wimps! Grateful Dead and Jerry Garcia Chatroom -- http://www.wemissjerry.org
  61. slightly o/t: best backup system for single user? by gavinhall · · Score: 1

    Posted by stodge:

    What's the best backup system - ie. hardware for a single user PC Linux system? The machine is just for my use at home, but I need to back up some files. Are Zip drives really worth it, and supported out of the box by Redhat (its what I use, ok?!)? Anyone hear anything from the Orb device that came out recently?

  62. Re:Commercial solution: arkeia by gavinhall · · Score: 1

    Posted by somar:

    Arkeia is great. You may be able to get a significant discount on the commercial product. I had communication with the president of the company and received a 54% discount on the mini lan product + 1 server class backup license for personal use. I just received my licensed product directly from the president of Knox Software. I previously tested the product to backup NT and Linux to a 8mm Exabyte Drive.

    -Scott

  63. NOOOOoooo..... by gavinhall · · Score: 2

    Posted by FascDot Killed My Previous Use:

    If ANYONE suggests Legato Networker, run away as fast and as far as you can.

    We had this piece of crap installed on Netware and it SUCKED. Oh, it backed up and restored just fine. But it was literally an all day event to restore a single file. And the "user interface" (in quotes because it barely qualified) was the WORST I have ever seen for ANY program. And it required numerous patched NLMs to handle file-locking correctly. And it still brought down our servers regularly.
    --
    "Please remember that how you say something is often more important than what you say." - Rob Malda

    1. Re:NOOOOoooo..... by Malor · · Score: 3

      Well, I'm just now installing Legato Networker for UNIX and it seems pretty good.

      We've had endless trouble with Arcserve for Windows NT. It backs up UNIX clients very slowly, and trashes its databases on a regular basis. I finally called up Arcserve to scream at them about it, and the upshot is that their database system cannot handle more than 16 million records, about 1GB, and the sheer volume of files we need to backup overwhelms their database engine. Of course, it doesn't gracefully fail, it just quietly corrupts itself without telling you. I have been dodging bullets with that system for months.

      Apparently, if you install a SQL server and use it to store your database of records, it works fine for larger installations, but I'm so pissed at them about not documenting such a basic limitation of their system that I've been exploring other alternatives.

      I did a test install of Legato Networker on a Solaris 2.5.1 box, and it seems to work pretty well. It spools multiple volumes to tape at the same time, seems to run A LOT faster than Arcserve did under UNIX (about the same speed as Arcserve/NT backing up NT files, about 2.5MB/minute to a 40GB SCSI DLT tape unit), and the restores run fine. It is a bit slow about responding to commands, sometimes taking a minute or so to set itself up for the next job, but it seems good on Unix at least.

      Unfortunately, there is no Linux client for Networker. :( Arcserve DOES have a Linux client, but that software is so bad I've never even tried it.

      Have you used Networker in anything other than a Novell environment? S'possible that their Netware stuff isn't so good, while their UNIX stuff is fine. I have all of one week experience with it, and while it seems fine to me, I haven't really loaded it down yet. :)

    2. Re:NOOOOoooo..... by yvain · · Score: 1

      Arcserve.... URGH!
      I remember using arcserve 5 on netware... Perhaps the only backup program that was more reliable was NT's backup. I could almost guarrentee that a backup run would result in an ABEND or corrupt files on tape. We switched to arcserve 6 when we moved to novell 4.1 - it didn't crash the server regularly either. But I have to say that Novell is not a brilliant platform on which to judge cross platform enterprise backup software.

      We currently use Legato Networker to back up a small constellation of suns, a couple of NT boxen and a few linux boxes - there are linux clients - for i386 and for sparc. They just don't have the graphical interfaces.

    3. Re:NOOOOoooo..... by SPiKe · · Score: 1

      This happens whenever you have your production file and print servers running the main backup server. I hope you guys aren't doing something as stupid as the Novell admin at my work.

    4. Re:NOOOOoooo..... by flight666 · · Score: 2

      We use Legato Networker in our enterprise backup environment. They have had an unsupported Linux Client for several months now, but just recently have release a supported Linux Client. We use it to back up a couple of our Linux boxes here, and it works great. Much more reliable than Arcserve or Alexandria. We use Legato to back up Oracle, Sybase, and MS SQL Server, Windows NT, HP-UX, and we are just now starting a pilot to move all of our Novell stuff to Legato.

      The Legato software so far has been rock solid, as long as you keep up with it. Restores have gone off without a hitch. We have NEVER lost a file with Networker.

      If anyone needs more info, email me and I would be happy to tell you about our experience.

      --Michael Brown

    5. Re:NOOOOoooo..... by nakaduct · · Score: 3
      Unfortunately, there is no Linux client for Networker.

      ftp://ftp.legato.com/pub/Unsuppor ted/Linux_Client/ has both 4.2 and 5.1 client kits, in .gz and .rpm formats. The clients are unsupported, but they work well for us.

      We use Networker to back up ~500 GB, spread across 30 clients (NT, Digital Unix/Tru64, and Linux). Backup performance is excellent (by interleaving sessions over two network cards and the local disks, it can keep two loaders running at ~5MB/sec each).

      I don't know what the maximum "partition" size is, but we've backed up 150GB file domains with no problems.

      Restore performance is slower, of course, but emphatically not an "all day event"; it takes a few seconds to find what you need in the database, and a couple minutes to load the tape (we're using twin 280GB DLT loaders). After that, the speed is the same as it would be for tar/dump/whatever; the tape drive must seek to your files and read, and that can take up to an hour.

      If your files are spread across mutiple tapes (either because you're using incremental or differential backups, or because a single saveset spans multiple tapes), then it can be as long as two hours. If you have only a few clients, these times are reduced somewhat.

      The only time I've spent an entire day doing restores is when we lost the Networker server (and its media indices), and had to use Networker's bootstrap procedure to bring back the index, followed by regular restores to bring back everything else. Because I hadn't bothered to keep hardcopies of the logs, Networker had to scan the tapes for a suitable bootstrap. The searching alone took a few hours.

      A couple caveats, though: it's not cheap, and it's not easy.

      Networker was designed for the kind of environment we've set up, and you may find it overkill for one or two clients. The GUI is marginal, but the command-line tools can completely eliminate it, and do more besides.

      Expect to spend a couple of weeks configuring it, and a couple more getting comfortable with the (extremely powerful, IMHO) command-line tools.

      You'll need a cabable server to hold the media indices -- we keep data in the index for a Quarter, and the database is over 2GB. We're using a dual-CPU Alpha 4100 @600MHz w/2GB memory, running Tru64 Unix (it's used for a number of other things, of course).

      NB: starting with Networker 5, you can have the tape devices and databases on separate machines, which reduces the need for one mammoth server to do backups and media management. It's also good if you have mutiple sites separated by sub-LAN-speed links; you can put a tape device on each LAN.

      cheers,
      mike

  64. >2G backup *media*? by Jamie+Zawinski · · Score: 2
    I used to do backups to a SCSI DAT, but I recently broke my drive. Anyway, I wasn't able to fit more than 2G on one tape, and I assumed that was the limit of DATs.

    If I want to backup 4G or 8G of data, and I don't want to have to switch media halfway through, what should I buy?

    Bonus points if it's media that is still likely to be in fashion (and thus easily readable) five years from now.

    I'm leaning toward the idea of not using tapes for backups at all, but just cloning everything to a spare hard disk, that is only ever mounted during the backup process.

    1. Re:>2G backup *media*? by kenn · · Score: 1

      Use DDS-2 or DDS-3 drives and tapes. I'll be
      ordering a HP DAT24 DDS-3 drive next week so I
      can let you know how I get on (it's for an NT
      box though :-(

      DDS-3 is 12GB (24 compresses) per tape. Something
      like 7GB/hour compressed. And the drives can
      read & write your original DDS-1 tapes as well.

      Kenn

    2. Re:>2G backup *media*? by Eric+Smith · · Score: 2
      I'll be ordering a HP DAT24 DDS-3 drive next week so I can let you know how I get on
      I've been using one for several months. It's great! I routinely write about 10G to the tape, and still have more than 1/3 of the tape to spare. The native capacity of DDS-3 is 12G, and they use compression to store more. Due to the nature of compression, your mileage may vary.

      If you shop around on the net, you can find name-brand DDS-3 tapes for under $14.

      Sony and HP have announced their DDS-4 drives. DDS-4 gets 20G native capacity. I was a little disappointed when DDS-4 was announced, as it is only a 67% capacity increase over DDS-3, while DDS-3 was a 200% increase over DDS-2. Anyhow, as near as I can tell, Sony is shipping their DDS-4 drive already but HP is not. I haven't seen any DDS-4 tapes at retail yet.

      Some people whine about the cost of DDS drives. It's almost irrelevant to me; I spend much more money on tapes. I find it to be a much better tradeoff to buy an expensive drive that uses inexpensive tapes. Most of the Travan and similar things are priced the other way around, so I have no use for them.

    3. Re:>2G backup *media*? by Chris+Siebenmann · · Score: 1

      I suspect that all of DAT, DLT, and Exabyte 8mm tapes will be around and usable in five years (all of them are sufficiently popular now). Current generations of all of them store over 2G. Second hard drives that are active are susceptible to various problems, such as overheating, that can take them out at the same time as the main drive.

      On the other hand: if you really want to make sure that the data stays usable over time, nothing beats keeping it on live disks. As JWZ mentions implicitly, various forms of backups risk either the hardware or the software (or both) no longer being capable of reading them a few years down the road. If the data is on your live disks, you will copy it around as you change and upgrade disks, machines, and operating systems, and will hopefully remember to keep being able to read it through this.

      This isn't strictly a problem for pure backups (where everything in the backup is just a duplicate of what you have on the active drive), but most people and places wind up using things both for backup and for archival purposes. It may also be a problem for disaster recovery; if your existing computer melts down, can you still buy a tape drive that can read the machine's tape backups? (The paranoid (or just cautious) will also check the head alignment periodically to make sure that it hasn't drifted so that the tape drive is making tapes only it can read.)

      The university I work at is currently going through this as we decomission the machine one of our last 9-track tape drives is on. We're having to sort through old 9-track tapes, work out what's important and what's not, coax the old hardware to read the tapes, and dump the data somewhere. (We're probably going to put much of it onto ISO-9660 format CD-Rs; hopefully that will have an equally good or better lifetime).

    4. Re:>2G backup *media*? by oracleofbargth · · Score: 1

      if you're having timeout problems, i suggest checking/rearranging the terminators on your scsi chain. same type of thing happened to me with an onboard AIC7870, and it turned out i had one too many terminators active.

    5. Re:>2G backup *media*? by Erik+Nygren · · Score: 1

      I just got one of the internal SCSI AIWA Travan NS20 drives for $300. I'm having problems with my on-board AIC-7890 SCSI controller having time-outs on my Ultra2 hard disk and hanging the machine. As such, I'm waiting for a second SCSI controller I ordered to show up before I really try running it through its paces. The drive holds 10GB uncompressed and does internal hardware compression and can also read the 4GB Travan NS8 media, supposedly. For home use, this seems like a cost-effective solution. As keeping off-machine and off-site backups is important to me, just having another hard disk doesn't make terribly much sense.

    6. Re:>2G backup *media*? by Ineversaidthat · · Score: 1

      With the dropping prices of fairly large scsi disks (assuming you're still running SGI), hang an external bay off the back of your box and go with the disk. When it's full, swap it out, date it, bag it, and throw it on the shelf.

  65. why all this seperate server crap? by tzanger · · Score: 1

    I don't understand. Our file/print server is one machine, and I'm installing the tape backup on that same machine.

    Maybe I'm missing something REALLY obvious here, but what's wrong with that?

    the email/dialup/firewall is a seperate box for security reasons but why *should* the backup server be a different computer? If the main server dies, you're fried either way. I have another machine I can throw the tape into and restore from, but I don't understand what's inherently wrong with using the same machine...

    please email me as well as send to this forum, /. really needs a "check replies to my thread" feature off the main screen

  66. Re:Why Tape? by jmalicki · · Score: 2

    If you take large amounts of backup and need to store it off site, you need tape. Tape is much cheaper than harddisks. Only maintaining one backup is also foolish. Media fails. So you need tow harddisks anyway. And are you going to have 20 of those 22GB harddisks for backups? When you transport the harddisk for offsite storage, the heads might fail. Tape has not been superceded.

  67. Snapshot filesystem backup solution for linux? by vinod · · Score: 1

    Could someone point to any efforts related to addition of snapshot backup capabilities for the ext2 (or ext3) filesystems, like the one that Veritas offers?

  68. I would recommend Amanda by krady · · Score: 3

    I would recommend the Amanda backup system which we have used in work for many years and can deal nicely with these problems.

  69. Re:linux problems with files > 2Gb must end. by Paul+Jakma · · Score: 1

    yes, but you can use tar to backup directly to the device. sounds like this guy is creating a tarball on an ext2 filesystem somewhere first.

    backup straight to the device!!!

    --
    I use Friend/Foe + mod-point modifiers as a karma/reputation system.
  70. Re:No backup solution? try ADSM by Colin+Smith · · Score: 1

    Yer still sposed to pay the license yknow!

    ADSM is very powerful; very "enterprise", but it's like repeatedly beating your head against a plasterboard wall. Legato's much nicer tho not as powerful.

    --
    Deleted
  71. Re:Commercial solution: arkeia by Palin · · Score: 1

    Arkeia is good...go check it out...www.arkeia.com...

    --Warning: Start of Shameless plug --
    We are also a reseller for the..so if you are interested give us a call +1-201-384-4444 x204

    --end Shameless plug --

    --
    Palin...
  72. Re:Jamie: Sony AIT - 25GB Native by Palin · · Score: 1

    The Seagate drive is actually just a sony drive relabeled...we had problems with the seagate drives and after a month of swapping and dealing with tech support we got a sony drive instead...and walla it was the same exact drive as the seagate...Even seagate tech support said they were waiting for a firmware update from sony for their drives...BTW: the sony firmware update will allow the drives to do 35/70GB or so instead of the 25/50 GB...Can we say free upgrade?

    You will need new tapes though...Also one other note...using the AIT drives with ARKEIA..Do not buy the Tapes with a MIC (in cartridge memory), ARKEIA does not use it..(nor does tar, BRU, etc)..Save yourself $20 per tape...

    Also...It looks like the casue of our problem was a bad batch of tapes from seagate (also made by sony, confirmed by a seagate rep.)..Unfortuanatly I don't have any more info to pass on (like lot numbers)..

    Good luck with backups...and the restores when need be...

    --
    Palin...
  73. Linux's 2GB limit by Zippy+the+Pinhead · · Score: 2

    The 2GB filesize is actually a VFS limitation, so it applies to all filesystems. But it only happens on 32-bit Linuxae. And there's a patch that addresses it. Other 2G limits: MS-DOS FAT partition size (addressed by FAT32 or a real FS), IDE on non-LBA BIOSes (can be worked around).

  74. Not on a 32-bit system it mustn't. by roystgnr · · Score: 4

    If you want to use files larger than 2 GB (the largest number that will fit in a signed 32-bit integer), then get a 64-bit system to put them on. If you want to have simple, efficient, easy to code and easy to port seeks within your files, then you're going to want to be able to use a signed integer to seek back and forth (let's not even mention the trouble with mmap() if your files are larger than a pointer can index...)

    The *last* thing Linux should do about 2GB files is try and use hack after kludge to satisfy people who want to use Intel chips but don't want to hear about their limitations.

    1. Re:Not on a 32-bit system it mustn't. by Hermanetta · · Score: 1

      I was Anon-Coward above, just decided to get account. Herman was taken. Heh.

      You _can_ randomly access large files with a 64bit+ file system. Think SGI and Starwars, or Oracle (databases) and you favorite bank (some of which are bound to be running some form of UNIX). You don't think that if they want to suddenly edit Starwars Episode I, frame 60,000 that they have to read the whole damn file again from the beginning, do you? :)

      File systems and OS support typically provide support for sequential acces, random access, and other such stuff. For instance in NT, asking for sequetial when opening a file is really just a hint for optimization for file cacheing. What you always get in a OS for a "File System" is some form of file pointer, an integer based number, where you use this to tell the os what part of the file you are talking about. There is usually a size specifier of some sort as well.

      The size and pointer are usually the same size (like both 32bit or both 64bit), so that you can do something like set the pointer to 0 and set the size to size of file (n) and reference the whole thing. On NT maximum read at a time is 32-bit and the pointer is 64bit.

      So, files that are meant to be streaming (or used as streaming, i.e., backup to tape) don't care and don't use pointers embeded in file and will be read sequentially. But using structured storage, like data bases, zip files, .bsp files for quake (maps), executable binares, etc..., all use embeded file pointers so a knowageable progam doesn't have to seek through a whole file to get what it needs, unless it just wants to.

      In 32-bit file system world on a 32-bit machine you can simply store the offset (file pointer) in the file data or it can be implied. What ever number you use would be able to reference the 4GB (2GB if you API only supports signed integer parameters for file pointers). You can just read the pointer number and plug it back in to the function to call up or write to the part you want. And, if you are reading / writing in 2K chunks just set the size to 2K. Simple.

      So the point of saying all this is that if your API is capable of supporing 64-bit pointers, it is just a little more hassle. If it's done like NT, you have two 32-bit intergers, high and low DWORD. This actually works great, and would probably still work wonderfully on a 64-bit system too. Its not a pure as getting to put in one number. Bit you can increment one number by one and suddenly you are talking about the second 4GB of a file. You dont have to even use the high DWORD, just leave it always zero, and you will always talk to/about a 2/4GB file. To get a pointer from instide the file its damn easy. Just read a section of the file. Set a 32bit mem pointer to the begining of the just read section in memory. (Assume mem pointer now points to the file pointer inside the just read file section.)
      Go like this:

      High=*p++;
      Low=*p;
      rval=SomeOS_ReadFile(FHandle,Hign,Low,Size,&buff er);

      as opposed to pointer setup for a 64bit API:

      HighNLow=*p;
      rval=SomeOS_ReadFile(FHandle,HignNLow,Size,&buff er);

      (note: the 64-bit API could also be used on a 32-bit macine by just passing a Quad-WORD or a two DWORD structure, or a float, or whatever)

      See, just what I said, a little bit more of a hassle, but you get like 2^64 file size, however big the hell that is, by adding one little parameter. And I think if Linux had better support at the kernel level for 64-bit file systems, you could use larger files. (Maybe it does, maybe the file systems suck, dunno.)

      I mean if everyone is saying a recompile is needed to use 64bit (theoretical) size files in Linux, because there is no way to specify file pointers above 32bit, then what do you expect. If 64bit file pointers exist in Linux but the kernel is messed then you dont neccessarily have to recompile; the interface remains the same. If the backend hooks for the file systems are too small, then extend the OS and fix and recompile the damn file systems. Whatever...

      But this really has nothing to do with Intel or Sun or IBM or 32bit or 64bit or 36 1/2 bit or anything. Just a damn upgrade. I like Linux; does anyone know why this was left out?

      - Herman

  75. COMMERCIAL: BRU 2000 by jsholovitz · · Score: 3

    On some Linux installations, I have used the BRU 2000 backup software. It costs a couple of hundred $$$, IIRC, but it is really excellent software with many features. So If you are willing to spend some money, that should work for you. I must defer to others, however, in the area of doing it with free tools.

  76. Re:what kind of tar? by mhoskins · · Score: 1

    Exactly, I've been using Tar and Cpio to do backups of 20 and 40 gig raid volumes with no hickups... Could it be simply that travan tape drives suck? I've never gotten them to perform up to my standards of reliability... Use dat or DLT
    ----------------------------------------------
    bash# lynx http://www.slashdot.org >>/dev/geek
    Matt on IRC, Nick: Tuttle

    --
    ----------------------------------------------
    bash# lynx http://www.slashdot.org >>/dev/geek
    Matt on IRC
  77. Re:Why Tape? by mhoskins · · Score: 2

    Tape is more reliable. And if you ever work for a company that is audited by its shareholders regularly you will find that you are required to keep backups around for quite some time... For instance. A financial inst. may keep backups of the transaction journal forever! An insurance company i once did work for was required to keep weekly grandfather and daily incremental backups for 5 years... thats a lot of storage for HDDs :-)
    --------------------------------------------- -
    bash# lynx http://www.slashdot.org >>/dev/geek
    Matt on IRC, Nick: Tuttle

    --
    ----------------------------------------------
    bash# lynx http://www.slashdot.org >>/dev/geek
    Matt on IRC
  78. Travan sucks? by Sesse · · Score: 1

    I have no problems with my HP Travan 4/8GB (internal) drive... Doing monthly full backup (of 2+4+2+0.5GB), and nightly incremental backup. I just don't see your point.

    /* Steinar */

    --
    (This comment is of course GPLed.)
    1. Re:Travan sucks? by cjs · · Score: 1

      Travan tapes are limited to several hundred backup-restore cycles, DLT/DAT are in the thousands.. which means that they produce less errors.
      I don't know about DLT, but DAT tapes certainly will not give you thousands, or even hundreds, of passes. Keep in mind this is a rotating head medium, and a medium designed for audio at that.

      cjs

      --
      The world's most portable OS: http://www.netbsd.org.
  79. ADSM by The+Mayor · · Score: 1

    IBM has a version of the ADSM client for Linux. I haven't checked into the situation in about a year, but back then the client was not officially supported. However, ADSM is really quite nice. This, of course, requires that the tape system be connected to a non-Linux machine.

    The company I worked for when I looked into this performs remote backups via network lines. They also offer Internet access over the network lines they use for the backups. It's really a nice system. If you happen to live in Houston, check them out: www.edms.net

    --
    --Be human.
    1. Re:ADSM by steveg · · Score: 1

      Where would the Linux client be located? The ADSM page I looked at had about every OS listed except Linux. Something like this that was cross platform sounds like it could be useful to us, but linux is a big part of our mix.

      --
      Ignorance killed the cat. Curiosity was framed.
  80. Re:Why Tape? Terabyte systems! by WonderClown · · Score: 2
    I'd like to see you cost-effectively backup a 5 TB database to disk. Oh, and don't forget that you have to be able to restore the system to its state on any day during the past month and at the beginning of any month in the past five years. And as others have mentioned, you want to make two copies and keep one off-site in case of disaster.

    Yes, for systems under 100 GB which don't need to keep historical data, backing up to disk is feasible, and in many circumstances is better than tape. However, that doesn't mean that tape has "been superceded" entirely.

  81. Re:Why Tape? by jmpvm · · Score: 1

    Yeah. The newest-gen DLT tapes on Ultra-LVD busses can hit near 400 MB/min. And this in on WINNT. I'd love to see what these baby's would do on a nice linux box. Unfortunately, that is not in our test plans. :(

  82. how about afbackup or burt? by sednet · · Score: 2
    haven't seen any thoughts on afbackup and burt. i auditioned both last summer for network backups of multiple 9GB machines over ssh and settled on afbackup to a DDS-2 DAT.

    afbackup is pretty painless to setup, speedy backups, can run over ssh, prompts by email when tape changes are needed, reasonable restores of entire backup sets, but is very slow for selected file restores.

    burt is wicked fast for backups, tcl-based interface, imho elegant, and can run over ssh. afbackup was better documented and offered an emergency restore option that i preferred at the time.

    i ruled out amanda because it is complex and tends to want a holding disk the size of an entire backup set.

    --
    about sean dreilinger
  83. CTAR recommendation: seconded! by CarlPatten · · Score: 1


    My company has been running CTAR for 3 1/2 years, and it has never let us down. It can run as command line with TAR-like commands, or via a very well-designed character-mode menu. I've been backing up 8+ gigs at a time on our main SCO Unix server with no problems.

    It's available for Linux as well as most varieties of Unix. $195 for the Linux version.

  84. what kind of tar? by h2odragon · · Score: 4

    I use standard GNU tar v 1.12 to back up several systems to 12G DAT tapes; and have never had a problem using the '--multi-volume' switch to put an 18G filesystem on 2 tapes.

    1. Re:what kind of tar? by jridley · · Score: 1

      I agree that DLT and AIC are awesome if you can afford them (anyone backing up a network had better figure out a way to afford them), but for personal use, I prefer Travan over DAT. Maybe it's just that my experience is with HP drives, but when we were using HP DAT's, we had to clean them daily, and even then sometimes they wouldn't get through a whole backup without blinking the "Error, I need cleaning" light. I've had numerous file restore failures and had to revert to older copies of the backups. Yes, we were using HP tapes too. Since then we have gone to DLT's and they're great.
      For my personal machine backup, I've had IOMega TR1 (400/800M), Seagate TR4 (4/8G) and now have an Aiwa NS20 TR5 (10/20G) drive. I've never had a single failure over hundreds of backups and dozens of restores. The Aiwa finishes a 5.6G backup in an hour, validates while writing, has hardware compression, and costs Oh yeah, I use GNU Tar to backup 6GB of data that lives on an 8GB drive, and I have no problems. I'm not spanning tapes, though; that might be an issue if there's a bug.

    2. Re:what kind of tar? by Doviende · · Score: 2
      At my previous job, i had a Linux box doing 30+ Gigs of backup every night. It read the info from the network via NFS, and used GNU tar to put it onto a 40G DLT. One of the servers involved had at least 13G of stuff on it alone...the others had 9G and 8G and stuff like that.

      I'd recommend the DLT drives to anyone who can afford them....they're awesome.

      "The value of a man resides in what he gives,
      and not in what he is capable of receiving."

      --
      "The value of a man resides in what he gives,
      and not in what he is capable of receiving."
      --Albert Einstein
  85. Try this by KBrown · · Score: 1

    for backup: tar cv /directory/you/want/to/backup | split -b 2048m - bak_mm-dd-yyyy_tar_ for restore: cat bak_mm-dd-yyyy_tar_* | tar xvf -

    --
    --
  86. Re:There are three good choices. by Hamhead · · Score: 1

    But the tar that comes with Slowlaris regularly reminds me of the advantage of going GNU. It's the first thing I do to a Solaris system, is replace /bin/tar with GNU's tar.

    --
    -- If you met me, you probably wouldn't remember me. I'm pretty hard to remember.
  87. How many files are you backing up? by Booya · · Score: 3

    I've had problems with Tar (and the other commands) as well when the number of files I had was extremely large. This is regardless of file size, eg: I had 200,000+ small files, but storage size was about 1 gig or so and I had problems with tar.. I ended up just breaking down my backups into batches. something like:

    tar -cf part-a-of-tree.tar /files/parta
    tar -cf part-b-of-tree.tar /files/partb

    etc...

    --
    -Booya "No Try Not. Do or do not, there is no try." -Yoda
  88. BRU by True+Dork · · Score: 1

    I know I've seen a few people mention BRU, but I'll put my .02 in as well. Someone mentioned the 24x6 from HP used in-house at estinc.com. I have one and loved it, but outgrew it. I'm on a 15 tape DLT robotic changer now, run by bru and a rather ugly hack I did to a changer driver. (I commented out an error, and it works fine hehe) The guys at bru are quite helpful and their product runs around 210 gigs per week across my changer on a linux 2.2 system for me. I plan on fixing my scripts for the backup (so they're not QUITE so ugly) and making them public on the truedork domain after I get off my butt, and getting estinc.com to link to me for them. Not fancy, but hell, they work. For single drive backups, they have a gui too. I'd love a gui, but it doesnt work for changers at the moment. Anyway, they took damn good care of me for the few hundred we spent on them. Check them out. Remember that commercial products are not all evil. Some companies have to do so to pay the rent :) See ya!

    The True Dork

  89. Re:Why Tape? by Krux · · Score: 1

    What we use is a hybrid system which backs everything up to disk quickly, then durring the day moves things off to tape. ADSM by IBM. When you have a dozen large UN*Xen boxes, about 60+ NT servers, and a dozen Novell servers, scattered about on different properties, it's the only way to do things.

    We were running into the 16million record database size limitation on Arcserve, and also our backups were not finishing in time. ADSM, while rather expensive, was a job saver.

    You could surely do something similar with the available free GNU utilities, and some smart perl scripting.

    --
    "One of these days... milkshake... BOOM!!!!" - emb
  90. BRU 2000 by Vskye · · Score: 1

    This software works well with our 24x6 auto changer from HP. There are 5 24GB tapes, and one cleaning tape. Matter of fact this is the same drive that is used in house by the folks that make BRU 2000. (I'm just a happy customer btw.)

    Dana

    --
    Life was hell, then I discovered Linux...
  91. IBM ADSM runs on about as many platforms as exist by Guy+Smiley · · Score: 1

    Have a look at the ADSM home page for more info. The ADSM server needs to run on a non-Linux box (AIX, HP/UX, Solaris, NT), but it has a Linux client, as well as clients for most OSes, and also lots of database back-ends (DB/2, Oracle, Sybase, SQL Server, Notes, Exchange).

  92. Backups & dump by Phexro · · Score: 0

    Hmm... I played with a few backup solutions on my Debian box a while ago. I found that plain `tar czf' is the best way to go. It's reliable and fairly quick. I messed with Taper, but it was definitely not ready for prime time. I regularly got segfaults when doing full backups.

    Can someone tell me how I might get dump to use all 4gb of my Travan-8 tape drive? By default it only uses a few megs before asking me to switch tapes, and I could not get it sorted out with the stuff in the man page.

  93. Re:Backup to CD-R? by Phexro · · Score: 1

    Hmm, probably not too feasible. You might see if cdrecord/cdwrite will burn from stdin, then you could `tar czf | cdrecord -v dev=0,0 speed=1' and then access it with tar.

    CD-Rs make good long-term backup media, but for most backup needs tapes are far superior. You can't make a coaster out of a tape.

  94. Legato works for us by rcgraves · · Score: 2

    15 Linux boxes, 2 HP9000s, 5 IRIX, a dozen NT, handful of Macs, 4 VMS, and one NetApp toaster backed up through Linux NFS-mount (poor Linux 2.0 NFS performance is an advantage here... when mounted on the HP it pretty much killed the network).

    The interface isn't that terrible. 5.5 is much better. We have had intermittent problems backing up a 36GB RAID filesystem (Linux 2.2) though.

    It's far from free, and the server requires NT or a commercial UNIX. We run it on HP-UX 10.20. But for multiplatform backup on a high-end tape changer robot, you need o go the commercial route.

  95. Re:Why Tape? by Delphis · · Score: 0

    Why the hell has my comment been marked as 'offtopic' anyway? ... bloody moderators.. my question was at the very point of the article IMHO

    Ho hum .. I hope Rob sorts this moderation crap out .. seems to be annoying everyone who posts valid things... just see the 'CPU Cooling Insanity' topic!

    --
    Delphis
  96. Why Tape? by Delphis · · Score: 1

    With the price and capacity of currently available hard disks, why on earth would you want to use tape anyway?

    Why not just get one or two large 20+GB HDs and backup stuff to them on a regular basis. I've been using a spare HD for backups for years. crontab scripts creating tar-gzips.

    I remember glancing through ads the other day that IBM makes a 22GB 3.5" IDE HD, which could backup that FTP server with no probs.

    Tape is just slow these days and it's been superceded.. it might be cheaper but then old tech always is (until it becomes antqiue! :))

    Just my 2 bits

    --
    Delphis
    1. Re:Why Tape? by Delphis · · Score: 1

      I didn't actually mean use the drive solely as a RAID1 device, but rather to create an archive file (or a few backups of different ages and/or parts of the filesystem) on for access whenever.

      Dunno about you, AC, but I find

      tar -zxvf Backup.tar.gz /

      fairly simple to do to get back any file I've ever needed to retrieve from my backups.

      Yea tapes are cheaper, but HDs are MUCH faster to access :)

      Getting a few HDs together and implementing a RAID5 system is much better for reliability anyway, RAID1 is probably a waste of time. Cheap IDE RAID is getting affordable.

      --
      Delphis
    2. Re:Why Tape? by Delphis · · Score: 1

      arse.. silly me SHOULD have used that preview button ..

      tar -zxvf Backup.tar.gz [path]/[file]

      was what I meant... doh :)

      --
      Delphis
    3. Re:Why Tape? by Delphis · · Score: 1

      Heard of removable disks? :) .. to store in aforementioned fireproof (earthed?) container.

      --
      Delphis
    4. Re:Why Tape? by belbo · · Score: 0
      Sorry, but your post is offtopic.
      The question was about backup software not hardware. Interesting suggestion, though ;-).

      Regards
      belbo

      --

      --
      "Just believe everything I tell you, and it will all be very, very simple."

    5. Re:Why Tape? by Detritus · · Score: 1
      Your PC (and all attached disks) gets fried by a lightning strike or other electrical fault. Your house catches on fire or is flooded. What do you do?

      A tape can be stored off-site or in a fireproof container.

      --
      Mea navis aericumbens anguillis abundat
    6. Re:Why Tape? by Chris+Siebenmann · · Score: 2

      I think that a lot of decisions will depend on what sort of disasters you want to recover from. Planning for only 'normal' aging drives having media failures is a lot different (and easier) than planning for your office burning down.

      There are all sorts of potentially problematic tradeoffs with various sorts of hard drives for backup. Things like:

      • a normal drive operating and in the machine is susceptible to common-mode failures that get both drives at once (such as overheating). If you keep it powered off it generally requires a reboot to see. External SCSI drives and some magic can avoid this.
      • Hot-swappable or otherwise removable HDs cost more and probably require rebooting the machine to use.
      • From what I understand, removable media like Zip and Jazz drives are not truly HDs, and are not as reliable as them (although they can be swapped in and out with the machine on). Repeated insert/write/remove cycles may expose you to a far-larger-than-HDs risk of media damage and data loss.

      I suspect that everything short of Zip/Jazz cartridges are more susceptible to damage when removed than tapes. Especially if they aren't mounted in an external enclosure and are carried around with the drive electronics bare.

      If you make more than one backup, tapes may become substantially cheaper than more and more HD space. If you store and handle lots of backups, again tapes may become easier and cheaper to deal with than other media.

      I have a fairly high trust for the long term durability of tapes sitting around. Manufacturers test this stuff and will tell you about it, including cautions for temperature limits and so on. Do disk manufacturers give equivalent figures for removed drives?

      For personal home use, I think that anything is better than nothing; a second HD is cheap and easy (especially if you aren't worried about things that would take both drives out at once, like overheating or fire). For professional use in an office or the like (even a home office), I'd trust tape more. The up-front costs are bigger, but the benefits can be substantial, and there are things that are far more convenient with tapes that are very important for professional use (such as periodic offsite backups).

      If you trust tapes they can also be used for archival purposes (where you delete the data off the HD after storing it on tape and verifying the tape) as well as normal backups. Depending on how much call you have for this, this may also represent a money savings with tape over disks.

    7. Re:Why Tape? by Clover_Kicker · · Score: 2

      Tapes are a great way to get a snapshot of your file system as of right now. With a sensible tape rotation, it's easy to go back and get a file as of a few months ago, i.e. before someone accidentally deleted the first 20 pages of the document or whatever, and nobody noticed until now.

      Keeping stuff online with big hard drives is a great way to backup data that is static, i.e. scanned documents, but is somewhat less useful for stuff that is constantly changing.

      My 0.02 $CAN

    8. Re:Why Tape? by Tyae · · Score: 2

      One more comment to add to the other replies re: usage of tape for backups:

      Not only do most companies need backup rotations, they also need a backup for disaster recovery.

      Ideally, you have a backup of your data kept offsite, in case of disaster. Tape makes this simple - it's readily portable. Every week night our DBA throws a couple tapes into his bag before heading out. If our building goes down in flames, we still have our data.

      Insurance money can help to rebuild, but it won't get back data!

  97. Re:Why Tape? Terabyte systems! by Delphis · · Score: 2

    True, put in that context the previous 4 or 5 posts are true and okay I see circumstances that tapes are useful (and probably necessary) .. for backing up financial records for tax purposes and for multi-terabyte databases.

    But,

    I was responding to the article about backing up just 22GB and for the home or semi-pro tape is just not worth the hassle any more (tapes stretch , to counter the 'heads crash' argument) when multiple disks (can you say RAID even if you want online reliability of data) can be had for much better performance.

    Okay I admit maybe 'superceded' was a bit strong given the arguments raised, but I think anyone has to admit that storage capacity of HDs (vs. cost) has shot up incredibly against tapes in the last few years.

    --
    Delphis
  98. kdat by CucKo0 · · Score: 2

    oops forgot to mention that kdat uses "tar" which is good if you have multiple servers incase you need to untar it on any unix box :)

    (did i mention that it has a nifty gui front end :)

  99. 2gig backup by CucKo0 · · Score: 3

    im currently doing a 8+gig backup using kdat from kde and it works well (no compression tho)

    we had some trouble with backing up to another server via nfs mount that would only allow me to do a 2 gig max file :(

    if i use the tar with compression i can get up to 24 gig backup to tape (12 gig without compression)

    but kdat allows you to span tapes and keeps a nice little index of all previous files backed up on that tape (very nice gui app)

    not sure if that will help but it's all i got rite now :)

  100. It's a 32bit thing. by raistlinne · · Score: 2

    That limit only exists on 32bit machines. 64bit Linux platforms, such as Linux/Alpha, have a truly astronomic file size limit, IIANVMM.

    --
    They laughed at Einstein. They laughed at the Wright Brothers. But they also laughed at Bozo the Clown. -- C. Sagan
  101. There are three good choices. by jerodd · · Score: 5
    I like to backup one of my filesystems which has huge files (a 6.4GB file and a 2.5GB file). I use GNU tar with it. Simply use the -M switch, and tar doesn't have any problems with the huge files. I'm not sure if SysV tar supports it (i.e. the tar in any non-GNU Unix system). Note that the filesystem I am using (JFS) supports big files and I had to use emx v0.9d to support 64-bit file lengths (I'm using OS/2).

    You can also use AMANDA backup, which I use on my GNU/Linux machines for backup. It seems to handle the large backup sizes acceptably.

    Finally, you can always just split up huge files using dd.

    Cheers,
    Joshua.

    --
    --jon. Postel is dead. May we all mourn his, and our, loss.
    1. Re:There are three good choices. by Surak · · Score: 1

      I regularly use both Solaris 5.5 and HP-UX 10.20's implementation of tar, both of which I -believe- are non-GNU (SysV), at work (GM). We regularly distribute a 4 GB Unigraphics parts library. I have never had a problem.

  102. Re:DUMP solution to +2GB by Chris+Siebenmann · · Score: 1

    Often, there's not much point in giving dump plausible numbers about the tape size. If you're dumping in a situation where a tape that is unexpectedly full won't get swapped, you might as well tell dump whatever lie it needs so that it never thinks it needs to change tapes.

    The Linux dump manpage I have handy even suggests that most of the time dump will directly notice end of media and deal with it properly, regardless of the tape size specified.

  103. Re:DUMP solution to +2GB by Chris+Siebenmann · · Score: 1

    The other problem with trying to be accurate (or slightly conservative) about the tape size to dump is putting multiple dumps onto the same tape. To get the sizing right, you need to tell dump how much tape is still left in the second and later dumps, which means capturing and parsing dump's output to find out how much of the tape has been used up by each filesystem.

    For relatively simple situations (where you don't plan to pack tapes to the brim and don't plan on tapes ever overflowing), this is a chunk of somewhat arcane work for little purpose. On the other hand, if one really needs to do this, I suspect that things like Amanda have the code already. And researching the wheel is a lot easier than reinventing it.

    Anyone looking for a project could always look into adding a switch to dump so that it produces program-friendly output that's easy to parse for this sort of information.

  104. Re:Are you writing straight to tape? by redhog · · Score: 1

    Is the 2Gb size limit an ext2-limit, or a Linux-limit?

    --
    --The knowledge that you are an idiot, is what distinguishes you from one.
  105. Re:Are you writing straight to tape? by redhog · · Score: 1

    So, when the Y2K-hype is gone, we still have a D2Gb-problem left to solve, in the entire UNIX-world. Perheaps NT have the same problem. I think so...

    --
    --The knowledge that you are an idiot, is what distinguishes you from one.
  106. Re:slightly o/t: best backup system for single use by redhog · · Score: 1

    I use a CD-RW writer as backup media. Works pretty fine, but I havent figured out yet how to make tar create multi-volumes when I have to pipe it to the cd-writer-program... But its nice to be able to put the CD in an ordinary CD-drive and tar -xvf /dev/cdrom... Someone have a acript or idea to make multivolumes?

    --
    --The knowledge that you are an idiot, is what distinguishes you from one.
  107. Re:DUMP solution to +2GB by Amoeba+Protozoa · · Score: 1

    We have found this not to be true with a number of tape drives (notably Seagate DATs), therefore the conservative limit. :)

    --When in doubt kludge it.

    -AP (Jordan Husney)


  108. Re:DUMP solution to +2GB by Amoeba+Protozoa · · Score: 2

    A couple people have written in asking how to do a restore opereration with "restore" (the companion program to dump). By far the easiest way is to do:

    # restore -i /dev/st0

    (where 0 is your tape drive number)


    However, If you have multiple filesystems on a single tape (like the example above), you must first use mt to fast-forward to the correct tape-mark. Let's say we want to get the second file-system off the tape:

    # mt fsf 1
    # restore -i /dev/st0

    This will then put you in restore's little "shell" for adding files/directories to be restored. For example:

    -- 8- *snip* ---

    restore > ls
    .:
    .automount/ bin/ lib/ proc/ usr/
    .bash_history boot/ lost+found/ root/ var/
    .mc.hot dev/ misc/ sbin/
    .mc.ini etc/ mnt/ tftpboot/
    .netwatch home/ net/ tmp/

    restore > ?
    Available commands are:
    ls [arg] - list directory
    cd arg - change directory
    pwd - print current directory
    add [arg] - add `arg' to list of files to be extracted
    delete [arg] - delete `arg' from list of files to be extracted
    extract - extract requested files
    setmodes - set modes of requested directories
    quit - immediately exit program
    what - list dump header information
    verbose - toggle verbose flag (useful with ``ls'')
    help or `?' - print this list
    If no `arg' is supplied, the current directory is used
    restore > add etc
    restore > extract
    You have not read any tapes yet.
    Unless you know which volume your file(s) are on you should start
    with the last volume and work towards towards the first.
    Specify next volume #: 1

    ( it will now restore from the tape to your cwd)

    Done!

    --- 8-
    Just to sum up, the example above opens the tape, lists the files, and adds "etc/" to the list of files to be extracted. Since this is a level 0 backup (a full non-incremental backup) I need not put in any other tapes and simply say "1" when it asks me for the next volume number.

    The etc/ directory (with all its sub-directories) will be in whatever directory I started restore from. If you are doing a system restore, do it from "/".


    -AP (Jordan Husney)


  109. DUMP solution to +2GB by Amoeba+Protozoa · · Score: 5
    For the ISP I run, we do a nightly cron-driven backup using dump. The key is that you must specify the total length of the tape. We have a 23GB tape drive, of which we conservitively say the tape is 20GB long.

    Here is our cron.daily/daily.dump file:

    #!/bin/sh
    ## Daily Full System Backup Givin with 20GB Tape Cap.
    mt rewind
    mt erase
    mt rewind
    /sbin/dump 0uBf 20000000 /dev/nst0 /dev/sdb1
    /sbin/dump 0uBf 20000000 /dev/nst0 /dev/sda1
    /sbin/dump 0uBf 20000000 /dev/nst0 /dev/sdc1
    mt rewind

    This dumps all three of our partitions out to a single tape. The 0 ("zero") option dumps the entire thing, as out tape drive is fast, vs. specifing a dump level > 0 (which is for doing various levels of incremental backups); The u, which updates a human-readable /etc/dumpdates file; B for the number of blocks ("kilobytes") the tape is long (this is your problem); and finally f: the device to dump to.

    One of the things that really gets people is how to pass arguments correctly to dump. A little diagram might serve as an aid:

    dump [arg name 1][arg name 2][arg name 3] [arg value 1][arg value 2][arg value 3]
    Hope that helps!

    We use the /dev/nst0 device to write to the tape three times without the thing rewinding. This is the key to putting more than one filesystem on per tape.

    If anybody has any questions about using dump, I would be happy to help.

    -AP
    jordanh@remotepoint.com

  110. linux problems with files > 2Gb must end. by atrox · · Score: 1

    The whole world is talking about Linus. Everyone wants to use it as server .. and so on ...

    I know, there are Patches to make Filesystems 2GB+ safe, but then most of the programs must recompile.. this is not the right solution!

    Support for large files should.. ahh. must be implemented in the normal kernel, so it can be used as server without patching the kernel and most daemons and fileutilities etc ...

    (I'm thinking of large databases or backups which both often need files > 2gb)

  111. Re:Backup to CD-R? by Bryan+Andersen · · Score: 1

    I recommend using a temp file. This is because a single spec of dust can cause you to make a coaster. Write the CD, then read it back, checking it's contents. If it's not correct, try try again...

  112. WARNING Re:Why Tape? by Bryan+Andersen · · Score: 3

    WARNING A tape or harddisk in a fireproof container will still be destroied in a fire. Most fireproof containers are designed to save paper from burning by a combination of steaming away water and thermal insulation. As such, the internal tempeture of the container will easily get over 210 degrees F. Most tapes and harddisks will be destroied at that point.

  113. Re:pe edition BRU also comes with Caldera 2.2 by Rock · · Score: 1

    bru pe also comes with Caldera OpenLinux 2.2. The home page for Enhanced Software Technologies is est. I believe Fred Fish (famous in Amiga circles for his "fish disk" software distributions) is one of the principles of est.
    --
    -- Rock
    -- http://home.att.net/~rocq

    --
    - - -
    "The sixth sick shiek's sixth sheep's sick."
  114. dump? by Jonas+�berg · · Score: 4

    I'd be interested to know what trouble you've been having with dump. I've been doing dumps of 2+ gig partitions for ages with dump and it works very well. Perhaps you're experiencing some other problem which is not related to the backup program you're using? If you can send me more information about where dump fails, I'd be happy to have a look at it.

    1. Re:dump? by victim · · Score: 2

      The man pages for dump on linux used to say it could not handle multivolume dumps. I note that at least in Debian 2.1 this is no longer true.

      I've personally restored my 20GB 6 way stripe set twice from dumps under linux from a single (large) tape with no problems.

  115. Re:Are you writing straight to tape? by tjrw · · Score: 1

    No, not even the "Unix-on-32-bit-processor" world.
    Most commercial Unices running on 32-bit processors have large file support. I know that DYNIX/ptx does. Max filesystem/file size is 1TB or something like that.
    glibc2.1 (RH6.0) seems to have the relevant library-level support. We just need filesystem support in Linux.

  116. Re:NOOOOoooo..... not on NetWare by Sharkeys-Day · · Score: 1

    We bought legato at my company, and found that its Netware support is virtually non-existant. The GUI is ugly and confusing, but it works.

    Our Netware/NT group has gone back to ArcServe,
    but we're keeping Legato for our HP servers (which have outgrown my custom cpio-based scripts).

  117. Commercial solution: arkeia by afx · · Score: 1

    Hi,
    if commercial is acceptable, check out
    http://www.arkeia.com/
    This thing is blindingly fast and doesnt seem to have a limit on file or backup size.

    --
    Life is too short for crappy pictures.
  118. Arkeia works fine for me by lydon · · Score: 2

    I use a commercial backup software called arkeia. You can get it from http://www.arkeia.com/.

    There is also a free for personal use version for linux server/clients. Just go to http://www.arkeia.com/downloadfree.html .

    Regards,
    Oliver.

  119. Re:Jamie: Sony AIT - 25GB Native by ondra · · Score: 1

    >Do not buy the Tapes with a MIC (in cartridge memory), ARKEIA does not use it..(nor does tar, BRU, etc.

    It does, the mt fsf 10 takes about quarter of time with MIC. If you rewind tape often, it quite useful.

  120. Re:Are you writing straight to tape? by -Surak- · · Score: 2

    Both, I believe. To access more than 2GB, you need to use 64 bit file access functions. fseek(), for example, uses an offset value that is a 32-bit signed int, so it can only address 2GB of a file. I think that SGI and others use special functions for 64 bit file access (fseek64() for example), and leave the traditional system calls alone. It's going to mean recompiling everything, at the least.

  121. Jamie: Sony AIT - 25GB Native by -Surak- · · Score: 4

    I highly recommend the Sony AIT drives (I think Seagate or Quantum also sells a variation of the same format). They do 25 GB native per 8mm tape, at 5MB/s. The drives are under $2000, and tapes are around $60. It may be a bit overkill for what you need, but the speed is VERY nice. It does compression as well, but people that quote compressed capacities should be shot.

    They also have a cool feature that allows storing directory info on NVRAM on the tape cartridge - 16 KB or so.
    And because it's Sony, it's definately likely to stay around. I think they still sell Betamax decks, and I kinda think they know what they are doing when it comes to helical scan recording equipment :)

  122. Are you writing straight to tape? by -Surak- · · Score: 5

    Tar and others should work fine as long as you are writing directly to tape, instead of to a temp file. Linux has a 2GB (2^31-1) maximum file size, so if your backup software is trying to spool to disk before streaming to tape, it may fail.

    Amanda handles this by splitting the disk files into 2 GB chunks and reassembling them when it writes to tape. It also deals well with network backups. The filesystem side backend is dump or GNU TAR, so it's fairly standard in that regard. I've had no problems with 8+ GB filesystems using Amanda.

    I would not recomend using e2fsdump - AFAIK, it's still beta, and I had problems with the interactive restore and some other issues. Because it accesses the filesystem at a lower level than standard file access (I believe), I'd be careful with trusting important backups to it.
    TAR definately a safer choice.

    BTW, I have a question myself... does anyone know how to get TAR (or something else) to restore permissions on symlinks? Typically it doesn't matter, but Apache uses symlink permissions for the SymlinksIfOwnersMatch directive, and every time I restore or copy a web partition, I have to go through and fix all the links that are now root owned.

    1. Re:Are you writing straight to tape? by acarey · · Score: 1

      NT's maximum file size = NT's maximum partition size = 16 exobytes. (1 exobyte = 1000 terabytes, I believe... can anybody confirm?)

      So not a big problem for a while, hopefully ;)

      Cheers
      Alastair

      --
      -- "I believe the human being and the fish can coexist peacefully." - George W. Bush, 29 September 2000
  123. BackupEdge by CSIP · · Score: 2

    BackupEdge is the most powerfull backup program for linux (or any other unix) that I've seen.
    It does do what you want, & has alot of other great features.

    -great automatic backup/verifys
    - backup recovery programs
    - bootdisk manager.

    downside: it's comercial
    see www.microlite.com for details

    --
    "Nyquil - The stuffy, sneezy, why-the-hell-is-the-room-spinning medicine."
  124. Re:No backup solution? try ADSM by cugel · · Score: 1

    Where I work, we add native linux clients to our IBM ADSM central backup system FREE, whereas all other platform clients require paying license fees. After the one-time cost for the server software, the massive cost is in client connections EXCEPT for linux. So at least in this context our linux boxes are "enterprise ready". In fact, the free linux clients include all the admin features as well as a gui backup/restore web-enabled interface.

  125. Re:No backup solution? try ADSM by cugel · · Score: 1

    Not So. Check with IBM. Look at the adsm license management system. Linux clients are FREE. They're not even tracked. The tradeoff is that the linux client is not supported. Good thing it works so well.

  126. Arkeia! by torment · · Score: 1

    Check out Arkeia here.

    1. Re:Arkeia! by anomalousCoward · · Score: 1

      Donald Ball wrote:

      "Er, it doesn't have Macintosh support as far as I can tell..."


      There really is only Mac backup program worth a darn... Dantz's Retrospect and Retrospect Express. You may already be familiar with it but what you may not know is... It will back up to an FTP server, so you can back up all your Macs onto a Unix system.

      I use the FTP backup option (the data is sent to a Digital Unix machine) and it works quite well.

      It saves all the Mac-specific goodies for you. And it writes multiple files per backup, so there is no problem if the FTP server has a 2-gig file size limit. (It seems to adjust the file size based on connection speed. For LAN backups it uses 16 meg files.)

  127. Re:And at only NZ$15,000 per license, it's a steal by flesh99 · · Score: 1

    We backup 1/2 a terrabyte nightly with Arkeia and it works beautifully. Legato costs too much, and is a bitch to configure. We are using ADIC Scalar 458's as the backup machines and dual 400mhz processor Linux baoxes with three SCSI cards each.

    --

  128. tar | dd by realyendor · · Score: 1

    Here's how I run my backups of several gigs. /dev/tape is a 20GB DLT, although when compressed, I'll load 180GB of web logs, easy.

    tar cvzf - filelist | dd of=/dev/tape bs=1000k

  129. Bug in GNU tar for incremental backups? by Pelerin · · Score: 1

    If anybody has any info on this, I would greatly appreciate it.

    I've used GNU tar v 1.12 with great success on my home machine, doing full and incremental backups. I've even rebuilt my system from the backups without any problems.

  130. And at only NZ$15,000 per license, it's a steal! by sam_vilain · · Score: 1

    It's a bit of a worry when the backup license costs more than a DLT7000 tape drive.

    --

  131. AMANDA by Wodin · · Score: 2

    AFAIK Amanda uses dump or tar behind the scenes.

    --
    -- Wodin
  132. Re:And at only NZ$15,000 per license, it's a steal by travisd · · Score: 1

    Yes, the main server *is* a bit pricey, but folks generally don't use it behind a single DLT drive. The client-licenses are actually about US$150/each once you are licensed for a particular type of OS - i.e. you'll pay a fee for Unix, for NT, and Netware but the licenses themselves can be used for *any* platform that your licensed for. Not cheap, but it gets the job done.

  133. tar-1.12.64011 for files > 2GB by gael · · Score: 2

    there is at alpha.gnu.org and mirrors a version of tar which supports 64 bit file access and which can be used with glibc >= 2.1 to make archives bigger than 2 GB on 32 bit hosts. I just have downloaded it and it compiles and checks succeed without any problem. I just can't really test it because I have only 50 MB left...

  134. Re:Backups & dump by Ineversaidthat · · Score: 1

    Sounds like dump has a misconception of your density. Try 'mt status' and 'mt densities'
    and use -d to tell dump the new number...

  135. Re:Kludge, but should work.. by [l0l]Bobo · · Score: 1

    Yes, we know that, but the guy was having probs with anything over 2Gb, so we can assume he has a 32-bit platform. So what you're recommending is that he changes his machine? Nice. But we were concentrating on less expensive solutions.

  136. Kludge, but should work.. by [l0l]Bobo · · Score: 2

    If you split your backup in a few different files (say, backup ~ftp/pub/mirrors and then ~ftp/pub/linux or whatever) then you should be able to get away with tar.

    This will change soon when SGI releases portions of xfs as open source, and when ext3fs is ready.

  137. 32 bit FS has nothing to do with 32 bit processor by Ace_ · · Score: 1

    The limitation is placed by ext2fs, which is a 32 bit FS. SGI is going to open source XFS soon though, and that is a 64 bit journaling FS, and it will probably (hopefully?) be incorporated into Linux.

    --
    -- Ace
  138. Linux Newbie Q: TR4 Backup by sp99 · · Score: 1

    Interested to see mention of TR4 tapes being used for Linux backup. Have a HP Colorado TR4 tape drive [parallel port connect] currently for Windoze. Can this be ued for Linux Backup? The HOWTO's I've read all refer to SCSI tapes. Any pointers? TIA.

  139. CTAR by jaimec · · Score: 3



    I have used a program called CTAR to cure these problems. Check it out here.

    http://www.ctar.com

  140. If you can't click on it it's not ready? by Olivier · · Score: 1

    I get the feeling that an OS is enterprise ready
    when you can click with your mouse.

    I always have had the feeling that an OS was
    enterprise ready when there was (hardly) any
    maintenance needed after setup. Perhaps I'm wrong.

    Another thing: tar is slow, and a backup over SMB
    is faster? You have the TCP/IP over SCSI running?
    (oh sorry, NT doesn't support that)

    just my 2%