Slashdot Mirror

← Back to Stories (view on slashdot.org)

'Zodiac Island' Makers Say ISP Worker Wiped an Entire Season

Posted by timothy on from the when-backup-day-comes-too-late dept.

itwbennett writes "The creators of 'Zodiac Island' say they lost an entire season of their syndicated children's television show after a former employee at their Internet service provider wiped out more than 300GB of video files. eR1 World Network, the show's creator, is suing the ISP, CyberLynk of Franklin, Wisconsin, and its former employee, Michael Jewson, for damages, saying CyberLynk should have done a better job of protecting its data."

45 of 228 comments (clear)

  1. Backups by Anonymous Coward · · Score: 5, Insightful

    This is why you need them.

    1. Re:Backups by bmo · · Score: 4, Informative

      "Only wimps use tape backup: real men just upload their important stuff on ftp, and let the rest of the world mirror it ;)" - Torvalds, Linus (1996-07-20)

      --
      BMO

    2. Re:Backups by White+Flame · · Score: 3, Funny

      Good thing Torvalds never used CyberLynk's FTP hosting.

    3. Re:Backups by mwvdlee · · Score: 2

      You mean like the backups that a company might pay an ISP to make?
      As was the case in this story, according to TFA.

      --
      Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
    4. Re:Backups by PacMan · · Score: 2

      "multiple offsite backups" aren't the problem. A single "onsite" copy would have served in this case. A pocket-sized USB drive would apparently have held all the data, recent home PC drives could have held multiple copies, for very little cost.

    5. Re:Backups by mr_mischief · · Score: 2

      It's the mirroring part that provides the safety net. In fact, if you're using an ISP that doesn't state explicitly in their ToS that backups are your responsibility and that any backups they provide are a best-effort courtesy service then find yourself an ISP that'll still be in business in six months to move to. Yes, I've worked in the ISP and hosting biz. I have for over a decade, and I now work for one of the big boys.

    6. Re:Backups by hairyfeet · · Score: 3, Interesting

      The sad part is if you read TFA (I know, but I got bored) then you'll see they paid for backups as part of their service agreement but the ISP lied and hadn't actually bothered to back up shit.

      Now considering how we have a "fuck everything but the quarterly earnings report!" attitude going on in businesses right now I have to wonder: How widespread is this? After all backup and the tapes or HDDs to put them on cost a pretty penny, so not actually spending that money makes your bottom line look good, at the same time saying you have a backup solution (which you charge extra for) is equally good for your bottom line.

      Now considering the fact that if these clowns would have followed best practices and changed the passwords when they fired this guy they probably STILL be getting away with charging for a service they don't actually have to incur the expense of actually providing I have to wonder, how many others are doing the same right now? I mean how many are actually gonna set up a test to see if their hosting company has the backups they say?

      It sounds to me like backup services are just one more way to cut expenses while making extra money, and sounds like it is ripe for abuse like in TFA.

      --
      ACs don't waste your time replying, your posts are never seen by me.
    7. Re:Backups by camperdave · · Score: 2

      Goes to prove the point that if you don't have two backups you may as well have none...

      Fry: What happened?
      Dr. Zoidberg: All six thousand hulls have been breached.
      Fry: Oh, the fools! Why didn't they build it with six thousand and one hulls? When will they learn?

      --
      When our name is on the back of your car, we're behind you all the way!
    8. Re:Backups by Comrade+Ogilvy · · Score: 2

      Right. But with how many insurance companies do you insure the same car, in case the first turns out to be criminally incompetent? And do you keep a ship-shape spare copy of your original car in the garage, just in case?

      This was a distributed, collaborative effort, with the copy of record constantly changing in small ways. It was perfectly reasonable to ask a professional to take care of the back up, and expect to never lose more than a week's worth of work in the worst case scenario.

      This will never see the inside of a court, because the ISP would simply lose. They were paid for a critical service that they chose not to perform.

  2. Torrents by White+Flame · · Score: 5, Funny

    They preserve culture.

    1. Re:Torrents by SuricouRaven · · Score: 4, Insightful

      A good point though. Businesses go out of business. Some television of the past has even been deliberatly destroyed for legal reasons, or because it is embarassing to the company today. Still more can no longer be shown for the same reason, and remains locked up in a vault somewhere. VHS tapes degrade quickly, but now the pirates have digital technology, they do serve to preserve - thousands of people with their own stores, independant, backups for each other. They can't be legally compelled to destroy anything, because they just don't care. Companies come and go, but so long as someone is willing to replace the occasional failed hard drive, a pirate collection is forever.

    2. Re:Torrents by dadelbunts · · Score: 2

      Kind of like abandonware. Most abandonware titles are impossible to find in a hard copy. In many cases the companies that own the rights arent around anymore. Kind of seems like digital media is harder to preserve than physical media to me sometimes as we take it for granted how easy it is to backup, then no one backs it up.

    3. Re:Torrents by Cederic · · Score: 2

      Some television of the past has even been deliberatly destroyed for legal reasons, or because it is embarassing to the company today.

      Or because it cost too much to retain it.

      There's a distressing amount of BBC material that's gone forever, very intentionally, primarily due to cost reasons.

    4. Re:Torrents by cffrost · · Score: 2

      "Stealing" does not mean same thing as "preserving".

      "Copying" doesn't mean the same thing as "stealing."

      --
      Thank you, Edward Snowden.

      "Arguments from authority are worthless." —Carl Sagan
    5. Re:Torrents by petermgreen · · Score: 4, Insightful

      True, even a huge chunk of the classic Doctor Who episodes would have been lost, had it not been for archives abroad.

      Even with the foreign "archives" (which afaict were often just rolls of film forgotten somewhere) some are still missing and many were recovered in poor condition requiring heavy restoration.

      And IIRC we only still have the famous silent film "metropolis" because people who were contractually obliged to destroy their copies didn't actually do so.

      --
      note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
    6. Re:Torrents by Jon_S · · Score: 2

      And according to MS what we need is a "hybrid cloud" model. Just keep nodding.

    7. Re:Torrents by PopeRatzo · · Score: 2

      "Stealing" does not mean same thing as "preserving".

      In terms of culture, it certainly does. There is no surer way of preserving, enlivening, spreading culture than "stealing".

      --
      You are welcome on my lawn.
    8. Re:Torrents by PopeRatzo · · Score: 2

      For a time it was thought that the safest/best place for the treasures of Egypt was England, whether the Egyptians liked it or not.

      The great thing about torrents (or filesharing generally) is that it leaves the original "treasure" in the hands of the owner. It allows more people to enjoy it than would be possible. Only a certain number of people can visit the pyramids or the artifacts of the pharaohs. Everybody can enjoy the torrent-copy of Keith Jarrett and Charlie Haden's Jasmine or the latest mix from Burial or the Hilliard Ensembles recording of the Bach motets. Or the latest from the great Die Antwoord.

      --
      You are welcome on my lawn.
  3. World Backup Day by erice · · Score: 2

    I guess they didn't hear that it was World Backup Day

    1. Re:World Backup Day by toastar · · Score: 5, Funny

      Who's fucking idea was it to make April fools day World Backup day?

    2. Re:World Backup Day by erice · · Score: 2

      TFA is very clear about this: the ISP was responsible for making backups, and failed to do so.

      Yes, but anyone who relies on ISP backups for important data, on an ftp site, no less, is an idiot. The only way this story makes any sense is if all they managed to trash all their local copies, including backups (if any), and then looked to the ftp site as a backup of last resort. The ftp site files were almost certainly not in condition to broadcast. Their loss means that the creators can blame someone else for the screwup and not have to redo all their work.

    3. Re:World Backup Day by cgenman · · Score: 2

      If you're relying upon the ISP to have backups, you don't have backups. What if that ISP goes under? Gets hit with a flood? Servers locked up by an FBI investigation? Or, as in this case, an employee goes on a deleting rampage?

      Don't just backup your data. Backup your providers. Backups are about redundancy.

      And never personally verifying that the ISP had backups? They might as well have used prayer as a data-protection methodology.

      --
      The ______ Agenda
    4. Re:World Backup Day by CharlyFoxtrot · · Score: 2

      Yes, but anyone who relies on ISP backups for important data, on an ftp site, no less, is an idiot.

      The only way this story makes any sense is if all they managed to trash all their local copies, including backups (if any), and then looked to the ftp site as a backup of last resort. The ftp site files were almost certainly not in condition to broadcast. Their loss means that the creators can blame someone else for the screwup and not have to redo all their work.

      A cheap idiot at that. We're talking about 300Gb of data. I'm guessing a production company can cough up 50 bucks for an external 500Gb hard drive. They might have even have had enough left over to splash out on a second one.

      --
      If all else fails, immortality can always be assured by spectacular error.
    5. Re:World Backup Day by Kilrah_il · · Score: 4, Insightful

      Re: April fools.
      I am now announcing that for the next 24h I will not believe any story not originating from Fox News. Since all the major (i.e. serious) papers print fake/prank stories today, I guess it's Fox's time to pull the major prank - print out a real, accurate, fact-filled news item, for once.

      --
      Whenever in an argument, remember this.
    6. Re:World Backup Day by mr_mischief · · Score: 2

      I believe you should be able to pay for that, too. But suing someone because they say "we backup our servers" and you think that means they guarantee 100% data retention is just the worst kind of careless oversight. If you find the ISP's website as I did and read the description of services and the ToS, there's no way an informed consumer would believe they are getting 100% guaranteed data backup. The company disclaims even being fit for use for any particular purpose.

    7. Re:World Backup Day by monkeythug · · Score: 2

      I can see you get you're concept of irony from Alanis Morrisette.

      It's not ironic, it's just unfortunate.

      --
      Don't you wish you hadn't wasted 3 seconds of your life reading this sig?
    8. Re:World Backup Day by maxume · · Score: 2

      The thing about that song is that it is ironic.

      --
      Nerd rage is the funniest rage.
    9. Re:World Backup Day by Entrope · · Score: 2

      Standard form disclaimers like the one you quote are typically superseded by the agreements that are specific to the relationship in question. If a customer pays extra for backups, judges tend to be unsympathetic if the vendor tries to weasel out by the kind of argument you suggested ("we never promised we could RESTORE backups! and we told them it was at their own risk!"). Otherwise the additional agreement and charge for the backups gets the customer nothing but a bill of goods.

      The show's creators really should have spent the hundred dollars or whatever to have an on-site backup of the data, though.

  4. Haven't you the master copy or just any copy? by VincenzoRomano · · Score: 2

    Backup is a very big word, guys.
    I mean, haven't you any other copy?
    Who designed your production processes, Pinocchio?
    Information technology is not just a bulb light that just works by plugging it in. It's (just a little bit) more complicated and yet (much) more powerful.
    Shame on you, then!

    --
    Maybe Computers will never be as intelligent as Humans.
    For sure they won't ever become so stupid. [VR-1988]
  5. Welcome to /. hell day!!!! by Dutchmaan · · Score: 3, Insightful

    Story post time is officially Apr. 1.... it's /. hell day...

  6. Re:Am I missing something here? by maxwell+demon · · Score: 2

    Probably the worker saw the FTP server full of copyrighted movies, and thought "better wipe them before we get any legal trouble; to be sure, better also delete the backups." :-)

    --
    The Tao of math: The numbers you can count are not the real numbers.
  7. Re:Sorry, but by White+Flame · · Score: 2

    They used the ISP's FTP hosting as a collaboration point between the different companies spread across the planet (animation studios, live action studios, editing, etc), and it was part of the deal that backups be done at the ISP itself. Yes, it's a non-redundant setup as opposed to having replication across all sites, but they did have a paid-for backup service that unfortunately didn't do their job.

  8. Re:Sorry, but by jjohnson · · Score: 2

    If I'm paying the hosting company to maintain backups, then that's my backup plan, and it's not unreasonable for a small business to rely on it if IT isn't a major part of their business. If they had 300GBs of storage in use, then they had a serious account that almost certainly included guarantees of regular backups. The ISPs admission that their backup system failed says as much.

    --
    Anyone who loves or hates any language, platform, or manufacturer, doesn't know what they're talking about.
  9. In their defense by Sycraft-fu · · Score: 2

    It sounds like their ISP was supposed to do that. I've nothing wrong with paying other companies to do something for you. Not every company has the resources to do everything. You outsource things to experts. However that means you presume they do it right, and do what they say. The ISP said "Ya no problem we back this up." And then it turned out they didn't.

    1. Re:In their defense by Cylix · · Score: 2

      It's fairly common practice to keep the raw video in case you need to do something with it. It's generally higher quality, free from effects and can be remixed as needed. In the event the finished product is wiped out then the show can be reproduced at some cost.

      With one of the previous companies I was with we spent so little on technology that it wasn't uncommon to lose the primary file server. Eventually, after the third or fourth reload plus reproducing they eventually opted to invest in some backup technologies and secondary file servers.

      While it is April fools day I have seen this scenario too many times. I believe there is some rule to humor that indicates a situation is humorous when it is on the unbelievable side. This doesn't strike me as terribly unbelievable.

      --
      "You should always go to other people's funerals; otherwise, they won't come to yours." -- Yogi Berra
    2. Re:In their defense by yeshuawatso · · Score: 2

      Mod parent up; someone read the article. This is exactly on point to why this will be settled out of court with a couple million exchanging hands, and A LOT of the ISP staff will be terminated for not doing their jobs. Plus, there's the PR stuff to reassure customers, dealing with those that lost non-essential data, and of course, the loss of customers who will simply exit and go somewhere else. The worse thing this could have done was make it to a lawsuit.

    3. Re:In their defense by Anonymous Coward · · Score: 4, Informative

      It's fairly common practice to keep the raw video in case you need to do something with it. It's generally higher quality, free from effects and can be remixed as needed. In the event the finished product is wiped out then the show can be reproduced at some cost.

      RTFA. This was not yet a finished product. They were files that had been passed back and forth between artist/animators/etc for the last 2 years while developing the show. It was a remote, collaborative effort that was still ongoing. So these were essentially the unfinished source files that got lost. The article says that while 300GB were wiped, they only permanently lost 65GB of data. I'm assuming the other 235GB were files that the various contributors still had their own local copies of.

  10. Re:Sorry, but by Eivind · · Score: 3, Insightful

    It's unreasonable to "rely" on ANY backup-plan whatsoever, without actually regularily testing RESTOREs.

    If you buy backup - which is fine - make sure to actually test a restore, and do so REGULARILY.

  11. RTFA and it does not make much sense by Ecuador · · Score: 3, Insightful

    It was an off-site FTP server for collaboration, are they telling us none of the collaborators had the full set of data? It was "just" 300GB, meaning it could fit easily on an average hard drive.
    Furthermore, they say they require all the data to reconstitute the episodes, so every time they needed the episodes, they would download all those 300GB of 6000+ files from FTP and rebuild their episodes? What kind of idiocy is this.
    And lastly, did that employee secure erase everything? It was more than a simple rm -rf ?

    --
    Violence is the last refuge of the incompetent. Polar Scope Align for iOS
  12. Q: because it breaks the flow of a message by DNS-and-BIND · · Score: 3, Funny

    A: Why is starting a comment in the Subject: line incredibly irritating?

    --
    Shutting down free speech with violence isn't fighting fascism. It IS fascism!
  13. The rules by sjames · · Score: 4, Insightful

    Rule 1, if you upload it to your ISP, keep a backup.

    Rule 2, if they say they keep backups, keep a backup, theirs may not be very good.

    Rule number 3, if they agree contracturally to make full backups, keep one of your own. They don't care as much about your stuff as you do and they probably have a get out of jail free clause buried somewhere in the fine print.

  14. Re:This is by martin-boundary · · Score: 3, Informative
    That's easy to say, but the trend is the other way. How many people here on slashdot even use gmail as their primary email provider and don't ever back up the messages?

    You might say it's not the same thing, but it's not so different for a company to keep some of their most valuable assets in the cloud in one place, and for a person to keep some of their most valuable communications and contacts in the cloud in one place.

    If something is valuable, never trust it wholly to the cloud.

  15. Re:Cost too much to retain it by TaoPhoenix · · Score: 2

    Nope.

    That's the beautiful trap. Companies love to moan about how long tail materials are "too expensive to retain", meaning they're willing for it to vanish forever, but skies alive if you create a college club around it! Copyright terrorists! Sue them!

    If some super-lawyer for EFF wants another angle to chip away at the copyright insanity, that might be an angle: get a statement under oath that something is "too costly to maintain", aka the retention value is negative, and then it becomes one of the CC licenses, perhaps Attribution-Only with commercial use allowed. (I don't think anyone wants to pretend that Da Mouse is their invention, they just want to make mashup derivatives.)

    To my vague recall that's what started the paper shredder industry - something labeled trash is no longer fully protected property.

    --
    My first Journal Entry ever, in 8 years! http://slashdot.org/journal/365947/aphelion-scifi-fantasy-horror-poetry-webzine
  16. 321 rule by TRRosen · · Score: 2

    3 copies
    2 mediums
    1 offsite

    PS stop talking about 'the cloud" like it exists. It's only an abstract concept. Everything is on a real piece of hardware that will fail and controlled by a human that will f*ck up.

  17. Great security and backup practices by grapeape · · Score: 3, Insightful

    And people wonder why fired IT workers are escorted to the door without being allowed to go back to their desks. All it takes is one idiot to make the rest of the company completely paranoid from that point forward. First rule of IT Staffing: When someone leaves...make sure their access leaves with them. The lack of backups however is inexcusable.