Slashdot Mirror
Mega-Uploads: The Cloud's Unspoken Hurdle
First time accepted submitter n7ytd writes "The Register has a piece today about overcoming one of the biggest challenges to migrating to cloud-based storage: how to get all that data onto the service provider's disks. With all of the enterprisey interweb solutions available, the oldest answer is still the right one: ship them your disks. Remember: 'Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.'"
Returning from a site with a tethered computer full of 80 MP 16-bit raw files from a day's worth of shooting would break most bandwidth bills if you tried uploading all these images.
Yeah, the bandwidth is great, but the latency SUCKS.
Second biggest challenge: trusting any of these places have the motivation to keep your data more secure than credit card companies do.
Getting around all the buzzwords
Well that's one hurdle.
The next is RECOVERY when ICE or FBI or some other 3letter agency walks in an takes your data because one tiny customer use the service for some allegedly nefarious purpose.
The key here is to use a service so big that even god himself would not dare take it down, although the Ayatollah might try. Small cloud services, even if multi-homed are a risky proposition. Even if you do manage to get all your data into them, they are not large enough to push back against any subpoena or search warrant that any misguided judge in some backwater jurisdiction may issue.
Sig Battery depleted. Reverting to safe mode.
Comment removed based on user account deletion
My last employer offered offsite backups to clients. For the initial seed, we always tried to get them to put it on an external HDD and ship it to us (or at least DVDs). The only major exceptions were clients that were also on FiOS - that was the only case where over-the-net transfer was faster than the backup-and-ship-it method for the initial seed.
Yes, never underestimate the bandwidth of a station wagon full of disks hurtling down the highway. The latency, on the other hand, leaves much to be desired, and I've heard the packet loss can be downright fatal.
--- Journals are boring; Go to my web page instead
No, the second biggest challenge is to come up with a viable exit strategy. Once you have several TB at this service provider, how will you move it out of there when the next provider has a better deal? That was one of the major big points for having a cloud in the first place, to have the freedom to move your compute requirements to a better, cheaper, faster (pick two) provider.
Even if you moved it in with a station wagon full of tapes or disks and your provider let you import it, I'm sure your provider will not be so helpful when you need to move it back out.
Blatant plug: Perhaps Actifio (www.actifio.com) can fix this for you, by replicating your data in, and also back out of production systems in deduped and compressed format.
To Terminate, or not to Terminate, that's the question - SCSIROB
... is getting it all back OFF again when you want to switch service providers.
The one thing you want never to happen is that you get locked in to a single cloud service. They might go bust, they might become uncompetitive. They may become politically "unfriendly" or tainted with customers you have no desire to be associated with - or any of a number of other reasons to say "adios".
Just like with disaster planning, all the processes and procedures, agreements and SLAs are worthless until you've actually PERFORMED the operation and done so without a major service interruption. How many cloud users have gone that far - and how many are locked in but don't know it?
politicians are like babies' nappies: they should both be changed regularly and for the same reasons
You don't know the exact dialogue between the journalist and the rep. I've been quoted in print in similarly stupid ways when what I said made absolute sense in context to what was asked. "Pressed if disks are accepted" could have been something like the rep telling them about a new CSV import tool they had built, the journalist saying "So if I mailed you a 5TB database on a disk, could you import that?", and the rep replying "Sure, but you'd need to export the data first...".
Bogtha Bogtha Bogtha
You you always use a UDP solution such as Aspera. Fast transfer speeds, bandwidth management and they have a specific AWS implimentation.
Other options to look at include Smartjog, whose new Bolt product looks quite interesting, Riverbed's Steelhead product, Filecatalyst and Signiant.
There are many solutions around now to deal with large file transfers for both small and large business. Most of them use UDP instead of TCP/IP, with Checksums to ensure all data is reliable delivered. Even with just 1Mbps upload speeds, something like one of the above named products will be advantageous. I've worked in the media industry for a number of years, and this type of thing is being used in Film and Television all the time. Of course, there are still tapes being shipped around, but in emerging markets, such as Russia for instance, the file transfer really beats a tape being stuck in customs for weeks or months.
Brought to you by the author of such childrens' classics as "Some Kittens can Fly!" and "All Dogs go to Hell."
How did you manage to fix armed FBI storming your servers located in another country problem?
That is just one of many of the hurdles.
Really, these problems are problems because most 'cloud' shit is done wrong.
It's a bit of a worn out record here on Slashdot, but anyone or any company which is fully dependent upon The Cloud for business continuity is a fool.
* First off, there is no such thing as 'utility computing', and probably never will be due to the volatile nature of storage and its ongoing cost of maintenance.
* Second, if you do not maintain primary physical control of something, to the best of your ability, you do not control it.
* For primary IT infrastructure, it will cost more to do "Cloud" than local. If you can afford 2-3 servers a year, but not much more, and a nominal IT operations budget, chances are you should have an in-house "cloud" with off-site replication.
* Bandwidth costs both ways will kill you, as will latency in many cases, will kill Cloud functionality.
At this point, I still strongly recommend against public Clouding your systems unless they are:
a) (very!) low volume with use-based billing. This only makes sense for a low-volume public-facing site where you don't already have IT infrastructure (on a cost basis)
b) off-site 'hot' replication. You've got your inside 'private Cloud' which replicates to off-site systems. (Cloud is basically just colocated virtualization, after all.)
c) Other geographic/distribution requirements (eg. multisite organization with none serving as a good central hub). In this case, colocation of your own equipment makes more sense in many regards.
~/ssh slashdot.org ssh: connect to host slashdot.org port 22: too many beers
In canada, unless you need low latency, the internet is about the most expensive method you could possibly use to transfer data. source
African or European?