Slashdot Mirror


The Risk of a Meltdown In the Cloud

zrbyte writes "A growing number of complexity theorists are beginning to recognize some potential problems with cloud computing. The growing consensus is that bizarre and unpredictable behavior often emerges in systems made up of 'networks of networks,' such as a business using the computational resources of a cloud provider. Bryan Ford at Yale University in New Haven says the full risks of the migration to the cloud have yet to be properly explored. He points out that complex systems can fail in many unexpected ways, and he outlines various simple scenarios in which a cloud could come unstuck."

48 of 154 comments (clear)

  1. It has to happen by Anonymous Coward · · Score: 5, Interesting

    At some point, there is going to be a massive failure. Someone big is going to lose *all* of their data. I still don't trust virtualization despite it being years old. It's still nascent in the grand scheme.

    Someone wake me when they invent the holodeck.

    1. Re:It has to happen by Chrisq · · Score: 5, Funny

      At some point, there is going to be a massive failure. Someone big is going to lose *all* of their data.

      I just hope its my mortgage company and not my bank.

    2. Re:It has to happen by Anonymous Coward · · Score: 3, Insightful

      Well, lots of people lose data or access to data "in the cloud" (i.e. "on the mainframe", to use '60s IBM parlance) all the time. Catastrophic losses happened in the '60s, happened for that short period of enlightenment when we thought we should have control of our own data, and will also happen as we revert to the past. The usual questions apply:

      - What are the motivations of your service provider? I.e. how much do /they/ care if your data is lost. This will determine how hard they try not to lose it. Recall that a businesses future depends on the interests of its owners and executives - there is no maxim requiring that the business last as long as possible, especially not in the field it is currently in;

      - Who else can bother you? I.e. who wants to access your data and who of those people has the resources to pay government or crackers?

    3. Re:It has to happen by gweihir · · Score: 3, Funny

      Well, as the holodeck is tactile, I expect it will kill quite a few people before the kinks are worked out.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    4. Re:It has to happen by Anonymous Coward · · Score: 4, Insightful

      If someone loses "all" their data in the cloud, their problem has nothing to do with the cloud. If you lose all your data, it's because you kept all your data in one place, with no backups in a different place, and all fault lands on you, not the cloud, not your cloud provider, and not on any given piece of technology. There have already been large failures, and some companies have already lost massive amounts of data, but it doesn't change anything, because these problems have nothing to do with whether you host your own servers or rent them from somebody else, which is really all "the cloud" boils down to.

      Also, inherently not trusting virtualization as a concept in 2012 is is moronic and baseless. It's a technology, just like any other. It can be implemented well or it can be implemented poorly, but as a concept it is not novel or revolutionary to any degree that it should engender trust or distrust, at least not any more than the hardware underneath it.

    5. Re:It has to happen by jedidiah · · Score: 5, Insightful

      If the cloud is not more robust than what your grandma could come up with on her own then what's the point really?

      Isn't the whole point of "the cloud" the fact that you aren't managing this stuff yourself? You don't have the burden? You don't need the expertise?

      If you push it back on the cloud consumer then a lot of it is really quite pointless.

      --
      A Pirate and a Puritan look the same on a balance sheet.
    6. Re:It has to happen by lightknight · · Score: 4, Insightful

      "If the cloud is not more robust than what your grandma could come up with on her own then what's the point really?"

      Money.

      --
      I am John Hurt.
    7. Re:It has to happen by datavirtue · · Score: 4, Insightful

      I keep remembering the Google applications that keep disappearing. Data is one thing, application providers going out of business or discontinuing a service is another issue entirely. Hopefully the competition still stands and you can migrate your applications over to their service.

      --
      I object to power without constructive purpose. --Spock
    8. Re:It has to happen by Anonymous Coward · · Score: 4, Funny

      But, but... you don't understand. You're money's not here. It's in Joe's house, and Jimmie's house and... it really is a wonderful life.

    9. Re:It has to happen by jriding · · Score: 5, Insightful

      Just a thought. Forget actual failure. What happens when they have IP data or licensed data that is being hosted by a cloud provider, or company to company lawsuit. Court case starts. Could or would they they seize all computers / servers that could house the data? What would happen to the other peoples data that resides on the same physical hardware?

      --
      love the taste, hate the texture
    10. Re:It has to happen by DarkOx · · Score: 4, Informative

      I am sorry but we have been virtualizing things by one name or anything going back to 1960's mainframes. In other words almost as long as commercial computing has existed.

      The cloud is a different matter. The issue is not with virtualization but with creating dependencies on and between parties who don't really talk to each other.

      --
      Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
    11. Re:It has to happen by DarkOx · · Score: 3, Insightful

      If the cloud is not more robust than what your grandma could come up with on her own then what's the point really?

      That is exactly it. The cloud is more robust than what grandma could come up with or what I have time to manage for her. Grandma has some family photo's, a cookie recipe or two, and *maybe* some financial statements etc. Some of it might be tragic to loose but its a loss we can live with.

      The F1000 I work at on the other hand certainly can build something more robust at least as relates to their specific needs; if they simply put up the dollars. Certain parties are trying bill the cloud as way to save money without sacrificing reliability. Perhaps it does offer good security against traditional risks of hardware failure, run away support costs, etc; but it brings new risks to the table as well. The truth is as an industry we know less about identifying, controlling and mitigating those risks than we do about in house solutions. That is a point that is being missed by lots of decision makers.

      --
      Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
    12. Re:It has to happen by CAIMLAS · · Score: 4, Interesting

      While I agree with you, a little perspective.

      I've seen systems, verified backups, and duplicate backups simply fall over. Every best practice was followed, backups were taken regularly, and the backups had been verified by Industry Leading Backup Software Everyone Still Uses (arg). But the system and data on the backups could not be restored.

      It eventually got fixed through some virtualized gyrations, but it took the better part of a week for the company's top engineer to figure it out.

      Shit happens.

      --
      ~/ssh slashdot.org ssh: connect to host slashdot.org port 22: too many beers
    13. Re:It has to happen by postbigbang · · Score: 4, Insightful

      Yes, a lot of people lose data on mainframes and the problem was generally behind the console. Online production systems and job-based systems need to be designed with practicality and component failure in mind.

      The cloud is a system, and the system needs redundancy, checks and reality checks, and quality ins-and-outs. That's right, just like what you've been doing all along. Same security, same backups, same contingencies-- as there are no shortcuts, just cheaper hardware.

      I have to dismiss the cited article as it's entirely ephemeral, with not one single citation to back it up. Financial markets, while important, are a somewhat unique context to cite; they're built differently than many cloud components. There is nothing tangible cited, no case history, just some Yale-y's bad, after-lunch tummy growling coinciding with thinking about the complexity of modern infrastructure.

      This whackamole approach serves no purpose, except to sell more inside, rather than outside, infrastructure.

      --
      ---- Teach Peace. It's Cheaper Than War.
    14. Re:It has to happen by Anonymous Coward · · Score: 3, Interesting

      I remember reading article a couple of years ago where the feds siezed everything at a hosting provider because *one* of their customers had been naughty. This naturally drove several unrelated businesses into the ground because they couldn't get to their data.

    15. Re:It has to happen by TekPolitik · · Score: 2

      What happens when they have IP data or licensed data that is being hosted by a cloud provider, or company to company lawsuit. Court case starts

      IP in the cloud is worse to deal with than you can possibly imagine. For starters, when somebody grants you a license to use IP, as often as not (and especially in the case of IP licensed to big companies) the licence is restricted to a particular country. This is in part because your IP is a different thing in each country, governed by different rules. If you go storing licensed IP in the cloud, you don't know where it is going to end up - you have a very good chance of breaching your licence. If you think "that's OK, I'm not storing somebody's licensed IP", think again - unless you are wrote it (or are simply using somebody else's IP without a licence, in which case you have the problem anyway), then you are.

      Then you have the problem that you likely haven't got the first inkling as to how intellectual property works in the, most likely third world country (if not now, then eventually), where the data is going to be stored.

      If you're dealing with confidential information, can you be sure some minimum wage flunky you have never even met is not going to be prepared to sell it for enough money to keep them and their family in comfort for years to come? Can you be sure the law in the country where it happened even cares? The criminal element that wants to sell your private data isn't so much sitting behind a keyboard in their mother's basement writing viruses or using skripts to break into your systems - they're getting jobs at places like Google in their data centres, possibly with a fake resume with their buddies giving fake references.

      Then you have the "cloud provider goes out of business or discontinues the service" issues (which are worse if the data is in a proprietary format).

      The biggest problems with the cloud are not technical issues (although there are technical issues any time you keep your data "there" rather than "here"). The big problems are the law and people issues. From that perspective, the cloud is a huge risk. If you are capable of safely storing your data and maintaining your systems without the cloud, then you should do so. Leave the cloud for people who cannot look after themselves.

    16. Re:It has to happen by Raenex · · Score: 2

      Even your grandmother can copy some files to a USB drive and chuck it in a drawer as a backup to the cloud. The cloud offers on-demand scalability, accessibility while traveling, and offsite storage should something happen locally. It's not an either-or proposition.

  2. Why not stick to real risks? by msobkow · · Score: 5, Insightful

    I don't understand the intent of the article other than to provide a knee-jerk chicken-little response to cloud processing and storage.

    Not one of the items mentioned is unique to the cloud. It can happen to any data center with more than two nodes involved in a cluster.

    But that's not surprising, because "the cloud" is just a distributed collection of cluster servers, the same as large multi-nationals have been running pretty much since their customer loads exceeded the ability of one server to span the global community.

    --
    I do not fail; I succeed at finding out what does not work.
    1. Re:Why not stick to real risks? by Anonymous Coward · · Score: 5, Interesting

      Problem is that half the people buying cloud services thing their system is imune to any new problem, and might not implement the same failback procedures they would have in a traditional way.

      We are in many way seeing a new generation of IT people and managers with far less understanding of the fragility of their system then the previus generation, emerge onto the scene.

    2. Re:Why not stick to real risks? by bonkeydcow · · Score: 2

      I agree. Too many people seem to think of computing as magic. Microsoft's cloud suffered a massive outage on Feb 29. All is well now, there is no ghost in the machine.

    3. Re:Why not stick to real risks? by NatasRevol · · Score: 5, Insightful

      Doubtful.

      More likely, bean counters don't listen to the IT people who say offline backups are important. Beancounters just hear extra expense, think everything in the cloud is secure, and deny redundancy/backups.

      --
      There are two types of people in the world: Those who crave closure
    4. Re:Why not stick to real risks? by gweihir · · Score: 5, Interesting

      In the cloud, you have no idea who else is on the same hardware as you and what their usage patterns are. You cannot be careful yourself anymore, you have to trust your service-provider. The past shows that unless you have huge contractual penalties in place, that is a losing game.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    5. Re:Why not stick to real risks? by Opportunist · · Score: 5, Funny

      This. A million times this!

      Our brass is also abuzz with "the cloud". Nobody has the foggiest (pun intended) idea what it is about, what it is or what the hell is going on, but it is the best thing since bread has been sliced. The cloud. It will help us safe millions. How? I dunno, I don't care, but it does! We hear it everywhere, we see it in our manager magazines, and there it is kinda-sorta explained but I didn't understa... I mean, I didn't read it throughly, I don't have the time, my time is valuable, ya know? But it will help us cut costs in a big way, we gotta push towards the cloud!

      The risks? They are not aware of the risks. They don't even know they exist. Why would they exist, they didn't exist so far, right? Redundan..what? Backup... whazat? Now why the hell do we need that suddenly?

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    6. Re:Why not stick to real risks? by Dan667 · · Score: 4, Insightful

      "the cloud" is just dumb terminals all over again. After businesses make a bunch of money on the cloud they will then start selling local solutions again to try and mitigate problems with being on the cloud.

    7. Re:Why not stick to real risks? by Lennie · · Score: 2

      Sill 4 year until the next leap day :-)

      --
      New things are always on the horizon
    8. Re:Why not stick to real risks? by lightknight · · Score: 2, Insightful

      Indeed. I can't believe that after we had finished ditching mainframes as a technological way of life, the pointy-heads want to go back to it. Like we missed something the last time we were there...

      --
      I am John Hurt.
    9. Re:Why not stick to real risks? by datavirtue · · Score: 2

      It isn't magic, but things do get spooky when you start hooking together disparate systems (none of them have ANY bugs, right?) and interdependencies. From my point of view "cloud" is a marketing strategy of large corporations aimed at managers and generally non-technical people. Microsoft has been pushing for subscription based software as soon as they determined the internet wasn't a fad. Sure, there are some neat cloud services and APIs available (Amazon, Google, FB, Twitter, etc...), but considering them for mission critical status is a whole other ball of wax. If it is not magical like a shiny iPhone then control-freak business owners and executives are not going to be interested in giving up control of their infrastructure. No magic == no deal.

      --
      I object to power without constructive purpose. --Spock
    10. Re:Why not stick to real risks? by eljefe6a · · Score: 2

      I published my research in to cloud costs. Maybe that will help your company. http://www.jesse-anderson.com/2012/02/ec2-performance-spot-instance-roi-and-emr-scalability/

    11. Re:Why not stick to real risks? by SecurityGuy · · Score: 3, Informative

      Yes, and the amount of data we work with is massively bigger. "Cloud" is a massive victory of marketing, not of technology.

  3. Oscillator by vlm · · Score: 4, Informative

    The TLDR version of the article is that load balancers can oscillate.
    Its spun into a cloudy-thing because thats trendy, but the basic argument is nothing new.
    Perhaps there's more "meat" in the original paper?

    One common thread is that nothing is ever really "new" in computer science / IT. Clouds are just a rehash of ye olde mainframe outsourcing from decades ago. I worked at a place that was doing that in the early to mid 90s.

    --
    "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
    1. Re:Oscillator by dkf · · Score: 5, Insightful

      The general point is that it's possible to get bad emergent behavior which is unexpected. This shouldn't be surprising to anyone (but it is, alas). We see it over and over in complex systems, and it's got pretty much nothing to do with what you implement the complex system with.

      What to do about it? Well, the only real fix is to stop the drive for efficiency at all costs. All those little inefficiencies that hit your bottom line, they also mean that when things go wrong you can weather the storm more easily. And yes, that resilience means things are going to cost more. How much more? Well, depends how much risk you want to take out of the system and how much you're willing to pay. Your call. (A local backup removes a lot of risk from things like cloud providers going belly up unexpectedly, but it does mean you're stuck with actually having to pay real money to do the backup and make sure it is working.)

      --
      "Little does he know, but there is no 'I' in 'Idiot'!"
    2. Re:Oscillator by cpu6502 · · Score: 2

      There's more to the article than just "load balancing". QUOTE: "An obvious example is the flash crashes that now plague many financial markets in which prices plummet dramatically for no apparent reason. Understanding how and why this happens is the focus of much research. Given that cloud is clearly becoming a network of networks that is rapidly growing in complexity, it's not hard to imagine that the computing equivalent of flash crashes are not just likely but inevitable."

      Flash crash - When the stocks, as measured by the DOW Index, suddenly drop several hundred points due to computerized trading. It's still unclear this happens.

      --
      My AC stalker: " I personally agree with your posts most of the time, but that won't keep me from modding you troll"
    3. Re:Oscillator by IcyHando'Death · · Score: 5, Insightful

      Pardon me mods, but +4 informative? This is a terrible summary from someone who doesn't seem to have understood what he's read. The novel "cloudy-thing" aspect of the article's argument is the very part the parent misses when he dismisses this as "nothing new".

      The cloud is an abstraction that intentionally hides detail. Cloud providers do that to make the service being offered simple to package, sell and use. They also do what they can to keep the tricks of their trade secret from competetors. But their infrastructure is actually very complex relative to what the average small to medium client would need for themselves. This is important in three ways:

      1. 1) Your own engineers can't take all aspects of a deployment into account when making decisions.
      2. 2) As a moderately sized company, using the cloud will expose you to the risks of emergent behaviour that would simply not be an issue on the smaller scale you would operate on if you ran your own infrastructure.
      3. 3) Your system may be humming along smoothly one moment, then start thrashing disasterously the next in the absence of any action on your part and for no apparent reason, simply because your cloud provider has tweaked some seemingly innocuous parameter (even after extensive testing)

      This is an important and novel issue and worthy of some real consideration.

  4. Cloud services should complement, not replace by sandytaru · · Score: 5, Insightful

    I think the safest bet is to have local copies in addition to copies in the cloud, even if all the processing and computing is actually done in the cloud. Companies should set stuff up to keep a local copy of critical services on a good old fashioned tape drive or backup server. This is sort of a reverse of the cloud based backup solutions, where local processing and databases took place on local servers, but had backups in the cloud in case of a local disaster. Same idea: Have a local backup in case of the meltdown of the cloud. You may find your primary app is temporarily useless, but you at least have all your critical data (hopefully in a format that can be transferred.)

    --
    Occasionally living proof of the Ballmer peak.
    1. Re:Cloud services should complement, not replace by __aajwxe560 · · Score: 3, Insightful

      As a SaaS Operations Manager, I believe you are spot on. While I have no major issue with sending data up to the magical "cloud" in many cases, shame on me if I am not taking some responsibility on my own for backups, etc. Even if a cloud provider provides an SLA and claims full backups, should that fall apart, its still going to be my butt in the sling for recovery.

      Their are of course numerous technical solutions to the problem, including cross-site replication, etc. The challenge is the more timely/accurate the replication, usually the higher the cost (bandwdith/limit latency expenses). When the finance folks start to see these numbers, or the customer, suddenly it becomes less "urgent" to them. The trade-off is often times a subtle expectation of HA due to potential direct impact on company profitability, but nobody wants to actually pay for it. I am aware of way to-many orgs that cut corners in this area and simply lie to the customer.

      Hence, to your point, hedge your bets and do the best you can with regular backups

  5. How funny, I was just talking about this yesterday by drinkypoo · · Score: 2

    Well, it's funny to me. But I was thinking that what you've got is a bunch of like computers operated by a single organization, connected by management networks, and depended upon by thousands of other organizations. If there was ever a sweeter target for a virus I don't know what it would be.

    --
    "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
  6. Obligatory XKCD by Neriak · · Score: 2, Funny
  7. Cloud Computing == Banking by ZombieBraintrust · · Score: 3, Insightful

    I think the best metaphor for cloud computing is banking. There are risks for doing it yourself and risks in trusting a bank. If you hold your money in a safe then you may be robbed by a family member, employee, or petty theif. If you keep your money in the bank you may be robbed by the goverment, bankers, or by proffesional bank robbers. This same dynamic is true of storing data. The big difference between cloud computing and banking is regulation. Federally insured banks are heavilly regulated. There are laws about home much money they can lend, who they can hire, and what rates they can charge.

  8. Example: Kindle's cloud by cpu6502 · · Score: 3, Informative

    A coworker discovered when he upgraded from Kindle 1 to Kindle 2, many of the items he had purchased were no longer in the cloud (as amazon had promised). Most of what he lost was periodicals like magazines, but also some books. He was not a happy camper and asked for a refund for those books he could no longer acces, but Amazon simply told him they are not responsible.

    That was back in 2009 if I recall correctly so maybe some of the bugs have been worked-out, but I stored it in memory as a reason why I won't trust the cloud to store any books I might purchase (or anything else). I try to back up these things to USB drive and googlemail storage.

    --
    My AC stalker: " I personally agree with your posts most of the time, but that won't keep me from modding you troll"
    1. Re:Example: Kindle's cloud by SecurityGuy · · Score: 2

      This is also why, after being briefly enamored of e-books, I've gone back to buying the real thing. I still sometimes reread books I bought 20+ years ago. Will the Kindle ones I've bought still work in 20 years? I doubt it. Are they required to? Certainly not. But I have dead-tree-and-ink versions ranging from days old to 130+ years on my bookshelf. Most of them work as well as the day they were purchased.

      I'll still buy them if they satisfy an immediate need and are significantly cheaper than the dead tree version. Last one I bought was three bucks, not stocked in bookstores, and I wanted something to read Right Now. Disposable entertainment. The problem with that is that it doesn't scale. I don't mind blowing $3 on a few hours amusement. I'm much happier having $20k or so in books lying about my house than spending $10k on digital versions that will one day go poof.

    2. Re:Example: Kindle's cloud by rtaylor · · Score: 2

      My diagrams include a cloud around my clouds. Supercloud for the win.

      --
      Rod Taylor
    3. Re:Example: Kindle's cloud by cpu6502 · · Score: 2

      Also with physical books or magazines you can resell them. I was just looking at the resale price of 3-year-old Fantasy & Science magazine, and they are going for $6 each. Not bad for something that only cost me $3 originally. (Good luck trying to resale an ebook or emag; you get a few pennies if that.)

      re: Googlemail. I use it because my house might burn down and take my USB drive with it. At least I'll still have a backup of my e-books (even if only partial) on my googlemail.

      --
      My AC stalker: " I personally agree with your posts most of the time, but that won't keep me from modding you troll"
  9. Serious problem, but not a surprise by gweihir · · Score: 4, Interesting

    Complex systems almost always exhibit surprising behavior. Cloud computing is no exception, and it is new in addition. This leads to a high level of risk of such events emerging without warning. Of course, people with a stake in the business side will never admit the risk. For examples of this happening in other fields, look at TEPCO, BP, RSA, ... All save and risk-free. Until things blow up.

    Put simple: "The Cloud - where other peoples servers can crash yours."

    Also appropriate:
        "A distributed system is one in which I cannot get something done because a machine I've never heard of is down." --Leslie Lamport
    This holds even more for the cloud.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  10. Nested cloud services by hawguy · · Score: 2

    I've always thought that the problem would be when companies start using cloud services that rely on cloud servers that rely on cloud services and one of those cloud services has an outage.

    A problem at one cloud provider can trickle up and affect your service that's running on a completely different cloud service, so for example, your website running on EC2 depends on order fulfillment software running on Rackspace's cloud, which uses back-end software on MS's Azure cloud.

    If Azure has a hiccup, then your web store goes down, and if enough sites are affected, it can make real changes in the load (maybe less load because people can't shop, maybe more load because users keep hitting "reload" to try to place their order) on EC2 and Rackspace which could cause additional problems as load balancers try to shift load around as they respond to the sudden and huge change in load.

  11. bandwidth will be bottle neck by e**(i+pi)-1 · · Score: 3, Interesting

    I do not know how broadband will be able to cope with this. No thanks, I try to keep independent and use good old rsync to keep my machines in sync. If things continue as they are we soon have bandwidth caps. Relying then on the cloud could become very expensive. Not only because of prize hikes in the cloud once the public is hooked, but also because faster internet service is needed.

  12. The cloud by javascriptjunkie · · Score: 4, Informative

    I've been working in the cloud since July. The company I work for really likes the idea of it. But I'll tell you something. As a programmer and systems administrator responsible for something that lives in the cloud, I'm just not seeing the value of it. At least the way it's implemented at Rackspace. We've had problems that are absolutely bizarre, that seemingly have no explanation, that take weeks to resolve, that don't originate on our side. We've had issues with data integrity that don't happen on regular servers, and while we're able to "scale," we're very limited in the ways we're allowed to do it. Maybe this kind of set up works for other companies and groups, but I can't see myself choosing a cloud provider over traditional collocation and the standard three tier server model for 99% of what I need to do.

  13. Re:Placing bets on first cloud company meltdown by DickBreath · · Score: 2

    Just to add a footnote: it's interesting to note the differences for this search term using google vs. bing.

    --

    I'll see your senator, and I'll raise you two judges.
  14. "This can never happen . . ." by NicknamesAreStupid · · Score: 2

    . . . until it does. No one with a vested stake can afford to speak the unspeakable of the greatest boon to the networking industry in the 21st century. This is the goose that is laying golden eggs, and the entire industry is cashing in on it. Before it is over, the bulk of all data will have shifted to the cloud at a cost of over a trillion dollars. The only people who can speak ill of this are those who have not learned to profit from it. Someday, there will be a great business in getting everyone out of the cloud. The marketing slogans can be easily predicted, "Get you head out the clouds and data out of the storm . . . don't let lightning strike . . . why is your precious corporate information floating away in the cloud?" Until then, everybody rides the train.

    Of course, it may eventually fail in a spectacular way, but it will weather the occasional failure (see Amazon's). Then again, California will have a 9.x earthquake someday, too. However, you make notice that the land values correlate closely to the major fault zones, too. Ironic, but such is life.