"There is also the question of how good a job they do with encrypting the data."
Most let you manage your own keys. So as long as you have a reasonable key management, it's up to YOU, not the provider.
"Are there regular security audits by an outside party who can affirm that the things the cloud company claims are in fact accurate?"
For the big players, yes. http://aws.amazon.com/complian.... Also "AWS has achieved ISO 27001 certification and has been validated as a Level 1 service provider under the Payment Card Industry (PCI) Data Security Standard (DSS). We undergo annual SOC 1 audits and have been successfully evaluated at the Moderate level for Federal government systems as well as DIACAP Level 2 for DoD systems."
Every one of those compliances requires auditing.
"What happens when an employee leaves the company? How is access controlled to prevent continued access?"
You federate your enterprise IAM with your cloud provider. Most support some form of SAML or OAuth. ADFS (an MS product) supports such things easily. You terminate the employee in your normal system and their IAM account is terminated. Also, you don't give deep credentials to most people but rather wrap them in services. You then stash those credentials in a secret/key server.
"To me, cloud is all smoke and mirrors."
That is because you haven't done the required reading.
This is wrong. The Torah is the "first" five books of Moses (Pentateuch). Septuagint also includes much of the later writings and Prophets. The Hebrew Bible (Tanach) includes Torah. But Torah is not the Hebrew Bible, but a mere (though important) sliver.
You make sure that all your devices block devices (at least if you're in a "Real Enterprise") are at least mirrored. Just because LVM2 writes across all disks, doesn't mean you're in a single-point-of-failure mode (or RAID0 or JBOD). You just make sure your "disks" are not just a single Physical Disk, but rather a LUN composed of a RAID6 with multiple paths anf power-supplies.
And if you're not an Enterprise, this can be done on the "cheap" with RAID6 commodity boxes running as iSCSI targets.
If you're ever relying on a single device for ANYTHING you're doing it wrong. But LVM is perfectly safe if you do it in a responsible way. It just isn't necessarily super-cheap.
given the poster's issue of VERY low cost massive storage, i like my solution. If he has the money for Fiber between all nodes, then rest assured, I agree very much with your idea. As I said for us, size and price and redundancy were really important and performance much less so. But, as I also said, if we needed to scale beyond what we have now, your way is a much better idea.
I'll put this out as a side point since I'm the OP: If we had to do more than 50TB, I think we'd go to a "real" solution like EMC or something like that. This has been very good for us, but given the need for that amount of storage, we also now have the money to spend on a superduper storage machine. Homebrew has been wonderful to get to this point, but unless we get the kind of employees necessary to really write our own FS a-la GoogleFS, I can't see us taking this solution that much further past where it is now only because I can't see myself putting THAT much scalable trust into something like NBD or software RAID5. At least not with really really close inspection of the limitations of that code.
We've scaled this to 30TB so far. I'm not sure about 1PB, though. For us, redundancy and storage size is key, performance less so.
Storage nodes: 7 x 2.8TB 2U RAID5+1 boxen with Serial ATA. The 2.8TB is logical, not physical. The OS for each of those machines is RAMDISK based (something we concocted based on what I read about the DNALounge awhile back) so it helps curb disk failures of the storage nodes themselves. We avoid disk failure by using RAID5. Of course that doesn't protect against mutiple simultaneous disk failure, but read on for more. Each of the storage nodes is exported via NBD.
Then we have a head unit, a 64-bit machine. This machine does a software RAID5 across the storage nodes using an NBD client. Essentially each storage node is a "disk" and the head unit binds and manages the sofware raid5. So let's say a whole storage node goes down (for whatever reason it does), all the data is still intact. RAID5 rebuild time over the gigabit network is about 18hrs, which is acceptable. We even have another storage box as a hot-spare.
On top of that, we have the whole cluster mirrored to another identical cluster via DRBD in a different geographic location. This is linked by Gigabit WAN. So if we have a massive disaster and lose the entire primary cluster, then we have a 2ndary cluster ready to go. We needed to purchase the Enterprise version of DRBD ($2k US) but that's worth it because they're neato guys.
We use XFS as the filesystem. This system gives us 14TB of redundant "RAID-55 with a Mirror" space. Both clusters together? $85k.
When the cluster starts running out of space (about 70% or so), we add ANOTHER cluster of similar stats to the initial one and use LVM to join the two units together.
This has scaled us to 30TB and we're pretty happy with it. The read speed is very good (hdparm says Timing buffered disk reads: 200 MB in 3.01 seconds = 66.49 MB/sec) and the write speed is about 32 MB/sec. For what our application is doing, that's a fine speed.
"The Christian faith (who's political wing is the Republican party) for some reason believe that sex is bad and that pornography is somehow immoral. I don't know how they reached that conclusion, after all, one need only look as far as Job's daughters antics in the book of Genesis to see that the Bible is no authority on sexual morality."
Um. You mean Lot's Daughters in the book of Genesis. Job has his own book called "Job". Before you denounce something (and rest assured I agree with your sentiment) you should read up on it. Otherwise you look like Tom Cruise. You just sound glib, Matt.
MAN IN LINE
(Even louder now)
It's the influence of television. Yeah, now Marshall McLuhan deals with it in terms of it being a-a high, uh, high intensity, you understand? A hot medium... as opposed to a...
ALVY
(More and more aggravated)
What I wouldn't give for a large sock o' horse manure.
MAN IN LINE
... as opposed to a print...
Alvy steps forward, waving his hands in frustration, and stands facing the
camera.
ALVY (Sighing and addressing the audience)
What do you do when you get stuck in a movie line with a guy like this behind you? I mean, it's just maddening!
The man in line moves toward Alvy. Both address the audience now.
MAN IN LINE
Wait a minute, why can't I give my opinion? It's a free country!
ALVY
I mean, d- He can give you- Do you hafta give it so loud? I mean, aren't you ashamed to pontificate like that? And-and the funny part of it is, M-Marshall McLuhan, you don't know anything about Marshall McLuhan's...work!
MAN IN LINE
(Overlapping)
Wait a minute! Really? Really? I happen to
teach a class at Columbia called "TV Media and Culture"! So I think that my insights into Mr. McLuhan-well, have a great deal of validity.
ALVY
Oh, do yuh? Well, that's funny, because I happen to have Mr. McLuhan right here. So... so, here, just let me-I mean, all right. Come over here... a second.
Alvy gestures to the camera which follows him and the man in line to the back of the crowded lobby. He moves over to a large stand-up movie poster and pulls Marshall McLuban from behind the poster.
ALVY (To McLuban)
Tell him.
MCLUHAN
(To the man in line)
I hear-I heard what you were saying. You-you know nothing of my work. You mean my whole fallacy is wrong. How you ever got to teach a course in anything is totally amazing.
anyhow, yes. There were companies who competed with VA after I left (angstrom microsystems), that i helped start (angstrom microsystems) that i eventually left (angstrom microsystem). They made rackmounts (and still do i think) specifically for rendering and we put them all over the place (rhythm and hues, pixar, dreamworks).
The reason why this is, or so I'm told by the tons of lawyers I work with/for, is that this allows the "little man" to sue the "big man". Yes, it seems dumb that anyone can attempt to sue anyone else for any reason, but how else can a "nobody" just issue suit against a "somebody". In Britain, the loser has to pay the costs of the winner. If the winner has a team from GibsonDunn or BlankRome or some other huge firm, this can be Millions of dollars. The little man can never attempt to sue if that is what's at stake.
Most lawyers do not work on contingency. most lawyers are paid per 10 minute or 15 minute interval of work. Personal injury attorneys are paid in contingency, but not in most other law.
If a suit is trivial, it is thrown out of court. It is only heard by the court if it is NOT trivial. A judge wouldn't hear a case like this unless he deemed it a worthy thing to hear. SO perhaps the judge is wrong, but that's another issue entirely. It's not the fault of the system at that point, but the fault of the lame-o judge.
This got modded up? Filth. The Yankees are store-bought filth.
Go Sox. Go Sox for the good of this country and humanity.
I'll be watching this game in a bar near Fenway tonight and I would still like to occasionally turn off the TVs just for shits and giggles. I don't see the device as malicious so much as it is fun.
All they are saying is that Red Hat Linux will no longer be released by RedHat. This means that a company won't spend lots of money supporting, for free, a free project. Companies that make money on open source tend to do so through charging for support. Updates and maintainence of software trees are a type of support. So I guess they looked at the bottom line and said, "hey free publicity is lots of fun, but it's just not worth it."
BUT They still have and fund the Fedora Project. This is essentially Red Hat linux. It's just no longer commercially supported. Just like debian.
You and slashdot are friends...
on
Ask Neil Gaiman
·
· Score: 4, Interesting
This is sort of the question I should have asked Warren Ellis, but I forgot to and asked something inane that he laughed at me for. Anyhow...
In ENDLESS NIGHTS, you make a reference to slashdot (the Destruction story. It's used as a threat. It's pretty funny for those of us who have been using this site for too long). That being said, you're aware of the tech/geek movement as you seem to get a great sum of cash from us. So you read slashdot. Cool. BUT what other sources of tech, science, etc do you read on a regular basis. Any cool magazines we don't know about? Any cool websites, links, etc, that Neil Gaiman checks to see where science is right now?
According to the Torah (the Judaic code of laws, the five books of Moses) God gave 613 Commandments, the bulk of which can be found in Leviticus and Numbers, with a repetition in Deutoronomy.
The most dramatic presentation were the 10 commandments in Exodus. Though they were given twice because Moses destroyed the first set. And then given again in Deutoronomy (in a slightly different hebraic and I assume, english, text).
You will find that the 10 are not as cut and dry as those that support posting the 10 commandments in school believe they are. How many people who want the 10 Commandments posted really support Exodus 20:8-11 :
20:8
Remember the sabbath day, to keep it holy.
20:9
Six days shalt thou labour, and do all thy work:
20:10
But the seventh day is the sabbath of the LORD thy God: in it thou shalt not do any work, thou, nor thy son, nor thy daughter, thy manservant, nor thy maidservant, nor thy cattle, nor thy stranger that is within thy gates:
20:11
For in six days the LORD made heaven and earth, the sea, and all that in them is, and rested the seventh day: wherefore the LORD blessed the sabbath day, and hallowed it.
That's part of the 10 Commandments. I don't see many bible thumpers giving their pets the day off.
I love watchmen. I just think that the average person doesn't want to read about superheros. If i'm starting someone out who says, "I don't like comics." Watchmen is the last thing I'd give them. It's an amazing superhero book. But it's a superhero book. When people say, "I don't like comics." they mean, "i don't like superheros." you're not going to turn them on by giving them a (really good) superhero book. My question for Warren was how do you get the people who don't like (or know) comics...
Slashdot Mirror
"There is also the question of how good a job they do with encrypting the data."
Most let you manage your own keys. So as long as you have a reasonable key management, it's up to YOU, not the provider.
"Are there regular security audits by an outside party who can affirm that the things the cloud company claims are in fact accurate?"
For the big players, yes. http://aws.amazon.com/complian.... Also "AWS has achieved ISO 27001 certification and has been validated as a Level 1 service provider under the Payment Card Industry (PCI) Data Security Standard (DSS). We undergo annual SOC 1 audits and have been successfully evaluated at the Moderate level for Federal government systems as well as DIACAP Level 2 for DoD systems."
Every one of those compliances requires auditing.
"What happens when an employee leaves the company? How is access controlled to prevent continued access?"
You federate your enterprise IAM with your cloud provider. Most support some form of SAML or OAuth. ADFS (an MS product) supports such things easily. You terminate the employee in your normal system and their IAM account is terminated. Also, you don't give deep credentials to most people but rather wrap them in services. You then stash those credentials in a secret/key server.
"To me, cloud is all smoke and mirrors."
That is because you haven't done the required reading.
The hebrew for wife and woman are the same word (isha).
Is it remotely possible you were kind of an asshole to him? Or are you without any flaws in personal confrontation?
What DOES God wants with a starship?
I have nothing meaningful to add other than I love the shoutout to Paladine.
Already exists:
http://www.theeroticreview.com/
It is fascinating reading. Just fascinating.
This is wrong. The Torah is the "first" five books of Moses (Pentateuch). Septuagint also includes much of the later writings and Prophets. The Hebrew Bible (Tanach) includes Torah. But Torah is not the Hebrew Bible, but a mere (though important) sliver.
You make sure that all your devices block devices (at least if you're in a "Real Enterprise") are at least mirrored. Just because LVM2 writes across all disks, doesn't mean you're in a single-point-of-failure mode (or RAID0 or JBOD). You just make sure your "disks" are not just a single Physical Disk, but rather a LUN composed of a RAID6 with multiple paths anf power-supplies.
And if you're not an Enterprise, this can be done on the "cheap" with RAID6 commodity boxes running as iSCSI targets.
If you're ever relying on a single device for ANYTHING you're doing it wrong. But LVM is perfectly safe if you do it in a responsible way. It just isn't necessarily super-cheap.
given the poster's issue of VERY low cost massive storage, i like my solution. If he has the money for Fiber between all nodes, then rest assured, I agree very much with your idea. As I said for us, size and price and redundancy were really important and performance much less so. But, as I also said, if we needed to scale beyond what we have now, your way is a much better idea.
I'll put this out as a side point since I'm the OP: If we had to do more than 50TB, I think we'd go to a "real" solution like EMC or something like that. This has been very good for us, but given the need for that amount of storage, we also now have the money to spend on a superduper storage machine. Homebrew has been wonderful to get to this point, but unless we get the kind of employees necessary to really write our own FS a-la GoogleFS, I can't see us taking this solution that much further past where it is now only because I can't see myself putting THAT much scalable trust into something like NBD or software RAID5. At least not with really really close inspection of the limitations of that code.
Storage nodes: 7 x 2.8TB 2U RAID5+1 boxen with Serial ATA. The 2.8TB is logical, not physical. The OS for each of those machines is RAMDISK based (something we concocted based on what I read about the DNALounge awhile back) so it helps curb disk failures of the storage nodes themselves. We avoid disk failure by using RAID5. Of course that doesn't protect against mutiple simultaneous disk failure, but read on for more. Each of the storage nodes is exported via NBD.
Then we have a head unit, a 64-bit machine. This machine does a software RAID5 across the storage nodes using an NBD client. Essentially each storage node is a "disk" and the head unit binds and manages the sofware raid5. So let's say a whole storage node goes down (for whatever reason it does), all the data is still intact. RAID5 rebuild time over the gigabit network is about 18hrs, which is acceptable. We even have another storage box as a hot-spare.
On top of that, we have the whole cluster mirrored to another identical cluster via DRBD in a different geographic location. This is linked by Gigabit WAN. So if we have a massive disaster and lose the entire primary cluster, then we have a 2ndary cluster ready to go. We needed to purchase the Enterprise version of DRBD ($2k US) but that's worth it because they're neato guys.
We use XFS as the filesystem. This system gives us 14TB of redundant "RAID-55 with a Mirror" space. Both clusters together? $85k.
When the cluster starts running out of space (about 70% or so), we add ANOTHER cluster of similar stats to the initial one and use LVM to join the two units together.
This has scaled us to 30TB and we're pretty happy with it. The read speed is very good (hdparm says Timing buffered disk reads: 200 MB in 3.01 seconds = 66.49 MB/sec) and the write speed is about 32 MB/sec. For what our application is doing, that's a fine speed.
Um. You mean Lot's Daughters in the book of Genesis. Job has his own book called "Job". Before you denounce something (and rest assured I agree with your sentiment) you should read up on it. Otherwise you look like Tom Cruise. You just sound glib, Matt.
(Even louder now)
It's the influence of television. Yeah, now Marshall McLuhan deals with it in terms of it being a-a high, uh, high intensity, you understand? A hot medium
ALVY
(More and more aggravated)
What I wouldn't give for a large sock o' horse manure.
MAN IN LINE
... as opposed to a print ...
Alvy steps forward, waving his hands in frustration, and stands facing the camera.
ALVY (Sighing and addressing the audience)
What do you do when you get stuck in a movie line with a guy like this behind you? I mean, it's just maddening!
The man in line moves toward Alvy. Both address the audience now.
MAN IN LINE
Wait a minute, why can't I give my opinion? It's a free country!
ALVY
I mean, d- He can give you- Do you hafta give it so loud? I mean, aren't you ashamed to pontificate like that? And-and the funny part of it is, M-Marshall McLuhan, you don't know anything about Marshall McLuhan's...work!
MAN IN LINE
(Overlapping)
Wait a minute! Really? Really? I happen to teach a class at Columbia called "TV Media and Culture"! So I think that my insights into Mr. McLuhan-well, have a great deal of validity.
ALVY ... so, here, just let me-I mean, all right. Come over here ... a second.
Oh, do yuh? Well, that's funny, because I happen to have Mr. McLuhan right here. So
Alvy gestures to the camera which follows him and the man in line to the back of the crowded lobby. He moves over to a large stand-up movie poster and pulls Marshall McLuban from behind the poster.
ALVY
(To McLuban) Tell him.
MCLUHAN
(To the man in line)
I hear-I heard what you were saying. You-you know nothing of my work. You mean my whole fallacy is wrong. How you ever got to teach a course in anything is totally amazing.
ALVY
Boy, if life were only like this!
-Annie Hall
anyhow, yes. There were companies who competed with VA after I left (angstrom microsystems), that i helped start (angstrom microsystems) that i eventually left (angstrom microsystem). They made rackmounts (and still do i think) specifically for rendering and we put them all over the place (rhythm and hues, pixar, dreamworks).
but it is a niche market and competitive as hell.
Most lawyers do not work on contingency. most lawyers are paid per 10 minute or 15 minute interval of work. Personal injury attorneys are paid in contingency, but not in most other law.
If a suit is trivial, it is thrown out of court. It is only heard by the court if it is NOT trivial. A judge wouldn't hear a case like this unless he deemed it a worthy thing to hear. SO perhaps the judge is wrong, but that's another issue entirely. It's not the fault of the system at that point, but the fault of the lame-o judge.
Go Sox. Go Sox for the good of this country and humanity.
I'll be watching this game in a bar near Fenway tonight and I would still like to occasionally turn off the TVs just for shits and giggles. I don't see the device as malicious so much as it is fun.
I did exactly this today and it was appreciated. How very timely.
The book was about Senator Joe McCarthy. Did you read it?
BUT They still have and fund the Fedora Project. This is essentially Red Hat linux. It's just no longer commercially supported. Just like debian.
In ENDLESS NIGHTS, you make a reference to slashdot (the Destruction story. It's used as a threat. It's pretty funny for those of us who have been using this site for too long). That being said, you're aware of the tech/geek movement as you seem to get a great sum of cash from us. So you read slashdot. Cool. BUT what other sources of tech, science, etc do you read on a regular basis. Any cool magazines we don't know about? Any cool websites, links, etc, that Neil Gaiman checks to see where science is right now?
http://www.k-otik.com/exploits/09.16.MS03-039-e
i'd post the code, but
The most dramatic presentation were the 10 commandments in Exodus. Though they were given twice because Moses destroyed the first set. And then given again in Deutoronomy (in a slightly different hebraic and I assume, english, text).
You will find that the 10 are not as cut and dry as those that support posting the 10 commandments in school believe they are. How many people who want the 10 Commandments posted really support Exodus 20:8-11 :
20:8
Remember the sabbath day, to keep it holy.
20:9
Six days shalt thou labour, and do all thy work:
20:10
But the seventh day is the sabbath of the LORD thy God: in it thou shalt not do any work, thou, nor thy son, nor thy daughter, thy manservant, nor thy maidservant, nor thy cattle, nor thy stranger that is within thy gates:
20:11
For in six days the LORD made heaven and earth, the sea, and all that in them is, and rested the seventh day: wherefore the LORD blessed the sabbath day, and hallowed it.
That's part of the 10 Commandments. I don't see many bible thumpers giving their pets the day off.
...and instulted me and threw some great information my way about some comics i never heard of, but will be picking up soon.
All that in a few paragraphs. Spider bless Warren Ellis.
Man. Good thing I bought mine in the original run. And everytime i wear them (which is too often) my head really hurts...
I love watchmen. I just think that the average person doesn't want to read about superheros. If i'm starting someone out who says, "I don't like comics." Watchmen is the last thing I'd give them. It's an amazing superhero book. But it's a superhero book. When people say, "I don't like comics." they mean, "i don't like superheros." you're not going to turn them on by giving them a (really good) superhero book. My question for Warren was how do you get the people who don't like (or know) comics...