One question: with 1588 what sort of hierarchy do you set up? Does everyone have a rubidium or cesium clock attached?
A typical way is to use a GPS GrandMaster clock that outputs PTP. Some of these have good oscillators in them to keep good time if GPS dies for whatever reason. The tricky part is getting this time to many machines without bad drift.
All the tools suck. Firewalls cause more harm than good. The platforms are all mediocre. In my world (low latency trading), pulling firewalls out is one of the highest priorities if it can be done (legally and reputationally).
When Paypal says you have $100 in your account, they actually have that money somewhere waiting for you to withdraw.
What in the world makes you think that? Not only do they probably have the money in various forms of liquid and illiquid assets, but there isn't even FDIC insurance to protect you if they screw up.
Don't get me started on the silly political fallout of merging a network & SAN team in a large organization:-( (hint: the SAN team will lose as there are less of us)
Not sure how it happened, but somehow, where I work (a large company), SAN was another networking product from day 1. A storage team handles the endpoints much like a server team handles the endpoints on the IP network. But, we manage all the MDS and maintain the relationship with Cisco as we do for Ethernet/IP. It works well.
Could you share with us your definition of a "crazy lookin [sic] thing"? I'm not very old and I brought homemade electronics into middle and high school on occasion and no one blinked an eye. If anything, this student's behavior should be encouraged and expected at a magnet school.
If wires sticking out of something are reason for alarm, then I think this clueless administrator would be very concerned if he walked around an engineering department at any university. This kid should not be punished for being precocious.
Though I worked throughout college, my sole internship was very well paid. I made approximately $30/hour plus housing stipend. This was working in technology at a financial firm. Companies who value technology are willing to pay for it. Never forget that.
I know for a fact that WSUS (Windows Server Update Services... basically a centralized patch server) would do "weird, interesting" things when two machines tried to check into WSUS with the same SID.
I don't even work with Windows servers and I happen to know this from engineering some network infrastructure (load balancing) for the folks in our organization who do manage WSUS. Long story short, what they thought was problematic load balancing across WSUS servers was actually the same SID being used from 1,000+ cloned VMs. WSUS thought they were one machine.
SSD's don't perform nearly as well as 4 SATA drives in a decent RAID setup.
What are you smoking?
These are conservative numbers, and they show a commodity (Intel X25-M) MLC flash drive doing over 1,000 IOPS, but a typical 7,200 RPM SATA drive doing 100 IOPS. (Note, I rounded down for the SSD and up for the spinning disk.) So, SSDs are an order of magnitude faster at IOPS than platters. To get a 10x increase of write IOPS on a spinning disk, you'd need to use RAID-0 across ten disks. Hardly a good idea.
I wonder how much resources it takes under OpenSolaris, and if a OpenSolaris virtual server, just for the ZFS, would make sense...
ZFS will always try to take up as much RAM as it can for the ARC (Adaptive Replacement Cache).
While ZFS on FUSE probably works fine, it will always make me a bit scared. But kudos if it works for you!
P.S. I like your Hilti analogy. The average do-it-yourselfer does not (and has no need to) know who Hilti is or what kind of products they make. Those who need to know, do.
Of course not. A project manager would look at this and go, "wow, we saved a lot of money!" It's pretty simple. ZFS does what most other filesystems do not; it guarantees data integrity at the block level by the use of checksums. When you're dealing with this many spindles and dense, non-enterprise drives, you are virtually guaranteed to get silent corruption. The article does not once have any of the words corrupt.*, checksum, or integrity mentioned in it once. The server doesn't use ECC RAM. The project, while well intentioned, should scare the crap out of anyone thinking about storing data with this company.
Google for Hadoop and read how it handles storing chunks of data (two copies in one rack on different servers, one copy on a node in a different rack).
I admit I don't know much about Hadoop, but how do you know a block wasn't corrupted? It very well might store checksum data at the object level, but ZFS stores at the block level. Without checksumming, even if you have multiple copies of your data, it's hard to know which is the right one if one copy gets corrupted silently.
My 20x1TB WD RE3 Raidz2 file/backup server registers a total space of just a touch over 17TB
As a side note, it's best not to have 20 drives in a single RAID-Z2 vdev. It would be much better to at least split this into at least two vdevs in the pool. You'd get better IOPS and a bit more redundancy.
So, going to the database to look up where a computer is every time there is a problem with communicating with it is less effort than renaming it every time you move it?
Renaming something tends to have knock-on effects if you have a lot of systems that reference that hostname.
"dig txt hostname.domain.com." isn't a lot of work to see where a device is located.
Just from looking at a WS ID, we know just about where it is. If a user calls and only knows their building and room, we can easily isolate the machine.
Until you move it, and then you have to rename it. A better idea is to give it an arbitrary name and then add location info in DNS TXT records if you want them to be easily accessible and not tied to the PC's actual identity.
Steam solves the backup problem too though. I can install any game I've bought on a computer where I log in, just by downloading it. Valve really is DRM done right.
There is no such thing as "DRM done right." Valve could just as easily provide you the privilege to redownload the game without DRM. DRM does not enable you to redownload it.
There's no place like Slashdot for broad, sweeping generalizations. Like most people, I was forced to write cursive in school up until 7th or 8th grade. I hated it every step of the way. It hurt my hand and was to me much less legible than print (and I got good marks on my writing either way).
At some point starting in high school, everything important I did was math/science related. Cursive doesn't do much good for clearly labeling diagrams. Nor is it good for pseudocode on tests in college.
Encrypt the entire drive with TrueCrypt or something. Use a strong cipher and a very strong passphrase. The laptop is as good as bricked to anyone who gets it.
but I also think it's important for a graduate of a 4-year CS degree to be an expert in at least one.
I've used this quote time and time again on here, and I'll use it again: "Computer Science is no more about computers than astronomy is about telescopes." -- Edsger W. Dijkstra.
And this one is completely appropriate to, as to why Knuth uses MIX/MMIX in his books and not some other language. The same thing applies exactly to a university program of computer science:
"Moreover, if I did use a high-level language, what language should it be? In the 1960s I would probably have chosen Algol W; in the 1970s, I would then have had to rewrite my books using Pascal; in the 1980s, I would surely have changed everything to C; in the 1990s, I would have had to switch to C++ and then probably to Java. In the 2000s, yet another language will no doubt be de rigueur. I cannot afford the time to rewrite my books as languages go in and out of fashion; languages aren't the point of my books, the point is rather what you can do in your favorite language. My books focus on timeless truths."
Being in expert in one programming language is fine if you just want to be a programmer. If you want to be a computer scientist, it's neither necessary nor sufficient.
QoS isn't a bad thing, but the user should be in control of it, not the ISP;
The problem is that I'd be afraid of other people prioritizing all traffic rather than just some. So now I'm gonna prioritize all of my traffic. So now everything is in a gold queue and nothing gets prioritized. It is somewhat of a prisoner's dilemma. Contrast this with a network with end to end control where you can trust DSCP or COS values along the way. A possible solution is maybe to allow end users to mark their packets, but make the queue they go into pretty small, so they can only prioritize a bit of voice or video and not much more.
A lot of money was funneled to GS by Paulsen (a GS alumni) and some of their major competitors were crippled.
You're an idiot and I can't let this stand without rebuttal. If you're referring to AIG bailout money going to GS, then you're complaining about GS getting paid for contracts that AIG knowingly and willfully entered into. Had AIG failed to live up to their end of the bargain, credit default swaps on AIG itself would have been triggered and even MORE companies would have had to pay for AIG's mistakes.
Goldman should be praised for properly hedging all their positions and this is what any shareholder of the firm would expect.
Slashdot Mirror
One question: with 1588 what sort of hierarchy do you set up? Does everyone have a rubidium or cesium clock attached?
A typical way is to use a GPS GrandMaster clock that outputs PTP. Some of these have good oscillators in them to keep good time if GPS dies for whatever reason. The tricky part is getting this time to many machines without bad drift.
All the tools suck. Firewalls cause more harm than good. The platforms are all mediocre. In my world (low latency trading), pulling firewalls out is one of the highest priorities if it can be done (legally and reputationally).
When Paypal says you have $100 in your account, they actually have that money somewhere waiting for you to withdraw.
What in the world makes you think that? Not only do they probably have the money in various forms of liquid and illiquid assets, but there isn't even FDIC insurance to protect you if they screw up.
Don't get me started on the silly political fallout of merging a network & SAN team in a large organization :-( (hint: the SAN team will lose as there are less of us)
Not sure how it happened, but somehow, where I work (a large company), SAN was another networking product from day 1. A storage team handles the endpoints much like a server team handles the endpoints on the IP network. But, we manage all the MDS and maintain the relationship with Cisco as we do for Ethernet/IP. It works well.
This kid brought a crazy lookin thing into school
Could you share with us your definition of a "crazy lookin [sic] thing"? I'm not very old and I brought homemade electronics into middle and high school on occasion and no one blinked an eye. If anything, this student's behavior should be encouraged and expected at a magnet school.
If wires sticking out of something are reason for alarm, then I think this clueless administrator would be very concerned if he walked around an engineering department at any university. This kid should not be punished for being precocious.
Though I worked throughout college, my sole internship was very well paid. I made approximately $30/hour plus housing stipend. This was working in technology at a financial firm. Companies who value technology are willing to pay for it. Never forget that.
So the behavior observed in our case was the clients get updated, but WSUS thinks only 1 client ever connected.
I know for a fact that WSUS (Windows Server Update Services... basically a centralized patch server) would do "weird, interesting" things when two machines tried to check into WSUS with the same SID.
I don't even work with Windows servers and I happen to know this from engineering some network infrastructure (load balancing) for the folks in our organization who do manage WSUS. Long story short, what they thought was problematic load balancing across WSUS servers was actually the same SID being used from 1,000+ cloned VMs. WSUS thought they were one machine.
SSD's don't perform nearly as well as 4 SATA drives in a decent RAID setup.
What are you smoking?
These are conservative numbers, and they show a commodity (Intel X25-M) MLC flash drive doing over 1,000 IOPS, but a typical 7,200 RPM SATA drive doing 100 IOPS. (Note, I rounded down for the SSD and up for the spinning disk.) So, SSDs are an order of magnitude faster at IOPS than platters. To get a 10x increase of write IOPS on a spinning disk, you'd need to use RAID-0 across ten disks. Hardly a good idea.
How about a MAC address?
MAC addresses are not known beyond your first-hop router, so this doesn't really work.
I wonder how much resources it takes under OpenSolaris, and if a OpenSolaris virtual server, just for the ZFS, would make sense...
ZFS will always try to take up as much RAM as it can for the ARC (Adaptive Replacement Cache).
While ZFS on FUSE probably works fine, it will always make me a bit scared. But kudos if it works for you!
P.S. I like your Hilti analogy. The average do-it-yourselfer does not (and has no need to) know who Hilti is or what kind of products they make. Those who need to know, do.
How about reading the section "A Backblaze Storage Pod is a Building Block".
I did read it. Black and white hardware failures are easy to deal with. Corruption is not.
are you a project manager by any chance?
Of course not. A project manager would look at this and go, "wow, we saved a lot of money!" It's pretty simple. ZFS does what most other filesystems do not; it guarantees data integrity at the block level by the use of checksums. When you're dealing with this many spindles and dense, non-enterprise drives, you are virtually guaranteed to get silent corruption. The article does not once have any of the words corrupt.*, checksum, or integrity mentioned in it once. The server doesn't use ECC RAM. The project, while well intentioned, should scare the crap out of anyone thinking about storing data with this company.
Google for Hadoop and read how it handles storing chunks of data (two copies in one rack on different servers, one copy on a node in a different rack).
I admit I don't know much about Hadoop, but how do you know a block wasn't corrupted? It very well might store checksum data at the object level, but ZFS stores at the block level. Without checksumming, even if you have multiple copies of your data, it's hard to know which is the right one if one copy gets corrupted silently.
why wouldn't you just build an entirely new pod with current disks and migrate the data? You could certainly afford it.
Maybe because there's no need to update and you just want to be able to replace broken drives?
Good luck with all the silent data corruption. Shoulda used ZFS.
My 20x1TB WD RE3 Raidz2 file/backup server registers a total space of just a touch over 17TB
As a side note, it's best not to have 20 drives in a single RAID-Z2 vdev. It would be much better to at least split this into at least two vdevs in the pool. You'd get better IOPS and a bit more redundancy.
So, going to the database to look up where a computer is every time there is a problem with communicating with it is less effort than renaming it every time you move it?
Renaming something tends to have knock-on effects if you have a lot of systems that reference that hostname.
"dig txt hostname.domain.com." isn't a lot of work to see where a device is located.
Just from looking at a WS ID, we know just about where it is. If a user calls and only knows their building and room, we can easily isolate the machine.
Until you move it, and then you have to rename it. A better idea is to give it an arbitrary name and then add location info in DNS TXT records if you want them to be easily accessible and not tied to the PC's actual identity.
Steam solves the backup problem too though. I can install any game I've bought on a computer where I log in, just by downloading it. Valve really is DRM done right.
There is no such thing as "DRM done right." Valve could just as easily provide you the privilege to redownload the game without DRM. DRM does not enable you to redownload it.
Cursive is faster to write for anyone
There's no place like Slashdot for broad, sweeping generalizations. Like most people, I was forced to write cursive in school up until 7th or 8th grade. I hated it every step of the way. It hurt my hand and was to me much less legible than print (and I got good marks on my writing either way).
At some point starting in high school, everything important I did was math/science related. Cursive doesn't do much good for clearly labeling diagrams. Nor is it good for pseudocode on tests in college.
Encrypt the entire drive with TrueCrypt or something. Use a strong cipher and a very strong passphrase. The laptop is as good as bricked to anyone who gets it.
but I also think it's important for a graduate of a 4-year CS degree to be an expert in at least one.
I've used this quote time and time again on here, and I'll use it again: "Computer Science is no more about computers than astronomy is about telescopes." -- Edsger W. Dijkstra.
And this one is completely appropriate to, as to why Knuth uses MIX/MMIX in his books and not some other language. The same thing applies exactly to a university program of computer science:
"Moreover, if I did use a high-level language, what language should it be? In the 1960s I would probably have chosen Algol W; in the 1970s, I would then have had to rewrite my books using Pascal; in the 1980s, I would surely have changed everything to C; in the 1990s, I would have had to switch to C++ and then probably to Java. In the 2000s, yet another language will no doubt be de rigueur. I cannot afford the time to rewrite my books as languages go in and out of fashion; languages aren't the point of my books, the point is rather what you can do in your favorite language. My books focus on timeless truths."
Being in expert in one programming language is fine if you just want to be a programmer. If you want to be a computer scientist, it's neither necessary nor sufficient.
QoS isn't a bad thing, but the user should be in control of it, not the ISP;
The problem is that I'd be afraid of other people prioritizing all traffic rather than just some. So now I'm gonna prioritize all of my traffic. So now everything is in a gold queue and nothing gets prioritized. It is somewhat of a prisoner's dilemma. Contrast this with a network with end to end control where you can trust DSCP or COS values along the way. A possible solution is maybe to allow end users to mark their packets, but make the queue they go into pretty small, so they can only prioritize a bit of voice or video and not much more.
A lot of money was funneled to GS by Paulsen (a GS alumni) and some of their major competitors were crippled.
You're an idiot and I can't let this stand without rebuttal. If you're referring to AIG bailout money going to GS, then you're complaining about GS getting paid for contracts that AIG knowingly and willfully entered into. Had AIG failed to live up to their end of the bargain, credit default swaps on AIG itself would have been triggered and even MORE companies would have had to pay for AIG's mistakes.
Goldman should be praised for properly hedging all their positions and this is what any shareholder of the firm would expect.