"Stand ups" where everybody gives a status update is best done in "virtual" sense (con-call, email,etc). These are the big time suckers where you wait 5 minutes for half of the folks to just show up.
On the flip side, design meetings, especially when dealing with physical layout issues, can be a different issue. For example, this past week I had a 20 minute meeting with our network team to discuss rack layout, positioning and the other fun issues that arise when you are low on space in a datacenter and have a ton of stuff showing up in a few weeks. It was a lot easier to just grab a conference room and go at it on the whiteboard. Within minutes we had a plan worked out and everybody was happy. Compare this to a conference call and the thing would have taken forever. Yes, you can use virtual whiteboard software, but why bother when a real one is so much cheaper and easier to use. (I'll spare you my usual rant about/.'ers always wanting to use a high tech solution to solve a low-tech problem.)
For the team that I work with (a bunch of system and network admins), we can realistically work from home 2-3 days a week. This gives us a good mix of "face time" for meetings where such a thing is important and the rest of the week we can be at home getting longer term projects done that just require us to be alone and crank out stuff (ever wonder why things are never documented they way they should be? )
Anyway, my 0.02 worth. your work environment can and probably will be different.
I happen to work for a county and support our GIS group (along with a dozen other county departments)
Lots of the folks in the GIS group use google maps and google earth for quick and dirty stuff. We even use a google maps mashup on our main site for anything that requires a quick and dirty mapping application (voting locations, locations of sex offendors, etc)
That said, it is not a replacement for the GIS department, but it does help keep the size of the department in check. There are a few gotchas with the use of google:
1) Google earth is not free.
It is free for non-commercial use only. Everybody else has to pay.
2) The imagery is old
We do flyovers every two years minimum. The stuff on google is often 5+ years old for some parts of the county (the copyright date gets updated, but the images do not)
3) The data is not nearly as accurate
For quick and dirty work, google earth is ok. But we have had to work on areas where google only has 1m or worse. We have 6" resolution for the entire county. It is also been rectified and fixed and things like plot lines and street centerlines are dead on. I've played with image overlays before, and google can be 20+ ft off in one direction or another. That is simply not acceptable when you are trying to figure out where you are going to put a street.
4) Ever try and plot a 6' by 42" map using google earth at full resolution with plot line overlays and dozen of other custom features that the customer wants for a presentation? Didn't think so.
So, if all the gis department does is provide non-rectified 1 meter satellite photos from 10 years ago... yeah, time to ditch them and use google. For anything else, you are going to need a gis group.... It does not have to be large, but it better exist.
With the current state of the art, they do what they are told to do. No more, no less. Given a set of inputs they produce an output. They are not some magical creature that will devise elegant solutions to a problem and save the day. The chances that they will devise a solution to a problem that they are not explicitly taught how to deal with is 0.
With a human you at least have the ability to learn. This gives us poor humans the ability to craft a new and novel solution to a problem that has yet to be seen. While we are not as good at is repetitive tasks, this ability to be creative gives us a huge advantage in new situations. How many years did it take until the computer folks could produce a system to beat a human at chess? That is an example of a system where the computer has perfect knowledge of the playing field. The real world of flying aircraft does not afford the programmers such an advantage.
As for the gimli incident, the 'computer' was blind to how much fuel was in the system, hence the original problem. Now, look back at all of the crashes blamed on 'pilot error'. Many were made because the pilot was not given correct information about the situation that s/he was in. Wrong altitude, wrong airspeed, direction, weather, flap position, etc, etc. What makes you think that a computer would not make the same decision given the same set of information? What magical sensor system will fail and provide the pilot with the wrong information and not provide a computer with the same (wrong) information?
Some day we will have an expert system that is capable of performing the same tasks under any given set of circumstances. Until then, we have pilots.
Thats the kicker. As evidenced by the air ram generator, the designers took into account full power loss. It is easy to account for the issues that have already occurred, and probably a handful of ones that have not yet occurred.
The fun thing is that in a system with that many components, the failure scenarios are endless. As soon as you think of all of them you will run into another. Accounting for all of them are next to impossible.
In the case of the gimli incident, not only would the computer have to know what to do about no engine power due to no fuel, and know how to do forward slip to cut some speed fast just before landing, it would have to know about a landing strip that was no longer in use. If you happen to account for the first two, your chances of knowing about the landing strip are zero. From the writeup, as the air speed dropped, so did the power generated by the generator. The computer would need to be on an independent power supply or else it would crash from lack of power during some of the maneuvers. (and then you would need multiple independent flight computers, with independent electrical systems. As somebody who just had two different 'fully redundant' disk arrays at two different sites totally fail because one controller took out the other via the sync channel, this is not an easy task. And yes, they both occurred on the same night a few hours apart from each other. totally random. )
In the gimli case, it was luck that one pilot happened to be a glider pilot and the other happened to be stationed at gimli a few years before.
If a computer was in charge of this incident, we would probably have 60+ dead people. In this case the plane was flown out of there 2 days latter after minor repairs and is still in use today. A few folks suffered minor injuries exiting the back of the aircraft.
They had internet access. First dial up, then a DLS line.
After a year or so, my siblings were the only ones using it, usually to download spyware and such while hitting myspace.
After walking my dad through reinstalling XP home on the computer to get rid of all of the crap, he gave up. The computer now sits in the corner of their home and is rarely used.
When they need internet access, they go to the library. It is not a major part of their life.
If I lived a bit closer, I would probably be able to put linux or lock down XP and make it a bit more secure on their system and set it up for them to use. Even then, the monthly cost of the dsl line was not worth it to them given the amount of use they would get out of it.
All of that said, I do see a market for something like a SunRay @ home for users like my parents. Small terminal that actually runs everything remotely. With higher speed internet connections (A sunray only needs about 1Mbps for very acceptable performance with a 1280x1024) and almost no power draw, it is perfect for things like this (yes, you can setup a similar setup with a linux terminal, but the sunray is actually simpler. I've done both in my life)
While such a setup would not be workable for most slashdoters, it would work fine for the rest of the world who dont care to become computer mechanics just to browse the web (think tivo users vs mythTV users)
The computer would only be good for scenarios that it has been told how to deal with. Running out of fuel at 41,000 ft is probably not one of them. Using a 767 as a glider is another. I can also bet that the computer would not have known that there was a decommissioned air force base within glide distance either.
We scored a nice setup by taking advantage of what we have, playing the public safety card and tossing the vendors out of the building:-)
The fiber was complements of the state's department of transportation. They were pulling fiber along the interstate and looking for money to help with their costs. Our traffic department was looking for fiber for their traffic cameras and one halway conversation led to another and the next thing we know we have 12 strands and an empty conduit (traffic got another 12). We had enough money to pay for our share of the fiber and everybody was happy. It also helps that the two buildings border the interstate, with the jail being about 300 yards away. Thus the latteral runs were not too expensive. By ourselves we would never have gotten past the right of way issues let alone the overall cost of the project.
As for financing the rest of it, we already had a SAN (a pair of Cisco MDS 9200's) at one site, so it was easy to pop a pair of CDWM gbic's into it and stretch it 2 miles north into another pair of 9200's. Note that these are not "director class" (too expensive). We simply run dual fabrics. In the event of a total switch failure we are still up and running. Note that we are right on the edge of the distance where latency will become an issue.
The real ace in the hole was the fact that the sheriff's department was replacing the servers that they use to run their computer aided dispatch system. For the cost of the annual maintenance on the old stratus systems we could purchase 4 new HP systems, disk arrays, and VCS licenses. Having them riding the DR infrastructure helped sell the entire thing (this _is_ 911 we are talking about). Once the SAN was in place, all we had to do was pick up one side of our cluster, drive it up north and plug it back in. Vola, HA w/ DR:-)
For the servers we simply use Veritas Cluster server in a stretched mode. One of our heartbeats is via a wireless shot and the other via fiber. Having the high speed network allows us to get away with more murder, since we also stretch the IP subnet between the two datacenters. This allows us to avoid having to play any DNS games.
Is it a perfect system? No. (I can tell you what happens when somebody uses a road grater to eat up 800ft of fiber along the road). I can tell you now that we have the wireless shot in place (it happened to be down during that fiber outage), we are where we need to be given the cost of downtime. That said, we still leave most things 'frozen' so that a short network blip will not cause the entire thing to bounce. We would rather suffer 30 minutes of downtime rather than have the thing bounce because a switch somewhere caused a spanning tree recalc (we are replacing all of our server network equipment this year and that issue will go byby as all of the new stuff supports rapid spanning tree)
As for backup, we again when the 'easy' and cheap route. Netbackup 6.x (which we already had) with simple disk storage units. The actual disks are Nexsan and just mounted up as a big file system. The idea is not to hold too much data on disk, but just enough for a weeks worth (we have seen the disks keep 3 of the LTO-3's feed at 80MB/s for hours on end, something not possible with a direct to tape backup over the lan.
Symantec/Veritas does not charge for DSU's, so that was the cheapest way to go for us. If we used a VTL product, we would have to license it as if it was a tape drive. Overall, we could get away with a 4 drive license. (the old system used 10 AIT-3's) Of course, those disks are mirrored between the two sites, netbackup is installed on the Solaris VCS cluster (which also runs oracle and a dozen other services) and we have a small take drive at one site in case the main site has issues. (We recently picked up an LTO-3 based ADIC i2000 for a song by getting it during their end of year. It also helps that they are made in the next county over, so shipping and installation fee's were waved)
What you describe is HA w/ DR and yes, it can get very, very expensive. (and complicated to get right. Trying to prevent false-positive failovers can be a pain in the ass)
Due to cost constraints and a few other issues, I run a 'stretched cluster' where the DR site is 3 miles up the road. Since we have dark fiber between the two sites, I can get away with things that other sites only dream about. My SAN is native FC the entire way, no FCIP or iSCSI bridges. We also have Gig-E between the two locations (soon to be 10Gig-E).
That said, this distance would not be good enough for many companies. I work for a local county, so anthing that takes out the two sites will likely destroy the entire county anyway. To make matters even more interesting, one of the two sites is also the county jail, so it is very well protected.
The other thing that I've had to pay attention to is the tape (yes, tape) backup system. No use in having a system that can replicate data in the blink of an eye if it is the wrong data. I like to joke that the system will commit your error to both sides of the cluster very quickly. In our case we use a standard D2D2T setup where we first dump to disk, and then destage to tape within 4 hours. We then duplicate and offsite the tapes for a month with a 3rd party firm. for us the weekly full backups come out to about 15 LTO-3's worth of data (7.1TB) It is the last thing i want to think about, but at least it is there when all else fails.
I can have a simple HA cluster that involves two nodes attached to a single disk array, all sitting in the same rack somewhere. Take a guess what happens when the power for the building goes out?
HA is nice, but will do nothing for you in the event of a disaster.
You can structure your site so that you get both, but doing so requires a lot more work (stretched clusters and SAN's spread over miles) and you have to be careful that you dont trash your performance while you are at it. (real time replication over distance involves latency, and you have to be careful about what that will do to your app)
Using wikipedia as the ultimate source of information, Yellowstone has seen eruptions 2.1 million years ago, 1.3 million years ago and 640,000 years ago.
This gives you a total of 3 data points, spaced 800,000 and 660,000 years apart. Thus, the average times is 730,000 years. We are 640,000 years into it so we should still have 90,000 years.
Of course, the with only 3 data points even that spacing is very speculative at best.
To make matters more interesting, the size of the 3 eruption's has varied in size, with the largest being 10x larger than the smallest.
Overall, I think i'll spend my time worrying about other things in life. If this thing does go, I probably wont care that much as I'll be totally covered in several feet of ash.
ever try and install something like oracle without any X libs installed? (no, using a tarball from another install does not count)
I've given up on trying to install a striped down system that lack things like X. Then again, none of my systems boot with X running, but it is usually critical to have it there for many applications. (yes, a tivial web server can go without)
We (local county government) use a company called Pictometry (www.pictometry.com) that sells us oblique images from 4 different angles with 6" or better resolution (satellite is often 1 meter at best). Basically, we can see any structure in our county from multiple angles. We can also compare them against time. (hey, when did this big deck show up, and where are the permits?)
As for those who think that having trees right up to the building is a good idea to block the camera, lets just hope that you dont have a forest fire anytime soon. In my neck of the woods, having trees right up to your house is a very bad idea since the threat of a forest fire is so high (same in parts of california where you often see the million dollar homes go up in smoke since somebody though that close in trees were "pretty"). Insurance companies are starting to catch on, bumping rates up for folks who refuse to clear out the stuff from around their house.
As I said, the cost is the same. I didnt say anything about the quality:-)
As you indicated, you can find a lot more info on linux via google. However, the solution may only pertain to Distro X, and not Y (as luck would have it, I'm usually running Y). Sometimes the problem is described as generic to linux, and thus you have to perform the search without using the distro as a keyword. You also have to wade though the various kernel sub-revisions since the problem you are seeing may be fixed differently depending on what version of the kernel you are using.
That can have the side effect that it actually makes it easier for solaris users. A google search for 'solaris ' will usually return things that pertain to the what you are running since there is only one "distro" to choose from. (there are a few other openSolaris distros now floating out there now, like BeleniX, SchilliX and NexentaOS, but they are distinct from solaris... at least for now:-)
I was not thinking of the process scheduler (I use the FSS a lot for our workloads since I pile 15+ zones onto some systems), I was thinking of things a bit lower level, like the IO scheduler that was changed in a recent linux "patch". See the article linked to this recent post: http://linux.slashdot.org/comments.pl?sid=219814&c id=17830154 for what I'm talking about.
Making major changes like the paging algorithm (another one that got swapped mid stream a while back) _and making them the default_ to the unsuspecting sysadmin can have some rather strange side effects. Its one thing to do it as part of a major kernel upgrade (2.4 -> 2.6) but not as a minor one (2.6.16 -> 2.6.17). Expecting the distro folks to keep a list of things straight just adds to the problems that the ISV's must deal with (the "we only support RedHat 4.x, not SUSE" crap). This potentially causes "Linux" to fragment and become a real pain for the enterprise. From a sysadmin point of view, it can get really bad: I cant just run "linux" in my environment, I need to run RedHat for some apps, SUSE for some others, Oracle Linux for another, etc, etc, etc. )
15 years ago 10GB db's were not the norm, they were huge. Today 100GB DB's are 'mid sized'. Large databases run 1TB or higher. Database benchmarks like TPC start at 100G and go up from there (to 10TB, and I've seen larger ones than that). Mid sized databases continue to run 10-20x the size of the 'commodity' hardware, and probably always will (otherwise nobody would need the latest greatest, thus the vendors would have stopped making larger boxes at 4 cpu and 16G of ram due to lack of demand). As soon as you give a DBA a bigger box, they tell the customer who then fills it with more data that they didnt want to put in before because the box could not deal with it. (and that is why I'm looking at a 1+TB ESRI SDE database in the next 6 months for our GIS group)
You will not find many X86 boxes that can hold a TB of ram (yet). Given that you cannot 'superchache' the database, you have to focus on things like Disk I/O, PCI bandwidth, memory interconnects, memory latency, etc. You will find large unix boxes (Sun 25K, HP Superdome, IBM who knows what) that can deal with that sort of load. The biggest "commodity" X86 box that I know of (that actually scales) is a Sun x4600, with up to 128G of ram and 8 Opteron CPU's. (the 32 cpu unisys version didnt scale very well and were very expensive so it was hard to call them commodity) It pales in comparison to the 72 CPU and 1.16TB of memory. (For some database loads, clustering a bunch of small x86 boxes actually decreases performance, no matter what the Oracle sales rep tells you). Besides, when you look at the cost of a 72 CPU oracle license, the hardware is no longer your highest priced item.
I've been using linux since '95ish (slackware something or other installed from floppies). I've been using solaris since '97 (2.5.1).
On the topic of servers, if given a choice of what to run on x64 hardware, its Solaris 10, hands down. Device management is much easier, kernel modules are a snap to deal with (no recompile with each kernel upgrade), folks dont change schedulers as part of minor patch releases, stable API's, etc, etc. Toss in things like zones and dtrace and I'm sold (and no, uml and strace are not the same). I usually dont need crazy hardware support on my servers, just fibre channel and AMD cpu's, so the "better hardware support" of linux does not buy me anything. These are servers, not toys in my basement. When they go down, I have 1000 people calling me and yelling. Its not worth the $250 savings to go with an off-brand NIC or anything other than a qlogic FC card.
Now, on the desktop, its linux. There availability of destop apps and hardware drivers for strange things that just work are much better (acrobat, firefox, flash, etc).
To make things even more interesting, if you want support, Solaris is actually cheaper (compared to redhat). Dont need support? Then they both cost the same.
I'm in the process of moving our Oracle environment from Solaris SPARC to Solaris x86/64 on a mix of Sun x4200's and HP 585's (or Sun x4600's if I can torture the sales rep enough). It involves about 60+ oracle instances that will be moved onto 4 systems. I know that solaris can deal with the load of 1000 procs all running at the same time.
In my environment, I have 500+ LTO-3 tapes (and I'm purchasing about 100 more each month). That cost about $25,000. I also have a Library with two LT0-3 drives. $3000 is for a very cheap one. Lets use a real library and get an ADIC i500 w/ 2 FC LTO-3 Drives. My cost for that library with a 4Gb dual port FC card was $30,000. Total cost for this setup is $55,000. Note that I dont need to keep all of the tapes in the library (most of them sit on a shelf since it is cheaper than the expansion slots for the library).
In order to be able to archive 200TB of data, I would need 200 of these drives (assuming no redundancy). That comes out to about $80,000 for the drives. The price does not include the price of the enclosure to actually connect the drives to the backup server. For the sake of argument, lets assume that cost $0 just to make the drive kids happy. Lets also assume that you dont pay anything for power to keep the drives spinning. Lets also assume that if you drop a drive from 6' nothing will happen to it.
Notice how the tape solution is $25,000 cheaper. The break even point is around 275 LTO-3 tapes worth of data, after which the tape solution becomes cheaper (using my overly simplified calculations that favor the drives).
Please note that the best solution depends on your environment. I run through about 20TB of data per week for backups (I consider that a small to miszied environment). We need to send tapes offsite (weekly), and my datacenter is limited on the power and colling standpoint. If you only have 500gb of weekly backups, then go ahead and toss 15 of these into a disk tray and have at it.
Notice how I still use disk as a front end. Its the only way I can feed my two LTO-3 drives. There is no way I can pull 200MB/s of data over the network from the clients and keep the two drives streaming. It also reduces the number of tape drives I need in order to run 200 backup jobs each day since I can multiplex 20 jobs at once to each disk array to try and drive overall system throughput up. But it is not cost effective for me to leave the data there.
And for the folks who think that shipping 20TB a week over the network to an off site company is a good idea... thats 34MB/s at 100% utilization. The cost of that link alone would pay for the entire tape system after a year. Never underestimate the bandwidth of a truck loaded up with tapes.:-)
Ok, so you are spending $399 for a 1TB drive. Compare this to a 400G (uncompressed) LTO-3 tape @ $50 per tape (price is good as of yesterday when I ordered another 100 LTO-3 tapes). Your drive is about 3x as expensive.
The tape is still cheaper. It also takes up less space on my shelf and I can drop it and not worry about loosing anything.
I am looking at these drives for the front end disk array that I use in my d2d2t setup (disk -> disk -> tape). Given about 40 of them I can keep 2-3 weeks of backups online in the disk and then destage to tape for the offsite vault and archive backups. This way restores of recent data is almost instant (no need to mount and seek to the spot in the tape), but the old archives cost me less and I save on power and cooling (the tape library expansion modules take no additional power. its just a shelf with tape slots).
Its not an either/or choice. Most folks with any real amount of data to backup use both.
Actually, the price of gas is elastic... kinda... there are two conditions:
1) the price has to get well above $3 per gallon as an average over the entire US (so it should be $4/g in places with traditionally high prices like california).
2) the price has to stay up that way for a long time (not $3 this month and $2 next). By 'long' I mean 5+ years. Long enough to give consumers the ability to change cars and homes. (running out to purchase a new hybrid does not make short or even medium term financial sense when compared to a 15mpg land yacht that is already paid for. the car loan makes the cost of gas cheap)
In the short term, most folks can take the financial hit up to a point, and at that time they start to drive smarter. But there is a limit to how much you will save by 'driving smarter'.
In the long term, folks will buy new cars that get better mileage, purchase a house that is closer to work, get a job that is closer to their house, etc. But the prices have to stay high. Today, folks are looking at the mpg ratings. If the price goes down tomorrow, they will start to look at things like legroom and horse power ratings (bigger is better if gas is $1.50 a gallon)
Thus, in the long term, higher gas prices will result in lower consumption. http://www.slate.com/id/2126981/ has a nice summary of why it is inelastic in the short term and elastic in the long term, and why we still like our 10 mpg cars. And you know that the folks providing the gas to us know that it is not in their best interest to keep the prices high for long.
All moderate to large employers have issues. Even within the company I work for (1000 employees), that are groups run by idiots and groups run by great people (even... gasp... managers...)
The trick is to find the right group who has at least 1 or 2 good levels of insulation (aka management) above you. Every large company has these groups. The trick is to find them. Its not easy, but if you manage to find one, you'll be much happier.
Just as a clarification. When i say "i have a tendandy to keep drilling on a topic", I dont mean for hours. It only takes about 5-10 minutes to take care of the issue. I'd rather cut it short rather than drag things on for hours. That is time i could be getting real work done, like surfing slashdot.
As for your particular issue.... I have been in the same situation before. In that case they didnt know what they needed, so i had to put my business analyst hat on to tell them that a) i was not the person that they were looking for and b) what they should be looking for. They didnt have much of a clue, and I'm sure that they are out of buisiness by now.
I've been involved with dozens of interviews. They all tend to go like this:
me: I see that you claimed to have skils in XYZ. How comfortable are you with implementing an XYZ system?
candidate: I'm an expert with XYZ
me: Oh, how did you solve the/Bogus XYZ/ issue?
or me: What were some of the design issues you faced in implementing XYZ? (make sure it is not something that is covered in chapter 1 of the XYZ cram guide)
candidate: Oh, simple..../5 minutes of bs/
If the candidate gives us a look of 'wtf are you talking about', then we smile and move on, drilling into real issues with XYZ (to determine just how much they know about XYZ).
The general idea is this: give them some bait, see if they take it.
As an average, I'd say that half of the candidates that get to my stage of the process are overstating their abilities. They dont get any further. I have yet to regret hiring somebody. (If they are faking it, I have a tendancy to keep drilling on the topic so that it soon becomes painfully obvious that even they realize that they are faking it and we all know it. Its a great way to make sure that the guy does not expect to actually get the job)
In general, lying will only get you a job at a place run by idiots. If that is the type of job you want, go for it. It can pay well, but the turnover can be a real pain. When layoffs come (and they will. Remember, you are working for idiots), dont expect them to keep "the best" people, just the ones that they _think_ are the best. i.e., the ones with the best golf scores.:-)
From reading the news article, it seems that they are selling $3,995 ATA -> ethernet converters (disk sold seperately). Each box will hold 15 drives and offer a simple raid controller inside. It still has the same performance issues of iSCSI (a bit lower overhead, but not by much).
I dont see what the point is other than the fact that they are offering yet another transport protocol. Given that one can install iSCSI target software on linux/solaris/windows... whats the point? Anybody who read the article on Sun's 4500 (48 500G drives w/ a dual proc opteron) know that you can cheaply assemble a box with a ton of drives, slap the iSCSI target software on it and call it networked storage. Redundancy sold seperately.
I've seen FC -> SATA arrays that run around $10K for 6TB. And thats for a system with active/active controllers, mirrored cache and 4Gb FC interfaces. And these systems had the ability to expand to 120 drives on the same controllers (final price: around $1 per GB). I think that FC has a bad rep for being expensive due to companies like EMC and HP. If you know what to look for, you can actually get the stuff for a relatively low cost. Given the performance boost compared to iSCSI, I'm going to stick with FC for the time being (I will still pay for the EMC kit when it makes sense from a support/reliability standpoint, but I also mix in things from companies like NexSAN when cost is more important)
Uh, I hate to tell you, but there is a place called the North Pole on the north side of Pikes Peak, which is only a 12 miles (summit to summit) from Cheyenne Mountain. Their website is http://www.santas-colo.com/
It would make a great place to hide a military compound. They could use the roller coaster as a hidden entry to the underground caverns:-)
Slashdot Mirror
It depends on the type of meeting:
,etc). These are the big time suckers where you wait 5 minutes for half of the folks to just show up.
/.'ers always wanting to use a high tech solution to solve a low-tech problem.)
"Stand ups" where everybody gives a status update is best done in "virtual" sense (con-call, email
On the flip side, design meetings, especially when dealing with physical layout issues, can be a different issue. For example, this past week I had a 20 minute meeting with our network team to discuss rack layout, positioning and the other fun issues that arise when you are low on space in a datacenter and have a ton of stuff showing up in a few weeks. It was a lot easier to just grab a conference room and go at it on the whiteboard. Within minutes we had a plan worked out and everybody was happy. Compare this to a conference call and the thing would have taken forever. Yes, you can use virtual whiteboard software, but why bother when a real one is so much cheaper and easier to use. (I'll spare you my usual rant about
For the team that I work with (a bunch of system and network admins), we can realistically work from home 2-3 days a week. This gives us a good mix of "face time" for meetings where such a thing is important and the rest of the week we can be at home getting longer term projects done that just require us to be alone and crank out stuff (ever wonder why things are never documented they way they should be? )
Anyway, my 0.02 worth. your work environment can and probably will be different.
I happen to work for a county and support our GIS group (along with a dozen other county departments)
Lots of the folks in the GIS group use google maps and google earth for quick and dirty stuff. We even use a google maps mashup on our main site for anything that requires a quick and dirty mapping application (voting locations, locations of sex offendors, etc)
That said, it is not a replacement for the GIS department, but it does help keep the size of the department in check. There are a few gotchas with the use of google:
1) Google earth is not free.
It is free for non-commercial use only. Everybody else has to pay.
2) The imagery is old
We do flyovers every two years minimum. The stuff on google is often 5+ years old for some parts of the county (the copyright date gets updated, but the images do not)
3) The data is not nearly as accurate
For quick and dirty work, google earth is ok. But we have had to work on areas where google only has 1m or worse. We have 6" resolution for the entire county. It is also been rectified and fixed and things like plot lines and street centerlines are dead on. I've played with image overlays before, and google can be 20+ ft off in one direction or another. That is simply not acceptable when you are trying to figure out where you are going to put a street.
4) Ever try and plot a 6' by 42" map using google earth at full resolution with plot line overlays and dozen of other custom features that the customer wants for a presentation? Didn't think so.
So, if all the gis department does is provide non-rectified 1 meter satellite photos from 10 years ago... yeah, time to ditch them and use google. For anything else, you are going to need a gis group.... It does not have to be large, but it better exist.
You place way too much faith in computers.
With the current state of the art, they do what they are told to do. No more, no less. Given a set of inputs they produce an output. They are not some magical creature that will devise elegant solutions to a problem and save the day. The chances that they will devise a solution to a problem that they are not explicitly taught how to deal with is 0.
With a human you at least have the ability to learn. This gives us poor humans the ability to craft a new and novel solution to a problem that has yet to be seen. While we are not as good at is repetitive tasks, this ability to be creative gives us a huge advantage in new situations. How many years did it take until the computer folks could produce a system to beat a human at chess? That is an example of a system where the computer has perfect knowledge of the playing field. The real world of flying aircraft does not afford the programmers such an advantage.
As for the gimli incident, the 'computer' was blind to how much fuel was in the system, hence the original problem. Now, look back at all of the crashes blamed on 'pilot error'. Many were made because the pilot was not given correct information about the situation that s/he was in. Wrong altitude, wrong airspeed, direction, weather, flap position, etc, etc. What makes you think that a computer would not make the same decision given the same set of information? What magical sensor system will fail and provide the pilot with the wrong information and not provide a computer with the same (wrong) information?
Some day we will have an expert system that is capable of performing the same tasks under any given set of circumstances. Until then, we have pilots.
Thats the kicker. As evidenced by the air ram generator, the designers took into account full power loss. It is easy to account for the issues that have already occurred, and probably a handful of ones that have not yet occurred.
The fun thing is that in a system with that many components, the failure scenarios are endless. As soon as you think of all of them you will run into another. Accounting for all of them are next to impossible.
In the case of the gimli incident, not only would the computer have to know what to do about no engine power due to no fuel, and know how to do forward slip to cut some speed fast just before landing, it would have to know about a landing strip that was no longer in use. If you happen to account for the first two, your chances of knowing about the landing strip are zero. From the writeup, as the air speed dropped, so did the power generated by the generator. The computer would need to be on an independent power supply or else it would crash from lack of power during some of the maneuvers. (and then you would need multiple independent flight computers, with independent electrical systems. As somebody who just had two different 'fully redundant' disk arrays at two different sites totally fail because one controller took out the other via the sync channel, this is not an easy task. And yes, they both occurred on the same night a few hours apart from each other. totally random. )
In the gimli case, it was luck that one pilot happened to be a glider pilot and the other happened to be stationed at gimli a few years before.
If a computer was in charge of this incident, we would probably have 60+ dead people. In this case the plane was flown out of there 2 days latter after minor repairs and is still in use today. A few folks suffered minor injuries exiting the back of the aircraft.
They had internet access. First dial up, then a DLS line.
After a year or so, my siblings were the only ones using it, usually to download spyware and such while hitting myspace.
After walking my dad through reinstalling XP home on the computer to get rid of all of the crap, he gave up. The computer now sits in the corner of their home and is rarely used.
When they need internet access, they go to the library. It is not a major part of their life.
If I lived a bit closer, I would probably be able to put linux or lock down XP and make it a bit more secure on their system and set it up for them to use. Even then, the monthly cost of the dsl line was not worth it to them given the amount of use they would get out of it.
All of that said, I do see a market for something like a SunRay @ home for users like my parents. Small terminal that actually runs everything remotely. With higher speed internet connections (A sunray only needs about 1Mbps for very acceptable performance with a 1280x1024) and almost no power draw, it is perfect for things like this (yes, you can setup a similar setup with a linux terminal, but the sunray is actually simpler. I've done both in my life)
While such a setup would not be workable for most slashdoters, it would work fine for the rest of the world who dont care to become computer mechanics just to browse the web (think tivo users vs mythTV users)
Perhaps you have never heard of the gimli glider: http://en.wikipedia.org/wiki/Gimli_Glider
The computer would only be good for scenarios that it has been told how to deal with. Running out of fuel at 41,000 ft is probably not one of them. Using a 767 as a glider is another. I can also bet that the computer would not have known that there was a decommissioned air force base within glide distance either.
We scored a nice setup by taking advantage of what we have, playing the public safety card and tossing the vendors out of the building :-)
:-)
The fiber was complements of the state's department of transportation. They were pulling fiber along the interstate and looking for money to help with their costs. Our traffic department was looking for fiber for their traffic cameras and one halway conversation led to another and the next thing we know we have 12 strands and an empty conduit (traffic got another 12). We had enough money to pay for our share of the fiber and everybody was happy. It also helps that the two buildings border the interstate, with the jail being about 300 yards away. Thus the latteral runs were not too expensive. By ourselves we would never have gotten past the right of way issues let alone the overall cost of the project.
As for financing the rest of it, we already had a SAN (a pair of Cisco MDS 9200's) at one site, so it was easy to pop a pair of CDWM gbic's into it and stretch it 2 miles north into another pair of 9200's. Note that these are not "director class" (too expensive). We simply run dual fabrics. In the event of a total switch failure we are still up and running. Note that we are right on the edge of the distance where latency will become an issue.
The real ace in the hole was the fact that the sheriff's department was replacing the servers that they use to run their computer aided dispatch system. For the cost of the annual maintenance on the old stratus systems we could purchase 4 new HP systems, disk arrays, and VCS licenses. Having them riding the DR infrastructure helped sell the entire thing (this _is_ 911 we are talking about). Once the SAN was in place, all we had to do was pick up one side of our cluster, drive it up north and plug it back in. Vola, HA w/ DR
For the servers we simply use Veritas Cluster server in a stretched mode. One of our heartbeats is via a wireless shot and the other via fiber. Having the high speed network allows us to get away with more murder, since we also stretch the IP subnet between the two datacenters. This allows us to avoid having to play any DNS games.
Is it a perfect system? No. (I can tell you what happens when somebody uses a road grater to eat up 800ft of fiber along the road). I can tell you now that we have the wireless shot in place (it happened to be down during that fiber outage), we are where we need to be given the cost of downtime. That said, we still leave most things 'frozen' so that a short network blip will not cause the entire thing to bounce. We would rather suffer 30 minutes of downtime rather than have the thing bounce because a switch somewhere caused a spanning tree recalc (we are replacing all of our server network equipment this year and that issue will go byby as all of the new stuff supports rapid spanning tree)
As for backup, we again when the 'easy' and cheap route. Netbackup 6.x (which we already had) with simple disk storage units. The actual disks are Nexsan and just mounted up as a big file system. The idea is not to hold too much data on disk, but just enough for a weeks worth (we have seen the disks keep 3 of the LTO-3's feed at 80MB/s for hours on end, something not possible with a direct to tape backup over the lan.
Symantec/Veritas does not charge for DSU's, so that was the cheapest way to go for us. If we used a VTL product, we would have to license it as if it was a tape drive. Overall, we could get away with a 4 drive license. (the old system used 10 AIT-3's) Of course, those disks are mirrored between the two sites, netbackup is installed on the Solaris VCS cluster (which also runs oracle and a dozen other services) and we have a small take drive at one site in case the main site has issues. (We recently picked up an LTO-3 based ADIC i2000 for a song by getting it during their end of year. It also helps that they are made in the next county over, so shipping and installation fee's were waved)
The final trick was to en
What you describe is HA w/ DR and yes, it can get very, very expensive. (and complicated to get right. Trying to prevent false-positive failovers can be a pain in the ass)
Due to cost constraints and a few other issues, I run a 'stretched cluster' where the DR site is 3 miles up the road. Since we have dark fiber between the two sites, I can get away with things that other sites only dream about. My SAN is native FC the entire way, no FCIP or iSCSI bridges. We also have Gig-E between the two locations (soon to be 10Gig-E).
That said, this distance would not be good enough for many companies. I work for a local county, so anthing that takes out the two sites will likely destroy the entire county anyway. To make matters even more interesting, one of the two sites is also the county jail, so it is very well protected.
The other thing that I've had to pay attention to is the tape (yes, tape) backup system. No use in having a system that can replicate data in the blink of an eye if it is the wrong data. I like to joke that the system will commit your error to both sides of the cluster very quickly. In our case we use a standard D2D2T setup where we first dump to disk, and then destage to tape within 4 hours. We then duplicate and offsite the tapes for a month with a 3rd party firm. for us the weekly full backups come out to about 15 LTO-3's worth of data (7.1TB) It is the last thing i want to think about, but at least it is there when all else fails.
HA is not the same as DR
I can have a simple HA cluster that involves two nodes attached to a single disk array, all sitting in the same rack somewhere. Take a guess what happens when the power for the building goes out?
HA is nice, but will do nothing for you in the event of a disaster.
You can structure your site so that you get both, but doing so requires a lot more work (stretched clusters and SAN's spread over miles) and you have to be careful that you dont trash your performance while you are at it. (real time replication over distance involves latency, and you have to be careful about what that will do to your app)
Ok, I live at 6,500 ft. Last time I checked, we still had fire stations because things still burned.
Even at the higher elivations (14,000) things still burn.
Its "regular" eruption?
Using wikipedia as the ultimate source of information, Yellowstone has seen eruptions 2.1 million years ago, 1.3 million years ago and 640,000 years ago.
This gives you a total of 3 data points, spaced 800,000 and 660,000 years apart. Thus, the average times is 730,000 years. We are 640,000 years into it so we should still have 90,000 years.
Of course, the with only 3 data points even that spacing is very speculative at best.
To make matters more interesting, the size of the 3 eruption's has varied in size, with the largest being 10x larger than the smallest.
http://www.solcomhouse.com/yellowstone.htm has some great details.
Overall, I think i'll spend my time worrying about other things in life. If this thing does go, I probably wont care that much as I'll be totally covered in several feet of ash.
ever try and install something like oracle without any X libs installed? (no, using a tarball from another install does not count)
I've given up on trying to install a striped down system that lack things like X. Then again, none of my systems boot with X running, but it is usually critical to have it there for many applications. (yes, a tivial web server can go without)
We (local county government) use a company called Pictometry (www.pictometry.com) that sells us oblique images from 4 different angles with 6" or better resolution (satellite is often 1 meter at best). Basically, we can see any structure in our county from multiple angles. We can also compare them against time. (hey, when did this big deck show up, and where are the permits?)
As for those who think that having trees right up to the building is a good idea to block the camera, lets just hope that you dont have a forest fire anytime soon. In my neck of the woods, having trees right up to your house is a very bad idea since the threat of a forest fire is so high (same in parts of california where you often see the million dollar homes go up in smoke since somebody though that close in trees were "pretty"). Insurance companies are starting to catch on, bumping rates up for folks who refuse to clear out the stuff from around their house.
As I said, the cost is the same. I didnt say anything about the quality :-)
:-)
As you indicated, you can find a lot more info on linux via google. However, the solution may only pertain to Distro X, and not Y (as luck would have it, I'm usually running Y). Sometimes the problem is described as generic to linux, and thus you have to perform the search without using the distro as a keyword. You also have to wade though the various kernel sub-revisions since the problem you are seeing may be fixed differently depending on what version of the kernel you are using.
That can have the side effect that it actually makes it easier for solaris users. A google search for 'solaris ' will usually return things that pertain to the what you are running since there is only one "distro" to choose from. (there are a few other openSolaris distros now floating out there now, like BeleniX, SchilliX and NexentaOS, but they are distinct from solaris... at least for now
I was not thinking of the process scheduler (I use the FSS a lot for our workloads since I pile 15+ zones onto some systems), I was thinking of things a bit lower level, like the IO scheduler that was changed in a recent linux "patch". See the article linked to this recent post: http://linux.slashdot.org/comments.pl?sid=219814&c id=17830154 for what I'm talking about.
Making major changes like the paging algorithm (another one that got swapped mid stream a while back) _and making them the default_ to the unsuspecting sysadmin can have some rather strange side effects. Its one thing to do it as part of a major kernel upgrade (2.4 -> 2.6) but not as a minor one (2.6.16 -> 2.6.17). Expecting the distro folks to keep a list of things straight just adds to the problems that the ISV's must deal with (the "we only support RedHat 4.x, not SUSE" crap). This potentially causes "Linux" to fragment and become a real pain for the enterprise. From a sysadmin point of view, it can get really bad: I cant just run "linux" in my environment, I need to run RedHat for some apps, SUSE for some others, Oracle Linux for another, etc, etc, etc. )
15 years ago 10GB db's were not the norm, they were huge. Today 100GB DB's are 'mid sized'. Large databases run 1TB or higher. Database benchmarks like TPC start at 100G and go up from there (to 10TB, and I've seen larger ones than that). Mid sized databases continue to run 10-20x the size of the 'commodity' hardware, and probably always will (otherwise nobody would need the latest greatest, thus the vendors would have stopped making larger boxes at 4 cpu and 16G of ram due to lack of demand). As soon as you give a DBA a bigger box, they tell the customer who then fills it with more data that they didnt want to put in before because the box could not deal with it. (and that is why I'm looking at a 1+TB ESRI SDE database in the next 6 months for our GIS group)
You will not find many X86 boxes that can hold a TB of ram (yet). Given that you cannot 'superchache' the database, you have to focus on things like Disk I/O, PCI bandwidth, memory interconnects, memory latency, etc. You will find large unix boxes (Sun 25K, HP Superdome, IBM who knows what) that can deal with that sort of load. The biggest "commodity" X86 box that I know of (that actually scales) is a Sun x4600, with up to 128G of ram and 8 Opteron CPU's. (the 32 cpu unisys version didnt scale very well and were very expensive so it was hard to call them commodity) It pales in comparison to the 72 CPU and 1.16TB of memory. (For some database loads, clustering a bunch of small x86 boxes actually decreases performance, no matter what the Oracle sales rep tells you). Besides, when you look at the cost of a 72 CPU oracle license, the hardware is no longer your highest priced item.
I've been using linux since '95ish (slackware something or other installed from floppies). I've been using solaris since '97 (2.5.1).
On the topic of servers, if given a choice of what to run on x64 hardware, its Solaris 10, hands down. Device management is much easier, kernel modules are a snap to deal with (no recompile with each kernel upgrade), folks dont change schedulers as part of minor patch releases, stable API's, etc, etc. Toss in things like zones and dtrace and I'm sold (and no, uml and strace are not the same). I usually dont need crazy hardware support on my servers, just fibre channel and AMD cpu's, so the "better hardware support" of linux does not buy me anything. These are servers, not toys in my basement. When they go down, I have 1000 people calling me and yelling. Its not worth the $250 savings to go with an off-brand NIC or anything other than a qlogic FC card.
Now, on the desktop, its linux. There availability of destop apps and hardware drivers for strange things that just work are much better (acrobat, firefox, flash, etc).
To make things even more interesting, if you want support, Solaris is actually cheaper (compared to redhat). Dont need support? Then they both cost the same.
I'm in the process of moving our Oracle environment from Solaris SPARC to Solaris x86/64 on a mix of Sun x4200's and HP 585's (or Sun x4600's if I can torture the sales rep enough). It involves about 60+ oracle instances that will be moved onto 4 systems. I know that solaris can deal with the load of 1000 procs all running at the same time.
Ok, here goes:
:-)
In my environment, I have 500+ LTO-3 tapes (and I'm purchasing about 100 more each month). That cost about $25,000. I also have a Library with two LT0-3 drives. $3000 is for a very cheap one. Lets use a real library and get an ADIC i500 w/ 2 FC LTO-3 Drives. My cost for that library with a 4Gb dual port FC card was $30,000. Total cost for this setup is $55,000. Note that I dont need to keep all of the tapes in the library (most of them sit on a shelf since it is cheaper than the expansion slots for the library).
In order to be able to archive 200TB of data, I would need 200 of these drives (assuming no redundancy). That comes out to about $80,000 for the drives. The price does not include the price of the enclosure to actually connect the drives to the backup server. For the sake of argument, lets assume that cost $0 just to make the drive kids happy. Lets also assume that you dont pay anything for power to keep the drives spinning. Lets also assume that if you drop a drive from 6' nothing will happen to it.
Notice how the tape solution is $25,000 cheaper. The break even point is around 275 LTO-3 tapes worth of data, after which the tape solution becomes cheaper (using my overly simplified calculations that favor the drives).
Please note that the best solution depends on your environment. I run through about 20TB of data per week for backups (I consider that a small to miszied environment). We need to send tapes offsite (weekly), and my datacenter is limited on the power and colling standpoint. If you only have 500gb of weekly backups, then go ahead and toss 15 of these into a disk tray and have at it.
Notice how I still use disk as a front end. Its the only way I can feed my two LTO-3 drives. There is no way I can pull 200MB/s of data over the network from the clients and keep the two drives streaming. It also reduces the number of tape drives I need in order to run 200 backup jobs each day since I can multiplex 20 jobs at once to each disk array to try and drive overall system throughput up. But it is not cost effective for me to leave the data there.
And for the folks who think that shipping 20TB a week over the network to an off site company is a good idea... thats 34MB/s at 100% utilization. The cost of that link alone would pay for the entire tape system after a year. Never underestimate the bandwidth of a truck loaded up with tapes.
Ok, so you are spending $399 for a 1TB drive. Compare this to a 400G (uncompressed) LTO-3 tape @ $50 per tape (price is good as of yesterday when I ordered another 100 LTO-3 tapes). Your drive is about 3x as expensive.
The tape is still cheaper. It also takes up less space on my shelf and I can drop it and not worry about loosing anything.
I am looking at these drives for the front end disk array that I use in my d2d2t setup (disk -> disk -> tape). Given about 40 of them I can keep 2-3 weeks of backups online in the disk and then destage to tape for the offsite vault and archive backups. This way restores of recent data is almost instant (no need to mount and seek to the spot in the tape), but the old archives cost me less and I save on power and cooling (the tape library expansion modules take no additional power. its just a shelf with tape slots).
Its not an either/or choice. Most folks with any real amount of data to backup use both.
Actually, the price of gas is elastic... kinda... there are two conditions:
1) the price has to get well above $3 per gallon as an average over the entire US (so it should be $4/g in places with traditionally high prices like california).
2) the price has to stay up that way for a long time (not $3 this month and $2 next). By 'long' I mean 5+ years. Long enough to give consumers the ability to change cars and homes. (running out to purchase a new hybrid does not make short or even medium term financial sense when compared to a 15mpg land yacht that is already paid for. the car loan makes the cost of gas cheap)
In the short term, most folks can take the financial hit up to a point, and at that time they start to drive smarter. But there is a limit to how much you will save by 'driving smarter'.
In the long term, folks will buy new cars that get better mileage, purchase a house that is closer to work, get a job that is closer to their house, etc. But the prices have to stay high. Today, folks are looking at the mpg ratings. If the price goes down tomorrow, they will start to look at things like legroom and horse power ratings (bigger is better if gas is $1.50 a gallon)
Thus, in the long term, higher gas prices will result in lower consumption. http://www.slate.com/id/2126981/ has a nice summary of why it is inelastic in the short term and elastic in the long term, and why we still like our 10 mpg cars. And you know that the folks providing the gas to us know that it is not in their best interest to keep the prices high for long.
All moderate to large employers have issues. Even within the company I work for (1000 employees), that are groups run by idiots and groups run by great people (even... gasp... managers...)
The trick is to find the right group who has at least 1 or 2 good levels of insulation (aka management) above you. Every large company has these groups. The trick is to find them. Its not easy, but if you manage to find one, you'll be much happier.
Just as a clarification. When i say "i have a tendandy to keep drilling on a topic", I dont mean for hours. It only takes about 5-10 minutes to take care of the issue. I'd rather cut it short rather than drag things on for hours. That is time i could be getting real work done, like surfing slashdot.
As for your particular issue.... I have been in the same situation before. In that case they didnt know what they needed, so i had to put my business analyst hat on to tell them that a) i was not the person that they were looking for and b) what they should be looking for. They didnt have much of a clue, and I'm sure that they are out of buisiness by now.
Anyway, thats my take on it.
me: I see that you claimed to have skils in XYZ. How comfortable are you with implementing an XYZ system?
/Bogus XYZ/ issue?
/5 minutes of bs/
:-)
candidate: I'm an expert with XYZ
me: Oh, how did you solve the
or
me: What were some of the design issues you faced in implementing XYZ? (make sure it is not something that is covered in chapter 1 of the XYZ cram guide)
candidate: Oh, simple....
If the candidate gives us a look of 'wtf are you talking about', then we smile and move on, drilling into real issues with XYZ (to determine just how much they know about XYZ).
The general idea is this: give them some bait, see if they take it.
As an average, I'd say that half of the candidates that get to my stage of the process are overstating their abilities. They dont get any further. I have yet to regret hiring somebody. (If they are faking it, I have a tendancy to keep drilling on the topic so that it soon becomes painfully obvious that even they realize that they are faking it and we all know it. Its a great way to make sure that the guy does not expect to actually get the job)
In general, lying will only get you a job at a place run by idiots. If that is the type of job you want, go for it. It can pay well, but the turnover can be a real pain. When layoffs come (and they will. Remember, you are working for idiots), dont expect them to keep "the best" people, just the ones that they _think_ are the best. i.e., the ones with the best golf scores.
From reading the news article, it seems that they are selling $3,995 ATA -> ethernet converters (disk sold seperately). Each box will hold 15 drives and offer a simple raid controller inside. It still has the same performance issues of iSCSI (a bit lower overhead, but not by much).
I dont see what the point is other than the fact that they are offering yet another transport protocol. Given that one can install iSCSI target software on linux/solaris/windows... whats the point? Anybody who read the article on Sun's 4500 (48 500G drives w/ a dual proc opteron) know that you can cheaply assemble a box with a ton of drives, slap the iSCSI target software on it and call it networked storage. Redundancy sold seperately.
I've seen FC -> SATA arrays that run around $10K for 6TB. And thats for a system with active/active controllers, mirrored cache and 4Gb FC interfaces. And these systems had the ability to expand to 120 drives on the same controllers (final price: around $1 per GB). I think that FC has a bad rep for being expensive due to companies like EMC and HP. If you know what to look for, you can actually get the stuff for a relatively low cost. Given the performance boost compared to iSCSI, I'm going to stick with FC for the time being (I will still pay for the EMC kit when it makes sense from a support/reliability standpoint, but I also mix in things from companies like NexSAN when cost is more important)
Uh, I hate to tell you, but there is a place called the North Pole on the north side of Pikes Peak, which is only a 12 miles (summit to summit) from Cheyenne Mountain. Their website is http://www.santas-colo.com/
:-)
It would make a great place to hide a military compound. They could use the roller coaster as a hidden entry to the underground caverns