Multiple servers for Active Directory, and off-site hot backup.
Multiple servers for terminal server farm (student information system - access/sql fat client..old school but it works)
Multiple servers for front-end/back-end exchange server.
SQL server
Terminal server for Quickbooks application
File servers at each location
IP camera video recording server
Multiple servers for network applications that require their own hardware (as per the software vendors). These probably will be consolidated to virtual servers in the next year or so.
Multiple Mac servers hosting Mac specific network apps and data, as well as Mac netboot system images.
Intranet server
Backup/remote admin server
It isn't hard to accumulate a room full of racks and servers as the school grows. We have 125 students and 70+ staff members excluding administration staff (special-ed school - low student to staff ratio). All staff members have remote access to their applications and data.
Consolidation is nice, if you can afford the downtime to consolidate and cost of consolidation. Virtualization is nice if you can afford the ability to make your hypervisor fault tolerant. All virtualization systems I've looked at have high costs when you start asking about HA/DR.
We enjoy good uptime because we have simple systems, and all our eggs are not in one "hardware basket".
Let me guess - you are one of those guys that loads up a server with lots of mission critical tasks and then prays that the box never fails.
I'm the "IT guy" for a small private school. We have 30 or so servers, two locations, and about 300 laptops and desktops.
If it plugs in somewhere in either of our facilities, I'm responsible for making sure it works - and continues to work.
Sure, I work on higher level things like security policy, network architecture, budgets, and reports for the board and administration, but I also fix printers and workstations.
My title is IT director since the state Department of Education REQUIRES that my pay grade be given a "Director" role. That's all well and good, but I don't get hung up about it.
Why? Because some days I'm the IT director and some days I'm the IT janitor. The reason I remain employed is that my attitude toward the job does not change on the days that I need to be the "janitor".
The article does not mention "backups" as in tape drives and off-site storage.
The article does mention lack of redundancy at the network carrier level.
My guess is that Northrop Grumman designed a network around single circuits connecting offices to data centers, and did not design the network to tolerate WAN link failures.
A stupid oversight for sure, but nothing that can't be easily remedied by ordering redundant WAN circuits from your telco of choice. Redundant routing gear would also be smart.
For all that are blaming government for this - they outsourced the design and implementation to a private company. That company screwed the pooch in design and implementation. Shame on both parties for not recognizing the risk of WAN failure.
So you've hit the 250GB cap, but you aren't a pirate. You pay for everything you consume - including bandwidth. Your only crime is that you went to another company for video service. You like your Apple TV and the iTunes store, or you like using a slingbox, or you like movies on demand from your Roku, or your DirecTV receiver.
All of these technologies may cause you to run over your cap, and they all have one thing in common - they directly compete with Comcast's video services.
Now Comcast appears to be using their broadband monopoly, in the form of transfer caps, to discourage the use of competing services.
If this isn't the very definition of an abusive monopoly, I don't know what is.
These idiots can't figure out how to secure the config pages of a cable modem, and we are to trust that they can implement QOS correctly? I've only been working on networks and IT stuff for a decade, so maybe I don't know what I'm talking about, but QOS seems a bit harder to do than securing a cable modem config page.
We need net neutrality for two reasons:
1. To keep the internet open to all that would want to use it.
2. To keep grossly incompetent network administrators' hands off of our data.
Without net neutrality laws, content providers, and network operators both have economic incentive to keep bandwidth scarce.
Network operators, thanks to monopoly or duopoly status, can keep their network capacity scarce, and still charge high prices, while deferring costly upgrades.
Content providers can lock out smaller competitors by purchasing "prioritized network capacity" at prices smaller content providers can not afford.
It's a lose-lose for consumers and the internet as a whole.
Look at the progress in industries where "capacity" was abundant - like silicon transistors in chip manufacturing. Chip designers paid little attention to transistor count knowing that future process technologies would allow their designs to be made in a cost-effective manner. This allowed the industry to produce very powerful chips at very low cost.
Similar progress will be made in network capacity and speeds to meet the demands of the network users, but this can only occur if the network treats all traffic/content equally.
During a CNBC interview, Steve Wynn stated his casino keeps 20 cents of every dollar bet.
The performance of his casino is strictly determined by the number of bets placed - not by the games played.
That tells me that EVERY game favors the house - as does blackjack including the card counters. If the games did not favor the house, they would eliminate them from the casino floor.
Having worked with plenty of enterprise grade raid (EMC symetrix, clarion, and Dell SAN devices) I can say that capacity and rebuild times are not a problem for high-end arrays.
What will bring the problem to the masses are these stupid consumer NAS boxes. It is very easy to build a 4 or 8 TB array for home use using relatively cheap hardware. Unfortunately, no home user/abuser, that I know, has the skill set to manage or protect such a large array of data.
My most recent experience with a Western Digital sharespace was awful. Here is a box with a Gigabit NIC, and 4 - 2TB hard drives in a RAID 5 array that has transfer rates around 9MB/sec at best. Combine that pitiful performance with a rebuild/reformat time of over two days - and you know where this is going.
Average joes are going to put their entire lives on these things and never back them up due to the time and space cost. When a failure does occur - it will take days to perform a rebuild of the array - vastly increasing the likelyhood of another failure and permanent data loss.
Crappy RAID's days are numbered - good RAID implementations will be with us as long as hard drives have ANY failure rate at all.
They are deploying these in public locations like parks and stores. Unfortunately they are only deploying this network in their service areas, and they are not public. You need to be a cablevision subscriber to access them.
Why yes, I am a subscriber - so let me tell you about them.
If you are outside they are great - assuming there is one near you. Once you go into a building - forget it. The signal falls off a cliff, and the service is unusable.
As cool as public Wifi would be, I'm not holding my breath for it. It's the wrong technology for the application.
This is American style management at its worst - and we are exporting this management trash to other countries. Hopefully other countries will see these management clowns for what they are.
Look at every failing or failed industry in the US, and you can point to the MBA managers that sucked the life out of the company, wrote big checks to the executives and shareholders and left a carcass behind for the employees and customers.
Autos, telecoms, steel, and soon to be IT services, and pharma. These industries are being squeezed for short-term profits and dividends and the expense of the future. Crap service and bail-out nation are the baby boomers' parting gifts to us.
Hopefully the rest of the world will figure out that smart, talented engineers can also make good managers and finance guys. The future success of their companies depend on it.
Boards of companies like AT&T and GM need to kick out the Harvard MBAs and move some engineers into those positions. They may even be able to save some money on executive payroll to do it.
There aren't many banking software vendors. They are all roughly the same in terms of quality. There is no software company in the world that will give you an iron-clad security guarantee at any price.
The market has determined that this type of software is "good enough". The software is "good enough" and the cost is tolerable. Unfortunately "good enough" = sucks.
All of our vendors were audited by multiple independent auditing firms, had SAS70 compliance, and were also audited by federal regulators (FDIC, and OTS). It is a federal requirement by our regulators that all of our vendors go through multiple security audits multiple times per year.
Further more, our applications WERE behind a managed security service (Perimeter security services) which included a web app firewall and intrusion detection.
How exactly do you audit code that is proprietary and not viewable by the public? Every application vendor in this space, that I know of, will not let anyone outside the company view proprietary code. Federal regulators are the exception - they are allowed by law to audit the code. I am not.
How is a small organization supposed to have the resources and the man-power to audit an entire company (let alone many companies) and their products? We were in the banking business, not the software development and auditing business.
In short - fuck off - you have no idea what you are talking about.
I was the network services manager for a small community bank a couple of years ago, and all of our online banking fraud was directly related to the insecurity of the online banking application - specifically SQL injection attacks.
The application vendor's solution was to encrypt everything in the database and block known SQL injection "patterns". I told them they needed to harden their application against SQL injection; encryption and pattern matching are not enough.
Sure enough, some Russian guys (I'm guessing by the originating IP addresses) figured out that if they opened an account with a known password, they could use SQL injection to copy the encrypted known password to an account with lots of money.
Our work-around for the crappy vendor's "security" was implementing RSA tokens (outside of the banking app) on business accounts that could electronically move money out of the bank. Non-business accounts could only transfer money inside the bank - a large fraudulent transaction would get caught by a human before the money left the bank.
Before anyone suggests switching vendors, consider two things:
1. Switching banking software vendors is EXTREMELY disruptive to business. In a business where customers complain about 5 minute drive-through times, a large software migration with downtime and training is intolerable.
2. All small to medium bank software vendors suffer from similar code quality problems. Moving to another product does not necessarily guarantee quality code.
When price is all that matters, it is a race to the bottom.
That bottom (in manufacturing) may well be Africa.
China and India may have sacrificed a generation to "undercut" the west, but that sacrifice will really hurt them when producers of goods move production to Africa.
Large oil reserves are being found in Africa - it is only a matter of time until industry moves from Asia to Africa.
People only value quality after they've been burned by the lack of it.
I'm not sure about these dire predictions. Technology is constantly evolving, and new uses for technology are constantly being created.
I'm the IT director for a small private school (less than 200 students, less than 70 staff, two locations). Our small little school has VOIP to the desktop in every classroom and office, network printers damn near everywhere, 20mbps fiber at each location, and a server room with two telco racks of network equipment and three 48-space racks of servers. We have laptop carts in most classrooms, and a few, well stocked, computer labs.
We are actually stressing our 400 amp service at our main site, and any further expansion will require an electrical service upgrade.
Have I mentioned that we are a SMALL school?
10 years ago we didn't even have one third of this stuff. I don't see any reason why this trend will not continue.
Frankly, I'm thrilled at the evolution of technologies like virtualization - they will make managing all this much easier.
IT is probably one of the few areas in our economy that will actually grow in the next 30 years. Will it grow like the last 30? Maybe not, but any growth is still better than none.
When Vista was released, we couldn't copy a few 16k files from our servers in under an hour. I have a hard time believing that Microsoft did not catch this bug during internal testing. Network file copy is a fairly important and frequently used function. This is a "showstopper" for anyone that uses a network file server, yet Microsoft shipped the product anyway.
I don't think a "showstopper" bug exists that would cause Microsoft to delay shipment of a product. Their motto appears to be "ship early and patch often".
If Apple or any other company uses this technology against their customers, I predict many phone batteries will "catch fire". It can't be too hard to make a Lithium-Ion battery go up in flames. Hell, it seems to be hard to PREVENT them from exploding.
So instead of hair-drying a wet phone, customers will cause the battery to go nuclear and then take the smoldering mess to the store demanding a replacement.
Short of building the sensors into an airplane black-box type device (which might hurt the phone's portability a bit) it is unlikely any of these sensors would survive a fire.
I had a Motorola Q that started smoking one day while it was charging. The battery was so hot, I couldn't hold it for more than a few seconds. The case started melting, and I'm sure if I hadn't been there, it would have caught fire.
The Verizon guy replaced the phone, no hassle, no questions.
Every day I read about how the world should be: wind and solar farms generating electricity, no more fossil fuels, everyone living in cities and can walk/bike to everything they need - and no more commercial, closed software - free and open software for all.
These are all nice ideas, but they fail in the exact same way - they aren't practical for most people.
We are going to burn every drop of financially viable fossil fuels that are in the ground - the sooner engineers and environmentalists accept that fact, the sooner we can start working toward REAL solutions to our energy problems (nuclear has my vote).
A world without Microsoft office, or Microsoft products in general might be a nice vision of your utopia, but for the vast majority of computer users, they are happy shelling out the cash for a refined product that they are comfortable using.
I like free and open products whenever possible, but replacing many Microsoft products, that people are comfortable with, has enormous costs beyond mere dollars.
Slashdot Mirror
When everyone is super - no one will be.
-ted
3 servers at one location about 27 at another.
Multiple servers for Active Directory, and off-site hot backup.
Multiple servers for terminal server farm (student information system - access/sql fat client..old school but it works)
Multiple servers for front-end/back-end exchange server.
SQL server
Terminal server for Quickbooks application
File servers at each location
IP camera video recording server
Multiple servers for network applications that require their own hardware (as per the software vendors). These probably will be consolidated to virtual servers in the next year or so.
Multiple Mac servers hosting Mac specific network apps and data, as well as Mac netboot system images.
Intranet server
Backup/remote admin server
It isn't hard to accumulate a room full of racks and servers as the school grows. We have 125 students and 70+ staff members excluding administration staff (special-ed school - low student to staff ratio). All staff members have remote access to their applications and data.
Consolidation is nice, if you can afford the downtime to consolidate and cost of consolidation. Virtualization is nice if you can afford the ability to make your hypervisor fault tolerant. All virtualization systems I've looked at have high costs when you start asking about HA/DR.
We enjoy good uptime because we have simple systems, and all our eggs are not in one "hardware basket".
Let me guess - you are one of those guys that loads up a server with lots of mission critical tasks and then prays that the box never fails.
-ted
I'm the "IT guy" for a small private school. We have 30 or so servers, two locations, and about 300 laptops and desktops.
If it plugs in somewhere in either of our facilities, I'm responsible for making sure it works - and continues to work.
Sure, I work on higher level things like security policy, network architecture, budgets, and reports for the board and administration, but I also fix printers and workstations.
My title is IT director since the state Department of Education REQUIRES that my pay grade be given a "Director" role. That's all well and good, but I don't get hung up about it.
Why? Because some days I'm the IT director and some days I'm the IT janitor. The reason I remain employed is that my attitude toward the job does not change on the days that I need to be the "janitor".
-ted
The article does not mention "backups" as in tape drives and off-site storage.
The article does mention lack of redundancy at the network carrier level.
My guess is that Northrop Grumman designed a network around single circuits connecting offices to data centers, and did not design the network to tolerate WAN link failures.
A stupid oversight for sure, but nothing that can't be easily remedied by ordering redundant WAN circuits from your telco of choice. Redundant routing gear would also be smart.
For all that are blaming government for this - they outsourced the design and implementation to a private company. That company screwed the pooch in design and implementation. Shame on both parties for not recognizing the risk of WAN failure.
-ted
So you've hit the 250GB cap, but you aren't a pirate. You pay for everything you consume - including bandwidth. Your only crime is that you went to another company for video service. You like your Apple TV and the iTunes store, or you like using a slingbox, or you like movies on demand from your Roku, or your DirecTV receiver.
All of these technologies may cause you to run over your cap, and they all have one thing in common - they directly compete with Comcast's video services.
Now Comcast appears to be using their broadband monopoly, in the form of transfer caps, to discourage the use of competing services.
If this isn't the very definition of an abusive monopoly, I don't know what is.
-ted
Mr. President, we must not allow a nuclear spaceship gap!
These idiots can't figure out how to secure the config pages of a cable modem, and we are to trust that they can implement QOS correctly? I've only been working on networks and IT stuff for a decade, so maybe I don't know what I'm talking about, but QOS seems a bit harder to do than securing a cable modem config page.
We need net neutrality for two reasons:
1. To keep the internet open to all that would want to use it.
2. To keep grossly incompetent network administrators' hands off of our data.
-ted
Well said TopSpin!
Without net neutrality laws, content providers, and network operators both have economic incentive to keep bandwidth scarce.
Network operators, thanks to monopoly or duopoly status, can keep their network capacity scarce, and still charge high prices, while deferring costly upgrades.
Content providers can lock out smaller competitors by purchasing "prioritized network capacity" at prices smaller content providers can not afford.
It's a lose-lose for consumers and the internet as a whole.
Look at the progress in industries where "capacity" was abundant - like silicon transistors in chip manufacturing. Chip designers paid little attention to transistor count knowing that future process technologies would allow their designs to be made in a cost-effective manner. This allowed the industry to produce very powerful chips at very low cost.
Similar progress will be made in network capacity and speeds to meet the demands of the network users, but this can only occur if the network treats all traffic/content equally.
-ted
During a CNBC interview, Steve Wynn stated his casino keeps 20 cents of every dollar bet.
The performance of his casino is strictly determined by the number of bets placed - not by the games played.
That tells me that EVERY game favors the house - as does blackjack including the card counters. If the games did not favor the house, they would eliminate them from the casino floor.
-ted
As a wise auditor once told me:
You can outsource the work, but you can not outsource the responsibility.
If your data is important to you - you must back it up, and you must test your backups.
The end.
-ted
We use it here to image over 200 Macs. Netboot a deploy studio image -> put image on destination drive -> reboot -> done.
It's easy to install, easy to use, and reliable.
-ted
Having worked with plenty of enterprise grade raid (EMC symetrix, clarion, and Dell SAN devices) I can say that capacity and rebuild times are not a problem for high-end arrays.
What will bring the problem to the masses are these stupid consumer NAS boxes. It is very easy to build a 4 or 8 TB array for home use using relatively cheap hardware. Unfortunately, no home user/abuser, that I know, has the skill set to manage or protect such a large array of data.
My most recent experience with a Western Digital sharespace was awful. Here is a box with a Gigabit NIC, and 4 - 2TB hard drives in a RAID 5 array that has transfer rates around 9MB/sec at best. Combine that pitiful performance with a rebuild/reformat time of over two days - and you know where this is going.
Average joes are going to put their entire lives on these things and never back them up due to the time and space cost. When a failure does occur - it will take days to perform a rebuild of the array - vastly increasing the likelyhood of another failure and permanent data loss.
Crappy RAID's days are numbered - good RAID implementations will be with us as long as hard drives have ANY failure rate at all.
-ted
They are deploying these in public locations like parks and stores. Unfortunately they are only deploying this network in their service areas, and they are not public. You need to be a cablevision subscriber to access them.
Why yes, I am a subscriber - so let me tell you about them.
If you are outside they are great - assuming there is one near you. Once you go into a building - forget it. The signal falls off a cliff, and the service is unusable.
As cool as public Wifi would be, I'm not holding my breath for it. It's the wrong technology for the application.
-ted
This is American style management at its worst - and we are exporting this management trash to other countries. Hopefully other countries will see these management clowns for what they are.
Look at every failing or failed industry in the US, and you can point to the MBA managers that sucked the life out of the company, wrote big checks to the executives and shareholders and left a carcass behind for the employees and customers.
Autos, telecoms, steel, and soon to be IT services, and pharma. These industries are being squeezed for short-term profits and dividends and the expense of the future. Crap service and bail-out nation are the baby boomers' parting gifts to us.
Hopefully the rest of the world will figure out that smart, talented engineers can also make good managers and finance guys. The future success of their companies depend on it.
Boards of companies like AT&T and GM need to kick out the Harvard MBAs and move some engineers into those positions. They may even be able to save some money on executive payroll to do it.
-ted
There aren't many banking software vendors. They are all roughly the same in terms of quality. There is no software company in the world that will give you an iron-clad security guarantee at any price.
The market has determined that this type of software is "good enough". The software is "good enough" and the cost is tolerable. Unfortunately "good enough" = sucks.
-ted
The FBI says that most Russian criminals have local accomplices. Many freelance between different crime gangs.
-ted
All of our vendors were audited by multiple independent auditing firms, had SAS70 compliance, and were also audited by federal regulators (FDIC, and OTS). It is a federal requirement by our regulators that all of our vendors go through multiple security audits multiple times per year.
Further more, our applications WERE behind a managed security service (Perimeter security services) which included a web app firewall and intrusion detection.
How exactly do you audit code that is proprietary and not viewable by the public? Every application vendor in this space, that I know of, will not let anyone outside the company view proprietary code. Federal regulators are the exception - they are allowed by law to audit the code. I am not.
How is a small organization supposed to have the resources and the man-power to audit an entire company (let alone many companies) and their products? We were in the banking business, not the software development and auditing business.
In short - fuck off - you have no idea what you are talking about.
-ted
I was the network services manager for a small community bank a couple of years ago, and all of our online banking fraud was directly related to the insecurity of the online banking application - specifically SQL injection attacks.
The application vendor's solution was to encrypt everything in the database and block known SQL injection "patterns". I told them they needed to harden their application against SQL injection; encryption and pattern matching are not enough.
Sure enough, some Russian guys (I'm guessing by the originating IP addresses) figured out that if they opened an account with a known password, they could use SQL injection to copy the encrypted known password to an account with lots of money.
Our work-around for the crappy vendor's "security" was implementing RSA tokens (outside of the banking app) on business accounts that could electronically move money out of the bank. Non-business accounts could only transfer money inside the bank - a large fraudulent transaction would get caught by a human before the money left the bank.
Before anyone suggests switching vendors, consider two things:
1. Switching banking software vendors is EXTREMELY disruptive to business. In a business where customers complain about 5 minute drive-through times, a large software migration with downtime and training is intolerable.
2. All small to medium bank software vendors suffer from similar code quality problems. Moving to another product does not necessarily guarantee quality code.
-ted
When price is all that matters, it is a race to the bottom.
That bottom (in manufacturing) may well be Africa.
China and India may have sacrificed a generation to "undercut" the west, but that sacrifice will really hurt them when producers of goods move production to Africa.
Large oil reserves are being found in Africa - it is only a matter of time until industry moves from Asia to Africa.
People only value quality after they've been burned by the lack of it.
-ted
I'm not sure about these dire predictions. Technology is constantly evolving, and new uses for technology are constantly being created.
I'm the IT director for a small private school (less than 200 students, less than 70 staff, two locations). Our small little school has VOIP to the desktop in every classroom and office, network printers damn near everywhere, 20mbps fiber at each location, and a server room with two telco racks of network equipment and three 48-space racks of servers. We have laptop carts in most classrooms, and a few, well stocked, computer labs.
We are actually stressing our 400 amp service at our main site, and any further expansion will require an electrical service upgrade.
Have I mentioned that we are a SMALL school?
10 years ago we didn't even have one third of this stuff. I don't see any reason why this trend will not continue.
Frankly, I'm thrilled at the evolution of technologies like virtualization - they will make managing all this much easier.
IT is probably one of the few areas in our economy that will actually grow in the next 30 years. Will it grow like the last 30? Maybe not, but any growth is still better than none.
-ted
My Etrade accounts have a traditional password with the requirement of an RSA token. This seems to be a great solution to the password problem.
The first part of the password is easy to remember, the second is changed every 60 seconds by the token.
It is a bit less convenient than a standard password, but that is the price to be paid to secure a bank account.
-ted
When Vista was released, we couldn't copy a few 16k files from our servers in under an hour. I have a hard time believing that Microsoft did not catch this bug during internal testing. Network file copy is a fairly important and frequently used function. This is a "showstopper" for anyone that uses a network file server, yet Microsoft shipped the product anyway.
I don't think a "showstopper" bug exists that would cause Microsoft to delay shipment of a product. Their motto appears to be "ship early and patch often".
The smart money waits for Windows 7 SP1 or SP2.
-ted
This is exactly one of the reasons why we run Trend Micro's content filtering on our Cisco ASA firewall. Works like a charm.
-ted
If Apple or any other company uses this technology against their customers, I predict many phone batteries will "catch fire". It can't be too hard to make a Lithium-Ion battery go up in flames. Hell, it seems to be hard to PREVENT them from exploding.
So instead of hair-drying a wet phone, customers will cause the battery to go nuclear and then take the smoldering mess to the store demanding a replacement.
Short of building the sensors into an airplane black-box type device (which might hurt the phone's portability a bit) it is unlikely any of these sensors would survive a fire.
I had a Motorola Q that started smoking one day while it was charging. The battery was so hot, I couldn't hold it for more than a few seconds. The case started melting, and I'm sure if I hadn't been there, it would have caught fire.
The Verizon guy replaced the phone, no hassle, no questions.
-ted
Every day I read about how the world should be: wind and solar farms generating electricity, no more fossil fuels, everyone living in cities and can walk/bike to everything they need - and no more commercial, closed software - free and open software for all.
These are all nice ideas, but they fail in the exact same way - they aren't practical for most people.
We are going to burn every drop of financially viable fossil fuels that are in the ground - the sooner engineers and environmentalists accept that fact, the sooner we can start working toward REAL solutions to our energy problems (nuclear has my vote).
A world without Microsoft office, or Microsoft products in general might be a nice vision of your utopia, but for the vast majority of computer users, they are happy shelling out the cash for a refined product that they are comfortable using.
I like free and open products whenever possible, but replacing many Microsoft products, that people are comfortable with, has enormous costs beyond mere dollars.
-ted