There are a lot of things you can do. If Googlebot is using too much bandwidth, you could easily (man tc) add an outbound QoS limit to your webservers.
The even better solution is one I did a long time ago.
All users were using sign-on, and then would sign-on to the squid proxy when they opened the browser. The firewall blocked most outbound traffic from workstations directly, and would only allow web traffic from the authenticated squid server.
This way we took care of most of the surfing problems simply by saying. "Everything you do is logged, and watched."
Of course there are tricks to get around this, but it takes care of most users. And the tricks are fairly obvious to spot.
I did a benchmark comparison of the HP P400 in a DL185 vs a simple LSI 1068E SAS jbod controller. (HP branded of course) HP said it wouldn't work with the 1068E in the 12x DL185 expander setup, but of course it only took me about 10min of looking at the LSI website to find the Initiator Target firmware and get it fixed.
Basically my benchmark showed that in all cases the JBOD setup with linux software raid was 10-50% faster than the P400 controller.
They didn't ship a BBU for the P400 so it wouldn't do any write caching in the RAID5 test. The write speed was 2% the speed of linux software RAID, so I basically ignored that test.
Eventually I gave up on the DL158 and bought a Supermicro based box (SC836 chassis, great box) for a bit less, and a lot less HP bullshit.
And if you are a big enough customer you'll likely have fiber to one of the various pops out there, and you can just buy a cross-connect directly to google's peering network. I don't know if anyone that size has yet to sign up for hosted gmail. I know a couple of schools have rolled it out.
Yea, openfire is huge, but it's probably the most complete open source jabber server out there. The enterprise plugin is also really useful for the corp admin that needs to do evil things like force all messages to be secured to the server, and then log them. The usage stats and clustering stuff is really good. The only thing missing that I want is multi-domain support.. it's on their top-of-the-roadmap feature list.
Goog apps for your domain works really well, and since it supports jabber clients/federation, it's really flexible about what you connect. There are a few enterprise features like "warn user if contact is not on your domain" and forced encryption.
I use openfire for my personal jabber server, it's been reliable, and keeps getting good updates.
I haven't used the spark client, and I haven't had good luck with the web client. That's probably the biggest thing I wish I could find was a good web client like gmail chat.
I'm in the process of building a new 8x 1T array. I'm not using any fancy raid card. Just a LSI 1068E chipset with a SAS expander to handle LOTS (16 slots in the case, using 8 right now).
I'm not putting the entire thing into one big array. I'm breaking up each 1T drive into ~100GB slices that I will use to build several overlapping arrays. Each MD device will be no more than 4-5 slices. This way if an error occurs on one disk in one part of a disk I will have a higher probability of recovery.
I may also use RAID 6 to give me more chance of rebuilding.
Disk errors tend to not be whole disk errors, just small broken physical parts of a single disk.
SMART will give me more chance to detect and replace dying drives.
Exactly, also my thought is this.. If you're working super hard to get really good PUE numbers, are you really going to stop there and not think about the system itself? Getting low PUE is mostly about getting the most compute out of your watt.
People that are working hard at PUE are most likely already thinking about the watts/{flop,io,etc}.
Thats's one of the specific things they had Lotus change about the Elise design. They lowered the door sills by several inches. They also had to beef things up a bit to handle the 900 pound battery pack.
I've worked with several "Open Source" software systems that were also for-pay. The place I worked for paid a support contract with the company (http://www.clusterresources.com/) so we could get several things:
1: direct phone support with the developers 2: voting rights for "next release" features 3: custom modules for our needs
Unfortunately, most religious groups also include the social problem of "one of US, or one of THEM" Where THEM can be damned, persecuted, or flat out killed.
We need to remove the "join or die" aspect from the major theologies.
But all machines do that anyway. Ram runs at 1.5V or 1.8V, the CPU runs at 1.2ish these days. Where does that come from? 3.3V or 5V rails..
This is why people are moving everything to the 12V rail on the PSU (ATX12V standard and other ideas) A single efficient conversion with a local on-board conversion is best.
Yea, I don't know who wrote that bit in the article, but they're just dumb. If you run any kind of system with a load balancer in front of it you can easily script starting up additional machines as soon as your monitoring says you reach 90% capacity.
As a year-round bicycle commuter, I completely agree. There are assholes everywhere you look. They're on bikes and in cars. No matter what we do there will always be assholes and accidents.
Thankfully part of my commute is on a walking/cycling path that is not part of any major road, and most of the roads I'm on have bike lanes.
I want the wires brought to my house to be maintained by the _local_ government. They don't have to provide the services on those wires, that can be done by any company. We already have government granted local monopolies like Comcast, and AT&T. I think a local city government would provide a better quality of service.
So the thing is, there is more than just a simple "eeprom write interface" on these chips.
Most of the time the the eeprom attached to the nic is a cheap small serial eeprom part, usually just a few kb.. maybe 32 or 64kb. It contains mostly things like a bit of boot strapping, a few "permanent" settings like the MAC address, and the PXE rom.
And that's where the problems come in. This serial interface is usually an afterthought, and if there is noise on that bus, bits can flip. Or if something bad happens in the NIC code, you could accidentally write when you meant to read.
Usually this is recoverable, but I haven't looked into this specific corruption situation. I've had to deal with this kind of thing before. It's not fun.
Flashing NIC eeproms isn't something a normal end-user does all the time. 99% of the time it's written at the factory, stuffed on the board, and forgotten about.
I agree. Communication technology becoming common to every day living. It's about time we turn it into something that governments service like the sewers, plumbing, and roads
Slashdot Mirror
Yup, a lot of work is put into reducing the energy cost per computer time.
See this site:
http://www.google.com/corporate/datacenters/
I suppose you would rather be simply called a "Team Member"
http://sites.target.com/site/en/corporate/page.jsp?contentId=PRD03-000521
At some places, there is a bit of honor in the title. They stopped giving out the title "Crayon" when SGI bought the company.
There are a lot of things you can do. If Googlebot is using too much bandwidth, you could easily (man tc) add an outbound QoS limit to your webservers.
http://www.google.com/search?q=googlebot+IP+range
If you're unable to do this, there is the GoogleBot webmaster tools that let you manage your hit rate.
http://www.google.com/support/webmasters
The even better solution is one I did a long time ago.
All users were using sign-on, and then would sign-on to the squid proxy when they opened the browser. The firewall blocked most outbound traffic from workstations directly, and would only allow web traffic from the authenticated squid server.
This way we took care of most of the surfing problems simply by saying. "Everything you do is logged, and watched."
Of course there are tricks to get around this, but it takes care of most users. And the tricks are fairly obvious to spot.
LOL, I was considering buying space in a HE building.. I'm now _really_ glad I am putting my stuff in layer42.
I did a benchmark comparison of the HP P400 in a DL185 vs a simple LSI 1068E SAS jbod controller. (HP branded of course) HP said it wouldn't work with the 1068E in the 12x DL185 expander setup, but of course it only took me about 10min of looking at the LSI website to find the Initiator Target firmware and get it fixed.
Basically my benchmark showed that in all cases the JBOD setup with linux software raid was 10-50% faster than the P400 controller.
They didn't ship a BBU for the P400 so it wouldn't do any write caching in the RAID5 test. The write speed was 2% the speed of linux software RAID, so I basically ignored that test.
Eventually I gave up on the DL158 and bought a Supermicro based box (SC836 chassis, great box) for a bit less, and a lot less HP bullshit.
Yea, I recently built an 8T server. It cost me about $5000, and has no raid controller, and uses linux software raid.
If I really wanted, I could buy a second $5000 server and do DRBD between them to have 2x redundancy.
And if you are a big enough customer you'll likely have fiber to one of the various pops out there, and you can just buy a cross-connect directly to google's peering network. I don't know if anyone that size has yet to sign up for hosted gmail. I know a couple of schools have rolled it out.
Yea, openfire is huge, but it's probably the most complete open source jabber server out there. The enterprise plugin is also really useful for the corp admin that needs to do evil things like force all messages to be secured to the server, and then log them. The usage stats and clustering stuff is really good. The only thing missing that I want is multi-domain support.. it's on their top-of-the-roadmap feature list.
Goog apps for your domain works really well, and since it supports jabber clients/federation, it's really flexible about what you connect. There are a few enterprise features like "warn user if contact is not on your domain" and forced encryption.
I also love openfire, I tuned the java memory usage down a bit, but I guess I don't have enough users to see if it's slow or not.
How many users and what hardware are you using?
It supports clustering, so I guess you can always scale it that way.
I use openfire for my personal jabber server, it's been reliable, and keeps getting good updates.
I haven't used the spark client, and I haven't had good luck with the web client. That's probably the biggest thing I wish I could find was a good web client like gmail chat.
I'm in the process of building a new 8x 1T array. I'm not using any fancy raid card. Just a LSI 1068E chipset with a SAS expander to handle LOTS (16 slots in the case, using 8 right now).
I'm not putting the entire thing into one big array. I'm breaking up each 1T drive into ~100GB slices that I will use to build several overlapping arrays. Each MD device will be no more than 4-5 slices. This way if an error occurs on one disk in one part of a disk I will have a higher probability of recovery.
I may also use RAID 6 to give me more chance of rebuilding.
Disk errors tend to not be whole disk errors, just small broken physical parts of a single disk.
SMART will give me more chance to detect and replace dying drives.
Exactly, also my thought is this.. If you're working super hard to get really good PUE numbers, are you really going to stop there and not think about the system itself? Getting low PUE is mostly about getting the most compute out of your watt.
People that are working hard at PUE are most likely already thinking about the watts/{flop,io,etc}.
Thats's one of the specific things they had Lotus change about the Elise design. They lowered the door sills by several inches. They also had to beef things up a bit to handle the 900 pound battery pack.
I've worked with several "Open Source" software systems that were also for-pay. The place I worked for paid a support contract with the company (http://www.clusterresources.com/) so we could get several things:
1: direct phone support with the developers
2: voting rights for "next release" features
3: custom modules for our needs
Thankfully I work for a really good manager who listens, and then takes action, or gives me good advice not only on projects but on my career.
Oh wait, I work for a company that has a good managers overall, and is very open source friendly.
Not every place has to suck to work for.
And it's not even really an Elise. It has similar styling, but major changes to the chassis have been made to the design.
Unfortunately, most religious groups also include the social problem of "one of US, or one of THEM" Where THEM can be damned, persecuted, or flat out killed.
We need to remove the "join or die" aspect from the major theologies.
But all machines do that anyway. Ram runs at 1.5V or 1.8V, the CPU runs at 1.2ish these days. Where does that come from? 3.3V or 5V rails..
This is why people are moving everything to the 12V rail on the PSU (ATX12V standard and other ideas) A single efficient conversion with a local on-board conversion is best.
DC power still has a lot of other issues.
Yea, I don't know who wrote that bit in the article, but they're just dumb. If you run any kind of system with a load balancer in front of it you can easily script starting up additional machines as soon as your monitoring says you reach 90% capacity.
As a year-round bicycle commuter, I completely agree. There are assholes everywhere you look. They're on bikes and in cars. No matter what we do there will always be assholes and accidents.
Thankfully part of my commute is on a walking/cycling path that is not part of any major road, and most of the roads I'm on have bike lanes.
I want the wires brought to my house to be maintained by the _local_ government. They don't have to provide the services on those wires, that can be done by any company. We already have government granted local monopolies like Comcast, and AT&T. I think a local city government would provide a better quality of service.
So the thing is, there is more than just a simple "eeprom write interface" on these chips.
Most of the time the the eeprom attached to the nic is a cheap small serial eeprom part, usually just a few kb.. maybe 32 or 64kb. It contains mostly things like a bit of boot strapping, a few "permanent" settings like the MAC address, and the PXE rom.
And that's where the problems come in. This serial interface is usually an afterthought, and if there is noise on that bus, bits can flip. Or if something bad happens in the NIC code, you could accidentally write when you meant to read.
Usually this is recoverable, but I haven't looked into this specific corruption situation. I've had to deal with this kind of thing before. It's not fun.
Flashing NIC eeproms isn't something a normal end-user does all the time. 99% of the time it's written at the factory, stuffed on the board, and forgotten about.
I agree. Communication technology becoming common to every day living. It's about time we turn it into something that governments service like the sewers, plumbing, and roads