Indeed, transparency is a problem. Although I disagree that you need to know where or specifically how the machines are configured. What transparency you do need is how the service is performing. What is throughput? Latency? Response times? Right now, you mostly only know they are on and off - having more of an early warning system is critical to getting your redundancy strategy in place - and more quickly understanding if the problem is really in the cloud or the app.
You pay the cloud service provider to organize the service (and the software that powers it). When it fails, they are responsible for correcting it quickly and paying, or crediting, your account as it exceeds SLAs. You are ultimately responsible for your own SLAs - that means making wise choices about if you can use the cloud at all, and make sure you have an automated redundancy process. Transparency and the tools that can help have been slow to emerge.
Full disclosure, I work for web monitoring provider http://www.hyperic.com/, who is actually working with the major cloud providers to develop a new tool that provides independent transparency to the performance of these clouds - http://www.cloudstatus.com/ and will be integrating that service into the open source Hyperic HQ complete monitoring and mangagement tool we also provide. Its definitely a work in progress, but we believe we're on a strong path to be solving this problem in the near future.
Gah... for ages now I have been trying to get sourceforge to put some decent monitoring software in to get ahead of these things. I am a self-confessed net-junkie, I need my RSS feeds, my sourceforge stats, and a general trust in when I need to get to a URL - it should just be there! Without it, I the net-bends - where my blood starts to boil with internet frustration.
To see that./ has the same problem... oh my. It'll take extra coffee and likely some chocolate to console myself that the earth will be right again....
I'll make the same self-serving (shameless) offer to you as we did to sf.net - free monitoring software so you can keep us net nerds happy. http://www/hyperic.com/ - we specialize in keeping web sites running, and while we can't overcome craptastic hardware, we can help you know when your site is about to implode, and hopefully mitigate it. Some of the biggest sites on the web use it - CNET, hi5, Ask.com, Microsoft... Let us know who you are and we'll make sure you get all the support you need in the forums.
I think Javier Soltero, CEO of Hyperic who Bob Bickel, Ringside Networks CEO is a board member to summed it up nicely:
There's a lot of talk these days about social networking and SaaS 'platforms'. Companies expose API's to their services and encourage people to build integrated apps which leverage the power of a CRM (like Salesforce) or social (like OpenSocial or Facebook) platforms into their apps. The nature of this integration typically involves two applications running on separate infrastructure, talking through the Internet through some variant of REST or web services. Aside from Salesforce's Force platform, the majority of the integration between these sorts of apps is done on the outer layers of an application. An app can devote an IFrame or some other piece of screen real estate so it can display content from a platform, or perhaps use Javascript calls to embed functionality. Even in cases like Force (where there's a richer API which can be embedded at any layer of the application), the interests of the platform provider (especially those of ad-driven social networking sites) restrict the ability of a customer of the platform to more deeply integrate the content from a social app into theirs. Ringside's open source Social Application Server is meant to remove that restriction.
I agree, that's crap. Once in a million are you going to find a diamond in the rough from a university that is going to fundamentally affect your business or strategic innovation in the next 5 years. College is great to hire bench strength. Serious talent is much harder to find.
Best idea is to leverage your personal network of the employees that you already have (this of course assumes that these people are quite talented already). Next is to never poison the well with crap. Don't hire warm bodies. The damage it will do to momentum and progress is immeasurable, and seriously deteriorates your ability to recruit high quality talent.
Finally, work with a good recruiter and don't just give them a job description. Tell them where you envision this person is working now. What other projects/companies have the caliber of development you need. And then let your recruiter be the pitbull they can.
My company, http://www.hyperic.com/about/careers.html, is an open source web infrastructure monitoring and management company. Part of being open source is we have also developed a strong community. This community is familiar with our product, many of which are Java developers already - and have made their prowess known to us. We also target our community for hires as well, which can be a great transition and a very low risk hire.
That said, if there are any java or UI programmers, sales consultants, professional services or product marketing folks out there - we're always hungry for great help!
MySQL's monitoring capabilities is based on open source vendor Hyperic (http://www.hyperic.com/). Hyperic's software, Hyperic HQ, is actually the software that powers many vendor's for sale monitoring and management services - including MuleSource, JBoss (soon to be all of Red Hat as well), SpringSource, Iona and more coming. Javier Soltero, the CEO of Hyperic, has been aptly quoted saying that management is the cash register of open source. Makes sense - people want to pay for a tighter relationship with a vendor providing support - Hyperic HQ provides the platform to base those services. Money for open source vendors is generally a good thing as it keeps new releases coming out and support on the other end of a phone.
Not sure about the asshole, but he didn't do his homework for sure. This comment is uninformed or intentionally misleading to cause flamebait:
with a closed source proprietary management software tool known as Enterprise Software Monitor
A little bit of reading would have shown him that MySQL's monitoring capabilities is based on open source vendor http://www.hyperic.com/. Hyperic's software, Hyperic HQ, is actually the software that powers many vendor's for sale monitoring and management services - including MuleSource, JBoss (soon to be all of Red Hat as well), SpringSource, Iona and more coming. Making money in open source is obviously a smart idea for all - it keeps the engineers and support staff employed to keep your software being maintained and improved. Plus, it may actually pay out someday with a $1B acquisition...
Full disclosure: I work for Hyperic, and think EVERYTHING that MySQL has done to date has been a series of extremely smart moves. I also think Jeff was being lazy when he wrote that article.
Jeff Gould is slanderously off-base with his comment:
with a closed source proprietary management software tool known as Enterprise Software Monitor
A little bit of reading would have shown him that MySQL's monitoring capabilities is based on open source vendor Hyperic (http://www.hyperic.com/). Hyperic's software, Hyperic HQ, is actually the software that powers many vendor's for sale monitoring and management services - including MuleSource, JBoss (soon to be all of Red Hat as well), SpringSource, Iona and more coming.
Full disclosure: I work for Hyperic, and think EVERYTHING that MySQL has done to date has been a series of extremely smart moves. Product parterns included!;)
I think your math is flawed. Your 500 physical servers just went to 500 virtual servers. Each one has a dependency that is now harder to dissect and is more abstract from the hardware.
Sure you may have a smaller room - but you definitely have more complexity. Now you have servers moving all around, because you really can't tell what their true capacity is on the physical server - it just looks like CPU or memory is getting busy. Now you need to move it. In pops VMotion. This is very fancy ooo-aaah stuff. You start doing this once, twice, eighty times a day (obviously depending on the size of your environment) However, soon you start with what I call "VMotion Sickness".
See, Vmotions create two problems. 90% of the time they are just moving a problem resource to somewhere else, creating a never-ending game of whack-a-mole, AND at the same time it consumes a serious amount of resources across your entire virtualization environment.
What you've done is add COMPLEXITY to those 500 servers by pushing them onto 100 physical boxes. You've also magnified risk of physical hardware problems so maintenance must be much more rigorous. And you've now added a huge learning curve to your entire team to learn how to triage any problems and avoid whack-a-mole.
Full disclosure: I work for a systems management software company, http://www.hyperic.com/, that specializes in managing virtualized environments and talk to these shops every day. Also, while I am at it - one of our customers, http://www.mosso.com/, a clustered hosting provider that is a division of Rackspace built a great case study on managing a 100% virtualized environment. And for the record, they were able to keep their staff the same, which they thought was a big achievement. They had actually thought that virtualization would add so much complexity, they would have to ADD staff to maintain SLAs. The case study can be found here: http://download.hyperic.com/pdf/Hyperic-CS-Mosso.pdf
Zimbra is a great alternative to Exchange. I have also used Doodle http://www.doodle.ch/main.html to schedule/suggest meeting times that work for people through a poll mechanism, however it doesn't integrate to your calendar to remind you that you have the meeting...
...shameless plug: if you're looking for open source monitoring software either your Exchange or Zimbra installation, you should check out Hyperic. Its open source and manages both of them right out of the box.:) Stacey Schneider http://www.hyperic.com/
Are you kidding? Blaming Windows for this? Microsoft may be an evil giant in your eyes, but what about blaming the people behind the attack? Computers as I know it still take code written by humans to do things both good and evil. This is the moral equivalent of blaming the manufacturer of a knife in a stabbing case.
I am curious - have you tried http://www.hyperic.com/ at all to help manage these swells in traffic? It's exactly what its designed to do - monitor and manage web infrastructure and help manage availability/capacity. Several large hosting companies that are experts in managing changing traffic patterns use it with great success - like Contegix, Mosso (a division of Rackspace), and USi.
You are right though - some of these traffic floods would take extra millions in infrastructure to really handle, but sometimes you can turn off services and reduce the complexity of the site to give a little extra cushion that can help accommodate some of it. Otherwise... I guess its back to the sneaker-net to get your shopping done.:)
Full disclosure: I work for Hyperic, so by working with them regularly I do know a lot about how these large managed hosting providers cope with swings in traffic - and how sometimes companies just aren't willing to pay/ever prepared for what it takes to survive them gracefully.
I admit it, I am a NorCal girl. But I happen to be down in Orange County at a meeting this week right in the middle of these fires. I am staying at the St. Regis in Laguna Niguel, which for those that don't know the area - is a really fancy $500/night style hotel. It is also a refugee shelter this week. The hotel has filled up this week with people trucking in their entire wardrobes, kids, dogs and I assume some laptops to this hotel.
Now, I have been through Cambodia, Uganda, and Kenya and seen real refugees living in camps. I never expected to think of this as one. The fires are real, the entire region smells like a campfire, and I sincerely feel sorry for the people that have lost their homes. But it is very surreal to think of people retreating en masse to a $500/night spa as a fire shelter. Something about that idea just makes me realize I probably don't belong in Southern California.
Also, while I am at it... I am actually attending a meeting at the Ritz-Carleton, which sold out of the room block before we reserved our rooms, so we're at the St. Regis. We were assured that this was across the street, and no problem. What I forgot to consider is that across the street in LA means driving. The "street" is PCH (Pacific Coast Highway). There are 4 lights between the hotels, and about 2 miles. It's not the same definition of across the street here as it is where I live in San Francisco. It is lovely though, and the drive is pleasant enough. Just a surreal part of the world. I am looking forward to going home tomorrow though!
The summary of their work explored one angle of how to create efficient markets, citing that the
"... the market implements fully efficient outcomes only under very stringent (unrealistic) conditions such as perfect competition, freely available information, private goods, and the absence of any environmental effects of production and consumption.
In his earlier research on software markets, Maskin cites
"...when discoveries are "sequential" (so that each successive invention builds in an essential way on its predecessors) patent protection is not as useful for encouraging innovation as in a static setting. Indeed, society and even inventors themselves may be better off without such protection. Furthermore, an inventor's prospective profit may actually be enhanced by competition and imitation."
Open Source develops code out in the open, encouraging reuse and improvement. This is an open economy of ideas, development and invention. Customers typically test and use products and understand its value faster and more completely, shortening sales processes. Feedback, through contributions and forum discussions happen openly and in real-time, correcting and improving product direction. Competition also occurs openly, where the vendors backing projects have an open view of their competition's activities and it is really up to the vendor to push themselves to "one-up" the rest of the market. This open competition spurs faster economics, which means better products at a better price.
I think as long as vendors manage to stay away from the whole "environmental effects of production and consumption" issue, we have a pretty good argument that Open Source is software's solution to creating the perfect efficient market.
I can't resist. Its ironic that people keep crying over not enough money/boredom/underappreciation, when employers like Hyperic, a booming open source systems management vendor, consistently struggles to find good people. We pay well, we're growing like crazy, and are always hunting for good people. If you're really that good and worth more than you are getting now - please check out our jobs at http://www.hyperic.com/about/careers.html.
We can't save all of you with employment - but who knows, maybe if more of you used our software you'd have more time and less work.:)
I have heard before that models are over the hill at age 25 - but now software engineers too?
When I was 12 I programmed a lemonade store where I could sell fictional lemonade to my brothers and sister. I made about $.45 off that puppy. I think I wrote it in BASIC and it ran on a Commodore 64. Somehow, even with a modern computer, I know I wouldn't have been much more capable in present day computing to sell my lemonade stand for anything more than a dollar - and that prob would've come from my mother. Sigh.
Back to work, a little sadder that I'm 33 and all washed up.
What's interesting is where there wasn't a winner. One area, Enterprise Monitoring, the editors decided that they couldn't arrive at a conclusion for. So no winner - just officially putting HP OpenView and IBM Tivoli "on notice".
Full disclosure: I work for one of the, um, finalists/threats mentioned in the article. Hyperic http://www.hyperic.com/. That said, I know they are doing a review of our product. I guess they will announce it later... another way for the BOSSIEs to just keep on keepin on...
That old saying comes to mind: Keep your friends close, your enemies closer.
By all means, I think they got a late start, but economics and the resounding voice of communities like slashdot, bloggers, and the media should work to keep a sharp eye and the conversation going. I'm not defending them actually, I mean to encourage staying critical. (A given on this forum!). I am saying that this opposition is on a collision course to their further adoption of openness and someday even real open source.
It is optimistic. And it will take a VERY long time, just look at Sun's example. But dismissing their efforts entirely is naive in itself. Their progress should be watched, and frankly, encouraged...even if that "encouragement" is criticism pointing them towards what the market is demanding.
I've long been saying to those goliaths, like Microsoft and IBM, that it's not a battle of who will win - closed source or open source companies, but that both are on a collision course to become one, evolving and taking on the best parts of each. Open source companies and projects need some level of commercialization to fund innovation and development. Closed source companies need to open up more to be trusted and stay relevant in today's fast moving market.
I work for an open source company, Hyperic http://www.hyperic.com/, and we make systems management software. Early on Hyperic embraced the fact that there is a demand to manage Microsoft techonologies, and we built our open source software to do just that (in addition to everything else we manage) - and not with some archane NRPE remote-watered-down mechanism. Natively against Microsoft's APIs - WMI. So we work with them.
True, their open source labs with channel25 and their codeplex efforts are very much behind the rest of the company. They are relatively new compared to the rest of Microsoft, and there's a lot of ballast to turn that steamship around. But it is making some inroads, and open conversation and criticism is getting attention. These companies have to listen or become irrelevant. True, Microsoft waited a long time to accept and embrace open source, but they are not that foolish to not make efforts in today's market. And with the amount of usage of Microsoft products out in the market, it would be even more foolish of us not to pay attention.
1. Remember your password
2. Fix your printer yourself.
3. If you get the message "Critical System Updates Available", don't ignore it. Take the updates.
4. Don't get your laptop stolen.
5. Use sudo, not root.
6. If it was working yesterday, something changed. Fess up.
7. Check to make sure its plugged in.
8. RTFM
9. Don't open that.exe your nice new stranger friend sent you.
10. If its 4:55 pm, let it go. It can wait until Monday.
I work for Hyperic, which is a systems management company here at 2nd & Mission in SF. Our website is run out of that colo on 365 Main... and it was up all day yesterday, even despite the manhole cover blasting off which resulted in the mad power outage... which was witnessed by a Fruit of the Loom commercial and an apparent dead guy that had been on a gurney in the street with a security guard for a couple hours. (for more on this with pictures, check out Javier's blog from yesterday: http://www.hyperic.com/blog/hyperic/2007/07/24/hyp eric-is-where-the-action-is/
Knowing a thing or two about running data centers here, that data center definitely has serious backup and disaster recovery - they are professionals otherwise we wouldn't have picked them and neither would other serious businesses like Yelp and Technorati. I don't know for sure - but the drunken idiot theory makes a whole lot more sense given how many other sites that I know are run out of there that were unaffected, ours included.
-Stacey Schneider
Hyperic
http://www.hyperic.com/
Hate to burst the "partner" bubble... but Hyperic is a systems management software provider (and a sister open source company by our investors) and also a Systems Management Partner for XenSource and VMware, and there is no systems management for Xen yet. Its in Beta now, and our CTO whose the first person to build a comprehensive management strategy for VMware is helping define the APIs with the Xen crew. VMware is more manageable today which is an unfortunate fact. And HP doesn't have a chance of catching up with the agility of Open Source to provide real support for ages. Its just a nice shiny name to have on the partner list... but it won't actually mean anything for a long time.
Speaking from a software long history of working for software companies - we couldn't deliver squat without technologies like VWware. The majority of the companies today doing virtualization are doing it in dev/test environments. Its had a long incubation period there and is catching on to production environments. Big named analysts generally converge that nearly 25% of companies are trying virtualization for production environments - and over 50% fail. Two big sore spots are that security and systems management were not adequately addressed in the incubation period - because they were behind the firewall and not mission critical. Now they are. When trying out virtualization in production - most people confess to a "hope" strategy of deploying slowly and only feeling confident by standing the test of time. This is because they have to manage the physical host with Virtual Center and inside the guests with something else... usually somewhat homegrown. Any problem is fixed with a ooo-ahhh vmotion to a less resource constrained location. This is single handedly slowing the growth of virtualization.
Full disclosure: I work for Hyperic, a VMware partner, and we built a management strategy that brings the virtualization software into the holistic picture of performance of your servers. We wrote a white paper on the subject called "Bridging the Virtual Divide". It is a product whitepaper - but hey, Hyperic HQ is open source, so its easily available to everyone. If you want to read the paper check it out here: http://www.hyperic.com/resources/whitepapers.html
I tried this once. I worked for a startup that evolved to a big company for 10 years, slaving away at a manic pace. Then Oracle gobbled us up. I left in the wake of the acquisition gracefully. I took a year off and became a scuba instructor in the west indies. It was great... for about 6 months. Then I would start looking at my dive manager and say... your website sux, want me to redesign that? Maybe do a little SEO? And then I started in on it - reorganizing information architecture... the works. About a week into my little IT project and I realized that once a geek, always a geek. So, I came back to IT and got a real nice screen saver and desktop background.
I recommend big vacations or little sabbaticals. Its not pretty when we try to run from who we are! (pathetic)
I don't get why whurley and hinkle are whining about commercialization killing open source. Its simple economics that help everything move forward. (of course this is only true if they use the money wisely... and not on ferraris and golf... which I assure you most VCs wouldn't allow, nor would customers pay for!)
Nagios, the project that is called out in whurley's blog, is an old school open source project. It was constructed by individuals slapping together code as they needed it, with no bent for making it easy or comprehensive for the community - it served their purpose and then they contributed it. It has no way of building valuation into its product roadmap. (this explains why the first step is compiling the code?!)
You start paying people to evolve it and its better for the community as a whole as they get products that are built with the professionalism, robustness, and usefulness that they desire. Nagios is an amazing project considering how much TLC it gets from anyone managing the project. Fortunately, their longevity on the playing field has given them a deep knowledge base, so tinkerers can fiddle with it endlessly. Conversely, it means that it takes forever to get it really running and you are always tinkerin with it. You need to have that commitment to working with the code to be even remotely successful with Nagios.
Its 2007, and Open Source has evolved. Money helps provide the economic building blocks to accelerate and improve the quality and experience for the projects out there benefiting from commercial backing. Net-net - its better for the maintainers and the community.
Full disclosure: I work for Hyperic, who is in the same general market as whurley and hinkle, and Nagios. We are very passionate about open source (most of our engineers were early committers on jboss, apache - or wrote useful things like mod_perl). Javier, our CEO, blogged about this issue around the same time whurley did (not in direct response) describing the new landscape of open source with commercial backing. Check it out: http://www.hyperic.com/blog/hyperic/2007/04/06/ope n-source-1997-vs-open-source-2007/
I have the Intel Core Duo 2 Ghz. I do marketing and products here - so I have to use some Microsoft Office stuff and Adobe Creative Studio. BOTH of which do not behave well on the Intel. I found using parallels solved some of my M$FT problems, but Adobe... well,... I was told to wait for CS3 - but now I find that I will have to pay dearly for it. Upgrading memory to 2 gig seemed to help a lot. Before that - everything, including parallels, was a dog and my system hung or crashed multiple times a day. 5 minutes to open excel twice in one day sent me over the edge and to the genius bar - credit card in hand. Also, installing the Mac Tools for parallels helped a lot with performance and usability of the software. That wasn't intuitive when I first tried parallels.
Since we're a software shop, and develop systems management software - most of our developers and QA need to test/work on a variety of operating systems - most everyone here uses Bootcamp and loves it. I unfortunately bounce between Adobe and Microsoft too much and am forced to live in parallels hell. Well, maybe its more like purgatory since I got my memory upgrade! (BTW, the guys at the genius bar said I'd be in heaven if I could afford 4 gb of memory...)
Slashdot Mirror
Indeed, transparency is a problem. Although I disagree that you need to know where or specifically how the machines are configured. What transparency you do need is how the service is performing. What is throughput? Latency? Response times? Right now, you mostly only know they are on and off - having more of an early warning system is critical to getting your redundancy strategy in place - and more quickly understanding if the problem is really in the cloud or the app.
You pay the cloud service provider to organize the service (and the software that powers it). When it fails, they are responsible for correcting it quickly and paying, or crediting, your account as it exceeds SLAs. You are ultimately responsible for your own SLAs - that means making wise choices about if you can use the cloud at all, and make sure you have an automated redundancy process. Transparency and the tools that can help have been slow to emerge.
Full disclosure, I work for web monitoring provider http://www.hyperic.com/, who is actually working with the major cloud providers to develop a new tool that provides independent transparency to the performance of these clouds - http://www.cloudstatus.com/ and will be integrating that service into the open source Hyperic HQ complete monitoring and mangagement tool we also provide. Its definitely a work in progress, but we believe we're on a strong path to be solving this problem in the near future.
Gah... for ages now I have been trying to get sourceforge to put some decent monitoring software in to get ahead of these things. I am a self-confessed net-junkie, I need my RSS feeds, my sourceforge stats, and a general trust in when I need to get to a URL - it should just be there! Without it, I the net-bends - where my blood starts to boil with internet frustration.
./ has the same problem... oh my. It'll take extra coffee and likely some chocolate to console myself that the earth will be right again....
To see that
I'll make the same self-serving (shameless) offer to you as we did to sf.net - free monitoring software so you can keep us net nerds happy. http://www/hyperic.com/ - we specialize in keeping web sites running, and while we can't overcome craptastic hardware, we can help you know when your site is about to implode, and hopefully mitigate it. Some of the biggest sites on the web use it - CNET, hi5, Ask.com, Microsoft... Let us know who you are and we'll make sure you get all the support you need in the forums.
In the meantime... keep on truckin'.
I agree, that's crap. Once in a million are you going to find a diamond in the rough from a university that is going to fundamentally affect your business or strategic innovation in the next 5 years. College is great to hire bench strength. Serious talent is much harder to find.
Best idea is to leverage your personal network of the employees that you already have (this of course assumes that these people are quite talented already). Next is to never poison the well with crap. Don't hire warm bodies. The damage it will do to momentum and progress is immeasurable, and seriously deteriorates your ability to recruit high quality talent.
Finally, work with a good recruiter and don't just give them a job description. Tell them where you envision this person is working now. What other projects/companies have the caliber of development you need. And then let your recruiter be the pitbull they can.
My company, http://www.hyperic.com/about/careers.html, is an open source web infrastructure monitoring and management company. Part of being open source is we have also developed a strong community. This community is familiar with our product, many of which are Java developers already - and have made their prowess known to us. We also target our community for hires as well, which can be a great transition and a very low risk hire.
That said, if there are any java or UI programmers, sales consultants, professional services or product marketing folks out there - we're always hungry for great help!
MySQL's monitoring capabilities is based on open source vendor Hyperic (http://www.hyperic.com/). Hyperic's software, Hyperic HQ, is actually the software that powers many vendor's for sale monitoring and management services - including MuleSource, JBoss (soon to be all of Red Hat as well), SpringSource, Iona and more coming. Javier Soltero, the CEO of Hyperic, has been aptly quoted saying that management is the cash register of open source. Makes sense - people want to pay for a tighter relationship with a vendor providing support - Hyperic HQ provides the platform to base those services. Money for open source vendors is generally a good thing as it keeps new releases coming out and support on the other end of a phone.
A little bit of reading would have shown him that MySQL's monitoring capabilities is based on open source vendor http://www.hyperic.com/. Hyperic's software, Hyperic HQ, is actually the software that powers many vendor's for sale monitoring and management services - including MuleSource, JBoss (soon to be all of Red Hat as well), SpringSource, Iona and more coming. Making money in open source is obviously a smart idea for all - it keeps the engineers and support staff employed to keep your software being maintained and improved. Plus, it may actually pay out someday with a $1B acquisition...
Full disclosure: I work for Hyperic, and think EVERYTHING that MySQL has done to date has been a series of extremely smart moves. I also think Jeff was being lazy when he wrote that article.
A little bit of reading would have shown him that MySQL's monitoring capabilities is based on open source vendor Hyperic (http://www.hyperic.com/). Hyperic's software, Hyperic HQ, is actually the software that powers many vendor's for sale monitoring and management services - including MuleSource, JBoss (soon to be all of Red Hat as well), SpringSource, Iona and more coming.
Full disclosure: I work for Hyperic, and think EVERYTHING that MySQL has done to date has been a series of extremely smart moves. Product parterns included!
I think your math is flawed. Your 500 physical servers just went to 500 virtual servers. Each one has a dependency that is now harder to dissect and is more abstract from the hardware.
Sure you may have a smaller room - but you definitely have more complexity. Now you have servers moving all around, because you really can't tell what their true capacity is on the physical server - it just looks like CPU or memory is getting busy. Now you need to move it. In pops VMotion. This is very fancy ooo-aaah stuff. You start doing this once, twice, eighty times a day (obviously depending on the size of your environment) However, soon you start with what I call "VMotion Sickness".
See, Vmotions create two problems. 90% of the time they are just moving a problem resource to somewhere else, creating a never-ending game of whack-a-mole, AND at the same time it consumes a serious amount of resources across your entire virtualization environment.
What you've done is add COMPLEXITY to those 500 servers by pushing them onto 100 physical boxes. You've also magnified risk of physical hardware problems so maintenance must be much more rigorous. And you've now added a huge learning curve to your entire team to learn how to triage any problems and avoid whack-a-mole.
Full disclosure: I work for a systems management software company, http://www.hyperic.com/, that specializes in managing virtualized environments and talk to these shops every day. Also, while I am at it - one of our customers, http://www.mosso.com/, a clustered hosting provider that is a division of Rackspace built a great case study on managing a 100% virtualized environment. And for the record, they were able to keep their staff the same, which they thought was a big achievement. They had actually thought that virtualization would add so much complexity, they would have to ADD staff to maintain SLAs. The case study can be found here: http://download.hyperic.com/pdf/Hyperic-CS-Mosso.pdf
Zimbra is a great alternative to Exchange. I have also used Doodle http://www.doodle.ch/main.html to schedule/suggest meeting times that work for people through a poll mechanism, however it doesn't integrate to your calendar to remind you that you have the meeting...
:)
...shameless plug: if you're looking for open source monitoring software either your Exchange or Zimbra installation, you should check out Hyperic. Its open source and manages both of them right out of the box.
Stacey Schneider
http://www.hyperic.com/
Are you kidding? Blaming Windows for this? Microsoft may be an evil giant in your eyes, but what about blaming the people behind the attack? Computers as I know it still take code written by humans to do things both good and evil. This is the moral equivalent of blaming the manufacturer of a knife in a stabbing case.
Operating systems are the least of your issues.
I am curious - have you tried http://www.hyperic.com/ at all to help manage these swells in traffic? It's exactly what its designed to do - monitor and manage web infrastructure and help manage availability/capacity. Several large hosting companies that are experts in managing changing traffic patterns use it with great success - like Contegix, Mosso (a division of Rackspace), and USi. You are right though - some of these traffic floods would take extra millions in infrastructure to really handle, but sometimes you can turn off services and reduce the complexity of the site to give a little extra cushion that can help accommodate some of it. Otherwise... I guess its back to the sneaker-net to get your shopping done. :)
Full disclosure: I work for Hyperic, so by working with them regularly I do know a lot about how these large managed hosting providers cope with swings in traffic - and how sometimes companies just aren't willing to pay/ever prepared for what it takes to survive them gracefully.
I admit it, I am a NorCal girl. But I happen to be down in Orange County at a meeting this week right in the middle of these fires. I am staying at the St. Regis in Laguna Niguel, which for those that don't know the area - is a really fancy $500/night style hotel. It is also a refugee shelter this week. The hotel has filled up this week with people trucking in their entire wardrobes, kids, dogs and I assume some laptops to this hotel.
Now, I have been through Cambodia, Uganda, and Kenya and seen real refugees living in camps. I never expected to think of this as one. The fires are real, the entire region smells like a campfire, and I sincerely feel sorry for the people that have lost their homes. But it is very surreal to think of people retreating en masse to a $500/night spa as a fire shelter. Something about that idea just makes me realize I probably don't belong in Southern California.
Also, while I am at it... I am actually attending a meeting at the Ritz-Carleton, which sold out of the room block before we reserved our rooms, so we're at the St. Regis. We were assured that this was across the street, and no problem. What I forgot to consider is that across the street in LA means driving. The "street" is PCH (Pacific Coast Highway). There are 4 lights between the hotels, and about 2 miles. It's not the same definition of across the street here as it is where I live in San Francisco. It is lovely though, and the drive is pleasant enough. Just a surreal part of the world. I am looking forward to going home tomorrow though!
-Stacey Schneider http//www.hyperic.com/
In his earlier research on software markets, Maskin cites
Open Source develops code out in the open, encouraging reuse and improvement. This is an open economy of ideas, development and invention. Customers typically test and use products and understand its value faster and more completely, shortening sales processes. Feedback, through contributions and forum discussions happen openly and in real-time, correcting and improving product direction. Competition also occurs openly, where the vendors backing projects have an open view of their competition's activities and it is really up to the vendor to push themselves to "one-up" the rest of the market. This open competition spurs faster economics, which means better products at a better price.
I think as long as vendors manage to stay away from the whole "environmental effects of production and consumption" issue, we have a pretty good argument that Open Source is software's solution to creating the perfect efficient market.
Full disclosure: I robbed the majority of this posting from a blog post I did on the same subject earlier today, and I work in Open Source. For the whole enchilada of the previous blog post see: http://www.hyperic.com/blog/hyperic/2007/10/16/nobel-prize-implies-open-source-is-a-perfect-market/
Stacey Schneider
http://www.hyperic.com/
I can't resist. Its ironic that people keep crying over not enough money/boredom/underappreciation, when employers like Hyperic, a booming open source systems management vendor, consistently struggles to find good people. We pay well, we're growing like crazy, and are always hunting for good people. If you're really that good and worth more than you are getting now - please check out our jobs at http://www.hyperic.com/about/careers.html.
:)
We can't save all of you with employment - but who knows, maybe if more of you used our software you'd have more time and less work.
I have heard before that models are over the hill at age 25 - but now software engineers too?
When I was 12 I programmed a lemonade store where I could sell fictional lemonade to my brothers and sister. I made about $.45 off that puppy. I think I wrote it in BASIC and it ran on a Commodore 64. Somehow, even with a modern computer, I know I wouldn't have been much more capable in present day computing to sell my lemonade stand for anything more than a dollar - and that prob would've come from my mother. Sigh.
Back to work, a little sadder that I'm 33 and all washed up.
What's interesting is where there wasn't a winner. One area, Enterprise Monitoring, the editors decided that they couldn't arrive at a conclusion for. So no winner - just officially putting HP OpenView and IBM Tivoli "on notice".
Check out the article: http://www.infoworld.com/infoworld/article/07/09/10/37FE-boss-enterprise-monitoring_1.html
Full disclosure: I work for one of the, um, finalists/threats mentioned in the article. Hyperic http://www.hyperic.com/. That said, I know they are doing a review of our product. I guess they will announce it later... another way for the BOSSIEs to just keep on keepin on...
That old saying comes to mind: Keep your friends close, your enemies closer.
By all means, I think they got a late start, but economics and the resounding voice of communities like slashdot, bloggers, and the media should work to keep a sharp eye and the conversation going. I'm not defending them actually, I mean to encourage staying critical. (A given on this forum!). I am saying that this opposition is on a collision course to their further adoption of openness and someday even real open source.
It is optimistic. And it will take a VERY long time, just look at Sun's example. But dismissing their efforts entirely is naive in itself. Their progress should be watched, and frankly, encouraged...even if that "encouragement" is criticism pointing them towards what the market is demanding.
I've long been saying to those goliaths, like Microsoft and IBM, that it's not a battle of who will win - closed source or open source companies, but that both are on a collision course to become one, evolving and taking on the best parts of each. Open source companies and projects need some level of commercialization to fund innovation and development. Closed source companies need to open up more to be trusted and stay relevant in today's fast moving market.
I work for an open source company, Hyperic http://www.hyperic.com/, and we make systems management software. Early on Hyperic embraced the fact that there is a demand to manage Microsoft techonologies, and we built our open source software to do just that (in addition to everything else we manage) - and not with some archane NRPE remote-watered-down mechanism. Natively against Microsoft's APIs - WMI. So we work with them.
True, their open source labs with channel25 and their codeplex efforts are very much behind the rest of the company. They are relatively new compared to the rest of Microsoft, and there's a lot of ballast to turn that steamship around. But it is making some inroads, and open conversation and criticism is getting attention. These companies have to listen or become irrelevant. True, Microsoft waited a long time to accept and embrace open source, but they are not that foolish to not make efforts in today's market. And with the amount of usage of Microsoft products out in the market, it would be even more foolish of us not to pay attention.
1. Remember your password .exe your nice new stranger friend sent you.
p py-national-sys-admin-appreciation-day/
./!
2. Fix your printer yourself.
3. If you get the message "Critical System Updates Available", don't ignore it. Take the updates.
4. Don't get your laptop stolen.
5. Use sudo, not root.
6. If it was working yesterday, something changed. Fess up.
7. Check to make sure its plugged in.
8. RTFM
9. Don't open that
10. If its 4:55 pm, let it go. It can wait until Monday.
Full disclosure - I work for Hyperic, http://www.hyperic.com/, and submitted this story which got beat by the one you are now reading... it was in a blog post Javier Soltero made this morning: http://www.hyperic.com/blog/hyperic/2007/07/27/ha
Just a fun conversation about all the stupid things admins have to put up with from their users. I know there's more out there!!! Bring it on
I work for Hyperic, which is a systems management company here at 2nd & Mission in SF. Our website is run out of that colo on 365 Main... and it was up all day yesterday, even despite the manhole cover blasting off which resulted in the mad power outage... which was witnessed by a Fruit of the Loom commercial and an apparent dead guy that had been on a gurney in the street with a security guard for a couple hours. (for more on this with pictures, check out Javier's blog from yesterday: http://www.hyperic.com/blog/hyperic/2007/07/24/hyp eric-is-where-the-action-is/
Knowing a thing or two about running data centers here, that data center definitely has serious backup and disaster recovery - they are professionals otherwise we wouldn't have picked them and neither would other serious businesses like Yelp and Technorati. I don't know for sure - but the drunken idiot theory makes a whole lot more sense given how many other sites that I know are run out of there that were unaffected, ours included.
-Stacey Schneider
Hyperic
http://www.hyperic.com/
Hate to burst the "partner" bubble... but Hyperic is a systems management software provider (and a sister open source company by our investors) and also a Systems Management Partner for XenSource and VMware, and there is no systems management for Xen yet. Its in Beta now, and our CTO whose the first person to build a comprehensive management strategy for VMware is helping define the APIs with the Xen crew. VMware is more manageable today which is an unfortunate fact. And HP doesn't have a chance of catching up with the agility of Open Source to provide real support for ages. Its just a nice shiny name to have on the partner list... but it won't actually mean anything for a long time.
In case it wasn't obvious, full disclosure - I work for Hyperic. http://www.hyperic.com/
Speaking from a software long history of working for software companies - we couldn't deliver squat without technologies like VWware. The majority of the companies today doing virtualization are doing it in dev/test environments. Its had a long incubation period there and is catching on to production environments. Big named analysts generally converge that nearly 25% of companies are trying virtualization for production environments - and over 50% fail. Two big sore spots are that security and systems management were not adequately addressed in the incubation period - because they were behind the firewall and not mission critical. Now they are. When trying out virtualization in production - most people confess to a "hope" strategy of deploying slowly and only feeling confident by standing the test of time. This is because they have to manage the physical host with Virtual Center and inside the guests with something else... usually somewhat homegrown. Any problem is fixed with a ooo-ahhh vmotion to a less resource constrained location. This is single handedly slowing the growth of virtualization.
Full disclosure: I work for Hyperic, a VMware partner, and we built a management strategy that brings the virtualization software into the holistic picture of performance of your servers. We wrote a white paper on the subject called "Bridging the Virtual Divide". It is a product whitepaper - but hey, Hyperic HQ is open source, so its easily available to everyone. If you want to read the paper check it out here: http://www.hyperic.com/resources/whitepapers.html
I tried this once. I worked for a startup that evolved to a big company for 10 years, slaving away at a manic pace. Then Oracle gobbled us up. I left in the wake of the acquisition gracefully. I took a year off and became a scuba instructor in the west indies. It was great... for about 6 months. Then I would start looking at my dive manager and say... your website sux, want me to redesign that? Maybe do a little SEO? And then I started in on it - reorganizing information architecture... the works. About a week into my little IT project and I realized that once a geek, always a geek. So, I came back to IT and got a real nice screen saver and desktop background.
I recommend big vacations or little sabbaticals. Its not pretty when we try to run from who we are! (pathetic)
I don't get why whurley and hinkle are whining about commercialization killing open source. Its simple economics that help everything move forward. (of course this is only true if they use the money wisely... and not on ferraris and golf... which I assure you most VCs wouldn't allow, nor would customers pay for!)
e n-source-1997-vs-open-source-2007/
Nagios, the project that is called out in whurley's blog, is an old school open source project. It was constructed by individuals slapping together code as they needed it, with no bent for making it easy or comprehensive for the community - it served their purpose and then they contributed it. It has no way of building valuation into its product roadmap. (this explains why the first step is compiling the code?!)
You start paying people to evolve it and its better for the community as a whole as they get products that are built with the professionalism, robustness, and usefulness that they desire. Nagios is an amazing project considering how much TLC it gets from anyone managing the project. Fortunately, their longevity on the playing field has given them a deep knowledge base, so tinkerers can fiddle with it endlessly. Conversely, it means that it takes forever to get it really running and you are always tinkerin with it. You need to have that commitment to working with the code to be even remotely successful with Nagios.
Its 2007, and Open Source has evolved. Money helps provide the economic building blocks to accelerate and improve the quality and experience for the projects out there benefiting from commercial backing. Net-net - its better for the maintainers and the community.
Full disclosure: I work for Hyperic, who is in the same general market as whurley and hinkle, and Nagios. We are very passionate about open source (most of our engineers were early committers on jboss, apache - or wrote useful things like mod_perl). Javier, our CEO, blogged about this issue around the same time whurley did (not in direct response) describing the new landscape of open source with commercial backing. Check it out: http://www.hyperic.com/blog/hyperic/2007/04/06/op
I have the Intel Core Duo 2 Ghz. I do marketing and products here - so I have to use some Microsoft Office stuff and Adobe Creative Studio. BOTH of which do not behave well on the Intel. I found using parallels solved some of my M$FT problems, but Adobe... well,... I was told to wait for CS3 - but now I find that I will have to pay dearly for it. Upgrading memory to 2 gig seemed to help a lot. Before that - everything, including parallels, was a dog and my system hung or crashed multiple times a day. 5 minutes to open excel twice in one day sent me over the edge and to the genius bar - credit card in hand. Also, installing the Mac Tools for parallels helped a lot with performance and usability of the software. That wasn't intuitive when I first tried parallels.
Since we're a software shop, and develop systems management software - most of our developers and QA need to test/work on a variety of operating systems - most everyone here uses Bootcamp and loves it. I unfortunately bounce between Adobe and Microsoft too much and am forced to live in parallels hell. Well, maybe its more like purgatory since I got my memory upgrade! (BTW, the guys at the genius bar said I'd be in heaven if I could afford 4 gb of memory...)
-Stacey
http://www.hyperic.com/