Anyone with any brain knows that in order to determine with any degree of certainty you need to test under real-world conditions. Real-world conditions include things like identifying beta users in several departments of a large company and allowing them to run beta software in order to test for the masses. Testing under virtualization is just smoke testing - verifying that your users don't have any serious difficulties with the new OS. Vista has a "burn in time" - from initial installation to some level of full capability. The file indexing database, for example, is a serious piece of technology which requires days upon days of use in order to reach a steady state. As operating systems become more complex and include more heuristic features, testers who take the attitude of just virtualizing and following a script will find themselves with more of a support headache than testers who opted to not only script test, but also put the beta product in real-world situations and find any issues.
Real-world testing uncovers many more issues (and more complex issues) than virtual script testing can ever hope to. This is, in fact, one reason that Microsoft chooses to release betas to the public.
In testing a beta product, a certain amount of resource overhead should be calculated in order to effectively allow for testing. For something as wide-ranging as an OS release, 10% of a beta user's time would need to be allocated to work with tech teams in diagnosing, reporting, and working around problems that are uncovered. Once the beta period is complete, I expect users to be able to upgraded to production with a minimal hassle. During SP1, we had users that were down for more than a day - not to mention loss of productivity post re-installation due to the OS's burn-in time.
I won't say that this approach doesn't have risk. Of course the action of placing real-world users on a beta product has profound implications especially with respect to an OS product.
Some ways to reduce risks associated with users running beta software under real-world conditions are:
- Smoke-testing in virtualized systems prior to even allowing the beta to continue. Pay attention to "show stoppers" AND their potential aggregate effects. Don't ignore issues because you think you have a work-around.
- Build contingencies for disastrous events (such as backing up or replicating offside more aggressively than for typical users).
- Have a reversion plan tested and in-place prior to beta go-live.
... and here's where we went wrong with SP1: Don't believe the vendor when they tell you there will be an upgrade path from SP1 Beta to SP1.
I, for one, will NOT be installing any beta service packs from Microsoft and I'll be recommending my company do the same. If you were unfortunate to test the SP1 beta, you'll recall that you were forced to re-front your machines after the beta period was over. Until Microsoft guarantees that it will provide a reasonable upgrade path from the beta to production, there is no point in testing until this becomes public.
We're all excited that someone has figured out how to "more easily" replicate a user interface element that has been a standard part of user interfaces since 1995?
Replace "Internet" with "Mainframe"
Replace "Browser" with "3270 Green Screen"
Replace "HTML/DHTML/XHTML/CSS/etc..." with "Terminal emulation protocol"
I'm currently typing this message into a box that covers about 1/10th of my total screen real estate and a scrollbar just appeared. I have to type 7 extra keystrokes to emphasize bold text. When are we going to grow up and stop reinventing complexity and start building systems for user interfaces that are truly useful??
I'm off to preview this post to make sure it looks right before I press the submit button./blech
Terry
I've always thought that the best way to deal with websites that have advertisements (ANY ads) is to *click* on the ad links...
Ok - before I get flamed, let me explain. The companies that are paying for ad space usually also pay a "per click" fee. Every time someone clicks on the ad, the advertiser pays the host site.
Some bright fool should write a web accelerator-type program that follows every link on the currently browsed site to at least one click deep. This should be done silently and in the background.
What it means is that clickthrough revenues for sites with ads would go through the roof, but no one would actually be reading and/or responding to the ads. The companies that are advertising would pay off like a slot machine and eventually go out of business because they would suddenly beleive that their internet advertising department is full of geniuses.
The central problem is that the advertisers are using the wrong metric to see if their ad was successful.
I'd buy the product for a $1.
Ravepunk
He's the title designer for "Da Ali G Show" for chrissake. I do not believe that the delicate sarchasm in Adams' work can be trusted to a music video director who designed the titles for a reasonably good TV show.
In both the radio show and the BBC TV Series, what made the jokes work was the voice characterization and acting. Without a good director at the helm who has a letter perfect sense of comic timing and voice characterization necessary to pull off the sarchasm, this movie will fail. My vote would have been to use a Disney cartoon director taking on live action for the first time if they wanted to save money.
Of course, I was the one that poopoo'ed the idea of Peter Jackson doing LOTR, so who the Hell am I to comment?
- ravepunk:/
Re:Anybody else want to see a night time picture?
on
Brine on Mars?
·
· Score: 1
I used to work for a company that wrote compilers. The runtime p-code intepreter for a particular product was hand-coded assembly with pretty dense comments and the main developer was known for... uh, smoking on the job let's say...
I remeber this one section of code where the comments just ended. 7 or 8 screens of assembly later was the following:
SOSXI: RET ; oops.
... then the comments started up again. It took me and some fellow developers about 3 months to figure out that code.
You want to obsfucate code better? Get high and write it.
:/ Ravepunk
Slashdot Mirror
Anyone with any brain knows that in order to determine with any degree of certainty you need to test under real-world conditions. Real-world conditions include things like identifying beta users in several departments of a large company and allowing them to run beta software in order to test for the masses. Testing under virtualization is just smoke testing - verifying that your users don't have any serious difficulties with the new OS. Vista has a "burn in time" - from initial installation to some level of full capability. The file indexing database, for example, is a serious piece of technology which requires days upon days of use in order to reach a steady state. As operating systems become more complex and include more heuristic features, testers who take the attitude of just virtualizing and following a script will find themselves with more of a support headache than testers who opted to not only script test, but also put the beta product in real-world situations and find any issues.
... and here's where we went wrong with SP1: Don't believe the vendor when they tell you there will be an upgrade path from SP1 Beta to SP1.
Real-world testing uncovers many more issues (and more complex issues) than virtual script testing can ever hope to. This is, in fact, one reason that Microsoft chooses to release betas to the public.
In testing a beta product, a certain amount of resource overhead should be calculated in order to effectively allow for testing. For something as wide-ranging as an OS release, 10% of a beta user's time would need to be allocated to work with tech teams in diagnosing, reporting, and working around problems that are uncovered. Once the beta period is complete, I expect users to be able to upgraded to production with a minimal hassle. During SP1, we had users that were down for more than a day - not to mention loss of productivity post re-installation due to the OS's burn-in time.
I won't say that this approach doesn't have risk. Of course the action of placing real-world users on a beta product has profound implications especially with respect to an OS product.
Some ways to reduce risks associated with users running beta software under real-world conditions are:
- Smoke-testing in virtualized systems prior to even allowing the beta to continue. Pay attention to "show stoppers" AND their potential aggregate effects. Don't ignore issues because you think you have a work-around.
- Build contingencies for disastrous events (such as backing up or replicating offside more aggressively than for typical users).
- Have a reversion plan tested and in-place prior to beta go-live.
I, for one, will NOT be installing any beta service packs from Microsoft and I'll be recommending my company do the same. If you were unfortunate to test the SP1 beta, you'll recall that you were forced to re-front your machines after the beta period was over. Until Microsoft guarantees that it will provide a reasonable upgrade path from the beta to production, there is no point in testing until this becomes public.
We're all excited that someone has figured out how to "more easily" replicate a user interface element that has been a standard part of user interfaces since 1995? Replace "Internet" with "Mainframe" Replace "Browser" with "3270 Green Screen" Replace "HTML/DHTML/XHTML/CSS/etc..." with "Terminal emulation protocol" I'm currently typing this message into a box that covers about 1/10th of my total screen real estate and a scrollbar just appeared. I have to type 7 extra keystrokes to emphasize bold text. When are we going to grow up and stop reinventing complexity and start building systems for user interfaces that are truly useful?? I'm off to preview this post to make sure it looks right before I press the submit button. /blech
Terry
I've always thought that the best way to deal with websites that have advertisements (ANY ads) is to *click* on the ad links ...
Ok - before I get flamed, let me explain. The companies that are paying for ad space usually also pay a "per click" fee. Every time someone clicks on the ad, the advertiser pays the host site.
Some bright fool should write a web accelerator-type program that follows every link on the currently browsed site to at least one click deep. This should be done silently and in the background.
What it means is that clickthrough revenues for sites with ads would go through the roof, but no one would actually be reading and/or responding to the ads. The companies that are advertising would pay off like a slot machine and eventually go out of business because they would suddenly beleive that their internet advertising department is full of geniuses.
The central problem is that the advertisers are using the wrong metric to see if their ad was successful.
I'd buy the product for a $1.
Ravepunk
He's the title designer for "Da Ali G Show" for chrissake. I do not believe that the delicate sarchasm in Adams' work can be trusted to a music video director who designed the titles for a reasonably good TV show.
:/
In both the radio show and the BBC TV Series, what made the jokes work was the voice characterization and acting. Without a good director at the helm who has a letter perfect sense of comic timing and voice characterization necessary to pull off the sarchasm, this movie will fail. My vote would have been to use a Disney cartoon director taking on live action for the first time if they wanted to save money.
Of course, I was the one that poopoo'ed the idea of Peter Jackson doing LOTR, so who the Hell am I to comment?
- ravepunk
Strange how the martian moons are so pixellated.
I used to work for a company that wrote compilers. The runtime p-code intepreter for a particular product was hand-coded assembly with pretty dense comments and the main developer was known for... uh, smoking on the job let's say... I remeber this one section of code where the comments just ended. 7 or 8 screens of assembly later was the following:
... then the comments started up again. It took me and some fellow developers about 3 months to figure out that code.
:/ Ravepunk
SOSXI: RET ; oops.
You want to obsfucate code better? Get high and write it.