What Is Fair Technical Support From a Manufacturer?

VincenzoRomano asks: "One year ago, I decided to buy some 'enterprise grade' firewalls, in order to replace the old ones used by a former ISP. Before buying, I did a bit of a survey. I browsed the product 'data sheets' from the manufacturer web sites, and in some cases, asked for more details by email. I finally choose a top product, that had been on the market for a year and a half, from a very well known and reputable company. The product showed a number of issues as soon as it was unpacked and put to work, that you would not expect from something 'enterprise grade', like not being able to keep a VPN up and running for more than a few minutes, or doing bad IP routing on our LAN. I've spent the last year to make the equipment working, accordingly to both their data sheets and the features expected from an 'enterprise grade' product. Important issues are still open while the technical support is actually relying on my own stuff and setup, and on my personal availability in order to do troubleshooting, firmware beta testing and other experiments. I've finally decided that the product was far from being ready to market or even usable for beta testers, and have requested some kind of compensation for all the job I had to do. What's your opinion about such a behavior in a company? Is it fair?"

Comments

Welcome to IT?

[EveryNickIsTaken]

What you're describing was fairly commonplace back in 1994-1997 when I was helping start up a regional ISP. To a certain extent, it was "neat" that we were finding bugs in brand new routers (Cisco + Bay were the usual culprits). Of course, it was a pain in the ass that it'd typically take hours, days, or weeks to realize that the hardware/software was bugged.

As technology has progressed, things certainly have gotten better. Regardless, you need to realize that "shit breaking" is part of IT. Don't like it? Leave the field.

Re:Welcome to IT?

[Sobrique]

Sadly so. Almost any product out there has bugs. If you're lucky, the really sucky ones are already gone. But only if you're lucky. I have memories of a NAS upgrade, that resulted in 8 TB of 'possibly corrupt' data, over long enough interval that just restoring a 'known good' backup just wasn't an option.

I'm afraid it's normal that 'new toys' have problems. Your only way of really avoiding, or at least migating this is to stay one release behind the latest and greatest. You've got good odds that by the time the 'next release' is finished, most of the real killer gotchas will have been found.

In part, it's laziness in testing. In part, it's the simple fact that it's definitely non-trivial to exhaustively test something in teh kind of intensive environment you see in the 'real world'. Things like race conditions typically don't occur often enough to be noticed in testing, but will start to crop up often enough to be a real problem in the real world.

Acceptable? No, not really. Fairly commonplace? Hell yeah.

Don't trust any .0 release. Don't trust anything that's sold to you as the 'newest and feature laden'. Ask yourself if you _really_ need that totally new and cool (and therefore almost certainly not properly tested) feature, or if actually, a revision or two back would do what you need.

With the best will in the world, a test environment will never really compete with a couple of hundred thousand people using it, for finding 'problems'.

You will almost certainly find that the EULA also includes a get out clause for exactly this kind of behaviour.

Re:Welcome to IT?

[Anonymous Coward]

Regardless, you need to realize that "shit breaking" is part of IT.

And that is why people in general have a dim view of computers... Not to mention the people who operate them.

I think the computer world in general needs to do a lot more in this regard - consider a television. It's got a relatively simple interface (depending on the model, but they're all usually pretty similar), and it's got "uptime" and MTBF that would make any server manufacturer more than proud. Why would the general population want to watch TV on their PC when (a) the TV is cheaper, (b) it works right out of the box, and (c) it's not going to be buggy or go obsolete suddenly (the impending HDTV changeover notwithstanding).

Software manufacturers (including you programmers who read Slashdot) - spend less time adding the next new feature and more time making sure the existing features work. Stop using us as your beta testers!

Re:Welcome to IT?

[Phreakiture]

Of course it's not fair.

It seems like you have taken every reasonable step, especially in spending a year on it. Now take the next step.

Name them.

Re:Welcome to IT?

[BVis]

I think the computer world in general needs to do a lot more in this regard - consider a television. It's got a relatively simple interface (depending on the model, but they're all usually pretty similar), and it's got "uptime" and MTBF that would make any server manufacturer more than proud.

A television is a far simpler device than a modern computer. No hard drive, no networking, etc. Comparing a television with a PC is about the same as comparing a digital watch to a PC. The more complicated and powerful the system, the more that can go wrong.

And what the hell is "the computer world in general"? I think it's pretty safe to assume you don't know what the hell you're talking about. I bet you've got 12:00 still flashing on your VCR.

Why would the general population want to watch TV on their PC when (a) the TV is cheaper, (b) it works right out of the box, and (c) it's not going to be buggy or go obsolete suddenly (the impending HDTV changeover notwithstanding).

My current television cost 5 times what my current computer cost. ($2000 HDTV versus about $400 for the PC I put together myself.) The TV has crashed on me a few times. I don't know where you're getting your information from, but I suggest you catch up to the times.

Software manufacturers (including you programmers who read Slashdot) - spend less time adding the next new feature and more time making sure the existing features work. Stop using us as your beta testers!

Stop yelling at the programmers. Programmers have about as much to do with when a product is released or how much testing it gets as the guy who empties the trash at an auto factory has with the quality of the product.

Granted, better programming USUALLY leads to fewer bugs, IF they're not crippled by terrible managers more interested in getting ahead based on other people's work, IF marketing doesn't make outlandish promises involving technologies that don't exist and/or statistics that mean absolutely nothing, IF there are enough programmers on a project to make things sane, IF IF IF.

Case in point: I work for a software company as a research engineer. In a meeting about a month ago, the VP of Sales (who tragically is making all the engineering prioritization decisions) asked our QA manager how long it was going to take to test a product before it could be released. The QA manager replied "Two weeks." The VP IMMEDIATELY said "One week." QA: "Two weeks" VP: "One week, you never give QA as much time as they want." No explanation, no justification other than "Oh QA always takes too long, just release it." Given the option, programmers write good code. They're rarely given the option.

Perhaps you should consider being part of the solution as opposed to just bitching about the problem. Failing that, try using some actual facts or reality-based information in your arguments.

Re:Welcome to IT?

[Phil246]

This is something that has come up in the last two weeks as part of the Software Engineering course im doing at Uni.
This sort of practise differs depending on the country and what is considered most important according to lecturer.

In the states, it is apparently considered that being first to market is most important rather then software quality since bugs can be fixed later on and customers are apathetic to switching even if the product is bad.

In the UK it is considered that quality of software is paramount, and that fixing it later is an undesirable option. You may be first to market but if you do a really sloppy job of it, then word will spread and you can forget about getting big contracts in the future. Also there aint no way you would get a BCS accreditation ( british computer society ).

Re:Welcome to IT?

[EveryNickIsTaken]

This is probably part of the reason why the UK is not known as a hotbed of IT/Software/Hardware innovation.

Re:Welcome to IT?

[Anonymous Coward]

and why botched jobs are rare?

Re:Welcome to IT?

[stu72]

All great advice but try telling that to customers.

People buy IT products on features, not reliablity or usability. If you stop developing new features to focus on reliablity, your customers will, in short order, abandon you for the guy pumping out feature after feature, however buggy.

I'm not saying this is rational, but it's how non-IT (and some IT) people make IT buying decisions, and that drives the industry.

Re:Welcome to IT?

[ScuzzMonkey]

For someone who had such a problem with the generalization "the computer world in general" you've gone on to make an even less accurate one with "Given the option, programmers write good code. They're rarely given the option."

I'll add mine: most code is complete crap. A small percentage of programmers write good code, the rest churn out drek that requires a lot of testing and review to pass muster (which, if anything, reinforces your point about QA). They are no more immune to featuritis or feature creep than their managers, no small number of whom are--you guessed it--former programmers.

The problems you have pointed out are very real and are a part of the overall issue, no doubt. But it's disingenous to suggest that simply allowing programmers to go their own way and take as long as they feel is necessary to complete the project would solve the issue. I suggest you amble on over to Sourceforge and peruse any random sample of the projects there whose coders have had as much time as they please to work on before release, and then come back and tell us how terrific it all is.

Re:Welcome to IT?

[gregmac]

You may be first to market but if you do a really sloppy job of it, then word will spread and you can forget about getting big contracts in the future. One word: Microsoft.

Re:Welcome to IT?

[BVis]

You're right about that. Looking at my own code is all the confirmation I really need :)

Re:Welcome to IT?

[fruitbane]

You know, this kind of scathing response doesn't help the discussion. The poster does have a point. We have allowed the complexity of computers and software to become an excuse for releasing crap products that have not been adequately developed or tested. It does not matter if the programmer or the manager or the CEO is responsible, if the user is the one who has to effectively help beta test a commercial product the model is broken.

Let's face it, people. We have allowed our industry to become one scandalized by poor reliability, poor build quality, and rampant problems that would simply not be allowed in any other industry. And frankly, very few companies have the balls to buckle down and try hard to get things right from step one.

Re:Welcome to IT?

[BVis]

I may have been a little strident. The point I was trying to make is that programmers aren't the only ones at fault, and the post I was replying to seemed to blame everything on lazy/incompetent programming, when the problem is more complicated than that.

"Buckling down" and getting things right from step one requires things that are anathema to modern business, such as allowing developers to drive development, giving QA the right to delay a release, and doing more than is absolutely necessary to still push out a product people will pay for.

Re:Welcome to IT?

[Eagleartoo]

I really wish I had mod points right now because you stole what I was going to say =) . Releasing a product that has to continually be patched because of it's verbosity is not my idea of a good product. When I switched to apple the only time it really screwed up is when I did something that taxed the system, and the software that was built in was like the AM/FM radio in your car. You turn it on and it works. What you are seeing in the bugginess of new hardware/software is simply greed. There hasn't been a better computer since unix.

Re:Welcome to IT?

[Scott7477]

While I agree in general with your comments, I have to disagree about testing hardware in real world conditions. I believe that any "top" manufacturer would have a decent-sized test lab where the device in question could be hooked up to the lab network and be put through its paces. I figure such a lab could have a secure link to the Web to suck down traffic to run through the lab network. This kind of setup would allow for fairly thorough testing that would closely approximate the kind of setups that customers would have.

I recall visiting Novell's test lab many years ago in Provo, UT where they had a warehouse sized room of PC's on racks(just aluminum frame racks, not the current rackmount type). I was told that they ran their products on this test network heavily before releasing their products. Granted, that was an internal network, but as I recall in its day Netware and its associated products were known for their reliability in production environments.

To sum up: I think that major manufacturers can test their products under real world conditions. If they don't, we should stop buying their products. To go back to my Novell case, apparently one of the factors in that company's decline was that their marketing people failed to realize that flashing a few slides in front of non-IT suits would get more sales than explaining their product to the IT people who would actually use it. So a big part of the problem is C-level execs making purchasing decisions without really paying attention to IT's concerns.

Re:Welcome to IT?

[raddan]

Firewalls have been around long enough where I simply wouldn't accept this nowadays. I put up with proprietary firewalls for years that were purchased before I was hired here. They were purchased because they had long feature lists and snappy GUIs. But in practice, they were garbage. Their state tables were small, so they often dropped connections. Their VPN implementations were buggy (in fact, a 'factory authorized field consultant' from this one particular company said that no one *really* knew how their IPSec stack worked; I know he was saying this to me in order to give me that "I'm on your side, buddy" kind of feeling, but it made me immediately lose faith in their product). Often the GUIs did not display the proper state of the machine. To add insult to injury, our paid support contract did not include 'premium' features such as access to their tech info library, where you would find out what error IDs actually meant.

After one particularly bad episode with the firewall, where we were required to replace a failing hard drive, we started looking for something new. That hard drive was a standard 2.5 inch Hitachi 40 GB-- but they charged us $500 for the "authorized" part so we wouldn't void our warranty. If they're going to charge you $500 for a part that should cost $100, fuck the warranty, man!

After evaluating our options, we settled on OpenBSD's PF. Several linux firewalls were considered as well, but we went with BSD because we were more familiar with it. We've been extremely pleased with our choice. PF gets better and better with each release, it is highly flexible and customizable, and the rule syntax was easy to learn. We have features that would cost us an arm and a leg in their proprietary counterparts (VPN, dynamic rulesets, firewall failover, and so on), and we can build it on commodity hardware. There's a huge community that has given us technical and moral support, and documentation is freely available. These are full computers, too, so if we need to write a custom monitor or report, piece of cake! And the savings in time and money have been enormous.

And this taught us an extremely valuable lesson-- don't be afraid of the learning curve of applications that don't use a GUI. The fact is, sometimes the problem space is complicated, and a text interface often handles that complexity better than a GUI. Sure, there's a higher nmemonic load, but we work with these machine every day, so we got used to it. You'd have to give me a really good argument at this point to get me to switch over.

Re:Welcome to IT?

[fruitbane]

I do agree with you here. These necessary steps do seem to go against the grain of the "modern" business approach, but they are much more prevalent in IT than in other industries. The automotive and children's goods industries are heavily regulated and recalls are mandates should certain standards not be met. Does that mean the only solution to this technology crisis is to implement a level of government regulation over the commercial software industry? Some of you may balk and say the market must be allowed to work itself out, but history has demonstrated that pure capitalism is essentially economic Darwinian anarchy. And if government regulation is not the solution, and I doubt anyone would call it a panacea, what is the answer?

Re:Welcome to IT?

[Cygnostik]

That's BS. Granted things are going to break or act unusual in some ways, because of vastly diverse situations - but when a company says pay us a million bucks and we'll give you this crap that does everything you could ever want, it does it perfectly and it'll leave you operating in such a way as to interface perfectly with your customers allowing everyone to ride off into the sunset and never have any worries ever again there's 2 problems.

1) Bullshit! Why would you even believe it?!?!
2) How dare they claim to ave the universal holy grail to offer if they just sell broken shit and expect the paying customer to fix someone elses product?

Re:Welcome to IT?

[scdeimos]

You're lucky anyone even bothered to ask how long it was going to take, even if they cut the estimate in half.

Where I work it's more-often-than-not a case of someone giving us a launch date as they hand us a dodgy, maggot-ridden request for development that doesn't have enough information to develop a solution, just a bunch of mockups or screenshots of how they want things to look to the end-users. After X gets released there come a bunch of billing and business rule changes that we'd thought we had already "settled" with the offender before coding had actually begun.

SLA

[Anonymous Coward]

I'd like to reply, but since you didn't sign an SLA, it would cost you $300 per incident.

Obligatory MS slam

[Jimbo God of Unix]

To be quite honest, this has been my personal experience with MS software. I've also had problems with various open source software, but at least with OSSs you don't necessarily expect it to be usable "out of the box" and sometimes requires a little fiddling. But with MS stuff you're paying real money for it.

James

Live and Learn

[ReidMaynard]

This is what "Evaluation Units" are for .. I have never brought a new enterprise solution on board before extensive evaluation via a loaner...most of the time we discover the product does not live up to advertizing.

Re:Live and Learn

[Sobrique]

Agreed. If you're both serious about the product and the sale, then run an eval. It takes some effort replicating your production environment, to a level where you can put this product 'in parallel' to see how it copes, but it's worth it, for exactly the reasons outlined above.

Sadly all too often, bosses don't see the 'value' of a test environment. There's plenty of large companies out there, that do, and they don't do it just because they like to waste money - My current employer buys everything in 4s. One for dev, one for testing/acceptance, one for production and one for disaster recovery.

Seems a bit wasteful, but the first time your test environment gets blatted by a 'major issue' you will cry tears of joy that you didn't have your production environment running that.

Re:Live and Learn

[tlhIngan]

Agreed. If you're both serious about the product and the sale, then run an eval. It takes some effort replicating your production environment, to a level where you can put this product 'in parallel' to see how it copes, but it's worth it, for exactly the reasons outlined above.

Of course, if your company is a fairly big one and you're thinking of doing a major purchase, many manufacturers will trip over themselves to let you do free evaluations for extended periods of time (easily 3 months, 6 months to a year maybe), with support and all that. They'll send the hardware, you set it up (or they may even set it up for you!), and let it run. If it works but doesn't do something critical, you can always return the equipment and say "I can't use this - it doesn't do X".

Firewall

[jwilhelm]

Assuming you have set everything up correctly (and I'm not saying that to be an ass, just that a lot of times people complain about hardware and it's configured incorrectly) and it is not working as advertized, then you certainly should be upset with the manufacturer. However demanding compensation from them is a bit much. Part of your job (I am assuming) is picking the right hardware for your company. That means testing it before deployment in a production environment. If you deployed the wrong hardware (even if on paper it looked right) that's not the fault of the company.

Re:Firewall

[Sobrique]

Actually, I'm fairly sure 'Fit for purposes' is defined in various sources of consumer rights, in various legislatures.

Sadly, it seems that IT is mostly exempt.

Or maybe it's just that _my_ definition of 'fit for purpose' which goes something like 'isn't flakey, unreliable and does wierd things' is slighty different from theirs 'operates as advertised in at least 5% of possible usage scenarios for at least 5% of the time'

Re:Firewall

[bernywork]

'Fit for purpose' is defined in Australian law, and is quite often sided on the side of the consumer and not the supplier / manufacturer.

In Aus, it's defined as part of the "Trade practices Act 1974". As the parent rightly said, it's in various consumer rights and various legislatures including the US.

I would turn around and ask for a refund citing as reference all the helpdesk cases you have logged and the fact that it is not working as advertised. I don't think however I would be asking for more. The only option if you want additional compensation I would suggest would be to start a civil suit for recovery of costs. Whether that's worth it or not probably won't be your decision.

Berny

Re:Firewall

[abb3w]

However demanding compensation from them is a bit much.

Mostly.

However... if you have a support contract; if you have well-written and up-to-date documentation complete with change logs, describing your network infrastructure, how the widget integrates into your systems, and which of its marketing-brochure functions it is expected to perform in practice; if you have created a maintenance user account with all necessary and sufficient permissions and access to allow their technicians to diagnose and maintain their product; and if you provide them with a procedure for requesting supplements to the aforementioned documentation and/or maintenance account permissions... THEN to the extent that they either are failing to meet the support level stated in the contract, or to the extent that their techs need assistance or support beyond a demonstration of the problem and the aforementioned supplement requests, you might (In My ta Hell with y'all Opinion) have some sort of basis for claiming reimbursement.

Of course, I strongly suspect VincenzoRomano is lacking one or more of those prerequisites, and that getting compensation without a visit to court is unlikely. In the future, odds of this problem might be reduced by an inquiry to the manufacturer before purchasing as to whether they have any satisfied customers using similar products for similar functions who might be willing to discuss their experiences. If there's one local, you can point your boss to this incident, and possibly get a company-expensed lunch for you and an opposite number at the "satisfied" customer for extended technical discussion. If you've the gift o' gab, you might even convince the boss that a few libations shared while on the expense account would be more likely to reveal the candid truth. Just make sure it's your company's account, not the sales critter's. =)

what part of

[Anonymous Coward]

What part of "there is no guarantee, express or implied" did you not understand?

Re:what part of

[Sobrique]

Which, lets face it, is pretty disgusting behaviour from someone selling you a product. I mean, imagine the uproar at someone who sold cars under that basis.

But hey, they all do it, so at least it's _expected_ that you'll end up with a pile of bug laden rubbish.

Re:what part of

[Anonymous Coward]

One of the first things a (good) tech support guy learns is managing expectations. If the customer expects too much, they'll be disappointed. Keep their hopes low, and they'll be presently surprised. Too low, though, and they'll decide you're an idiot and give up and you'll lose a customer.

Re:what part of

[Anonymous Coward]

http://en.wikipedia.org/wiki/Misrepresentation

It was your decision to purchase

[ip_freely_2000]

"I finally choose a top product, that had been on the market for a year and a half, from a very well known and reputable company."

Something sounds fishy here. A 'top' product from a reputable company on the market for 18 months but it doesn't work?

(a) Since there are no names mentioned, maybe it's not a top product from a reputable company.
(b) You are trying to use the product for something it was not designed.
(c) You're a customerzilla that is a networking legend in his own mind.

In any event, you chose the product so you've got to deal with it. You either toss the device or continue your CYA exercise and get something out of your 'investment'. Apart from an apology and updated firmware, they really owe you nothing. Maybe they can offer you a job. But would you want to work for a company that would allegedly ship bad products?

Re:It was your decision to purchase

[Aladrin]

On top of all that, he either totally ignored all the bad reviews for the product, or there were none. If there were no reviews at all, that's a sign in itself, and can be considered a bad review.

So let's assume that mysterious, but very apparently very popular firewall X did indeed have a ton of good reviews. Doesn't that pretty much leave him as an edge-case? Someone who is either using the product as it was not intended, or so incompetent that nothing the company can do will straighten the problem out? The company is probably relying on his 'availability' to troubleshoot because they cannot replicate the problem in their labs.

Re:It was your decision to purchase

[Anonymous Coward]

In any event, you chose the product so you've got to deal with it.

Yeah because advertising your product does something that it doesn't isn't illegal at all.

Re:It was your decision to purchase

[ip_freely_2000]

ummm...if you buy IT products based on advertising
  (A) you get what you deserve
  (B) you'd better jump out of the industry before you get pushed

Re:It was your decision to purchase

[Jah-Wren Ryel]

So let's assume that mysterious, but very apparently very popular firewall X did indeed have a ton of good reviews. Doesn't that pretty much leave him as an edge-case?

Or the reviews were astroturf and/or advertiser influenced.

You took too long

[Overzeetop]

You bought it. It didn't work. Tech support couldn't fix the problem.

You had two options: work with the reduced functionality or send the product back. It sounds like you chose the former and are now regretting the decision.

Maybe you needed a custom solution that was actually outside of your budget.

What the market decides.

[Shivetya]

If a manufacturer starts losing sales due to bad word of mouth then they will either increase support or get marginalized in the environment they compete in. If people continue to buy their stuff then obviously they aren't doing enough research before buying. This is the biggest issue I have seen where bad companies persist, people love to claim doing great research and get stuck on a product because of one or two things that pique their interest losing focus on the bad sides.

If you feel as if your getting the short end of the stick then why continue to use their product ? Also, make sure others know about it. If your not willing to take the time to let others know about the issues then it can't be a major problem.

Re:What the market decides.

[Anonymous Brave Guy]

How about what the law decides? In most places, there is a legal requirement along the lines of "fitness for purpose" or similar. If the goods aren't, then their technical support needs to fix that, or they should expect the goods to be returned as unfit.

Re:What the market decides.

[Sobrique]

You are correct. However most EULAs have also got the weasel out clauses built into them.

Relative enforcibility is a different matter. It's pretty easy to say that this vacuum cleaner I bought, isn't fit for purposes, and make an assertion/demonstration as to why. It's a bit harder with a piece of software that 'works-ish'.

Re:What the market decides.

[Anonymous Brave Guy]

You are correct. However most EULAs have also got the weasel out clauses built into them.

You are correct. However, as you seem to have noticed yourself, there is this teensy weensy little issue of having no legal weight whatsoever. In many jurisdictions, you can't disclaim warranties of fitness for purpose etc. however many lawyerly weasel words you put into the "contract". Of course, whether EULAs can constitute valid contracts in the first place is a mostly untested question in many jurisdictions, too. In any case, if someone's text editor or video game, for which I paid good money, proceeded to hose my PC because of programmer negligence, I rather doubt any court in my country would side with the software vendor and uphold all the disclaimers, limits on damages to the original purchase price, and so on. But we won't know for sure until someone really does make that mistake, or the damage caused by crappy security and/or reliability in mainstream products results in a high profile case that gets (or doesn't get) a big damages award in a high-level court.

What's your issue?

[Anonymous Coward]

The product showed a number of issues as soon as it was unpacked and put to work, that you would expect from something 'enterprise grade', like not being able to keep a VPN up and running for more than a few minutes, or doing bad IP routing on our LAN.

Well, if you expect these problems, why are you complaining?

Grammar nitpicks aside, it sounds like you are using Sonicwall. Your best course here is to ditch it and go with a reputable brand.

Oh it is such a hard lesson to learn

[Anonymous Coward]

I was actually in the same place that you are a few years back. My company needed a consolidated log manager and we picked up what seemed to be a reputable product from a reputable company. In a smaller environment the product would have been excellent however it did not scale well for our large network. My team spent the next 6 months to a year (off and on) trying to make this product functional. The company sent engineers to our Site, they provided us with free training in their classrooms but it really came down to the wrong product for the job.

The lesson we learned is that anytime an important or expensive piece of hardware/software needs to be purchased we make the company demo it in our environment. Most companies will send out a demo unit for you to test with and a lot of times they will send a tech to help make sure all goes well. If all goes well congrats but if it doesn't don't rely on empty promises of "the next firmware or driver" to fix your problem.

Finally I would like to actually answer your question on the topic. We pay a lot of money every year for support and any reputable company will provide that support. Over the phone troubleshooting, timely RMAs, and in rare cases sending a tech onsite when all other options have been exhausted. It sounds like the company is doing their part by continually troubleshooting with you. Hopefully in the future you will learn from this mistake just as I learned from mine and make sure to get a proper demo before paying money to a company based on whitepapers and sales pitches.

Evaluation

[Neil Watson]

Vendors are usually willing to lend you a model for evaluation.

Name 'em, already...

[dos]

Let's see, an "enterprise-grade" firewall that also supports VPN connections, that in reality isn't really as good as its rep... 's' key on your keyboard broken so you can't type "SonicWALL"?

Well...

[frank_adrian314159]

What's your opinion about such a behavior in a company? Is it fair?

I'd probably call it poor, but look on the bright side - the company made money, the salesman made money and two out of three ain't bad...

Trivial solution

[Sigma 7]

The product showed a number of issues as soon as it was unpacked and put to work, that you would not expect from something 'enterprise grade', like not being able to keep a VPN up and running for more than a few minutes, or doing bad IP routing on our LAN.

Defective within 30 days. Request a return at the 29-day mark.

I've finally decided that the product was far from being ready to market or even usable for beta testers, and have requested some kind of compensation for all the job I had to do.

Don't bother requesting compensation, unless you have a documented attempt to return the product within the 30 day period.

What's your opinion about such a behavior in a company? Is it fair?

It might not be fair (depending on which viewpoint you have), but it is not possible to get a defective product to work.

Test Labs can't test Real Life

[TBone]

As an app engineer for a major web services company (if you've ever bought software or ringtones for your cellphone or handheld, there's a good chance you've used our software), I guarantee that no testing/QA process will effecively exaust the space of possible real world examples. There's just too make devices with too many configurations in too many places for a test cycle to even be able to test all the permutations.

We have binders full of test cases...that are run across multiple devices, or through automated testing tools. And we still see, on the day after a code upgrade, code and device issues that "could" have been caught but weren't because that wasn't one of the primary test cases.

Unless your target audience is using a specific device in a specific configuration for a certain task, you'll never be able to replicate your real-world use cases in a manner that provides you'll be able to exhaustively test them. You test for the largest volume of potential use cases, and hope that your decided-upon "edge cases" really are edge cases and dodn't fall under the bell of the curve..