Slashdot Mirror

← Back to Stories (view on slashdot.org)

Software QA and Load Testing Solutions?

Posted by Cliff on from the stress-it-until-it-breaks dept.

tekiegreg asks: "I've been asked by the boss to evaluate Load Testing and QA solutions for use by our company. Google Searches have yielded TestComplete and Mercury's solutions. However prices are very steep. Has anything in the Open Source world even come close to this level of functionality in a testing suite? Searches of Sourceforge and Freshmeat reveal nil. Are there any other solutions that people have tried, out there?"

23 comments

  1. JRunner by Joe+Helfrich · · Score: 2, Informative

    The only open source solution I'm familiar with is JMeter. I haven't worked with it much, but in the few projects I've used it, it's been helpful. I't part of the Jakarta project. You can find it at http://jakarta.apache.org/jmeter/index.html

    1. Re:JRunner by Anonymous Coward · · Score: 0

      This doesn't even begin to approach the commercial tools he mentions...

    2. Re:JRunner by Intron · · Score: 1

      So what does the poster need that jmeter doesn't do? Here's the thing: ITS OPEN SOURCE. Maybe an additional needed capability could be added and returned to the OS community.

      People (and I won't say just Windows users, mind you) have a tendency to look at OS software and say: This doesn't do what I need, so I'll just buy the commercial product. It might be cheaper to write the extensions to jmeter.

      --
      Intron: the portion of DNA which expresses nothing useful.
    3. Re:JRunner by Anonymous Coward · · Score: 2, Insightful

      This could be true, however

      1) It could be possible that the organization might not have the resources to develop (not just money, but expertise) such extensions

      2) The time to develop, test and deploy is probably going to be longer than a buy (or download) and implement. What is the time frame?

      3) The cost to benefit might still be better in buying a commercial package

    4. Re:JRunner by dubl-u · · Score: 1

      The cost to benefit might still be better in buying a commercial package

      It might. But beware. If you don't have the source, then you'd better be doing something firmly within the envelope of what the designers expected.

  2. JMeter by NastyNate · · Score: 2, Informative

    JMeter

  3. you need something like LoadRunner by mozkill · · Score: 3, Informative

    The closest open source testing tool I have found, that resembles Load Runner the most is OpenSTA: http://opensta.org/

    --

    -- Betting on the survival of the media industry is a serious risk. I advise investing elsewhere.
  4. Mercury is worth every dollar by Bravo_Two_Zero · · Score: 1

    Of course, experiences vary, as do needs. Our experiences with Mercury's managed testing were worth every dollar. We did investigate open source options at the time, but we just didn't have the available developers to work out the test scripts.

    --


    Amateurs discuss tactics. Professionals discuss logistics.

    1. Re:Mercury is worth every dollar by commanderfoxtrot · · Score: 1

      I use Mercury Loadrunner frequently, with thousands of virtual users.

      It's a very powerful tool but the essence of it is very simple.

      I looked at OpenSTA, but it is simply not in the same league.

      The Mercury licences cost many tens of thousans of pounds, but do note you can buy short-term licences which can help reduce costs. When you're working with multi-million-pound projects, the costs are well worth it.

      --
      http://blog.grcm.net/
  5. Uhmm, perl? by joto · · Score: 2, Insightful

    I don't think you'll find many open source testing tools. Usually open source software is tested by its users. While that is hopelessly inadequate if you have 5 users, and intend to charge them big bucks, it tends to work well in the open source world.

    1. Re:Uhmm, perl? by kwoff · · Score: 1
      Usually open source software is tested by its users. ... it tends to work well in the open source world.

      For example, witness that paradigm of open-source code, slashcode, tested by hundreds of thousands of users every day. Quel joie!

  6. Not open source.... by Nick+of+NSTime · · Score: 1

    Worksoft has a proprietary solution. I don't know if it's cheaper than TestComplete. I just happened to see it today.

  7. Rational? by CptChipJew · · Score: 2, Informative

    I use Rational's (IBM) stuff in my work. It's expensive for sites, but I think pretty reasonable if there is only going to be one tester.

    Rational Robot does automated testing. I'm pretty sure they have load and performance tools too.

    http://www-306.ibm.com/software/rational/

    --
    Vonal Declosion
  8. Do it yourself by abradsn · · Score: 2, Interesting

    I have 5 years of experience in this field.

    Get it over with and just write the software yourself. Most of the work is custom software anyways.

    Log everything to an xml file and or a database.

    Write your own front end to the whole test harness.

    There really isn't very much good help available.

    Even though nunit is good for unit tests, there is nothing that stops you from implementing load/stress and performance tests with it.

    You can even use the nunit user interface as a testharness until you get your own written.

    In the end you'll spend some time and money on your own automation, but that is what you would have ended up doing down the road anyways.

  9. Some Open Source Testing Tools by pfdietz · · Score: 2, Informative

    Watier

    The Grinder

    Selenium

    Last January there was a workshop on open source web test tools in Austin.

  10. Depends on Your Requirements by matheny · · Score: 1

    Load testing and QA have various meanings. If your QA simply means that you track the delta between load tests, and if the delta isn't significant a release is approved then something like OpenSTA should be just fine for you.

    However if you need to be able to do regression testing, performance tuning, and code profiling then you will either need to drop some cash (Mercury can do all these things) or spend some considerable time doing your own development. My company developed something similar to Mercury's load runner and quick test, and looking back it would have probably been less expensive to just spend the money up front.

    So, if you don't have money but your requirements aren't real strict then a hodge podge of open source tools with a bit of custom development will work fine. If you have strict requirements then be prepared to spend a bunch of time or money getting these tools up and running.

  11. Have you considered... by jo42 · · Score: 1

    Have you considered Linux and some Perl scripts? Even little olde wget can generate a considerable amount of traffic for you.

  12. Emily Latilla might say by ClosedSource · · Score: 1

    Since IBM is so pro-OSS they're releasing the source for Rational Visual Test a popular product they discont... What is that? You mean IBM is not releasing the code? But I thought you couldn't buy any more licenses for it? You mean IBM says OSS stands for Old Slippery Snake-oil?

    That's very different. Never mind.

  13. At my last programming position by brokeninside · · Score: 1

    My employer used an modified (in house) version of CCUnit to automate testing on Solaris. For Windows testing, we used WinRunner. But that was two years ago. CCUnit claims to be platform dependent, so it might work on Windows.

  14. Write it yourself using Perl and IE by cerberusss · · Score: 2, Informative

    I've seen a lot of comments mentioning you should write a testing tool yourself. If you decide to do this, maybe this article could be useful; it talks about automating IE using Perl scripts.

    --
    8 of 13 people found this answer helpful. Did you?
  15. JMeter by unboring · · Score: 1

    Have you looked at JMeter? I've used it to test web/server applications, but it can be used for other applications too.

  16. Grinder by petard · · Score: 1

    For web apps, we've had a great deal of success with grinder. Its scripts are easy to write, and its recording mode is sometimes all you need to gin up a quick load test. For one project, we had Mercury as well... we found that grinder was much more helpful, at least for the development team. (Though I appreciated having both.)

    --
    .sig: file not found