Slashdot Mirror
Testing Frameworks in Python
An anonymous reader writes "This article looks at Python's two standard modules for unit testing: unittest and doctest. These modules expand on the capability of the built-in assert statement, which is used for validation of pre-conditions and post-conditions within functions. The author discusses the best ways to incorporate testing into Python development, weighing the advantages of different styles for different types of projects."
...in Bret Pettichord's Scripting for Testers one day class.
It talks about eliminating some of the tediousness from testing web applications, mainly by using automated solutions like WTR.
He's also got a list of testing resources that's got some good stuff in there...
The Army reading list
A few thoughts:
1. I've never had a problem with memory management in python. Can't say it doesn't exist, just never impacted my production applications.
2. Implementing great built-in test frameworks doesn't need to wait for memory management improvements. I'm seeing almost immediate pay-offs from this kind of built-in testing.
3. I'm implementing python as an alternative to java in large applications - with complete success. Easy to learn, easy to maintain, fast enough to handle millions of rows of data a day - what's not to like?
Circular references are what weak references are meant to handle. But it'd be nice if the garbage collector would handle those cases automatically.
Ita erat quando hic adveni.
It's Guido van Rossum.
As for your claims about the GC, I was under the impression that current versions of Python have a hybrid GC that cleans up circ-refs. I certainly could be mistaken, though, as I've been exiled to Java-land for the last several years.
I hate to break it to the hack and slashdot crowd, but Testing is actually a whole career in itself, and the application of different testing processes and methods to different projects is a critical part of ensuring projects succeed.
This article covers NOTHING about the different types of testing on a project, or indeed how test cases should even be constructed. Its basically about some UnitTesting elements that could be done by testing.
I know its unpopular here on Slashdot to claim that there are more developers working on big projects than people hacking in Python. Buts its articles like these that underline the difference between professional software development and hacking.
This is about hacking.
An Eye for an Eye will make the whole world blind - Gandhi
Judging from some previous comments, I see that some fail to grasp that modern computer languages form a large ecosystem. Each language has its purpose, and one can not easily dismiss a language as dead, just because some other, ostensibly more powerful language has appeared on the block. Monkeys, whales, cockroaches, ants, and plants continue to coexist with humans.
.
When I want to solve a program I choose the language I will use, taking into account the abstractions and facilities it offers. * I chose Java when I wanted to leverage the javadoc applets (doclets) to convert a Java-like syntax into UML with my UMLgraph tool
* I chose C++ to implement the CScout refactoring browser for C programs. In this case I wanted rich and efficient data structures, with minimal speed and space overhead. CScout datasets can require more than 1GB of RAM, and runtimes can span more than a day; any overhead of object boxing, garbage collection, or bytecode interpretation would in this case be unacceptable.
* I chose Perl to o Convert digital photographs and GPS track logs into annotated photo albums and trip maps
o Examine the availability of 4500 URLs cited in computer science research papers.
o To create the diagrams and the index for my book Code Reading: The Open Source Perspective.
In all the above cases, I needed a typeless language with a rich set of operators, functions, and libraries to minimize the time I would spend to convert my ideas into code. Ruby and Python would have served me equally well.
* Finally, I chose C to write
o the *BSD sed implementation.
o The socketpipe zero-overhead network pipe tool.
o The Outwit Windows-Unix shell integration tool suite.
o The fileprune backup file prune utility.
o A device driver for interfacing with my home's alarm system.
In these cases, I did not require any fancy data structures or framework APIs, but I did want tight integration with the underlying system, absolute efficiency, and minimum-fuss portability. For code that will be executed billions of times on tens of thousands of systems, spending some additional effort to provide the absolute efficiency and reasonable portability that are possible in C, is a proposition one should take into account.
Are you saying that this is an inherent language problem, or simply an implementation problem? i.e. Does the Jython interpreter display this issue, or only the native Python compiler/interpreter?
Javascript + Nintendo DSi = DSiCade
> Please go an get a decent, non-language specific book on testing before reading and listening to this
> article.
Sure usability, user acceptance, system, string, and integration testing are all valuable. But why can't developers without any knowledge of these start with built-in unit testing?
And keep in mind that few books on testing cover the fairly recent phenomenon of test-driven-development (or test-centered-development).
The use of built-in test harnesses and focus upon developing tests as documentation of requirements is probably the biggest single improvement in testing in twenty years.
In one fell swoop the debugger is rendered obsolete...
> probably the biggest single improvement
> in testing in twenty years
Right on. Being able to make some changes and run a suite of 500+ tests to make sure things still work is a thing of beauty.
And when a bug slips through, that's just an opportunity to write another test and make sure that bug never happens again. Good stuff.
The Army reading list
According to this you are correct.
BUT.. please note it also says "You can run gc.collect() to force a collection, but there are pathological cases where objects will never be collected."
It also mentions the use of weakrefs.
True programmers never test their code, that is for people who are unsure of themselves and who should not be trusted
Despite that fact that you are clearly trolling I think there's a valid point in what you are saying: there are programmers who think that they are too good to write unit or system tests for their code. And that's a real danger.
The adage "a line of untested code is a line of broken code" is so often true that I still find it scary when examining untested code. It's just amazing how much of a difference the discipline of writing automatic unit tests makes in improving code.
If you think you are too good to write tests, then perhaps you are too good for the software industry?
John.
You're not a "long time Python developer." Far from it. You may qualify as a "python developer from a long time ago," though -- in fact, that seems likely, given that the page you linked was last modified in 1999.
:)
It's Guido van Rossum, not Guido van Sustren. And Python's garbage collector works fine on cycles, as it has done since 2.0, iirc. The only way you'll get a memory leak with Python these days is by writing a C module that forgets some DECREFs or by writing cyclic objects with __del__ methods, and even in the latter case, you can easily take care of your leaks by breaking the cycle yourself.
You're uninformed and incorrect; take your troll to the next Perl story, please
Jeremy
Looking for a Python IRC bot?
Despite all of Guido Van Sustren's claims to the contrary, Python's garbage collector just doesn't work correctly, allowing the programmer to create a circular reference which never gets resolved. This is a critical impediment to writing mission critical applications in Python, as they will eventually run out of memory and fail.
Neither of those references point out he weakref module that allows you to work nicely with circular references.
Python's memory issues need to be fixed before the language can break out of its niche.
Yep, it's a niche language. Crawl back under your bridge.
It is clear you do not know anything about Python. You are not a long-time Python developer but rather a troll. The first hint is that you misspell Guido Van Rossum's name.
The second hint is that you cite a page from 1999. A Python developer would know that that was before Python had cyclic garbage collection. Here is an article from 2001 that describes how the issue was fixed.
The third hint is that you point to a page that describes how to avoid creating a memory leak by appending an infinite number of items to a list. Guess what: appending an infinite number of items to a list causes a memory leak in Java, C++, C, assembly, Scheme, sh, Perl and every other programming language in the world. If you ask the computer to store a continually growing list of items, it will do so...in any language.
If you think that Python can leak memory in circumstances where other languages would not, post an example program and we'll all test it out.
For using a niche language, Portage sure seems pretty popular.
I use Jython regularly for unit testing Java. The expressivness of jython is great for writing shorter unit tests, and the lack of compilation makes the whole write-test-debug cycle short. And, dare I say, it's just more fun to code in jython than Java.
I agree that those topics are worthy of deeper discussion, but I think the article stands well on its own. I'm learning Python and I think the unittest section will help greatly while stumbling through the process of building applications.
The links he provides in the Resources section at the end of the article provide well-rounded background information on testing in general.
What you're saying puzzles me. You're absolutely right that this article is not about testing as a whole. The title should have been "Unit Testing Frameworks in Python."
But your statement that "this is about hacking" and not professional software development puzzles me.
I believe unit tests are a very legitimate piece of testing - a kind of first-line defence. They're intended to test individual software modules for their low-level behaviour. Typically, a developer would be expected to run them before submitting any change or bugfix, as a kind of "smoke test" to make sure things are okay. Certainly, some organizations might make the mistake of thinking that this kind of testing is all that's required - which is dreadfully wrong - but I don't think there's anything hackish about it.
In a large organization, the testing team might not consider it testing because unit tests are necessarily maintained and performed by the developers only.
But I would argue the exact opposite with regards to underlining the difference between professional software development and hacking. If you don't have unit tests, I would say that what you're doing is closer to hacking.
Accountability on the heads of the powerful.
Power in the hands of the accountable.
Call me silly here, but that article talks about developer ONLY testing, and doesn't seem to discuss different types of projects at all. This was about basic code testing, and mainly unit-test.
Yes, this is exacly what this article is about. What is wrong with that?
No UAT, no System Test, no Integration Test... no how test cases should be defined.
No it is assumed that anyone programmer worth his salt knows some test methodology, and this usefull article explains tools that make that task easier.
Please go an get a decent, non-language specific book on testing before reading and listening to this article.
Yes I agree, if you don't know these things you should learn them. But you act as thought this article is a waste of the electrons it traveled over, while it is actually quite usefull. I program python, learned about test methodology years ago but didn't learn about these tools until today.
I don't see how you got +5 Insightfull for pointing out that this is article is not the all-inclusive source of testing knowledge. Everything ever written is limited in scope, that doesn't make it less usefull for the purpose it was written.
Guess what: appending an infinite number of items to a list causes a memory leak in Java, C++, C, assembly, Scheme, sh, Perl and every other programming language in the world.
Properly implemented, Scheme should never leak memory. Even if you try to add an infinite number of items to a collection, you'll get an out-of-memory error before you'd get a leak.
A deep unwavering belief is a sure sign you're missing something...
For example, it's easier to see what's going on in:
than in the equivalent C or Java code. Also, terseness isn't the ultimate goal--I could make this more terse (and less readable or efficient) by saying:
The biggest benefit of Python over Java, as far as I'm concerned, is that code written in Python looks like what it does--that's the source of the "Python is executable pseudocode" meme you often hear from Python fanboys. The lack of a compile step and the dynamic typing help, but they're secondary to me.
FWIW, I'm the author of the referenced testing article. Though that's not really germane to the following comment.
b.foster's misguided comment gives a link to a 1999 article about problems in Python with cleaning up circular references. Well, yeah: in 1999, Python 1.5.2 could leak on circular references. It was actually less of a problem in practice than advocates of mark-and-sweep collectors tended to complain about; but it really was a limitation. That was a long time ago.
Nowadays, Python still uses references counting as its primary collection mechanism. It's a nice system: fast, deterministic, etc. (despite what you might think, refcount *really is* faster than Bohm generational; and is well-tested as such by very smart Python developers). However, the last half-decade of Python versions also perform occassional checks for orphaned objects, and cleans them up too. The "problem" pointed to is of historical interest, at most.
Buy Text Processing in Python
Argh...the garbage collector does handle circular references automatically and has done so for at least two years. Weak references are something totally different and are used e.g. for caches.
How is the parent "insightful"?!
I wrote a 2300 word article for a column on Python! I didn't write a book. Well, actually, I did write a book, but it isn't the above article (and it's not about testing frameworks). It's certainly not a good idea to think my short article is the alpha and omega of testing. But I am confident that my article -does- address a topic that some Python programmers can benefit from. And other installments of _Charming Python_ each address similarly small, but useful, topics.
Buy Text Processing in Python
Unfortunately you had to use HTML to render an outline, when you'd have used PostScript if you had a choice. The reality of programming is the necessity of working with other people, including programmers thinking in the paradigm of the moment (think 1990s Perl), graphic designers/artists with Flash on the brain, marketers with buzzword colored glasses, legacy systems, and customers seeing their businesses modeled explicitly for the first time in any terms. I'd prefer compiling everyone's flowcharts to C++/IDL/VHDL, but that environment is at least 5 years away. Meanwhile, all we have it this year's hammer, and everything looks like a nail.
--
make install -not war
Sorry bud, I think I've heard that line before.
It is a very short article about using the unit testing framework for Python, not the definitive guide to testing. Do you even understand how amazingly large a subject software testing is?
I think that unit testing is an important tool, it allow you to easily continue to do integration test and regression testing, if you know what you're doing.
I will however agree with you on the "get a book on testing" part. I think it's important that anyone who develop software, understand what is involved in testing.
First off, I had mod points and I feel bad about not modding you up, and you don't deserve a 0:offtopic score.But I though a responce would be better.
Can someone help me understand why I would want to use Python as opposed to Java?
Python is not just about brevity, but I think you underestimate writing less lines of code. Take the fact that if you are writing twice as much code, you're probably writing twice as many bugs. Also a consise 50K LOC program is much easier to get around in than a 200K program.
But anyway, Python is an interpreted, interactive, object-oriented programming language., where as Java is only the last of those three. Python also has dynamic documentation, meaning that the documentation string can be inspected at run-time, without the need to maintain a separate doc tree, ie Javadoc.
Classes are dynamic ie. you can change their behavior at runtime. All python objects can be marshalled or serialized, while in Java you have to define the interface, and account for anything that you use that can't be serialized.
There are also thing which I am indifferent about like tabbed blocks, and some things which I don't like, like not having enough compile time type checking. IMHO you should always define the type of a function parameter, it makes learning the API faster.
In general, Python is great for script-strength stuff, ie. in situations where you would use perl. I can see it getting unwieldly as it got bigger but it generally scales much better then perl. Java tends to be too complex and too restrictive. In short there are many situations where you would choose one over the other.
Why, o why must the sky fall when I've learned to fly?
Here is a presentation, with PDF article, about integrating the two approaches, straight from recent PyCon:
Literate Testing: Automated Testing with doctest
Try reading ESR's Why Python?, also check out the Quotes on Python from python.org :)
It's a beautiful language, I'm sorry I can't code in it more...
Any technology distinguishable from magic, is insufficiently advanced.
I've been running Zope (implemented in python) on production servers for years. The zope process typically has uptime of weeks and downtime is typically only of the scheduled variety. In the past 20 months I've been at my current job, I can count the number of Zope crashes on one hand. OTOH, we also run Java app servers (formerly BEA, now Jboss) and they crash all the time... frequently with out-of-memory errors.
Snide remarks about "dirty little secrets" notwithstanding, I know which platform I trust to keep running.
You know, the funny thing about this is that a good coder must want to see their code actually run, right? After all, everyone wants immediate gratification. And the beauty of unit tests is it provides just that... you can actually run your code in a controlled environment easily without the need to start up an entire application/framework/etc. This is especially true of large and/or distributed applications where there's a great deal of overhead in running a full system test.
Moreover, unit tests make the test-develop cycle a *lot* shorter, especially on large projects, since you can easily run your code. Thus, you're wasting less time testing and more time coding... sounds like a great deal to me!
So, in my mind, any coder that doesn't feel like writing tests isn't, in fact, a particularly good coder. Or, perhaps, they're doing their job for all the wrong reasons. I code because I want to see the fruits of my labour, and unit tests are a great way to quickly and easily see the results of my work.