Python 2.4 Final Released

Eventh writes "The final release of Python 2.4 was just released. Python 2.4 is the result of almost 18 month's worth of work on top of Python 2.3. New features are, but not limited to, function decorators, generator expressions, a number of new module and more. Check out Andrew Kuchling's What's New In Python for a detailed view of some of the new features of Python 2.4. "

Comments

18 Months

[nijk]

It took them a while, but it's worth it. Here's some of the new features:

* multi-line imports - when using imports in the form from foo import bar, baz, bing, bang, you can surround the imported names with brackets, and they can be split across lines. This is part of PEP 328.
* Farewell to OverflowWarning - as documented in PEP 237, Python no longer generates OverflowWarnings.
* function/method decorators - function and method decorators, first described in PEP 318, have been added to the language, using 'pie-decorator' syntax. Decorators are on the line before the 'def', and prefixed with an '@' sign. (PEP 318)
* Assigning to None - the compiler now treats assigning to None as a SyntaxError.
* Failed import cleanup - when a module import failed, versions of Python prior to 2.4a2 would leave a broken module in sys.modules - subsequent attempts to import the failing module would silently succeed, but use the broken module object. The import machinery now removes the failing module from sys.modules if the import fails.
* The -m command line option - python -m modulename will find a module in the standard library, and invoke it. For example, python -m pdb is equivalent to python -m /usr/lib/python2.4/pdb.py

Re:18 Months

[MikeBabcock]

Its not that they're hard to parse, its that they're unnecessary and ugly.

if (x == y)
{
if (!do(something))
{
printf(failuremsg);
return -1;
}
return 0;
}

vs.

if x == y:
if not do(something):
print failuremsg
return -1
return 0

Linus' comments in the kernel's CodingStyle document are relevant too -- try to keep your functions so that you can read most of them on one screen. Python allows you to do this more often (I find) without resorting to strange brace positions.

Re:18 Months

[ReelOddeeo]

Linus' comments in the kernel's CodingStyle document are relevant too -- try to keep your functions so that you can read most of them on one screen. Python allows you to do this more often (I find) without resorting to strange brace positions.

My functions do each fit on one screen.

For any sufficiently large definition of screen.

function decorators

[koi88]

Finally I can make my functions so much prettier.

OK Trolls...

[Anonymous Coward]

If you can keep your "significant whitespace sucks" comments as children of this one, I'm sure everyone will get along just fine...

Re:OK Trolls...

[pclminion]

Do you really want to go through each line, scrolling all the way to the end because your language has significant whitespace?

There is absolutely nothing that says you can't break things across lines. In most cases you don't even need a backslash to escape the linebreak. TRY the damn thing before criticizing it.

Re:OK Trolls...

[asdfghjklqwertyuiop]

Say it's someone else's script. You'd have to write another script to process the original script to add the escape characters where they're needed and regular linebreaks where they're not.

Someone else's script? Written in what, python? Well then I don't need to process anything. It is either valid python or not. If you're just talking about including some pre-formatted text into a python script, then just enclose it in a multi-line string (which in python are inclosed by """ (three double quotes in a row)).

And I still think that if there's not a visible character there, it shouldn't mean anything. It goes against logic: Nothing isn't something.

I used to believe python's whitepsace-sensitive C syntax would really get on my nerves if I ever tried python, since I'm used to C and perl. Like many people, I wouldn't even give python a shot because of that whitespace issue. But one day I did because I needed to help a friend, and I actually found the language quite pleasant.

The whitepsace sensitivity didn't get in my way like I thought it would (which is to say it never got in my way, I didn't even notice it). Python's rules for whitespace are quite logical and intuitive and follow the way you're already used to indenting for readability in C or perl. When I started using python, I didn't even have to learn how the whitespace worked like I would have to learn any other syntax. The whitespace rules already worked the way I always programmed, so once I started python I had no trouble at all.

heh

[boschmorden]

import overlords

print "I for one welcome our new %s overloads.!" % overloads.get_random()

department of redundancy dept.

[Anonymous Coward]

The python final release of python 2.4 is finally released for python users out there who wish to use the final release of python release final.

Just in time for Xmas!

[FooAtWFU]

No! It's the Christmas season, so decorations are in right now! We should all get the new version of Python and decorate our source code to celebrate the spirit of holiday cheer!

cue grumbling about how Christmas is overcommericalized+etc

genexps

[ultrabot]

You forgot the most important improvement, the "generator expressions".

From the AMK's excellent (as always) overview:

print sum(obj.count for obj in list_all_objects())

The important part is that no intermediate list is generated, because we are dealing with generators.

Generators in general kick so much ass it's not even funny.

Re:genexps

[ultrabot]

It seems like 90% of these are created because lamdba sucks and def isn't anonymous.

Uh, you do realize that generators have nothing to do with anonymous functions, and that it's rather a simplified, understandable approach for providing coroutines? Generators are significantly more powerful than Ruby code blocks.

Re:genexps

[CoughDropAddict]

At a language level, it's true that generators don't have anything to do with lambda or code blocks. But when you consider what these features are used for, there is some overlap.

Consider the example you gave:

print sum(obj.count for obj in list_all_objects())

In Ruby, you can accomplish the same thing by writing:

print (objGenerator.map { |obj| obj.count } ).sum

where "objGenerator" is an object that mixes in the "Enumerable" module. An even better way (that really avoids building an intermediate list, even of integers) is:

a = 0
objGenerator.each { |obj| a += obj.count }
print a

There is one thing that generators give you that blocks can't, that is very, very cool. With a generator, you can create programs with a pipeline architecture, where different steps of the pipeline all can be written as if they have the main loop.

Imagine you are writing a compiler. You could write (be easy on my syntax, I haven't written Python in a while)

def lexer(input):
state = STATE_BEGIN
ch = input.read(1)
if state == STATE_BEGIN:
if ch == '(':
yield TOKEN_LEFT_PAREN
(...)

def parser(lexer)
token = lexer.get()
if token == TOKEN_LEFT_PAREN:
yield parse_expr(lexer)
# gobble TOKEN_LEFT_PAREN
lexer.get()
(...)

The beauty is that both your lexer and your parser can be written as if they have the main loop, and all they have to do is yield a token/expression when they find one. In reality, it's pipelined and you get the efficiency of not having to build up the entire list of tokens before you start parsing.

Re:genexps

[CoughDropAddict]

Oops, I realized that probably the best way to do this in ruby is to use inject:

print objGenerator.inject(0) { |sum, value| sum + value }

No intermediate list, and you don't have to rely on having a function like "sum" defined -- it could be any operation.

You could also add this to the "Enumerable" mixin:

module Enumerable
def sum
return inject(0) { |sum, value| sum + value }
end
end

Now you can call "sum" on any object that implements "Enumerable", and the example becomes:

print objGenerator.sum

generator expressions?

[deathazre]

2,500KVA @ 0.8 PF? (note: some electrical background required to appreciate the above)

On Decimal and floating point

[Frater 219]

The What's New document gives a somewhat inaccurate description of the importance of the Decimal type. Naturally, switching from binary to decimal changes which fractions can be exactly represented, but it does not change the fact that some cannot.

The importance of decimal arithmetic is not that it is more accurate than binary, but rather that it conforms more closely to what people expect from using decimal in daily life. These expectations are codified into various social rules such as accounting practices.

There's been some heated discussion of Python 2.4's Decimal, in this very regard, on the Lambda the Ultimate languages blog.

Re:On Decimal and floating point

[pkhuong]

So, obviously, we should be using base (apply '* *n-first-primes*). How about 30 or 210? At least it's not an arbitrary base.

Re:On Decimal and floating point

[Frequency Domain]

So, obviously, we should be using base (apply '* *n-first-primes*). How about 30 or 210? At least it's not an arbitrary base.

It doesn't matter which base you use. All your base are belong to us.

frozenset

[hey]

Instead of having a special keyword for immutable sets (frozenset) wouldn't it be better to have an "immutable", "final" or "const" keyword?!

Re:frozenset

[Anonymous Coward]

in short, no.

python doesn't have modifiers for anything else, so it would require syntax changes. Whitespace after a python reserved word seperates it from its expression.

"frozen x = Set()" complexifies things because since day 1, an assignment returns 'None', and 'frozen None' is nonsense.

You could of course write it as a property:

x = Set()
x.become_frozen()

But it's better to consider 'frozenset' as a subclass of 'set' (or vice versa), since 'frozenset' very well fits the 'is a' paradigm, because a frozenset IS A type of set!

Dive into Python

[cbelle13013]

I had absolutely no interest in Python until I read Slashdots review of Dive Into Python. Its right on target and got me excited to run home and see what I could do. Of course that only lasted for about a month, but I'll be sure to head home and take a peak at it again.

Re:I *want* to be enthused, but...

[ultrabot]

after Linus's comments I am inclined to get more profficient with Bash and C and almost ignore Python completely. It's so dissapointing though - I really wanted to learn Python; it's such a neat language.

Linus explicitly mentioned that he doesn't do anything "in the middle" - it's either kernel hacking or something trivial enough to do with bash. Just go ahead and learn Python - you will find that it's *easier* than bash, especially if your programs might have errors (which they do).

BTW, why would you want to get more proficient in C? Programmers are abandoning C in droves. It's just not programmer-time efficient to do things in C anymore. It's one thing if you are maintaining a project that was written in C originally, but for new projects, C is a non-starter.

Go read ESR's "The Art of Unix Programming", available online for free.

Is it just me?

[digitaltraveller]

Or has python strayed from it's original philosophy of 'one best way to do it'?

I used python in the 1.5 days. The syntax was incredibly clean. Nowadays the language has tremendous idiomatic power, any programming language researcher should be familiar with it.

But that power has brough alot of complexity. At the end of the day, languages are tools and the learning curve to understand (particularly others) python code seems to be increasing.

Re:Is it just me?

[pclminion]

If a newbie writes something that can be re-written with a list comprehension for instance, it's pretty much given that nobody will actually answer his question, choosing instead to re-write his program for him using tricks and shortcuts.

List comprehensions are not "tricks," they are an extremely powerful language feature. Newbies should be taught how to use them, and translating their loop-based code into list comprehensions is probably the best way to do that.

Python can be written from either an imperative standpoint or a pseudo-functional one. Most of the highly skilled Python programmers code in the pseudo-functional style, because it is more efficient and (arguably) more elegant.

Sure, you can get into some pretty scary territory with various combinations of sum(), map(), reduce(), and list comprehensions but that's your choice. I admit that there should not be such a big performance gap between the different styles... This is due to not enough effort being spent on improving the VM.

Re:Is it just me?

[Defiler]

Take a look at this, and the very elegant Scheme dialect that was designed to go along with it.
How To Design Programs
One semi-unique feature is that it asks you what your skill level is when you launch the environment, and tailors the feature-set accordingly.
Novices see a simple subset of the language, and experts get the full power inherent in a real-deal Lisp dialect.

Re:Is it just me?

[Jerf]

Or has python strayed from it's original philosophy of 'one best way to do it'?

To a degree, yes. Largely to the extent it has it is a result of backwards-compatibility; while the Python designers do not make the mistake of enshrining reverse compatibility above all else, they do try to avoid gratuitously compromising it.

As a result, as better ways to do things have emerged, sometime the old ways hang around and muss things up. However, for any given version there is always a "core" that you can stick to that is very close to "one best way to do it"... and despite what a first reading tells you, you really don't throw much of the language away.

For instance, while in modern Python you can say either

a = []
for i in range(5):
a.append(i*2)

or

a = [i*2 for i in range(5)]

the latter is the "one right way to do it". Few, if any, new additions truly leave you with two equally good choices.

(One of Guido's examples is the "lambda" statement, but a lot of people, including me, rather like not needing to replace

self.register("event", lambda event: self.keyPressed())

with

def handler(event):
self.keyPressed()
self.register("event", handler)

but hey, that's life. (While that isn't code for any GUI toolkit in particular that is a pattern common to all of them.))

By and large, none of these things have affected the difficulty of learning Python from scratch and using its libraries. It has affected the difficulty of reading other people's code, but I find the alternative, "keeping the language stagnant indefinately", completely unacceptable, and frankly, reading code is hard anyhow. (I was writing code for others long before I was reading other's code.)

In fact, given as that is the alternative, "keeping the language stagnant indefinately", while I concede it is somewhat sad that we can't jump to an optimal language immediately so that nobody ever has to learn anything past their first impression (not sarcastic, that would be the ideal), that doesn't seem to be working for folks. You might try LISP, though, I gather that hasn't changed syntax, much. (Though I also gather mature programs in that language tend to start looking like their own languages themselves, so that just may move the pain...)

Re:Is it just me?

[tungwaiyip]

The 'one best way to do it' mostly mean basic language operations, like iterating over a list. Many recipes in the Python cookbook are really mildly complex utility programs. They are differ in features, complexity, performance and intented use. The are naturally many ways to do similar things. And if you take these variables into account, you will find they are not really doing the 'same' thing.

Since Python 2.x there is a move toward iterators and generators. Rather than seeing this as bloats I would say this Python is maturing to handle real world applications. If you are handling simple data and performance is not an issue, use regular lists. If you writing application to handle arbitrary large data set or performance is an issue, go for iterators and generators. List comprehension is a neat language construct but I don't see any need to rush rewriting anything.

Re:Pah

[Anonymous Coward]

python -c 'print "55 5 5";'

Re:I *want* to be enthused, but...

[oexeo]

BTW, why would you want to get more proficient in C? Programmers are abandoning C in droves. It's just not programmer-time efficient to do things in C anymore. It's one thing if you are maintaining a project that was written in C originally, but for new projects, C is a non-starter.

I think one of the problems is too many people are still spreading the myth that it's essential to learn C before moving on to C++, which is totally false, C++ is a language in itself, and can be treated as such. Learning C (unless you intend to use it) is a waste of time, and I would go so far to say learning C first will make you less producive in C++, because it teaches concepts which are not applicable, or are actually bad habits when used in C++. At least that's my view on the subject.

That said, C still has its uses, but for many projects (like parent said) it's a "non-starter"

Re:I *want* to be enthused, but...

[Anonymous Coward]

Here's a novel idea: Use the right tool for the job. There's a very large class of problems for which neither c nor bash are particularly appropriate programming languages. I suspect that Linus was attempting to comment on the partcular classes of problems on which he typically works, not on the merits of any specific programming language. If everything you're doing now can be done with bash, then use bash. However, don't rule out Python just because Linus doesn't use it. One day, you'll need to do something that's impractical with bash, and you'll find Python quite useful.

I use Python extensively in my day-to-day work. It's the most versatile tool in my toolbox, and I'm continually amazed that so few people, at least around here, use it.

Re:Python is a pathetic language.

[tuffy]

I dare python lovers to write something like that in python in one line.

for f in glob.glob("/tmp/*").sorted(): print f

Though you will need to import glob in a seperate line. But I fail to see how doing trivial things trivially is a helpful measure of a language's usefulness.

Re:Writing extensions...

[imroy]

The Boost project has Boost.Python. I haven't used it yet, but the docs make it sound very interesting. It looks quite simple to write base classes in C++ and then subclass them in Python.

Re:Supprised

[kirkjobsluder]

Can one tell me why I should learn python and not any other programming language anyway?

Well, some of the things I like about python:
1: tightly structured code.
2: less punctuation.
3: more readable syntax.
4: full OOP from the ground up (in contrast to perl where the OOP is bolted on with references).
5: short production cycle.

Many of these things can be found in other languages as well. So it largely comes down a matter of preference.

From the little I have seen, python seems to be a command line language. Is it anywhere similar to Visual Basic, which I have come to see and experience through a GUI?

Check out tkinter and wxPython.

Re:Sets in Python

[tuffy]

Sets can operate on anything. But you need to pass in a list of those things to build a set out of, such as:

>>> s = sets.Set(["foo","bar"])
>>> s
Set(['foo', 'bar'])
>>> 'foo' in s
True

Not a Python Programmer...

[pragma_x]

...but I am impressed with the whole funciton-decorator addition to the language.

Function Decorator Proposal/Specification

Its nice to see a language evolve in favor of use without sacrificing readability and overall utility.

Before:

def foo(cls):
pass
foo = synchronized(lock)(foo)
foo = classmethod(foo)

After:

@classmethod
@synchronized(lock)
def foo(cls):
pass

Readable is of course in the eye of the beholder... and I am a C and Java programmer with a sizable fetish for D. So while I don't find the syntax all that pleasing to me in terms of my own work, it certainly changes things for reading python code.

I was also stunned to learn how flexible decorators were in the previous version of python. It's refreshing to be see functions treated as objects... unless I'm mistaken about the concept.

However its a huge shame that the new decorator syntax isn't supported for classes in 2.4. Seems like that's going to become a wart on a rather consistent language syntax, IMO.

Re:CommonLisp for the 21st century?!

[Frater 219]

How do you return an anonymous function from a function in Python?

You don't use lambda. You define a function (with def) under a temporary name and return it. The temporary name is local to the enclosing function, so it doesn't pollute the namespace: Python def differs in this regard from CL defun.

But seriously, Common Lisp is the Common Lisp for the 21st century. It's doing quite nicely; in the past couple of years folks have taken this "old" programming language and added excellent support for all kinds of current needs -- a Web application server (TBNL), a remarkable SQL interface (CL-SQL), general-purpose dataflow programming (Kenny Tilton's Cells), PDF typesetting and generation (CL-PDF, a rival to TeX!), OpenGL ....

(Peter Seibel's Practical Common Lisp is highly recommended for the "modern" programmer interested in this "old" language. Practical examples include spam filtering, an MP3 database, and a Shoutcast server.)

sigils sneaking in

[CatGrep]

So now that Python is using the '@' character for function decorators the Pythonistas can no longer crticize other languages like Ruby (which uses sigils responsibly) for their use of '@' (I'm not including Perl in the list of languages that use sigils responsibly, btw :)

Welcome to the sigil club! Pandoras box is now open! '$' can't be too far behind :)

Re:sigils sneaking in

[jkujawa]

Done.

Re:Writing extensions...

[William Stein]

One can write extensions in Pyrex, which is a language that is very similar to Python. Pyrex code is converted to C, which is ccompiled into an extension to Python. One can also access any C libraries from Pyrex.

Re:CommonLisp for the 21st century?!

[geg81]

Python looks NOTHING like common lisp.

Python is a lot like CommonLisp: dynamic typing, reflection, eval, lexical scoping, extensive iteration and looping constructs, strings-as-sequences, and on and on.

For once thing is the "big difference" you describe: you can't transparently process code as data. That means no MACROS, which is what makes Lisp so damn powerful.

Go back and read the original papers on hygienic macros: you don't need Lisp syntax or code-as-data in order to have a macro facility as powerful as Scheme's (and Scheme got by for many years without macros anyway). I wouldn't be surprised if Python at some point gets a hygienic macro facility. Furthermore, there is a separate data syntax for Python that takes source that looks very similar to Python code and represents it as a DOM tree.

How do you return an anonymous function from a function in Python?

In the obvious way:

A ``def'' form executed inside a function definition defines a local function that can be returned or passed around. Free variables used in the nested function can access the local variables of the function containing the def.

That part is actually more natural than doing the same in CommonLisp (Python, like Scheme, but unlike CommonLisp, does not have separate value and function slots on symbols).

How do you build a function at run time? It's not easy or obvious.

There are two things you could mean by that. The first is to build it from source or structure. You can do this:

exec "def c(x): return x*x*x"

The second is to build complex functions using functions that take functions as arguments. You do that the same way you do in CommonLisp, since Python supports the same primitives: lexical closures, dynamic typing, and functions that take functions as arguments.

I think people see "lambda" and they somehow think that Python has something to do with Lisp.

I think Lisp zealots incorrectly think nothing that isn't exactly Lisp could even come close. Python, in fact, is very close to Lisp; the two big differences are syntax and lack of macros. Lack of macros can be addressed, and there are separate Python-like syntactic representations of data.

WTF?

[ArbitraryConstant]

Linus is not some kind of deity. He (by his own account) does nothing between kernelspace and small scripts.

I'm very proficient with C, Python and Bash, and I can tell you for a fact: Bash and Python barely compete against each other.

Bash has nothing in the way of nice datatypes. Bash is very slow while Python can be nearly as fast as C (I've gotten it up to 80% when relying heavily on C libraries like regex). Bash can't do more than the most trivial things without helper programs, which while useful, takes forever because it has to keep spawning processes.

For a high level language, Bash has pathetic memory management. Pretty much the only way to get some things done is tempfiles, which is worse than malloc because they're not removed if you don't clean them up.

If you can't hold more than two languages in your head, go home and learn Java and C#. You're only going to get made fun of on slashdot.

python - awesome

[hashmap]

In some of my last projects I had to analyze a Perl and a Java program. I programmed a few years in both of these, but now after a year with Python I was truly suprised of how primitive these languages felt.

All those funny symbols, casting back and forth in perl just getting in the way yet don't really say anything useful ... here is an example:

foreach my $val(@{$G->{spec_val}}) {
...@{$G->{species}} = $G->{dbh}->selectrow_array($G->{sth_species},undef ,$val);
...if(@{$G->{species}}) {
......$G->{spec_label}{$val} = $G->{species}[0]. '; '. $val;
...}
}

whether or not this is good code is not the point, I have to make it work, look at all that pointless markup, in python this same thing would look like this:

for val in G.spec_val:
...G.species = G.dbh.selectrow_array(G.sth_species, None, val)
...if G.species:
......G.spec_label[val] = G.species[0] + ';' + val

(leading . stands for a space )which version would you rather read?

or that uselessly verbose java where you have to write X number of lines before any action starts ...

Python is a simple, clean and powerful language where the real value comes tomorrow or next month, when you have to understand and modify what you wrote today. There are no objective measures of this quality you have to try it to believe it.

Re:Supprised

[Junks Jerzey]

Can one tell me why I should learn python and not any other programming language anyway?

I expect this isn't the answer that true Python devotees would express, but here goes anyway: It's a very high-level, very dynamic, language that's reliably cross-platform.

"Very high-level" differentiates it from Java, which I see as more mid-level. It's also different than Perl 5, which is higher-level than Python in some ways, but convoluted and crusty in other ways (anything involving nested data structures, for example).

"Dynamic" means you can test code interactively, you don't a build process, you don't waste time enumerating things and creating redundant headers and so on.

"Reliably cross-platform" is the key. This is where Scheme and Lisp and Haskell fall down. Lisp has a standard definition, but the community is fractured by there not being a standard implementation. You can argue that diversity is good and all that, but it does tend to hurt overall. Python has a huge number of standard library modules as a result.

You need to learn the language before complaining

[tungwaiyip]

You example is a very powerful one liner!

You need to learn a few Python idioms. Anyone who work with Python for a while should have no problem understanding this.

1. x and y or z is Python's version of ternary operator. If is similar to x ? y : z in C.

2. " ".join(list) is similar to string.join(" ", list). I won't go into detail here but your expresion collapse a string with multiple whitespaces into single space.

3. The lambda construct is a function factory. Thing of callback in C but Python allow for much more elegant functional style programming.

The entire expression create a function which can collapse or not collapse a string base on you choice. The best thing is to see it in action with two more lines:

for line in file('xyz.txt'):
print processFunc(line)

This will output a files with whitespace collapsed or unmodified depends on the collapse flag.

Every language has its syntax and idiom to learn. Is the syntax below readable? Yes if you actual know the language.

for (i=0; in; i++) {

If you expect 0 learning and that each line of Python maps to something you are familiar with such as C, that's not what Python really is.