Slashdot Mirror
Core Python Programming
Ravi writes "Python, the dynamic object oriented programming language created by Guido van Rossum is known to deliver both the power and general applicability of traditional compiled languages without the complexities accompanying them. Coupled with its ease of use, programs written in Python can be run on multiple Operating systems and system architectures which gives it the same portability of any interpreted language. My first brush with Python was when I tried out a beautiful game called PySol — which is more like a collection of over 200 card games and PySol is entirely coded using the Python language. Nowadays a variety of Python web frameworks have also cropped up which promise the same kind of rapid application development that is possible using other programming languages." Read the rest of Ravi's review
Core Python Programming - 2nd Edition
author
Wesley.J.Chun
pages
1050
publisher
Prentice Hall
rating
9
reviewer
Ravi
ISBN
0-13-226993-7
summary
An excellent book to learn the Python programming Language
I found the book titled "Core Python Programming" authored by Wesley.J.Chun and published by Prentice Hall to be an ideal book to learn the wonderful Python language. This book is quite voluminous, with 23 chapters spanning 1050 pages. The book is divided into two parts the first part titled Core Python which contain 14 chapters which impart a sound understanding of the semantics of the language and the second part titled "Advanced Topics" which contain a collection of 9 chapters which give a good introduction to the specialized uses such as in database programming, network programming, using threads in python, GUI programming and so on.
In the first chapter of the book, the readers get to know the various features of Python and the steps needed to install Python on ones machine. When you install Python on your machine, it also provides its own shell where you can execute pieces of python code. The author has taken advantage of this functionality of Python in narrating the concepts throughout the book. Each concept and syntax is followed by bits of code which the readers can try out in the Python shell in its entity. I found this process much easier in learning this language as one need not go through the write — compile — execute cycle which is more prevalent in the traditional languages.
In depth coverage has been provided for important concepts such as lists, tuples and dictionaries as well as data-types and string sequences and they have been provided separate chapters of their own. The sixth chapter titled "Sequences: Strings, Lists and Tuples" is the second largest in the book and is quite detailed in its coverage of the topic.
Chapter 9 deals with file manipulations where the author introduces all the built in functions available in Python which allow one to open, read from and write to a file. Interestingly, the functions are also illustrated by aid of short easy to understand examples. A couple of modules related to file handling are also introduced in this chapter.
Errors and exceptions form the basis of the 10th chapter where different errors and exceptions supported in Python are explained. This chapter also explains how programmers can create custom exception classes which I found quite informative.
One of the biggest advantages of Python is that all its functionality is split up into modules. A module could be just a single python file which contain a collection of functions or classes which can be re-used in programs written in Python. And all one has to do is import the module into ones programs to start using those pieces of code. The chapter 12 titled Modules give a firm understanding of this concept and also introduces different ways in which you can import external pieces of code into the Python program.
Chapter 13 titled "Object Oriented Programming" is by far the largest chapter in this book and spans over 100 pages. In this chapter, the author endeavors to give a sound base to Object oriented concepts as well as how they relate to programming in Python. The author introduces a large number of Python classes, methods and descriptors in this chapter.
Regular expressions play a very important part in programming verily because manipulating text/data is a necessity. And it is possible to easily modify and mould data to ones choosing by way of regular expressions. Python has strong support for regular expressions. The second part titled "Advanced concepts" of the book starts with a chapter on regular expressions. In this chapter, one gets to know about the regular expression module and many functions associated with the module. The author also provides a couple of examples which give insights into the ways in which regular expressions can be used in Python to reformat data.
The next two chapters give an introduction to the world of sockets and how Python can be used to write client server programs.
Multithreaded programming forms the basis of the 18th chapter. Here the author introduces a couple of modules available in Python which make it quite easy to create threads in ones Python program.
I found the chapter titled "Web Programming" very interesting to read. Especially since Python is used in a large way to create dynamic websites. And the next chapter titled "Database programming" gives a sound introduction to the Python objects which allow one to easily connect to and retrieve data from the databases.
I found this book really informative, especially suited for upcoming Python programmers. At the end of each chapter, there is an exercise section which imparts a nice touch to this book as it allows you to test your knowledge. Even though the advanced topics (second part of the book) are not covered in depth, the author succeeds in providing enough knowledge about the relevant Python modules and functions followed by a couple of examples which whets ones appetite without overwhelming the reader. This is the second edition of the book and it has been significantly revamped to include new features introduced in Python 2.5.
The author Wesley J. Chun has been a former employee at Yahoo and has played a major role in creating Yahoo Mail and Yahoo People Search using Python. He has over 20 years of experience in the IT field with over a decade of experience programming in Python.
Ravi Kumar likes all things related to open source and free software and enjoys sharing his experiences and thoughts through his blog All about Linux.
You can purchase Core Python Programming - 2nd Edition from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
I found the book titled "Core Python Programming" authored by Wesley.J.Chun and published by Prentice Hall to be an ideal book to learn the wonderful Python language. This book is quite voluminous, with 23 chapters spanning 1050 pages. The book is divided into two parts the first part titled Core Python which contain 14 chapters which impart a sound understanding of the semantics of the language and the second part titled "Advanced Topics" which contain a collection of 9 chapters which give a good introduction to the specialized uses such as in database programming, network programming, using threads in python, GUI programming and so on.
In the first chapter of the book, the readers get to know the various features of Python and the steps needed to install Python on ones machine. When you install Python on your machine, it also provides its own shell where you can execute pieces of python code. The author has taken advantage of this functionality of Python in narrating the concepts throughout the book. Each concept and syntax is followed by bits of code which the readers can try out in the Python shell in its entity. I found this process much easier in learning this language as one need not go through the write — compile — execute cycle which is more prevalent in the traditional languages.
In depth coverage has been provided for important concepts such as lists, tuples and dictionaries as well as data-types and string sequences and they have been provided separate chapters of their own. The sixth chapter titled "Sequences: Strings, Lists and Tuples" is the second largest in the book and is quite detailed in its coverage of the topic.
Chapter 9 deals with file manipulations where the author introduces all the built in functions available in Python which allow one to open, read from and write to a file. Interestingly, the functions are also illustrated by aid of short easy to understand examples. A couple of modules related to file handling are also introduced in this chapter.
Errors and exceptions form the basis of the 10th chapter where different errors and exceptions supported in Python are explained. This chapter also explains how programmers can create custom exception classes which I found quite informative.
One of the biggest advantages of Python is that all its functionality is split up into modules. A module could be just a single python file which contain a collection of functions or classes which can be re-used in programs written in Python. And all one has to do is import the module into ones programs to start using those pieces of code. The chapter 12 titled Modules give a firm understanding of this concept and also introduces different ways in which you can import external pieces of code into the Python program.
Chapter 13 titled "Object Oriented Programming" is by far the largest chapter in this book and spans over 100 pages. In this chapter, the author endeavors to give a sound base to Object oriented concepts as well as how they relate to programming in Python. The author introduces a large number of Python classes, methods and descriptors in this chapter.
Regular expressions play a very important part in programming verily because manipulating text/data is a necessity. And it is possible to easily modify and mould data to ones choosing by way of regular expressions. Python has strong support for regular expressions. The second part titled "Advanced concepts" of the book starts with a chapter on regular expressions. In this chapter, one gets to know about the regular expression module and many functions associated with the module. The author also provides a couple of examples which give insights into the ways in which regular expressions can be used in Python to reformat data.
The next two chapters give an introduction to the world of sockets and how Python can be used to write client server programs.
Multithreaded programming forms the basis of the 18th chapter. Here the author introduces a couple of modules available in Python which make it quite easy to create threads in ones Python program.
I found the chapter titled "Web Programming" very interesting to read. Especially since Python is used in a large way to create dynamic websites. And the next chapter titled "Database programming" gives a sound introduction to the Python objects which allow one to easily connect to and retrieve data from the databases.
I found this book really informative, especially suited for upcoming Python programmers. At the end of each chapter, there is an exercise section which imparts a nice touch to this book as it allows you to test your knowledge. Even though the advanced topics (second part of the book) are not covered in depth, the author succeeds in providing enough knowledge about the relevant Python modules and functions followed by a couple of examples which whets ones appetite without overwhelming the reader. This is the second edition of the book and it has been significantly revamped to include new features introduced in Python 2.5.
The author Wesley J. Chun has been a former employee at Yahoo and has played a major role in creating Yahoo Mail and Yahoo People Search using Python. He has over 20 years of experience in the IT field with over a decade of experience programming in Python.
Ravi Kumar likes all things related to open source and free software and enjoys sharing his experiences and thoughts through his blog All about Linux.
You can purchase Core Python Programming - 2nd Edition from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Syntax wise it seemed like a fresh change from C-style languages. I'd use it again if I had a new project to do.
One might try out Alan Gald's fine Learning to Program site. Short on pretty. Long on usefulness.
Oh, and all the thick books in the world will never help a budding python coder more than the tutor mailing list.
I'm just starting to learn programming, starting with Python. It's actually very easy, but you can do quite a lot with it. I also like that the interpreter's a small download, and easily available on the 3 different OS's (Windows for gaming, Mac at school, and Linux for work at home) a I use daily, which isn't the case, for, say, BASIC (As far as I've found).
has documentation too...
Reality is nothing but a collective hunch.
Sometimes it's worth spending nights writing and optimizing C programs to gain just a few milliseconds, sometimes you just need to write a small script in either Python or Ruby to automatize a task you'll never do again.
In this case, would you really care about execution speed? I'll just go for the ease of string/files/array/hashes manipulation, write a powerful script in 10 minutes and go drink a coffee while my script works for me and you're still struggling with your char str[80];.
Then dont pretend that Python is an all purpose programming language
and admit/accept it for what it really is : An overhyped, scripting
language, a better PERL in some aspects if you will.
People who complain about speed issues with Python are not creating
3d engines or trying to squeeze milliseconds out of their algorithms.
They fall for the hype and try to create general-purpose applications
only to be extremely disappointed in the end (that is if they are objective)
Unless Chun has changed the focus a lot, the book takes a fairly introductory stance on things. There's a lot of material explaining programming practices, OO and so on. If you're experienced with C/Perl/*nix development, you're probably better off with an O'Reilly book, or something else that has more reference material.
Where's the review? This was a sketchy synopsis. How about the writing style? Does the code use modern idioms? What did the examples cover -- are they real-world or contrived? Hell, is the typeface clear?
It's no wonder I skip straight to the comments -- these reviews are just about the most content-free section of slashdot.
Done with slashdot, done with nerds, getting a life.
Q: How many Python programmers does it take to screw in a lightbulb?
A: That joke's not funny.
After a year of fiddling with Python, I find I quite like it for some things, and it's not the right fit for other things. That's fine, there's no reason to be rabid pro- or rabid con- about any language.
I happen to use Linux, OSX and Windows, and the number one annoyance for me is a lack of clear "native-looking" GUI or OpenGL toolkit offering that is reachable by scripting languages like Perl and Python.
I'd like to see PyGame in a MacPort. The Images module was pretty easy to build but it should be in MacPorts too. I don't want to rely on X11 when on a Mac, but to use X11 when on Linux, which seems to be like bringing antimatter and matter together in the same universe. A GladeXML-like module that relies only on Aqua would be great.
Similarly on Windows, you're relegated to some fugly Tk choice if you want portable GUI. I can't think of a toolkit that looks less native than Tk, unless you count the early Java Swing "Metal" theme.
The fact that OSX still isn't standardized on python2.4 makes it hard to support python-rich applications like Blender, and we still run into problems where a 2.3 parser can't let scripts use 2.4 features (even smart scripts that try:except: to fall back gracefully). I think Leopard will be python2.4 but don't recall seeing official confirmation on that.
I'm sure I've misstated something technically-- I don't have my notes from home to double-check a few things from my last gui toolkit search a couple weeks ago.
[
Guido van Rossum himself as well as Fred Drake wrote an amazingly concise and complete introduction to Python. This book will get you up and running quickly. The book is called "An Introduction to Python" and it can be found here (free html, a $5 pdf, or isbn):
http://www.network-theory.co.uk/python/intro/
This book is only 100 pages long! It was written for Python 2.2.2, but it is perfectly useful for any Python 2.x. This book is amazingly concise, but it is also quite complete. You will get the most out of this book if you have a background in C/C++.
I am not affiliated with the book in any way, just a satisfied customer.
Not necessarily a "char str[80]". Many people forget that C++ isn't just "C that lets you do proper OOP", it's a better C and plenty of string classes, file access classes, array and hash classes exist for C++.
/. crowd familiar with enough to whip up something almost as quick as you can in Python.
Of course I doubt any of them are as easy to use as Python... but I wouldn't mind betting there are some C++ library guru's out here in the
Save some money:
Amazon.com $32.99
Overstock.com $31.34
Half.com $28.95
Regarding the indentation-as-syntax, consider the following C code, where someone tried to add the baz() function to a loop:
while (foo())
bar();
baz();
This code has an obvious bug. Baz() is not in the loop body, because the braces are missing. It's obvious from the indentation that that was the coder's intention. Why does one need an orthogonal and error-prone syntax for marking basic blocks, when the formatting already makes the intention obvious?
Or, consider this example from the paper Algorithm Education in Python:
Here is the Insertion Sort algorithm exactly as presented in Cormen, Leiserson, Rivest, and Stein's "Introduction to Algorithms":
Insertion-Sort(A)
for j <- 2 to length[A]
do key <- A[j]
i <- j - 1
while i > 0 and A[i] > key
do A[i+1] <- A[i]
i <- i - 1
A[i + 1] <- key
Here is the same algorithm, implemented in Python:
def InsertionSort(A):
for j in range(1, len(A)):
key = A[j]
i = j - 1
while (i >=0) and (A[i] > key):
A[i+1] = A[i]
i = i - 1
A[i+1] = key
I haven't left anything out. Paste that into a Python interpreter and run it:
>>> x = [2,7,3,8,1] # create test case
>>> InsertionSort(x) # call routine
>>> x # look at result
[1, 2, 3, 7, 8]
There are about a thousand other cool things, and of course there are inevitably some warts and poor design decisions too. Bottom-line for me, at least, it's the power of Perl, but at the extreme other end of the spectrum in terms of readability. The Google folks think so too.
And Java is far slower yet it gets all the ink why? Because Sun was seen as an upstart taking on Microsoft in the days before open source was the big thing, and now Sun is not so loved. In the aftermath of the hullaballoo, Java remains a slug of a language implementation.
Meanwhile, I can whip up apps in Python in no time and they run far faster and with less nonsense and hassle. Slower than C? Yes. Better for rapid prototyping and quick problem solving on the fly? Yes. Best language I know to bridge structured and object oriented paradigms? Yes.
If I haven't already bought this book, and I may have and just not remember given the stacks of them on my various desks, I will then do so and give a copy to my godson who is just learning about computers. The archaic school system might still be teaching LOGO and I want him to see something that he can actually use.
If my grammar and spelling are off, I am [distracted/tired/careless] (take your pick)
Python and its retarded syntax, what the hell where they thinking?
# First, as far as web programming is concerned, the users are always slower than the slowest of all programming languages : they read the text, fill in the forms, etc.
What matters is how fast responses are once forms are submitted.
# Second, as Python is used in games to implement AI, so I guess it isn't THAT slow.
Python is almost always (when used in this circumstance) used to script what happens in game AI, not the AI itself. When Python is used for such coding, performance-critical functions are usually implemented in C or C++.
Third, "scripting language = slow" has been proven false for long now...
If this were true, then developers needing performance would not need to implement extensions in C or C++.
I really, really wanted to like Python. I even got over the ridiculous whitespace issue. But the slowness issue was definitely what killed Python for me. It is just god-awful slow. For equivalent tasks, my Java code is faster than Python. My Tcl code is faster than Python. Even my Ruby code is faster than Python. Until Python gets a lot faster, I won't be using it for anything. Speed isn't always the most important factor, but with Python the slowness is just too much to ignore.
It really depends on the application. Python is really easy to use and for applications that aren't cpu/io intensive it could prove better to use over a language such as C just because it's much easier and quicker to put together something relatively small together. However, once the requirements start to grow it's a completely different scenario. As soon as you're going to be writing an application that's going to take a few seconds to execute an operation, then a faster language such as C might be a better solution.
I am quite involved in algorithm competitions and have for the past couple of years started helping in the setup of a few. One of them recently introduced Python as amoungst the original C/C++, Java and Pascal. From this I got a good experience in the speed differences between the four languages.
C/C++ and Pascal have always been pretty much neck-and-neck as the fastest, with Pascal just loosing out. Java used to be about 3-4 times slower, but it has been improving over the past couple of years and now with 1.5 it is about 1.5 times slower than C/C++ and Pascal. Not too bad when you take everything into account. With Python however, the gap varies tremendously. In some cases it's less 3-4 times slower (comparing to C/C++ from here on), but in some cases it's as bad as 100 times slower!!! And that's taking the exact same algorithm, the exact same code even and translating it between the four languages!
My conclusion is that while Python is slow, the decision as to whether to use it or not depends on the problem.
...and it need not be said.
Between Pyrex and Psyco, there's really very, very few applications that a language like Python isn't appropriate for. Premature optimization is the far more common programmer sin these days than choosing a language that's "too slow." Except for a few, specific application domains (that only a minority of coders are writing) dynamic languages like python are an excellent choice on today's hardware. And if you need it to be faster, just profile, find the couple of spots that matter, and pull out Pyrex.. or even the C/Python API if it makes you feel manlier. :-)
Plus, with things like PyPy + LLVM, Parrot, and IronPython emerging, things are only going to get better.. don't be the last one on the dynamic boat!
Maybe you shouldn't take the algorithm and use it direct? Each language has a way of doing things and some special methods for major speed boosts. Look at the number of python programs around coding up the usual way and never any list comprehension in site which would dramatically speed that part of the code up.
java is quite competitive these days re: execution speed.
Do keep up old chap.
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
If you've ever written code before Dive into Python is the way to go. It is free on the web and also available in print from APress. I am a big fan of books of this style which assume that you already have some knowledge of basic programming and focus on the specifics of the new language.
You really are a literary genius, have you thought about writing childrens novels?
I had a couple of people check the code in the case where it was 100 times slower and the best they could do was reduce the time from 30 seconds to 28 seconds. On top of it none of the students that participated could get their Python solutions any faster. So it wasn't the case of not using the most efficient functionality.
Its simply not true that Java is "far slower" that Python. And Java gets "all the ink" because its not only a fast language, but also has proven again and again it can scale very well on many different levels.
Java has a very very fast and advanced virtual machine much much faster than python's. The advantage python has is that a lot of the libraries are implemented in c where java implements most in java itself. A lot of what is already implemented in the operating system java implements it again.That makes also the startup time of java very slow (it is like booting up a new computer). But simply saying java is slower than python is not true at all.
It is worth looking at the way that Python is used in the game industry. Quite a lot of shops use it in a heterogeneous environment where the core functionality, e.g. graphics, physics, control/hardware, etc are in highly optimised C with inline assembly but Python acts as a glue pulling them all together and scripting most of the actual gameplay. There are books on this, but it is more than just theoretical. I interviewed at a shop that does exactly this for the console games they produce.
P.S. which algorithm competition allows python? really curious since I have always been pissed that these competitions don't accept languages like bash/ruby/python.
Ruby faster than Python? Hogwash.
Ruby is easily, measurably, slower than Python - its a big topic of discussion in the Ruby community and the subject of numerous projects to address.
For anyone who has ever programmed before, the O'Reilly Python Pocket Reference is all that you need to start. I say that because Python is so simple to learn that I am afraid that some giant book might end up being a hindrance rather than an asset at the start. You might lose sight of just how simple a language it is- yet its very powerful and extensible. (the Pocket reference is around $9 or $10)
It also depends heavily on whether you're using idiomatic Python. If you're doing a straight translation of algorithms from another language, chances are you're unnecessarily introducing performance problems. Quite often, the optimal approach in C/C++/Java is not the same as it is in Python. Often the "obvious" approach in Python is the fastest (in particular - use the built-in types!).
For example, in C/C++/Java a switch is often the fastest way to make a multi-choice decision. Python does not have a switch, and so Python beginners tend to use an of if/elif[/elif...]/else statement. A better way to do this is to encapsulate each branch into a function, then build a dictionary of values to functions (once - there's no point building the dictionary every time), then simply do a dictionary lookup and call the result.
Also, check out Psyco: http://psyco.sourceforge.net/ - it can often give a significant speedup.
However, most of the time Python will be significantly slower than well-optimised C/C++/Java. OTOH, because Python is so easy to write, it's a lot simpler to get it right first, then optimise.
Might not work as a direct substitute, but Python also has an extensive online tutorial under the documentation section of their website, http://www.python.org./
> It is just god-awful slow.
Really? hmmm, my team runs a huge data warehouse in which all data is transformed via python programs. This means 40+ million rows a day get hash table lookups, business rule validations, type changes and various transformations.
Then most of that data goes through an additional metadata-driven validation process that checks and confirms every single field. Again, metadata-driven - so a oouple hundred lines of various hash table lookups to determine exactly what validation applied to each field.
Ok, now let's talk performance:
1. io performance (most critical factor here)
a. same as c, in other words, extremely fast
2. cpu performance
a. much slower than c, but workable
b. psycho helps a ton
c. the ability to rewrite key functions and classes is cool
Let's also talk about productivity
1. great for ease in training part-time or junior programmers
2. great for immediately understanding old code
3. great for writing robust test-driven code
4. good for very quickly assembling a solution
The above combo of ease of writing, ease of maintaining puts it far ahead of perl, tcl and java. Ruby is really the one other one in this niche I can think of off the top of my head. And even there Python appears stronger on the maintenance side.
So, yeah - I'm sure there are many applications that have more significant performance requirements than ours does. But for this large application it is completely fine.
Not so sure. I am a C++ guru, and I still reach for Python first in most cases. It's without a doubt the language that enable me to express the problem that I'm working on in the most clear and succinct fashion. Furthermore, it has certain features that don't really have obvious counterparts in C++, such as generators and list comprehensions, that I find extremely useful.
And when I really need speed for that expensive inner loop, or image processing algorithm, or whatever, I just reach for boost::python and incorporate my C++ code nicely exposed as Python functions and classes.
Is a car slow because an airplane is much faster? If you're going to write a ray tracer or a weather simulator, C++ would be a better choice than Python. If you're going to write a server than has to deal with a dozen requests per second, a simple game, a GUI app, a code generator or anything else that will not stress your CPU to its limits, Python is fast enough. And you will win in productivity: no compile/link phase, far less code to write, exceptions instead of crashes when your program fails.
The key to getting good performance out of Python is to know and use the libraries well. Most of the performance sensitive library calls are implemented in C and have been optimized. As long as you use those instead of reinventing the wheel, you will get pretty decent performance from Python. Python's standard library is very complete ("batteries included"), for example it includes a unit testing framework and a profiler.
Additionally, by being able to implement things in a small amount of code, you'll get a better overview of what your program is doing. This means it is easier to spot where the performance bottlenecks are. If your program is so complex that you don't realize you're using an O(N*N) algorithm where O(N*ln(N)) is also possible, no fast compiler will be able to compensate for that unless your data sets are really small.
"Python is SLOW"
:)
Shrug. The "official" python implementation is indeed slow.. compared to say, C++ or Java. But, is it too slow for some particular application? That's what matters, and in many cases it is not too slow.
I've done some work on a spim (IM spam) filter for AIM - pyboticide. It was written entirely in python, and was essentially a stripped down OSCAR server proxy. It was certainly fast enough for that, and since it was written in python it was very easy to work with the source code.
An application only needs to be as fast as it needs to be - no faster.
Psyco 1.5 speeds up python 2.4.x to about 50% of java which is good enough to make most web apps I/O bound.
1999 called, they want their comment back!
Sorry, but Java is much faster than Python here in 2006.
An interactive (has an immediate mode prompt) language like Python is very helpful for first learning. BASIC used to fill that role, but I like starting with structured concepts. Kids have no problem "getting" that indentation marks blocks.
Some experts hate python because while the grammar is "context free", the lexing isn't. The grammar is defined in terms of INDENT and OUTDENT taking the place of '{' and '}' (or BEGIN and END). However, whether a given indent is an INDENT or OUTDENT depends on context. So you can't use a traditional lexical analyzer when programmatically generating/modifying python code. You have to translate leading whitespace to INDENT and OUTDENT based on lexical context when parsing, and translate INDENT and OUTDENT to the appropriate amount of leading whitespace when pasting code blocks.
Personally, I don't find that this is a problem. Sure 'lex' doesn't work out of the box, but the custom lexing code is pretty trivial. And the indentation sure is intuitive. Maybe that means I'm not an expert.
I'm sorry but Python is a VERY easy language to learn plus inherently object oriented (although it impressively supports the functional paradigm as well). How is it that you can get all the way to chapter 13 in a book about python before talking about its OO features? I'm not seeing much value here...
Somebody was going to say that too.
Another big issue is the memory usage. Man is Python a memory hog! For a list of integers it takes up about 20 bytes per integer! Now you just imagine the processing overhead of managing all that additional data. On top of that it has to go figure out what types are used in before it can perform the operation - and this is all happening at runtime.
> In defense, this super dynamicness does have some very useful results. namely that everything is dynamic including classes and functions which believe it or not, can actualy be useful sometimes.
I agree 100% with you on that point. And this is one of the reasons I believe Python is where it is. It's one of the easier languages in which you can use functional programming.
> P.S. which algorithm competition allows python? really curious since I have always been pissed that these competitions don't accept languages like bash/ruby/python.
Unfortunately it's not one I think you'd be participating in. It's a South African only event and the only reason we have Python is because of Mark Shuttleworth pumping in the money. As an organiser it's a painful language to have to support, simply due to the not-so-easy-to-measure speed difference. Last year we had a 10x time factor and this year a 5x time factor while dramatically reducing the constraints on some of the slower Python problems.
The ICFP contest allows any language you want. It's once a year, with 3 days to work on the problem. Typically it's a single optimization problem for which a perfect answer is not feasible in terms of computation, so the best heuristic wins.
The Internet Problem Solving Contest also allows any language, since you only have to submit the output, not the program. It's also once per year. It has multiple problems and you have to solve as many as possible in 5 hours.
It *is* a general purpose programming language. Well, it is as well suited to 3d games programming as C is to a web application; and nobody would dare to say C is not a general purpose programming language.
I know you said "all purpose programming language", but then again, for that matter no language is "all purpose" (not even all-purpose flours are really that)
Based on my own experience, I wouldn't say Python is overhyped, and it IS a MUCH MUCH, MUCH better Perl (I used Perl for 2 years, discovered Python and never looked back.)
You called Python a "scripting language". What newer language or implementation runs on the bare hardware, or the bare OS Libraries, apart from C/C++... let's see: Java? nope, VB? nope any .NET Language? NOPE You see a pattern there?
Also remember - taking the exact same algorithm between different languages may be misleading. In the case of python, there are more 'pythonic' language constructs that obviate the need to use particular algorithms - and also may provide speed above and beyond what you would find using the traditional algorithm.
So, this, for instance, is a Pythonic way of getting multiple return values from function:
def something():
return "Foo", "Bar"
foo, bar = something()
whereas this is not:
def something(l):
l.append("Foo", "Bar")
l = []
something(l)
foo = l[0]
bar = l[1]
Someone coming from a language like C or C++ might however have designed something like the latter.
Here is a link to comparisons of python with other languages for your edification: http://wiki.python.org/moin/LanguageComparisons
Lodragan Draoidh
The more you explain it, the more I don't understand it. - Mark Twain
Read my post above wrt the use of particular features of Python improving performance.
As for the switch statement in C/Java, it's actually no longer any faster then a bunch of if statements since the modern compilers optimize them to the same machine code. As for Python though it's a different story since it's interpreted. Building up a dictionary might actually be faster, but I would have to test it out first before accepting it as fact.
> OTOH, because Python is so easy to write, it's a lot simpler to get it right first, then optimise.
Don't get me wrong, I really like Python. I'm just highlighting the fact that it's better suited to particular applications than others.
Well C was written as a readable assembly language and there ;-).
has been 20 plus years of optimisation going on there, so,
you have got to be a shit hot assembly programer to beat pure
C on speed.
C++ is considerably slower and harder to read ( flambait
The interesting thing is that well written Pyhon applications
run about the same speed as well written Java.
The key phrase here is "well written" and you really do have to
be aware of what pyhon is doing the classic being
python " for x in range(1,1000)"
is not the same as C "for (x=1;x1000;x++)"
The python statment actually means
" build a list consisting of all the numbers from 1 to 1000
then for each entry in the list assign it to 'x' and do the
loop body "
This is obviously going to be slow compared with a classic for loop
and I have seen this statement used in many a brain dead performance
comparision.
Also note that in real life C and C++ solutions can actually be
slower than Python because its just to complicated to code stuff
like a dictionaty of functions which is all built into python.
So write it in Python and if you have a performance problem
write better Python and if you still have a performance problem
write it in C.
Old COBOL programmers never die. They just code in C.
It really depends on what your doing with it. If you come from the Java world and write Python code like you would Java code it is quite slow.
...)
....
Also, keep in mind Python does not use a virtual machine at all in the same way Java does -- which means it is somewhat like comparing apples to oranges.
I was a full time J2EE engineer for a while and a recent convert to Python for general programming. The things I noticed are:
- Java is great for large teams of programmers working together across the globe but as good as Python for smaller groups in the same place.
- Python's web frameworks are amazingly fast and very powerful.
- Java is almost text book perfect when it comes to programming. Unlike most other languages kids can come out of university and not have to change hardly anything to get started in the real world of Java.
- It's very rare to see stock Tomcats or even stock JVM's meaning that most (commercial) Java apps send the entire 'Java OS' (VM) with their products.
- Java web apps are very slow on startup, but scale well after the startup time.
- Python has a better RORA system, though Java isn't to far behind.
- Java terminology is very confusing compared to Pythons esp. since it is generally termed by marketers (Java2 is Java1.4 and then we hit Java 5 which is Java1.5
Here is how it seems companies leverage the languages
1. Java
Manager's reason for Java: "I have a large IT infrastructure and want to do enterprise applications with teams across the globe."
Engineer Translation for Java: "I have an over all large group of engineers and want to make an application designed by the business with at least some of the team out sourced."
2. Python
Manager's reason for Python: "I have a small IT budget and want to agile and business reactive application development."
Engineer Translation for Python "I don't have a budget to get more engineers and the requirements state that we need to be able to change and update the application on the business whim."
In scenario number 1 it is a growing company who wants to hire lots of new engineers so the plus side of using Java is:
1. Just about every engineer (good or bad) knows it
2. It enforces C++ like development cycles
3. If done right, the objects are reusable and simple for an architect to understand
4. You can hire fresh outs
In scenario number 2 it's a growing company who is not investing more money in the IT department but expects better results for the buck put into it. The plus side of Python would be:
1. It's a RAAG language (read at a glance) which means you can skim through the code quite quickly without needing tracers or debuggers.
2. By design each engineer is a mini architect
3. If done right, the objects are reusable and simple for engineers to grasp quickly
4. The quality of engineers you get tend to be quite high since most Python engineers are converts from other languages with X years of experience in the field. (though it also means you can't hire as quickly).
Bottom line there is no right or wrong answer as to which is better or faster. Python is faster in some cases and Java is faster in others and it largely depends on how you run your business and the money that you want to spend in IT as to which one would help your company.
> python " for x in range(1,1000)"
> is not the same as C "for (x=1;x1000;x++)"
In place of range(), you should use xrange() as it doesn't build up the whole list as range() does.
Once you've actually used the python for any period of time you come like this feature. In reality whitespace does matter. When a person reads a block of code, what the person thinks the code does is at least as important if not more important than what the compiler/interpeter builds the code to do. Making whitespace a functional part of the code just aligns the reader and the interpeter/compiler which removes a source of errors.
Instead of saying all this, which is pretty much 90% FLUFF....
"Chapter 9 deals with file manipulations where the author introduces all the built in functions available in Python which allow one to open, read from and write to a file. Interestingly, the functions are also illustrated by aid of short easy to understand example"
You could write....
Chapter 9 explains files.
same shit. just shorter. time is lines of code around here, cowboy.
http://wiki.python.org/moin/PythonSpeed
Lodragan Draoidh
The more you explain it, the more I don't understand it. - Mark Twain
This is awfully debatable. The explicit static typing and ease of static analysis makes error-checking much easier in Java, which makes for much easier QA if you don't like testing. Java also have very good IDEs, which it doesn't deserve. But the verboseness of Java and it's lack of many (syntactic or other) niceness such as first-class-object functions, tuple-unpacking, litteral syntax for common containers (lists, hashes, ...), ... make it much slower to use as a developer. It also requires the users (of the language) to think at a much lower level, akin to C versus Java. If "smaller groups" have a good knowledge of the language, Java just can't beat Python.
They're not that fast (even though most of them are fairly fast). And Python has the "more frameworks than built-in types" issue making it a pain to pick a framework.
Are you joking? Java's a textbook piece of shit when it comes to programming. Python is not *that* much better by the way. If you want "textbook perfect", go check functional languages. Or Smalltalk if you're into OO.
The fact that most unis and schools now teach "computer science" in Java may have something to do with that one...
"The way we can tell it's C# instead of Haskell is because it's nine lines instead of two." -- wadler
That simple stupidity just made everyone who knows anything about Ruby and Python dismiss your post as utter nutjobbery.
Java is indeed faster than Python by a large margin (unless the runtimes are short, in which case the sheer monstruosity of the JVM's startup time kills Java's perfs).
TCL compares very favourably to Python, even though it's mostly slightly slower, unless you're using Regexp (the Tcl regex engine is just wicked fast).
But Ruby on the other hand, is one of the slowest languages available today. The current baseline implementation is a piece of crap, it's wasteful, it doesn't have any optimization, it's more or less the slowest and worst way to implement an interpreted language. Even though Ruby is a quite nice language, speed is definitely not it's forte.
The only way for Python to be slower than Ruby would be to code Ruby in Ruby while you're coding Java in Python (using Java idioms and techniques while producing Python code, which is a bad idea).
"The way we can tell it's C# instead of Haskell is because it's nine lines instead of two." -- wadler
> As for the switch statement in C/Java, it's actually no longer any faster then a bunch of if statements since the modern compilers optimize them to the same machine code.
I think you mean that some compilers may optimize a bunch of if statements into jump tables, the same way they've been doing for switch for well over a decade. Python could perhaps do jump table optimization for a switch opcode, but you'd be micro-optimizing something that would either barely benefit or would benefit more from a redesign to proper OO or functional code. Switch is a crappy primitive construct with marginal expressive utility -- now if Python had ML or Erlang-style pattern matching, then we'd be talking.
Done with slashdot, done with nerds, getting a life.
So many editors switch tabs/spaces freely which creates problems with indentation syntax. Because of python and make, I have reverted to using e3 as an editor. I like that it is small and simple, but it does not have anything fancy like syntax highlighting. For programming syntax highlighting helps a lot. I used jed before and loved it, but it mucks with spaces and tabs, so python and make wouldn't work. Yeah, I think you can change the settings to make it work, but it is a huge pain in the ass.
I worked on a TP project in 2000. We converted the project from C to python, dropped 40,000 lines of code and bench-marked the python implementation as 3 times faster than the C implementation. The reason - we were able to focus on the problem when programming in Python, whereas in C we spent all our time (and code) getting it solid.
Actually, Python is byte-code compiled at run-time. You can see this in action after you run a Python app one time, you will see a
Lodragan Draoidh
The more you explain it, the more I don't understand it. - Mark Twain
A minority of applications are actually CPU-bound. Most applications are I/O-bound and can't be optimized. Most "general purpose" applications won't benefit from being written in C.
Please, for the good of Humanity, vote Obama.
> - Python's web frameworks are amazingly fast and very powerful.
Python has a pile of MVC frameworks with 99% overlapping functionality but little else. Show me Python's answer to JSF, Tapestry, or Wicket.
Done with slashdot, done with nerds, getting a life.
Few web-hosters have python to begin then. Then I read that for python to perform well requires mod_python 3.X, which in turn requires Apache 2.X. Practically no web-hosters use apache 2.x.
Popular frameworks like TurboGears, or Django, also require mod_python 3.x and apache 2.x.
No problem if you are developing internally, but if you want a hosted solution, I'm not sure if python is practical.
I'll takle the flamebait! Studies have shown that C++ can be faster than C. I don't have the links and would be happy to simply remove considerable from your statement. C++ is about the same as C for speed. In some cases a little faster, in others a little slower. If it is considerably slower than the code was not well written in the first place. It may well be true that, in general, C programs tend to be faster as the language is smaller and less likely to confuse the programmer with advanced features. I suppose any program you write in python, I can write to be slower in C++, and to be even slower with C. This reflects my genius and not the quality of the C++/C programs I have produced. Inherently, well written C/C++ will be faster than python, but it may well be faster to write the application in python. If my server is not a busy server, why should I care about the application speed? I will be more concerned with my programming speed. If my server is busy I might well prefer funding the overtime to have the application well written in C/C++. In reality the trade-off between C/C++ and python isn't about speed or development time but strong typing.
That is why they allow you to link with components in C. The critical time components can be written in C while the rest can stay Python, if you want. I think python is more useful for prototype design or just quick scripts. I agree it is slow, but it would be faster if someone made a compiler--I think someone did make a JIT one, but for one off things JITC is slow too. Some components don't need to be blazing fast, but an interpeter sucks with slowness.
Python is a serious piece of problems for me:
"Python, the dynamic object oriented programming language created by Guido van Rossum is known to deliver both the power and general applicability of traditional compiled languages without the complexities accompanying them."
Sounds like a typical spamvertisement really!
"Coupled with its ease of use"
umm hell, python is notoriously known for bad coders, writing broken applications based on underdeveloped and unmature modules, breaking forward compatibility.
Sadly enough, in the last two years, I have not seen any progress making python-based applications more stable. Programs like BitTorrent(!) have serious issues running on python-2.5(!)
"programs written in Python can be run on multiple Operating systems and system architectures which gives it the same portability of any interpreted language."
nothing new here, move along...
"My first brush with Python was when I tried out a beautiful game called PySol"
FOR CRYING OUT LOUD!!! Pysol is the ultimate proof that python programs are incredibly impossible to maintain forward because the library/module layer is so incredibly unmature. Pysol is no longer developed ! It doesn't even work with Python-2.4 !!! Even if the original developer stopped working on it, somebody should have been able to pick it up and maintain it easily, and the fact that this didn't happen is proof again that python is barring its own progress. See http://www.pysol.org/ - last release was in 2003.
Python has wasted my time as a packager/distro developer many many times, and ruined quite some weekends for me.
The world would be a better place without Python.
Scripting language just means it is interpeted. That is why python is slow. If they made a compiler for python, it would be much faster, though likely not as fast as C.
This is why there is so much interest in a Python compilier--compiliers are much faster than interpeters. There are some starting projects--such as psycho and Python 2 C Module Generator
Java and TCL have certainly had many more years and manpower to optimize thier implementations. I don't know much about Ruby. I'm sure in a few years, Python will improve too.
django and turbogears both run fine on apache1 too. or on lighttpd... or basically anywhere where there is fastcgi or scgi:
e nts
http://code.djangoproject.com/wiki/ServerArrangem
http://docs.turbogears.org/1.0#deployment
generally speaking, when you want to run a python web-application, you want to avoid to have to reload the whole application by every request.
because of this, CGI is not suitable, but there are many other ways:
- scgi
- fastcgi
- mod_python
- standalone python http servers (afaik turbogears and zope do this)
(of course, it's still true, that it's easier to find php-hosting than python hosting)
What are the pyc files for?
So a beginner programming in Python has to known the internal implementation details of every function?
Any sect, cult, or religion will legislate its creed into law if it acquires the political power to do so.
Only if you don't know much about python, try looking at stackless python. eve online, the mmorpg with the current world record for concurrent players was written almost entirely in python.
This is a joke. I am joking. Joke joke joke.
It's fast enough to multiplex (conference call) multiple voip streams.
See "Scripting Language" My Arse: Using Python for Voice over IP
http://www.python.org/pycon/dc2004/papers/6/
Except for when I'm developing something. A month or so ago I needed a small script to generate an RSS feed from a webpage. Now, I had two (OK, more, but stay with me) choices: I could write it in C, taking days and writing, oh, 200+ lines of code, worrying about pointers, declaration, and script typing, or I could develop it with less that 100 lines of python in about half an hour. I'll let you guess what I chose.
My new blog
"First, as far as web programming is concerned, the users are always slower than the slowest of all programming languages : they read the text, fill in the forms, etc."
Right.... Try applying the "slow user" argument to a server that has to handle thousands of simultaneous requests from said "slow" users.
The user can afford to be slow. The server can't, and if it can't scale you're in deep trouble.
Any sect, cult, or religion will legislate its creed into law if it acquires the political power to do so.
I don't know about python in general, but I have been going through the SciPy tutorials and plotting even a simple sine function and saving directly to file and not displaying a gui is about 20 times slower than piping data into grace or using pgplot and perl. My CPU usage spikes at 99% during the task, where as grace barely uses 10%. Matplot lib does some really great map projections that I would like to put in a web appp. But I'm having serious doubts. I may have to stick with piping data to grace.
Premature optimization was a sin in the days when people were arguing about the efficiency difference of particular assembler instructions, or if it was wiser to use post- or pre-increment. In current days of dynamic languages, on-the-fly typing, dynamic dispatch, remote procedure calls and multi-Ghz processors churning through billions of unnecessary instructions, the mantra should be revisited. If the tool you use sprinkles millions of unnecessary instructions around at random, no amount of profiling will allow you to optimize the whole program. Everything runs slow, and only some particularly obnoxious bottlenecks are found. The end result: a program that is slow as mollasses, even on the fastest hardware. Some 'premature' attempts at making the entire thing efficient could have avoided such a nightmare, but alas, the wrong application of Knuths comment on (for him) contemporary programming allows programmers to get away with sloppy thinking and therefore sloppy coding. Speed matters, if only that with more speed, you can pack more functionality in the program.
Interestingly: try to ask a runtime typed language programmer how big their objects are, and they usually they don't have a clue. This general lack of knowledge leads to bloated memory apps and, by the caching argument, to lack of speed.
One recent thread about the book (which also comments on why things like functions and OOP appear later in the book than one would think):r owse_thread/thread/b8366618c4547978
3 7 (it seems like Amazon, Buy.com, Bookpool, and Overstock rotate for having best overall price, i.e., no tax [depending on where you live] and free shipping)
http://groups.google.com/group/comp.lang.python/b
Also check the Amazon page for reviews and other feedback plus the author even posted a comment:
http://amazon.com/o/ASIN/0132269937
To reply to some previous comments:
- It's *much* faster than it used to be: http://shootout.alioth.debian.org/gp4/python.php
- The indentation only bothers you for 1-4 months. (I didn't like it either at first.)
- It *is* interpreted but byte-compiled like Java to make successive runs faster
- Why ESR likes Python: http://www.linuxjournal.com/article/3882
- Native look-n-feel GUI: http://wxwidgets.org/ and http://wxpython.org/
- Compare to other languages: http://wiki.python.org/moin/LanguageComparisons
- Shopping: http://www.bestbookdeal.com/book/compare/01322699
- Bad code: Python is attractive to first-time programmers because of its ease, so that's what you may be seeing. Also, bad code is language-independent, regardless; Python does not go out of its way to make this happen. However, Python also attracts long-time programmers because they discover they are more efficient and productive in it.
FWIW, I switched to Python a few years ago (after lots of C/C++, Java, Perl, Tcl, etc.), and I don't want to program in anything else again. The naysayers can pound on me all they like, but from my point of view, I enjoy what I do, I get decent pay, and I can get home on time to feed my kids then hack some more for fun after putting them to bed.
-A.C.
True, Ruby is slow, and I'm looking forward to 2.0. But to misquote Guido, sometimes it's Fast Enough. I've done a few things here at work where rewrites in faster languages were negligibly faster, really. It all depends on what you're trying to do, and how over-engineered your solution is ;-)
Stating on Slashdot that I like cheese since 1997.
Google Code Jam allows Java, C++, C#, VB.NET and .
mod_python is most definately not a requirement, although it is an option, there are several other ways to host python web apps, and fastcgi
seems to be one of the more popular, there are also a large number of hosting providers that explicitly support python web apps , including
some that specialize in it.
"In Xanadu did Kubla Khan a stately pleasure dome decree"
Comment removed based on user account deletion
Anyway, the whole question is rendered moot by the ability to embed Python in C++ and call C++ libraries from Python. It might seem ironic to you given your preconceptions, but Python brings C++ much closer to being an all-purpose programming languages than it could ever be on its own. Python makes C++ programming tolerable by allowing you to strike the unpleasant parts of Greenspun's Tenth Rule of Programming (which I take to apply to C++.)
I give you Greenspun's Tenth Rule, Boost.Python version: "A C++ program that uses Boost.Python gets a mature, well-documented, slow implementation of half of Common Lisp without the burden of maintenance."
I prefer core programming in qore: http://qore.sourceforge.net/ :-)
i once wrote a numerical simulator of weakly coupled harmonic oscillators. i started out writing it in python. when it got to the point that the simulations took days, i re-wrote it in c++. upon running it the first time, it exited within one second. i was a bit unsettled, that it did not complain about a segmentation fault or some other error condition, which i had expected for a first run. turns out it ran correctly. a relative speedup of about 10000. so yes, python is slow. and yes, you are right, the ability to re-write performance-critical parts in C/C++ is a great boon to python (though it's possible in most other languages as well).
on the other hand, i discovered that numerical code written in java can approach the speed of C/C++. the added bonus here (if you come from python-land, and using jython is an acceptable choice for you), is that you can call the time-intensive routines in a safe language, and without having to write any glue code. in the absence of other constraints, i would recommend that combo, if you want to script in python, but need performance in a cpu-bound environment.
check your speling
oops, got the jython link wrong. also, if you're into (CPU) performance, it might be useful to take a quick look at a relevant benchmark over at the great language shootout.
check your speling
Hey, that's cool.
I literally copied (^c) and pasted (^v) your python implementation of InsertionSort into a python session in a Terminal and it worked exactly as advertised!
What's cooler is that I then tried it with letters which also worked as you would expect...
>>> y = [ 'm', 'p', 'r', 'a', 'z', 'w', 'hello']
>>> InsertionSort(y)
>>> y
['a', 'hello', 'm', 'p', 'r', 'w', 'z']
>>>
Try doing that in C!
Way to go Python!!
Someone used "verily" on slashdot?!
Gushing review of 1000+ page book that can't even begin to be as concise as K & R... again.
Yawn.
Send the author back when they can get the size down to something the size of K & R while maintaining the same level of usefulness.
+++OK ATH
Reference counting memory management does not allow cyclic data structures, and since most programs have them (especially business apps with object models where nodes point to children or parent) or any GUI app (children widgets point to parent and vice versa), I fail to see how it is "easy to use". It is as easy to use as C++ with smart pointers, regarding memory management.
Python doesn't have anything like that, and I'm guessing it never will. The problem is, that it's way too easy to whip up a custom web framework by picking up a templating system, back end and data storage and gluing them together.
The good news is that many curent internet applications have trained the user to expect much slower responses, so they are less likely to keep hitting the [Enter] key or press the [Submit] button on client server systems.
:)
You really don't know my users
Other posts stated "it depends on what you are going to do with it" and that Python is of course not suited for every task (like squeezing out every frame of GL apps). But let's pick a very basic daily omnipresent task: substitute some text.
Feel free to look at png2wx.py and compare with the perl counterpart, png2wx.pl. Or maybe encoded() and octal() were just poorly coded, but I doubt that, given their simplicity.
Read the FAQ. Python's garbage collector does check for cyclical references, just not as frequently (unless you tell it to via the gc module).
Ita erat quando hic adveni.
The book teaches the principles of Python programming by letting the reader create simple games. I found this method to be more concrete and rewarding than other books whose examples were very abstract. My one complaint is that the last three chapters on object oriented programing and gui graphic programing are a little confusing and use a watered down version of PyGames and LiveWires modules.
However, it provided me, a self-proclaimed programming idiot, a good solid foundation in the basics of programing.
Thanks for your post. Can you tell me more about how you use python to validate and transform the data? Are you using in house developed tools or off the shelf components? Can you tell me more about the metadata-driven validation process. I am currently investigating techniques to use to validate data in our db's and some pointers in the right direction would be greatly appreciated.
We're using multiple methods here:
1. ETL (extract/transform/load) processing in which source data is translated to the warehouse format. This consists of several components:
a. python-based etl framework - responsible for managing our processing queues. This code takes a decentralized approach quite different than most commercial etl tools - that are large and awkward monolithic applications. This has worked very well.
b. python-based transformation - typically simple custom-written programs that read a single file and write a single transformed file. Execution of these programs is handled by the etl framework - based on existance of data in the queue. Lookups are via python hash tables. This has proved to be an extremely easy to develop and maintain solution.
c. python-based, metadata-driven validator and transformer - warehouses are easy to load quite a lot of data quickly into (hitting easily over 100k rows / second) but very slow to delete data from. Additionally, many of our source systems will change business rules without informing us. So, we have a completely separate python validator that is handed each field individually, then writes the output row. It assumes that all business rules are handled by the calling transformation program - so it only double-checks type, date format, value ranges, default values. It also is ultimately responsible for final file formatting - this is because due to so many control characters in my data I need to go with fixed-length files, and formatting so many fields this way was so error-prone.
2. testing framework - we've got a few testing challenges in the warehouse:
- server configuration consistency between multiple read-only servers
- data consistency between sources and targets and between multiple read-only servers
- data quality due to lack of enforced foreign key & check constraints
- configuration policy compliance (corporate security and departmental standards)
- performance of complex queries
Our tester generates tests for many of the above, and allows us to easily run tests for all of the above. We use it in both development for QA and then daily for continual QC (emails us alerts if there are any).
The python command shell (and irb) is great. But it'd be nice to be able to edit multi-line commands when you make a typo and then be able to separate execution from core commands. If they would just add a line numbering scheme and some simple commands like 'RUN' and 'LIST', you'd have a truly modern work environment.
- You can't delete all the messages in one folder.
- The search function never works as intended (I want to look for messages only in one folder, damnit!).
So assuming this guy is very clever, Python must be broken or too dificult to use.
IANAL but write like a drunk one.
Exactly ... but it's NOT easy to whip up a custom web framework that gets things done like Tapestry or JSF, where you don't have to route everything through a dispatch controller that has to route all the form values back to the domain objects and deal with the transaction demarcation by hand if you're even lucky enough to get such a thing in the framework at all... (inhale)
I guess Zope is pretty close, but I don't know what to make of Zope. Zope3 seems to be doing its darndest to keep itself obscure, and it competes with Turbogears to see who can have the least useful documentation. Django is actually getting there with its admin interface, but it's sort of a whole extra framework inside an existing classic MVC one. If PHP can get itself a component-oriented framework with Prado, I should have hoped that Python could fill the bill. I wonder whether a port of Wicket would be doable?
</ramble>
Done with slashdot, done with nerds, getting a life.
Python "compiles" to a python specific bytecode which is intepreted by the python program. When I say compile, I mean the code is translated into the computer's native machine code for the processor. Interpeting and runnnig bytecode is still much slower than running a native program with the processor.
.pyc files are for python code translated into a python specific bytecode. .pyo files are an optimized version. The bytecode is interpeted by the python program in every case. It only loads faster because python translates the source code anyway, and has to convert from source to bytecode if you run a .py file alone. With pyc/o files, the first step is already done. Running directly from raw source would suck for many reasons--more complex and even slower.
The pyc and pyo files are not native machine code like what comes out of gcc or nasm. Native machine files don't require a separate program to run, though they need something to load them into memory and usually support libs. pyc/o files always require the python binary to run them in addition to any support modules.
Premature optimization was a sin in the days when people were arguing about the efficiency difference of particular assembler instructions, or if it was wiser to use post- or pre-increment. In current days of dynamic languages, on-the-fly typing, dynamic dispatch, remote procedure calls and multi-Ghz processors churning through billions of unnecessary instructions, the mantra should be revisited. If the tool you use sprinkles millions of unnecessary instructions around at random, no amount of profiling will allow you to optimize the whole program. Everything runs slow, and only some particularly obnoxious bottlenecks are found. The end result: a program that is slow as mollasses, even on the fastest hardware. Some 'premature' attempts at making the entire thing efficient could have avoided such a nightmare, but alas, the wrong application of Knuths comment on (for him) contemporary programming allows programmers to get away with sloppy thinking and therefore sloppy coding. Speed matters, if only that with more speed, you can pack more functionality in the program.
I disagree: premature optimization is an even bigger sin today. I didn't mean to imply that you needn't worry because Python can be made as fast (as, say , C), only that Python can often be made fast enough for the task as hand... and often is, even without profiling or extensions in C.
As far as sloppiness is concerned, needless infatuation with speed tends to be the greatest offender, insomuch as the programmer misses—or never accomplishes—the high-level goals and design of an ambitious program by focusing too much on non-issues like performance when the application makes relatively low demands of modern hardware.
"Slow" languages have, in my experience, have no correlation with sloppy thinking. In fact, I'd assert it's just the opposite: most of the best programmers I know work with languages like Python, Ruby, Haskell, Scheme, Dylan, etc. These languages allow those programmers to concentrate on the real problem (the application) rather than the distractions (the hardware).
Careless memory use, another bigger offender. This one cropped up with popular garbage collected languages. If apps grow, careless use of memory will byte you. Again from practice, I've seen many apps that will simply max out memory, on tasks that should not be that memory intensive. This includes python apps. In many languages, the average programmer actually wouldn't know how heavy things are. In python: how much does a class weight? How much does an instance weight? Ask a programmer, and often he wouldn't know. I asked.
Heavy objects have another downside: lack of speed. If every object has an 8 or 16 byte overhead attached to it (Java numbers, normal objects and serializable objects), arrays of these objects will more quickly flush the L1 and L2 caches, leading to a slower application. Again, it takes a whole lot of profiling to fix that problem. Of course, if these are not your concerns and your app is waiting for IO, that's fine. But still, having 500 Megs committed on something that will just push a few bits around will become a problem some day.
This is not against Python per se. I think Python apps are usually of very high quality. Problems start when you try to scale these things up. This is usually the point where Python is abandoned, because it's overall speed falls short.
In this day and age, I would not say anymore that premature optimization is the root of all evil. It is still evil, but there are bigger evils now: needless abstraction and careless use of memory. In Knuth's days, those were not even an option. In 1K of memory, there's not much abstraction or memory bloat that is tolerable.
This works for anything that has a quack function. How does Python figure out that it has such a function? Hash lookup, followed by a function call or an exception being thrown. Compare with:
void myDuck(DuckInterface duck) { duck.quack(); }
Translates to a direct virtual function call at (bytecode) compile time. Psyco is capable of getting rid of the hash lookup by compiling specializations of the myDuck function. These specializations take up space, need to be maintained, compiled, etc. All to the detriment of speed.
I see where you're driving--and to a certain extent, I agree. Java is probably the best example of the particular kind of evil to which you allude... languages like Python and Ruby don't place as much emphasis on "architecture" and "abstraction", don't insist that OOP is the answer to every problem, don't produce such deep, thin inheritance schemes. And yes, when your application reaches a certain level of complexity, Python can break down. I guess the point I'll re-emphasize is, I think far too many developers (still) predict a complexity for their applications that's not appropriate and that will never emerge. They base their tool decisions on this perceived complexity, they choose lower-level tools for a project with a limited scope and little performance demands, and the project simply never gets completed. Many applications _should_ be relatively simple applications, and dynamic languages are excellent tools for these tasks.