Slashdot Mirror
Python 3.4 Released
New submitter gadfium writes:
"Python 3.4 has been released. It adds new library features, bug fixes, and security improvements. It includes: at standardized implementation of enumeration types, a statistics module, improvements to object finalization, a more secure and interchangeable hash algorithm for strings and binary data, asynchronous I/O support, and an installer for the pip package manager."
That would be nice.
And everyone will keep using 2.6/2.7, the windows XP of python.
Yup, Global Interpreter Lock so Python is still fundamentally single threaded -- only a single thread can be executing any python code at any given instance.
Its 2014 and we still can't have a multi-threaded python, this is ridiculous.
If you read Guido's criteria for getting rid of the GIL, he lists so many things that are specific to the current single threaded system (which is evidently perfect) that the only solution that meets his criteria is the current system.
I guess the only solution is to either live with single threaded system or fork it.
.. and 2.7 works better and can more easily be deployed so when I write code in python, it can ACTUALLY GET USED.
Python 3 is effectively worthless unless I can deploy it; and right now none of the major, fair priced hosts can support large websites support it.
There's the old parable about two programmers that are told to double the speed of their program. One guy spends a month rewriting the core in assembly and using hardware acceleration. The other guy waits 6 months and buys a new computer. They both succeeded.
I realize that python will probably never be used in post production of HD video. But if I want to deploy a web site with readable reliable code that can be easily maintained, I'm not using java or php. So long as I can scale by throwing machines at the problem, python wool solve my problems wonderfully.
Waaaayyyyy too much effort went into their treatment of string hashing for their dictionary objects. It seems like a useless distraction and a sad opportunity for endless bike shedding.
Python, like Perl, is just ridiculously slow. (Disclaimer: I love Perl.) Even a tight loop doing nothing is slow compared to some other, non-JITd scripting languages**, let alone JITd or compiled languages. A 5% change in string hashing performance is not only imperceptible in real work loads, it's nearly immeasurable in something like Python.
You use Python because you enjoy the language and its ecosystem of libraries. You most certainly do not use Python because of it's choice of micro-optimizations.
** Proof: Lua is an order of magnitude quicker even with just a simple loop. Compare
local x = 0
for i = 1,(20 * 1048576) do
x = x + 1
end
to
x = 0
for i in range(0, 20 * 1048576):
x = x + 1
Only 3.4? The Python in my pants is 9 inches and that's when flaccid!
Sorry AC you like many americans are easily confused when you see metric you were looking at the millimeters side of the ruler.
it shouldn't have been a surprise that it would spell doom for Python to fork it into two incompatible branches for a couple of "it would be nice" type features.
No.
The Python community, overall, approves of Python 3.x. The major breakages have to do with Unicode, but that's because Python 3.x does it right and Python 2.x didn't.
If you don't think Unicode matters, my guess is you are an English-speaking American. Others disagree.
There are efforts underway to port the major Python projects to support 3.x. SciPy will be the big one... Django already has support for Python 3.x.
Perl6 never went anywhere, Python 3.x is in wide use.
PyPI download stats indicate that Python 3 packages account for less than 3% of all Python package downloads. That's hardly "widespread use" for something that's been released for over 5 years.
I have recently started bathing in the waters of Python. What I have realized is that it is a core group within Python who are rightfully proud of their 3.x accomplishment. But they are solidly ignoring the fact that only a tiny percentage of people are using it. The reasons are quite simple people will need 8 modules for their system and 1 barely works with 3.x and the other says something like "mostly works" Well most people aren't willing to depend upon "mostly".
Now module after module is going 3.x but the other problem is that for most people having two pythons on their machine is a pain in the ass. I know there are tools to make this less painful but I can tell you an easy way to make it painless, Don't have two versions.
Then there is this call that you should begin new projects in 3.x; but the problem again is the two versions issue.
What bothers me about all this is that I come from a C++ / PHP world. With C++ I have upgraded countless times over many years and had close to zero problems with my code. I don't even know which compiler XCode is even using right now. With PHP my various upgrades have broken exactly one module and I hear rumours that the next big version of PHP will break one module in my older code. But I don't care as I am replacing my PHP with Python.
Where I am worried is that the core Python people will do something stupid like announce an end of support date for 2.7. The problem there is that it might be easier for some people to install a whole different language to sit alongside Python 2.7 and start playing with that instead of smashing their machine in the teeth and simultaneously installing 3.x.
Didn't you actually mean to say, "FUCK BETA!!"?
And he asks nicely that Python programmers stop invoking his name in vain.
Honestly, this entire stack is so non-deterministic, what is it doing in the Enterprise again? I haven't seen anything this Rube Goldberg since the 50s. Shit-can this stinking pile of merde for almost anything and the world will be a better place.
I am personally more interested in the PyPy release that will bring transactional memory.
The project I work on right now is proudly PyPy compatible and I hope to keep it so :-)
Have they fixed the whitespace bug yet?
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
(no text)
And python has supported it (at least on unix) virtually since it was first released.
I've never really seen much virtue in multi threading - its useful in a limited number of cases but usually it creates more problems than it solves (compared to multi process) and is usually used by people who don't really know what they're doing. Essentially multi threading takes all the advantages of protected process virtual memory and throws them in the bin.
There are plenty of languages that are equally expressive while still being far faster and more efficient.
And available on entry-level web hosting? The only language I know of that's more widely supported on shared hosting is one that's been called a fractal of bad design.
Most Linux distributions are shipping 2.x and 3x in parallel and have been doing so for years without issues.
That's fine if your audience is willing to install Linux, either as a dual boot or in a VM. It took until Python 3.3 before Windows could practically run 2.x and 3.x in parallel.
If a, b and c can be int or float, how can a programmer implement "a = b divide c" without a lot of ugly type checking?
By determining what type you want in the result and choosing the operator that produces that type. And you can get Python 3 division behavior in Python 2.6 or 2.7 using from __future__ import division.
Of course I know about multiprocessing. Why have one copy of the interpreter and libraries loaded when you can have N, plus its so much more efficient to marshal data across process boundaries than to access a global shared memory block.
I've heard this processes are so much better because we can't do threads for so long. Kind of like if I cut off my right arm, its so much better to only have a left arm because you only need to move 5 fingers instead of 10.
...a fanboi talking.
"Of course I know about multiprocessing. Why have one copy of the interpreter and libraries loaded when you can have N, plus its so much more efficient to marshal data across process boundaries than to access a global shared memory block. "
*snort* You obviously don't have a clue about multiprocess. Look up copy-on-write then get back to me.
As for a shared memory block , uuuh , guess what , theres something called "shared memory" specificially for multiprocess access. Its been around since the 70s in unix but I wouldn't expect a clueless (obviously Windows) programmer to know about it though ironically even Windows supports it now.
Here, educate yourself:
http://www.cs.cf.ac.uk/Dave/C/...
http://en.wikipedia.org/wiki/I...
Guido is the textbook definition of an egomaniac primadonna. The moron seems to think he knows best how everyone should be using the tool known as Python. He is completely clueless to the fact that a tool needs to adapt to the needs of its users, rather than the other way around. Before Python can ever get its shit together that fucking delusional asshole has got to go.
I take it you are also against private variables and interfaces too? For that matter, why not just expose the entire system memory to every process, let programs screw with each other or alter kernel data directly!
No, the language fetishist community approves of Python 3.x. In general people who want a tool for doing something are using Python 2.x. This might change over time, but for the moment Python 2.x offers more then Python 3.x does from a developer perspective when one's goals are result oriented rather then language oriented.
sometimes the programmer implementing "a = b div c" does not know whether the result "a" is int or float because he's writing a general library function
If the output of the general library function shall be a float, then use /. If the output of the general library function shall be an integer, then use //. If the output of the general function shall depend on the types of the arguments, then I'm having trouble understanding in what sort of case this would prove useful.
where depending on the application "a" may be int or it be float.
On what aspect of the application would it depend? As far as I can tell, whether a should be int or float depends on what kind of quantity b is intended to represent and what kind of quantity c is intended to represent. In what specific "general library function" would "use floor division if neither b nor c is a float; otherwise, use true division" be helpful? It would prove easier for me to see your point if you can give a concrete example.
I think it comes down to the libraries and what you're using python for. For me, I use python as a sys admin language so I really depend on fabric. When fabric works complete in 3.x I'll consider moving.
It's similar to upgrading your desktop os. Some folks will upgrade to whatever is latest because they like doing so. Even if the latest happens to be vista. Others wait until all the bugs are worked out and vista becomes windows 7.
Yes, copying can be faster than using shared memory. Why? Because locking. Mutexes kill CPU pipelining. If there are too many mutexes taken by an interpreter, then copying data can easily outperform sharing, even large chunks of data.
If you care about performance, then you want to reduce the amount of interaction between two threads as much as possible. In C or C++ this is easy. In a scripting language it's not easy because the interpret has to take locks (e.g. when updating a dictionary, or when garbage collecting) regardless of whether there might be any thread contention.
PyPy will try to use transactional memory to do this, but it will be a gigantic failure because it will bloat memory in an epic way, making it useless for large applications. And without hardware transactional memory it'll be relatively slow, too, because it's still using mutexes (just not as many).
If you call C code from python the GIL is released
Only if the library explicitly releases the GIL. Pillow (Python Imaging Library), for one, happens not to.
As I understand it, the GIL is mainly needed because the CPython reference counting model needs to touch a whole bunch of reference counts when churning objects, and this needs to not screw up. Thus Jython and IronPython, leveraging the garbage collection of their respective underlying VM platforms, didn't need a GIL.
But these implementations pay a penalty for relying on the VM's tracing garbage collection, and this penalty is loss of finalizer functionality.
Memory managed by the VM isn't the only resource that a program uses, but it's the only resource that some garbage collectors anticipate. Other resources include open files, open network connections, open database connections, and the like. One common design pattern for ensuring that unmanaged resources get released is try/finally, which works with JVM, CLR, and CPython, but doesn't work so well with resources held longer than one method. Another design pattern is Resource Acquisition Is Initialization, which releases the resource in a "finalizer" method associated with the object. CPython, JVM, and CLR all support finalizers in theory (__del__, finalize, and IDisposable respectively), but platforms that use pure tracing collection don't guarantee that they'll get called at all, which causes an unmanaged resource leak. In CPython, so long as an object isn't part of a reference cycle, its finalizer gets called once it's unreachable, which allows closing the file, closing the connection, freeing the unmanaged bitmap, etc.
Perl clones the entire interpreter for every thread.
So does Python if you import multiprocessing. But then you run into other problems, such as it being hard to send objects back and forth between processes unless they're small and simple.
Another problem is that anything sent through a connection between processes has to be picklable with a bytes representation smaller than about 32 MB, so no passing big images around. Does this cause an actual problem in practice?
If a hosting plan "can support large websites", then it's probably a virtual private server or larger, and you can compile your own Python in your own VPS. Or what am I missing?
New computers are not 'multithreaded'
Tell that to the single-core yet simultaneous-multithreaded Atom N450 in my laptop, which injects the second thread's instructions into the first thread's pipeline bubbles and vice versa.
There's the old parable about two programmers that are told to double the speed of their program. One guy spends a month rewriting the core in assembly and using hardware acceleration. The other guy waits 6 months and buys a new computer.
The second guy failed because the deadline was in three months. Or the second guy failed because other customers running the program on their own computers weren't likewise willing to buy a new computer just to run one program.
Then why the hell are they downloading python - a programming language???
To run an application written in that programming language. For the same reason that\ downloads Flash Player to run SWFs and Java to run JARs and a web browser to run web applications, one downloads Python to run Python applications.
What should a well-behaved program do with bytes objects pulled from a database that already contains plenty of invalid encoding?
various internal applications and systems not needing to support more than one language
If a program supports just one language, and that one language is either standard written Chinese or Hindi, then how should it work with even one language without Unicode?
One common design pattern for ensuring that unmanaged resources get released is try/finally, which works with JVM, CLR, and CPython, but doesn't work so well with resources held longer than one method.
And this is why it is best practice in Python to use the with statement
This is explicitly a syntactic sugar for the try/finally approach (PEP 343) and shares some of its disadvantages. Like the lifetime of a resource allocated in try and cleaned up in finally, the lifetime of a resource allocated at the start of a with block and cleaned up at the end of said block is limited to the lifetime of that block. If you allocate an object in one method, such as the __init__ method of a class, and retain it, how do you ensure that it gets freed? For example, a class that reads a stream of objects out of a file may implement the iterator protocol: open the file in __init__, return an object in __next__ (next in Python 2), and close the file when?
Shutting down!
I remember when Perl beat everything I or others benchmarked against it (way before Lua existed.) When it started losing was when other languages began to cheat by dynamically compiling themselves. If you remove the compiling, Perl probably still wins. Sadly, the complexity and power of perl to hack with it's own interpreter makes it extremely difficult to implement a dynamic compiler for it. Somebody with the resources probably could pull it off... but nobody seems to be trying and Perl 6 effectively dead ended the language. So we will never get fixed threading in Perl 5 or whatever constraints are needed to allow dynamic compiling... which is what Perl 6 should have done instead of making a Virtual Machine that the language can hack with like it did with it's own interpreter. Although, if they ever get Perl 6 and Perrot done... It will be quite powerful to have a VM run mixed languages seamlessly and possibly get a dynamic compiler as well.
If you are writing an object designed to work with with, you do the cleanup in its __exit__() method function.
So how does the object ensure that every other object that owns it wraps its allocation with with?
sugar can be sweet. I think it's an improvement
I agree with you that with is a syntactic improvement. I was arguing about program semantics that make with necessary, especially when object lifetimes don't correspond directly to program blocks.
You would close the file inside __next__() when you hit end-of-file and right before you raise the StopIteration exception. Am I overlooking something?
You are overlooking the case in which the caller stops using the iterator before the iterator completes, such as a for loop that hits a break statement or an exception not caught within the loop.
def average(seq): return sum(seq) / len(seq)
Or in Python 3.4 or later: from statistics import mean as average
Can you implement a simple average() function in Python 3 that satisfies the specifications mentioned above?
I'm not clear what average([-10, -11]) should return under your specification. Or would it raise a ValueError? In general, the average of integers is not an integer but a rational, because integers are not closed under mean. Mean would have to promote to a type capable of representing (or at least approximating) rationals. Python has no built-in exact rational type, but it does have float.
As mentioned in the spec, int list should return an int.
Please clarify the spec further: If the list contains an int and a float (such as average([10, 12.0])), should the result be returning a float or raising a ValueError? I admit that my process of requirements elicitation sounds like I'm asking a lot of nitpicky questions, but it's necessary to avoid an underspecified function. "What Python used to do prior to the addition of true division in 2.2" is still underspecified if the function is ported to any language other than Python.
So your function should return func([-10,-11]) --> (-10-11)/2 = -11 under the rules of integer division.
Until now you have left "the rules of integer division" unspecified. Floor division in Python 2.2+ rounds toward negative infinity, but C integer division rounds toward 0. So you've added to the spec.
If your spec is "do what Java does", then Java integer division rounds toward zero, and Java integer addition wraps within (signed) 32 bits. This means the naive Python 2 implementation gets it wrong too, and we're back to square one where we're scanning the list for being all ints and using the "act like Java" method if no float or complex types are found in the array.