Slashdot Mirror
Interview With Python Creator Guido Van Rossum (techrocket.com)
The online programming school Tech Rocket just published a new interview with Guido van Rossum, the creator of Python. "Looking back I don't think I ever really doubted Python, and I always had fun," he tells the site. "I had a lot of doubts about myself, but Python's ever-increasing success, and encouragement from people to whom I looked up (even Larry Wall!), made me forget that."
He describes what it's like being Python's Benevolent Dictator for Life, and says that the most astonishing thing he's seen built with Python is "probaby the Dropbox server. Two million lines of code and counting, and it serves hundreds of millions of users." And he leaves aspiring programmers with this advice. "Don't do something you don't enjoy just because it looks lucrative -- that's where the competition will be fiercest, and because you don't enjoy it, you'll lose out to others who are more motivated."
He describes what it's like being Python's Benevolent Dictator for Life, and says that the most astonishing thing he's seen built with Python is "probaby the Dropbox server. Two million lines of code and counting, and it serves hundreds of millions of users." And he leaves aspiring programmers with this advice. "Don't do something you don't enjoy just because it looks lucrative -- that's where the competition will be fiercest, and because you don't enjoy it, you'll lose out to others who are more motivated."
Because he understands the problem Dropbox is trying to solve better than you do.
I was expecting an interview with John Cleese.
When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
He meant to say 2 million functions in one line of code.
When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
It's the "serves hundreds of millions of users" part. I know "web scale" is an overly used buzzword but it is in fact very difficult to serve huge numbers of users. There's also collaboration tools, an API for other developers to integrate Dropbox into their own software, etc.
You had me up till you said Java.
Works as long as what you make in that language won't get too big.
Two million lines of code for DropBox is pretty impressive for a script kiddie language.
Works as long as what you make in that language won't get too big.
I think that's true of all languages.
My impression (having written O(10k) line Python programs for fun) is that it's quite capable of implementing the kind of O(1m) line COBOL, PL/1 & C systems I worked on in the 80s and 90s - probably in a tenth of the size too. It's also infinitely better for "Rapid Application Development" (is RAD still a thing?) than the 4GLs and BASIC variants we used back then.
Python is one of the few bright spots in the evolution of programming languages over the past 30 years. Too bad van Rossum fucked up the transition to version 3.
Learn how to use a compiled language like Java.
If you have a need for speed, compile Python code to C binary with Cython.
http://cython.org/
"seen built" didn't say he built it himself.
Website Just Down For Me? Find out
Java is a horrible language to use compared to Python. At least say something smart sounding like Go or Haskell.
Like the java virtual machine, for which you compile to BYTE CODE!!!1!
Serpent, maybe?
Serpent is a real-time scripting language inspired by Python but completely reimplemented to support real-time garbage collection and multiple instances of the virtual machines running on independent threads.
https://sourceforge.net/projects/serpent/
He meant to say 2 million functions in one line of code.
We're talking about python, not perl.
#DeleteChrome
If you're still running Python 2 code, you should into the mirror see where the problem lies.
The problem lies far beyond the mirror. It lies in the dozens of libraries that I rely on that have not upgraded to Python 3 (despite pleas to the original authors). It relies on the hundreds of hours of porting, testing, and double checking that I will need to do to move my code over to Python 3. The improvements in Python 3 are real, but they are not worth the enormous burden imposed on everyone to get their code to be Python 3 compliant.
Fast Federal Court and I.T.C. updates
I know this article was focused on Guido's softer side, but would have liked them to mention the elephant in the room: the move from Python 2 to 3. This been a huge resource drain on the entire community and many (including me) remain unconvinced that it was the right decision. It would have been nice if the topic was broached.
Fast Federal Court and I.T.C. updates
QED. Search your feelings, Luke. You will see that Perl is the path to enlightenment.
What I'm most curious about is why the Python community is so much nicer to deal with than the Rust programming language's community.
I've seen so much help and compassion from the Python community. I'm not just talking about questions that I've asked, but questions and problems that I've witnessed other people ask about. The Python community is always so supportive, and in all of the right ways. They aren't smug or holier-than-thou. They don't tell you "you're doing things wrong". They may ask for more information, but then they always provide a helpful response without resorting to snark or a bad attitude. They're friendly and pleasant people to deal with.
Then there's the Rust community. In my opinion and experience, it's so much worse than the Python community to deal with. The smugness is pretty extreme from them. They will often tell you that you're "doing it wrong", even when you're doing something a certain way for a very good reason that they just don't/can't comprehend. It's almost like they're more concerned with their own feelings than with helping somebody who is in needed. And you'd better hope that you don't say the wrong thing at a site like Hacker News or reddit, because the Rust downvote squads will take you out quite rapidly! The last thing they want to hear is legitimate criticism of Rust. All of the emphasis that the Rust community places on the Rust Code of Conduct and the Rust Moderation Team apparently doesn't do any good, because the Rust community is not one that I like to deal with!
Maybe it's a maturity/age thing that causes the Python community to be so much nicer to deal with? The Python community members are generally older and more experienced. The Rust community members are typically much younger, with some of them not even being out of high school yet. Maybe the Python community is better to deal with because the Python community members have more professional real-world experience, while Rust community members often have none at all? The Python community members are more laid back and confident with providing help, while the more immature Rust community members are on a quest to inflate their sense of self worth?
Actually, Guido didn't "fuck up" the transition to Python 3. It was done almost perfectly.
People using Python 2 haven't been affected by it much. They can upgrade whenever they want, and Python 2 has still seen excellent support from the Python maintainers.
Meanwhile, Python 3 has been able to resolve some serious problems with Python 2, providing a much richer and more useful language. Developers have been moving to Python 3 at their own pace, which is exactly how it should be!
Now that Python 3 has proven itself, it's seeing widespread adoption within the Python community. It's what almost everyone is using for new code.
If you want to see an example of a real fuck up, look at Perl 6. That's a real disaster! Here we are, over 15 years later, and there's still so little to show. Rakudo is an awful joke. Even the Perl 6 logo is goddamn childish.
Another example of a really fucked up programming language is Rust. They kept promising Rust 1.0, and then it kept getting pushed back again and again. But while this was happening the language was changing significantly almost every day. So code you wrote on a Monday often wouldn't compile at all by the following Wednesday, and would need significant reworking by the following Friday! The first impression a lot of people got about Rust was a lack of stability and having to rewrite their code every week. Even the 1.0 release was pathetic. Despite being "stable" releases, it took them up until Rust 1.6 to get their libcore stable!
Here's a list. Only 30 out of the top 360 packages aren't ready for Python 3.
http://py3readiness.org/
I was just talking to an old Co-Worker from a C++ company I worked at a few years back. He asked "So what are you doing lately" and I told him I'm working on my thesis, which is titled "Ruby's a Terrible Programming Language, And You're A Terrible Programmer For Liking It". Then I cited a number of my complaints -- being able to add arbitrary functions to a live object, never knowing where to look for the interface definition of parameter objects, need to extensively test all execution paths of production code (Which no one ever does,) odd syntactic quirks and changes in syntax between language versions. He laughed and said he had exactly the same complaints about Python. You see, Object Oriented Programming was invented to reduce maintenance costs for completed projects, because that's where 90% of your expenses with the project will be. Ruby, at least, and apparently Python as well according to my friend's complaints, were invented to make the cheap part of the development process "easier", while at the same time letting the language fanboys pat themselves on the back about what clever programmers they are. This is exactly the opposite of software "engineering".
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
Hey I could do that in C with ten million lines of code!
http://michaelsmith.id.au
Comment removed based on user account deletion
Why 2M lines? At a guess, core functionality is a few thousand lines or maybe a few tens of thousands of lines. Plus a LOT of management software -- report generators, collecting garbage at the application level, etc, etc,etc, . And it has to talk to its users through HTML which has a lot of virtues, but is extraordinarily klunky at times. It does probably email. And it replicates ftp sorta? It distributes work over a vast complex of servers. It presumably does billing, and payments, and all that stuff -- 500 million users is a few more than I'd entrust Intuit's off the shelf software to do. (Actually, 1 account may be more than I'd trust to Intuit). And it tries to be secure which means you can multiply the complexity of even the simplest undertaking by a factor of about seven.
But you're right. 2M lines is a lot of code.
You can't see ANYTHING from a car, You've got to get out of the goddamned contraption and walk...Edward Abbey
Its funny. I have heard Java described as the new Cobol. Its not so much that the java language is verbose. Its more that its ecosystem of APIs is verbose in the way you go about using them, and the scalabilty of the language seems to cause software architectures to grow out of control.
http://michaelsmith.id.au
Perl is just right except for the multiple ways of returning an error. Is it undefined? Or zero? Or a null?
There's no null in Perl (5). Well, 'undef' is technically a null, but that's covered by your 'undefined'.
Both zero and undefined are false when evaluated in a boolean manner. So return either. It doesn't really matter most of the time.
I prefer to return an empty list: just a plain return; does that.
"Slow Down Cowboy! It's been 58 minutes since you last successfully posted a comment" -- slashdot, driving users away.
I can easily see half that change coming just from gaining access to Jinja2. I've worked extensively with a huge array of templating languages in recent years, and while they all have basically the same syntax none of them are anywhere near the sheer unadulterated power that Jinja2 offers.
It took me a while to figure out just what it was that Jinja2 had that they all lacked and which made such a massive difference. I finally nailed it down. All of them have, right in the beginning of the manual guides on how to write helper functions to extend the language. Jinja2 barely mentions that. The reason is simple: it hardly ever needs them. This is because Jinja2 exposes any data you pass to it as the original python object - with all that object's methods available inside the template.
That's extremely useful. The problem with the "helper methods" approach is that you're screwed whenever the thing running the template is not code you have access to (or can afford to alter). In a corporate environment you may be using a tool like consul-template which you really don't want to maintain a local derivative off, so you can't use any functions in your templates that their derivative of golang-template did not already include. With Jinja2 even with somebody else's code you still have access to every method the data type object exposes.
So if you have to get the overlap of two lists in your template from two disparate upstream data sources you don't have to hope the template language includes the right list operators, you just use the built-in methods of the list objects directly.
Have you seen what a list-overlap seeker written in pure golang-template looks like ? It's a hellish maze of deeply nested loops and if statements and the output enmeshed inside all that gunk because golang lacks a sufficient set method to create new variables that can let you pre-construct your new list before looping over it.
I had to do just that recently, the golang template was well over 500 lines of unreadable and barely navigatable junk - I did the same thing in less than 5 lines of Jinja2.
So I can easily imagine that for a web app that's highly template driven you could end up with a huge amount of code that exists within the templates (or as helper-functions to the template library) which you can throw away if you redo it in python.
Unicode killed the ASCII-art *
I suspect a lot of that code lies in redundancy and load-sharing systems. Python is notoriously bad at multithreading (so much so that most python coders prefer a library called multiprocess that fakes multithreading by spawning new python processses entirely) - so load-balancing, load-sharing and redundancy under heavy use are problems python is particularly bad at (not the language so much as the architecture of the implementation to be fair).
So it's likely a great deal of the code is dedicated to solutions to those difficult problems. It also rather depends on how you count it. Other things that could contribute large amounts of code:
- They likely use a custom application server (in order to implement all that redundancy and load-sharing)
- There's likely a significant amount of debug logs in there, and extensive logging throughout (you need that if you're going to keep something like this maintained and find/fix problems quickly).
Finally - the comparison is not actually very fair. The Kernel is written in C - a language designed for brevity, while python is much more verbose. Python is not a language that encourages lots of one-liners, except where they can be used to avoid deep nesting (which is actively discouraged) or in purely functional calls (which are available but only encouraged for specific use-case determined algorithms where the non-functional version would actually be harder to read).
So the exact same algorithm is likely to use more lines in python than in C - and be a lot easier to read. I was not surprised to learn from this interview that a lot of his own early work was in Pascal and Algol - you can see a lot of the Wirth philosophy in Python. Python in a very real sense struck a perfect balance between readable verbosity and cruft. Java is too far to the other side, you need about 100 lines just to show a simple "exit on click" button on the screen and almost none of them have anything to do with the task at hand. The same thing in TCL/TK will take one line, in python it's about 3 lines (depending what GUI library you use).
I recently developed a fairly comprehensive GUI library in python. I was building an RPG in Pygame (need to pick that up again sometime) and pygame has no gui elements and none of the libraries are maintained, so I wrote my own. Python made this ridiculously easy for the most part. Hell at one point I actually wrote a recursive object (that is to say - an class in which one of the methods would instantiate another instance of the same class it's a method off). The class was a box containing a list of items you can select (used for things like the inventory) but when you need to add something to a box like that (say in the game editor - adding treaures into a chest) it instantiates another instance of itself, with different parameters to show you the list of possible things you can add.
That took no clever trickery whatsoever, the method just instantiated the class and operated on the object, python handled all the recursion magic entirely transparently.
That gave me (as a long time python developer) an entirely new respect for just how powerful the language really is. I also strongly suspect that in most languages that activity would have been far more complex, many languages don't even allow recursive classes after all.
Unicode killed the ASCII-art *
He joined Dropbox quite late on, they had a fully developed syncing platform for years before he joined.
He might have advanced that platform on, but its difficult to claim that he "built quite a lot of Dropbox". From what I gather, his main work is around optimising their python runtimes, compilers and interpreters.
The best advice is to find work you enjoy doing that others don't enjoy doing, which is also desired by clients. At any rate, I pursued an enjoyable thing and made a success of it so your sample size is up to 2 now.
This space intentionally left blank
Interview With Python Creator Guido Van Rossum
Well, I tried to read it, since I'm a huge fan of Python. But one of the paragraphs was indented in a slightly different way than all the others, so I couldn't.
The Kernel is written in C - a language designed for brevity, while python is much more verbose. ... So the exact same algorithm is likely to use more lines in python than in C
I dunno, YMMV, but to me the opposite seems to pretty much always be the case - for any non-trivial chunk of code, the Python version tends to require far fewer lines than the C equivalent. At several different companies we've ported various C modules to Python and it's common for the Python version to have only 20% (or fewer) LOC vs the C original. The reason is just the usual stuff: Python, being a much higher level language, introduces a lot of overhead but in exchange you get powerful built-in data types and have to do basically zero manual memory management.
This tends to show up in not just large libraries or apps, but even in small chunks of code. For example, below is a function from one of our network monitoring agents; as background, basically there are a bunch of different server clusters and a job monitor spits out an hourly file that lists on each line the IP address of each server and the number of errors it encountered (this is part of some legacy thing we're hoping to replace, it's kind of goofy). Anyway, those files get aggregated to a central monitor, that in turn looks for various conditions and alarms if e.g. the error rates are too high.
Anyway, here's a function that reads those files and tells you which servers are seeing the most errors (it returns a list of server IP address and number of errors encountered, in descending order of number of errors):
def ServerErrorRates(reportFiles):
counts = {} # ip addr --> total errors
for filename in reportFiles:
for line in open(filename):
ip, errors = line.strip().split()
counts[ip] = counts.get(ip, 0) + int(errors)
return sorted(counts.items(), key=lambda x:x[1], reverse=True)
Nothing fancy, but doing that in straight C is likely going to take far more than 7 code statements. There's just no way to perform the same work in C in anything close to 7 statements. It's not a knock on C, the two tools are just optimized for different things.
As another example, I just took a peek at our HTTP server library. The whole thing is a single file of less than 800 LOC, and that handles all HTTP request/response handling including header reading/writing, file uploads, cookies, websocket support, request routing, etc., etc. without using any of the HTTP stuff from the Python standard library. The C equivalent would certainly be several times as many lines of code.
I think you could maybe argue that individual statements in Python are more verbose than C, but it's common for each Python statement to be the equivalent of several C statements (and/or many statements in C are simply not required in Python), so on the whole Python programs end up being way more concise. Occasionally I'll see an exception, but it's just that - an exception.
This has improved quite a bit recently. I still write all my stuff to run on both 2.7 or 3.x, but more and more I'm able to use 3 exclusively. Yeah, there are a few try: blocks and imports at the top of all my files, but it's not so burdensome. It's not like they are forked - 2.7 can be made to behave very similarly to 3.x. Just run your tests in both versions.
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
Parent is being modded into oblivion, but it's a fair question: Why would a file server need 2M lines of code?
Seriously, even with all the user stuff and housekeeping, it seems like an insane amount of code.
Just cruising through this digital world at 33 1/3 rpm...
Thats actually a good point I had not considered. But I do think at the level of something as close to the metal as file management that pythons verbose names and syntax would make it bigger than C. At application level the higher level of abstraction should win for python.
Unicode killed the ASCII-art *
Yeah, that's a good point. I think an even better example might be low level bit twiddling of a byte stream and conversion back and forth between a struct in memory and a packed binary file format - C rules there and in Python you end up jumping through weird hoops a lot of times. The struct module helps a lot (though to your point it's more verbose than the same task in C), but only to a point.
There's probably working, and there's actually working. When you have a complex system that a business actually depends on, running 2to3 absolutely requires that everything be re-tested. The larger the system is -- and that can mean interactions with libraries, databases, other languages, etc. -- the larger that testing job is, and it gets larger in a nonlinear fashion with the amount of code as interactions multiply.
However, Python 2.x isn't going to go anywhere. If you have a substantial system or systems written in it, and it's doing what it supposed to be doing, there's actually no reason to move it over. If you want to, you can write new stuff in 3.x; no reason they both can't exist on the same machine(s), either. Either one can call the other. Nothing to it.
Personally, so far at least, I have no specific need for 3.x, and so haven't bothered with it for anything serious, but I'm not against using it if some reason arises that makes it more useful than 2.x. I can't say I really object to 2.x becoming stable because development is going towards 3.x, either. Again, it reduces the need to re-test, and it keeps the unit testing mechanisms stable as well.
I've fallen off your lawn, and I can't get up.
You're thinking of Emacs. C-x M-c M-facebook.
Check out my sci-fi/humor trilogy at PatriotsBooks.
It's not about "a space", it's about levels of indentation.
The answer is basically that braces can match at different places on different lines and be perfectly valid; and many people write code just that way.
if condition
doThis
if otherCondition
doThisToo
else
humAFewBars
scratchItch
else
doThat
is clearer than:
if condition { doThis ifOtherCondition { doThisToo }
else { humAFewBars scratchItch }} else { doThat }
Structured whitespace delineates code blocks in a much more visually obvious manner. This, in turn, makes understanding what you're looking at much easier. Such understanding aids in writing, maintaining, and altering code. Even more so when it is otherPeoplesCode.
I've fallen off your lawn, and I can't get up.
#!/usr/bin/python3
Shebang lines like that didn't work on Windows from 3.0 through 3.2. Windows determines which executable to run by examining the portion of the filename after the final period character. Only in version 3.3 did Python gain py.exe, a shebang line processor for Windows implementing PEP 397.
Python has Sqlite support and javascript does not
You were saying? node-sqlite3
I think every successful programmer is probably an example of applying this advice. I've never met a successful programmer that hated programming, and most people hate programming. But the field has gotten more crowded, so perhaps many people are now doing it purely for the money.
I think we've pushed this "anyone can grow up to be president" thing too far.