TurboGears: Python on Rails?

gcantallopsr writes "If you liked Ruby on Rails and its 15m intro video (.mov) you will probably like TurboGears and its 20 minute wiki tutorial. (.mov) It shows you the development of a simple wiki in just 20 minutes, and there is a text version of the tutorial. TurboGears uses Python, SQLObject, CherryPy, Kid, MochiKit and some extra pythonic glue to help you to (in their own words) 'Create a database-driven, ready-to-extend application in minutes. All with designer friendly templates, easy AJAX on the browser side and on the server side, not a single SQL query in sight with code that is as natural as writing a function.'"

Comments

20 mins vs 15 mins!!

[VC]

How do the python crowd expect to get taken seriously when implementing a wiki takes an whopping 125% as long in python as in ruby!!!!!!????

(oh wait, they did ajax as well.. ;-)

Re:20 mins vs 15 mins!!

[jurt1235]

How does anybody get taken seriously anymore when creating just another wiki?

Re:20 mins vs 15 mins!!

[gijsvanswaaij]

Some wikis can be very useful. Take, for example, the wikis that often go on the sites of opensource projects. They often contain info that is just not suitable for Wikipedia.

Re:20 mins vs 15 mins!!

[timmarhy]

i think you mean an extra 25%.

Re:20 mins vs 15 mins!!

[Anonymous Coward]

wouldn't that be 33% longer?

Re:20 mins vs 15 mins!!

[Gherald]

I think he's refering to just another wiki engine.

Wiki sites are _extremely_ useful for all sorts of things far transcending OSS and Wikipedia both.

Re:20 mins vs 15 mins!!

[gmf]

How do the python crowd expect to get taken seriously when implementing a wiki takes an whopping 125% as long in python as in ruby!!!!!!????

How do you expect to get taken seriously when you don't know that it's 133%, not 125%?

Re:20 mins vs 15 mins!!

[Anonymous Coward]

If you are able to get Frist Psot, clearly you didn't WTFV.

Re:20 mins vs 15 mins!!

[VC]

125% is 25% extra..

Re:20 mins vs 15 mins!!

[ratpack91]

You still got the percentage wrong though. My advice is stop digging the hole deeper.

Re:20 mins vs 15 mins!!

[badmammajamma]

Yeah and it only takes 4 lines of Ruby code whereas in Python it takes 15!

Re:20 mins vs 15 mins!!

[Anonymous Coward]

The original "125% as long" is correct. It means the same thing as "25% longer." If you said "25% as long," you would be talking about something shorter.

no sql?

[ambilio]

...not a single SQL query in sight...

Why is this an advantage?

Re:no sql?

[quigonn]

It shows a high level of abstraction when you access the DB by simply loading/persisting objects instead of having to handle queries, result sets, records and all the other low-level stuff. And abstraction (in this case) is good, as it helps the developer concentrate on the relevant parts of the program.

Re:no sql?

[wootest]

Other people have answered in-depth, but the short answer is:

a) You do not need to worry about which vendor's dialect of SQL syntax you're using - provided you know how to create and populate the tables in any database system, you can switch at the drop of a hat if you need or want to.

b) Provided the layers are stable, it protects you from SQL query injections. The abstraction layer does the escaping for you.

c) Abstracted queries makes queries 'just another function/method call', and you get ordinary data structures back. This in combination with a) and b) and a competent framework (Rails, Django, TurboGears, Cake, Trax, WebObjects) makes coding much quicker as you don't have to keep the semantics of SQL and your database in mind - just the model itself.

There are *many* nuances to this, but the above three are some of the most pertinent ones. Peruse the other comments if you want to get in-depth.

Re:no sql?

[VGPowerlord]

Why is this an advantage?

It makes it easier for the programmer.

However, I really wrote this to say what it is not an advantage. I have two words for you: Outer Joins.

First of all, an Outer Join (of which LEFT JOIN is the most common) is used to get information from two tables, whether or not the information in one table has matching information in the other table. A real world example would be a list of customers and the number of orders they've placed with your business. With a normal (inner) join, customers who have never placed an order would not be listed.

I keep seeing people mentioned SQLObject, so I'll pull up a selected reference to the SQLObject FAQ: "How can I do a LEFT JOIN?."

The Simple method on that page is what I believe is referred to as the "1+N" problem. 1 Query becomes 1+N queries, where N is the number of results in the first query's result set. Needless to say, this doesn't scale well.

The Efficient method is actually much more complicated than the SQL to do the query, and it still takes 2 queries to boot.

This is why you'll never see a large business relying on an SQL builder to build queries. In the hands of someone who knows what they're doing, hand written queries are optimized much better than generated ones.

Re:no sql?

[mixmasterjake]

Persistance layers are cool for a lot of reasons. Though, it definitely takes some getting used to for those of us who have developed a lot of SQL driven applications.

Don't let anyone fool you into thinking that a persistance layer will be less development work for you - I have found this to be untrue. I can use automated tools to get myself 80% of the way there. For anything substantial, though, it always seem to wind up being more work as I figure out how to configure & trick the persistance layer into giving me my data in the most efficient way. This can be frustrating when you know how to accomplish the same thing in 5 seconds using plain SQL. Maybe it's just me?

But, if you do manage to get over the hump, the benefit is that your business logic layer is very clean with no DB code whatsoever. If you use it properly, you can get 100% separation between these layers.

If you're using a strongly typed language, you get the added bonus of compile-time checks for certain illegal assigment errors. You don't have to fumble around with things like converting dates into SQL. You don't have to check for SQL injection. Lots of other little things.

Re:no sql?

[ammoQ]

Because SQL with its subqueries, joins, unions, aggregate function etc. is way to powerfull. It doesn't match the OO paradigm, therefore it must be bad.
It's obviously much more efficient to retrieve a million records, create a million objects and count those objects with a certain property than runnig a simple "select count(*) from x where y=z" against the database.
If this assumption does not hold, you should build caching, optimizations etc. into the persistence framework, until you have a database sitting on top of a database.

Re:no sql?

[Craig Ringer]

SQL its self isn't the problem. SQL in apps, however:

- is an absolute PITA to assemble dynamically, involving lots of icky string mangling;
- is often rather ugly; and
- is not trivial to do safely - it's way too easy to expose SQL injection holes.

Re:no sql?

[tweek]

Finally someone after my own heart.

Re:no sql?

[sunya]

found this on http://www.eod.com/devil/

Ruby on Rails, proper noun: The VisualBasic of the
Web, making the first ninety percent of what you need to do easy by
making the last ten percent impossible.

Re:no sql?

[ammoQ]

String mangling is only necessary and SQL injection is only possible if you don't have an API supporting host variables.
Therefore (and for some other reasons) one should use host variables whenever possible.

Re:no sql?

[jrcamp]

For anything substantial, though, it always seem to wind up being more work as I figure out how to configure & trick the persistance layer into giving me my data in the most efficient way. This can be frustrating when you know how to accomplish the same thing in 5 seconds using plain SQL. Maybe it's just me?

If there is something you need to specifically query by hand in SQL you just use find_by_sql in Rails. I'm not sure how this is configuring and tricking it?

Re:no sql?

[Anonymous Coward]

Nobody has mentioned that abstracting database rows into objects means that you can use your application logic to cache data. This means that you do not have to transfer static data from the database on each page request and you can much much more easilly cache data than you can using the SQL cache. SQL server side caching has its place but it doesnt save you the network overhead.

I have used this method and it creates incredibly scalable and fast applications with only a few specific simple queries.

Dont get me wrong, returning 10,000 results from the database, and triming that down to 1,000 is very stupid. Something like SELECT product_name, product_description, product_price FROM product WHERE id=10; is something that can benefit hugely from this style.

Re:no sql?

[Craig Ringer]

For simple cases that's true. If you're doing more complex, dynamic queries that's not really accurate. In particular, conditionally specifying clauses for WHERE and HAVING, and additional fields for ORDER BY, is annoying.

I've used languages with built-in database integration. In particular I did a bit with Progress 4GL. Progress isn't magic by any stretch (and it's tied to a database that makes MySQL 3 look advanced), but in my experience it did significantly reduce the frequency with which I had to start fiddling with query strings. Too bad it was such a dog in every other way.

These days I suspect many of the things I've been doing by fiddling with query strings may be better done with the application of a few stored procedures, possibly procedural stored procedures. There must be some better way.

Re:no sql?

[holle2]

It shows a high level of abstraction when you access the DB by simply loading/persisting objects [...]

Last time we used an object-to-sql mapper was quite some time ago, so my info might be outdated:

We attempted to create objects (in Python) to store a lot of attributes ( > 30 ). The design explicitly asked for a "flat" database.
All this happened inside Zope/Plone, so we tried out the Archetypes, which come with a hand attributes (or better PropertySheet) to SQL mapper. But the code was that ugly, it created a single SQL insert/update statement for each property (attribute) even if it did not change.
This resulted in an extremely long running "save" operation that you could simply throw the code out of the project ... . The data is now stored in the Zopes own ZODB and gets archived after a while in a SQL DB.
For archiving purposes a hand-crafted SQL statement is used that runs lightning fast :-) .

On the other hand, I have seen Java code querying an SQL DB using the Oracles TopLink product to abstract the underlying database. The code was so gross, you simply felt the urge to go and wash yourself after looking at it.

The result of this experience is:

• If you want to concentrate on your object oriented approach, stick to databases that can handle objects, like the ZODB (usable w/o Zope if you like).
• If you have to access mass-data from legacy systems (like e.g. master product data), you have to program SQL, since nothing is faster than that. Make sure the results can be converted into something like a list of dictionaries so handling of the results is more natural in an object oriented manner.
• Every good programmer should be able to switch between programming languages and be able to access data stored in legacy systems as well. Sometimes the data comes only via an ODBC connector from an old AS/400, but if that is the source of the master data, you have to live with it.
• SQL databases are not the primary choice to store objects and/or tree like structures. They were designed it the "good ole' times" when everybody though that relational databases were cool. Sometimes it can be much more useful to store a tree in an object persistent storage and put the node data into an SQL DB (for performance).
• For debugging: If you use SQL code in your project you can actually print out the statements before they get send over to the database so you can simply copy them from your logfile (you have a logfile, do you ?) and run them manually to see if they produce the outcome you want. This is often more useful than enabling query logging on the server since digging through thousands of lines of log-messages can be rather time consuming.
• For designing: You can build your queries step by step in a "Query Analyzer" using some example data and then embed the results into your code (including the exchange of fixed teststring with variable names).
• For performance: You might know that you have to run a query 3000 times in a row with different variables (e.g. insert into ...) but you object mapper does not know that fact. Thus you can use one prepared SQL statement and fill in the variables upon execution. This saves a hell lot of time (depending on your database system, some systems are slow either way).

Ok, I got my flame shield up, let 'em come ...

Re:no sql?

[multi io]

For simple cases that's true. If you're doing more complex, dynamic queries that's not really accurate. In particular, conditionally specifying clauses for WHERE and HAVING, and additional fields for ORDER BY, is annoying. In any decent programming language it's a two-or-less-hours-job to hack up a library that makes code along the lines of

stmt = new SqlStatement("select foo,bar from table where x=")+x+" and y="+y
if (something) {
stmt += " and (hi="+hi+" or ho="+ho+")"
}
stmt += " order by bar"

dbconn.execute(stmt)

work, including automatic quoting of all the parameters (hint: overwrite "+" on the "SqlStatement" class). Unless the language/library already has some such facility.

I even did this in Java once, and while that didn't end up as syntactically pleasing as the above, it was still straightforward to write and use.

Re:no sql?

[Ian Bicking]

Well, no points for the docs on this one (hey, it's open source) but there is some help for outer joins noted here.

Re:no sql?

[mixmasterjake]

Well, my personal experience is with hibenate. It might not work quite the same as SQLObject.

hibernate does have a feature that allows you to use direct SQL calls. However, it's definitely no magic bullet for solving every problem. Also, using native SQL seems to be recommended only as a last resort by OO purists and ORM snobs such as myself. The more you add, the more you lose your DB independence.

One simple example of a tricky situation is that I may want data from tables that requires loading 75% of my entire graph. I practically have to load the whole database just to get a few fields. So I have to dig deep into the mapping files & loading strategies. I may optimize these for one view at the expense of another. So, I may have to compromise at times.

To me, it is worth the trade-off. I'm pro-ORM. But, not because it necessarily saves me time. I would still say that trickery, voodoo and a bit of black magic are all required when using an ORM on a complex project.

Re:no sql?

[IAmTheDave]

Thus you can use one prepared SQL statement and fill in the variables upon execution. This saves a hell lot of time (depending on your database system, some systems are slow either way).

This "abstraction" has been handled for a lot of us at the database level for some time. In all of our projects (some close to 100k lines of code, not huge but not small) there is no SQL in the actual compiled software. It exists in stored procedures in the database itself. These procedures run nice and fast, and "abstract" queries from the software. Software asks for a set of data, ANSI (or T/P)SQL that the software has no care or knowledge about save the name and params, executes on the DB side.

IMHO, queries generated automatically by Rails - Ruby or Python - will never be able to be quite as efficient as writing them yourself. Love stored procs - cmon MySQL 5! (and yay Postgres, MS SQL, Oracle, etc...)

Re:no sql?

[feNIX77]

In my experience ORM frameworks are fantastic for simple DB schemas/applications. For more complicated cases they may work, but at a performance cost - such as pulling considerably more data than required. Putting into consideration hardware and time restrictions, this may be (more than) acceptable. Ideally, I would prefer to create the most efficient solution, which has thus far not been via an ORM framework...maybe this is a sign I'm getting old :)

My Experience

[HogynCymraeg]

I've used Python/SQLObject/CherryPy on a project before. It's very quick to code something useful. SQLObject will change the way you think about how you integrate DBs to web applications. All in all, it's well worth checking out.

Re:My Experience

[Anonymous Coward]

What are the performance tradeoffs? Personally I find the kind of abstraction presented via frameworks is uncalled for and leads to slow, bloated webapps but prototyping is another story alltogether.

What is natural about that?

[barcodez]

as natural as writing a function.

So what is so natural about writing a function? I would have though if it is based on Python it would be OO with behavioral methods rather than procedural function calls.

Why is everyone clambering to find the 'next' language for programming in the small when quite clearly a good language for programming in the large is what is required - at least for enterprise applications (I'm going to include wikis in that for now).

Re:What is natural about that?

[Anonymous Coward]

Because no language yet can out-do Java in that domain. This is most likely due to there being relatively low demand for things that Java can't already do.

Hopefully, this kind of thing will spur more interest and eventually result in something more than a toy framework.

Re:What is natural about that?

[Anonymous Coward]

.. OO with behavioral methods rather than procedural function calls.

Me thinks you really didn't understand either one.

Re:What is natural about that?

[llamaguy]

Python isn't 'pure' OO - it can do functional programming as well with only minor irritations (ie, the lack of genuine anonymous functions - lambda in Python doesn't cut it!). That's why you have to include the brackets in a function call, because otherwise you're just referencing the function itself, as in:

def foo():
    print 'Foo!'
    def bar():
        print 'Bar!'
    return bar
baz = foo
qig = foo()

Functions are first-class objects as well - end this class war!

There are too many ways to answer that

[Anonymous Coward]

SQL is an awful language, and don't let anyone tell you otherwise. It is highly non-portable, insofar as it is non-trivial, it is not easy to parse, it is difficult to debug, it is a lot of work to test, it's tedious, it's time-consuming, and, worst of all, it's an awful mishmash of "human-readable" code with grammars. The resulting mess is something that is cumbersome, ugly, and definitely not human readable.

Re:There are too many ways to answer that

[ceeam]

As opposed to...?

Re:There are too many ways to answer that

[cpghost]

As opposed to...?

As opposed to using transparent persisting of objects, e.g. with ZODB (which doesn't use SQL at all) or other persistance frameworks (which translate everything to SQL behind the scenes).

Re:There are too many ways to answer that

[Hackeron]

As opposed to python!

Re:There are too many ways to answer that

[Bogtha]

As opposed to SQLObject, which was mentioned in the summary. It allows you to make the objects in your program persistent by transparently storing them in a traditional SQL database. All you have to do is inherit from a base class and set/get the object's attributes normally.

Re:There are too many ways to answer that

[tigersha]

Ok, let's play manager and go through that checklist.

a) It is highly non-portable.

Python has ONE (ok, one and a half if you count Jython) implementation, and between the two parts its not portable either. So the argument does not really apply. Draw.

b) non-trivial.

As opposed to Python? Go to you bookshelf, pick a book on SQL and book on Python. Weigh them. Score one for SQL.

c) it is not easy to parse.

SQL is almost certainly easier to parse than Python. Score one for SQL.

d) it is difficult to debug.

Python is easier to debug than SQL? They are both pretty much the same. Draw.

e) it is a lot of work to test.

Lets see. Untyped language means YOU write the checks. Equally bad here. Draw

f) it's tedious.

This is the part where you going into fantasy land. Try writing something like

SELECT * FROM address, transaction, lineitem where lineitem.cost > 200 and address.city = 'New New York'.

That is ONE line of SQL. ONE. In Python it would be a tad more than that.

This is entirely the point. SQL's main power lies in these set-like things. NO OO language is good at that. And to boot, SQL only gives you the datayou need, not the entire object and all its unecessary paraphenalia, which Python (and Java and Ruby) does. Working with sets of things that contains only projections of objects is the main things with relational databases and a major, major problem with OO languages in general. Use each language where it is strongest.

Score one for SQL

g) it's time-consuming

Repeat the previous query and see what is faster. Hint: it's not going to be Python. Score one for SQL.

h) it's an awful mishmash of "human-readable" code with grammars.

As opposed to what? Python? Its a draw. All computer languages are like that. Ok, admittedly brainfuck, assembly language and The lambda calculus are exceptions here, but most languages try to be a balance between machine and human readable. That is the entire friggin point of having languages.

So ladies and gentlemen, we have a winner. SQL.

TO get back to the real world, the thing with SQL is the fact that you can work with sets like your tap works with water. That is amazingly powerful. And it allows very much better optimizations than loop-based stuff for load of reasons not worth going into here. Of course Python is a good languaes. But it's not better than SQL, it's different and is used for different things.

Re:There are too many ways to answer that

[afd8856]

you can do the one line sql statament in one line python, using list comprehension.

Re:There are too many ways to answer that

[Anonymous Coward]

Wheras database abstraction lets developers write code that interacts with databases as badly as design tools let non-developers create websites without thinking about the technicalities / consequences.

D/B abstraction only works for the most trivial of databases before it becomes non-trivial itself, and you end up with people unnecesarily moving data between layers because they can't / don't know how to manipulate it directly.

I'm certainly not advocating that every developer should learn SQL, but the implication that there is a tool that allows developers to write an application without understanding the data model (or indeed that developers don't do SQL) is much like saying 'here's a tool that lets you write code without needing to know logic'.

It's certainly feasible, but it's not going to result in the best application ever. Which may be good enough for what you want to do, especially if doing it quickly is the essence.

That said, for user config and page contents and other systems where you do little more than update and search by ID, it is probably enough.

My biggest complaint about SQL is the way JDBC interacts with it - absolutely horrible - all those positional ?, little syntax validation.

Re:There are too many ways to answer that

[Anonymous Coward]

You mean step back to the 1960's era of network databases? Throw 30 years of database research out the window? Limit myself to 1-1 or 1-many relations only? No thanks. I prefer to explicitly ask the DB for what I need, not "navigate" between "objects" all the time. I can't even imagine how to handle referential integrity or constraints in an object database. Or even what those terms mean in that context.

Re:There are too many ways to answer that

[swimmar132]

Rails supports many-to-many relationships as well. And provides support for many other relationships (see acts_as_list, acts_as_tree, acts_as_nested_set, etc).

Re:There are too many ways to answer that

[Ian Bicking]

...Try writing something like

SELECT * FROM address, transaction, lineitem where lineitem.cost > 200 and address.city = 'New New York'.

OK. That's actually buggy SQL since you'll get the cross product of those tables, but I'll assume you mean:

SELECT * FROM address, transaction, lineitem where lineitem.cost > 200 and address.city = 'New New York' and lineitem.transaction_id = transaction.transaction_id and transaction.address_id = address.address_id

In SQLObject you would typically write this as:

Transaction.select((Address.q.id == Transaction.q.id) & (LineItem.q.transactionID == Transaction.q.id) & (Address.q.city == 'New New York') & (LineItem.q.cost > 200))

The code and the result are pretty much the same.

Re:There are too many ways to answer that

[pthisis]

Python has ONE (ok, one and a half if you count Jython) implementation

I'm not disputing your basic point, but there are many Python implementations. Off the top of my head:

CPython (aka "standard" Python)
Jython
Stackless Python
IronPython
PyPy

CPython, Stackless, and Jython are real production implementations.

PyPy is rapidly headed that way (and already self-hosting and passes more than 95% of the Python compliance tests). One of the primary developers is Armin Rigo, who did psyco (the specializing dynamic Python compiler that achieved speedups of up to a factor of 100 for numeric code).

IronPython looks to be abandoned.

Re:There are too many ways to answer that

[Ian Bicking]

IronPython isn't abandoned, it's just in some weird Microsoft limbo. It's by the original author of Jython, who got hired by Microsoft and has apparently been caught in some bureaucratic limbo where he spends most of his time in meetings (though another programmer internal to MS has also been allocated, from what I understand).

Re:There are too many ways to answer that

[VGPowerlord]

SELECT * FROM address, transaction, lineitem where lineitem.cost > 200 and address.city = 'New New York' and lineitem.transaction_id = transaction.transaction_id and transaction.address_id = address.address_id

That's still too complicated. JOIN and ON/USING makes it much easier to read:

SELECT *
FROM address a
JOIN transaction t USING(address_id)
JOIN lineitem l USING(transaction_id)
WHERE l.cost > 200
AND a.city = 'New New York'

As to the grandparents query, you should never ever use

SELECT *

in an application. Adding an additional row to any of the involved tables generally does bad things to the application's output.

Re:There are too many ways to answer that

[PDHoss]

Dood...

IronPython 0.9.2

Re:There are too many ways to answer that

[Anonymous Coward]

Wheras code abstraction lets developers write code that interacts with the computer as badly as design tools let non-developers create websites without thinking about the technicalities / consequences.

Code abstraction only works for the most trivial of applications before it becomes non-trivial itself, and you end up with people unnecesarily moving data between layers because they can't / don't know how to manipulate it directly.

I'm certainly not advocating that every developer should learn Assembly, but the implication that there is a tool that allows developers to write an application without understanding machine code (or indeed that developers don't do Assembly) is much like saying 'here's a tool that lets you write code without needing to know logic'.

It's certainly feasible, but it's not going to result in the best application ever. Which may be good enough for what you want to do, especially if doing it quickly is the essence.

Re:There are too many ways to answer that

[tigersha]

You can filter thigns from a list comprehension, yes. But when you have things like joins between different object lists, fugetaboutit.

Although list comprehensions are nice. Thank Haskell for that.

Video software

[The OPTiCIAN]

What software do people use for making these neat videos? (I realise this is bordering on off-topic :) )

Re:Video software

[jrockway]

Since it was made on a mac, my guess is Snapz Pro X.

http://www.ambrosiasw.com/utilities/snapzprox/

Re:Video software

[mykdavies]

What software do people use for making these neat videos?

vnc2swf

And to bring us (nearly) back on topic, the latest version is written in python!

Re:Video software

[puddpunk]

So what exactly are we supposed to moderate valid, interesting and useful questions as? I modded up a question below (thats going to be reset now that I posted this) that I was interested in and I modded it "Underrated".

What do you suggest? Funny?

Re:Video software

[Anonymous Coward]

"So what exactly are we supposed to moderate valid, interesting and useful questions as?

What do you suggest? Funny?"

Why not "Interesting"?

Re:Video software

[zootm]

Personally, I'd say "Interesting". The guy you're replying to appears to be trolling there though, don't worry about it.

Re:Video software

[Anonymous Coward]

So what exactly are we supposed to moderate valid, interesting and useful questions as?

Certainly not "insightful." What insight is there in asking what software was used? "Interesting" is the natural one, especially since you included that word in your description of the question.

Re:Video software

[daegol]

gvidcap is a nice one. Captures to standard video files.

Re:Video software

[Anonymous Coward]

Why do I appear to be trolling? Just because someone modded me 'troll'? I don't think so. And yes, interesting would have been a perfectly appropriate mod for his question.

Re:Video software

[tazzzzz]

Yes, I used Snapz Pro X. I had a couple of people ask for some form of captions for those with sound off or those that are hard of hearing... something like Camtasia on Windows could make that easier, but then I couldn't use TextMate.

Snapz Pro X was nice to use, on the whole.

Re:Video software

[Chonine]

Im also curious as to where I can get more video tutorials like this...

There are a number of programs that I want to learn how to use, and aside from learning by doing, being shown a 10 minute video like this beforehand can really get me started. This is a potential contribution to free software that people should start considering. Its a lot easier to watch a short video than read a howto or man page.

SQLObject rocks!

[SimHacker]

One of the nicest features of SQLObject is that it insulates you from the peculiararities of the database's SQL syntax, so you don't need to put any SQL code directly into the Python code (but you can if you need to for efficiency or if you're willing to write non-portable code).

The SQL database abstraction layer is an important feature of SQLObject, that Ruby on Rails doesn't currently support -- you have to write database dependent SQL code mixed in with your Ruby code.

SQLObject lets you write generic SQL queries with normal Pythonic expressions and operators, which are automatically translated into the database dependent SQL syntax by the database driver. So you don't have to change any of your Python code to port it to a different database, and you don't have to mix together two different notations, or quote a bunch of SQL strings in your Python code. It's a much more "pythonic" way of database programming than raw SQL.

The great thing is that it's so convenient and the syntax is so simple, that you can use the interactive Python shell to browse and test out and edit your database. It's trivial enough to type in some Python code on the keyboard that loops over the results of a query, performs some complex logic, and validates and edits a bunch of rows in the database. Much more powerful and easier to use than anything you can do with raw SQL.

-Don

Re:SQLObject rocks!

[VGPowerlord]

I haven't used much of Ruby on Rails, or Ruby in particular, but doesn't ActiveRecord do all this stuff for you?

Re:SQLObject rocks!

[ambilio]

Sounds like an Object-Relational Mapper. You can find these for just about any platform out there: Ruby - ActiveRecord, Java - Hibernate (and *many* more), PHP - Propel. Explicit SQL is also optional with these.

Re:SQLObject rocks!

[timmarhy]

the only problem is you miss out on all the advantages of databases such as triggers and stored functions, which ultimately shit all over your abstration layers and the "do it in code" mentality.

Re:SQLObject rocks!

[Hackeron]

stored procedures and triggers *are* code - its a hack to follow the "do it all in the database" mentality.

Its pretty nice when you have different interfaces accessing the same database with a history trigger for instance that stores every change on modify or create (not possible with delete though), but personally, I dont want anything that alters my database to be the database itself.

Call me old fashioned, but I use databases to store information in, not to program in.

Re:SQLObject rocks!

[Osty]

The SQL database abstraction layer is an important feature of SQLObject, that Ruby on Rails doesn't currently support -- you have to write database dependent SQL code mixed in with your Ruby code.

There's your problem -- you're mixing SQL code in with your app code. Bad developer! Bad! You're introducing SQL injection holes (I don't care how well you think you're filtering your input, chances are you missed something, especially if you're trying to do so in a DBMS-agnostic way). While an abstraction layer like SQLObject may be better in that regard because it's main focus is interfacing with SQL and thus should be more focused on catching these kinds of bugs, if it's mixing in dynamic SQL queries there's a good chance it also has SQL injection holes.

There's a reason why stored procedures are important, and it doesn't have anything to do with perceived performance increases from not having to re-parse the same SQL code every call (that's definitely a benefit, but it's minor). In this case, stored procedures shelter you from SQL injection attacks (assuming, of course, that your stored procedures aren't just dumb "take a string and 'exec' it as SQL code" queries). You may not agree with the people that tell you to use stored procedures to keep your business logic near the data, or that using stored procedures allows you to easily expose only the permissions required (GRANT EXECUTE on the stored proc, rather than having to figure out what tables need INSERT permissions, which ones need SELECT permissions, etc for each app that uses this database), but I don't see how you could argue their protection against injection. Yes, parameterized queries allow you to write dynamic SQL code with some level of injection protection, but that still leaves you with the downside of having SQL mixed into your code.

It's trivial enough to type in some Python code on the keyboard that loops over the results of a query, performs some complex logic, and validates and edits a bunch of rows in the database. Much more powerful and easier to use than anything you can do with raw SQL.

I call BS. First off, SQL is a set-based language. Very rarely do you need to loop over a result set (if you find yourself looping in SQL code, you're not thinking hard enough). Whatever "loop and operate" action you'd take with Python can be done quicker and more efficiently with SQL code than with app code. Second, for Python code to loop over a result set means it has to get that result set. If your database is not local to your web server (bad idea!), that means network I/O costs. You can and should avoid that cost by processing the data first at the SQL Server (hey, stored procedures again! Why return 100,000 rows that you're going to process down to 1,000 when you could just return the 1,000 processed rows in the first place? Why return anything at all when you're going to insert/update data based on what's in a different table?). Finally, while it may be "easier" for a developer proficient in Python and not the SQL dialect used by your chosen DBMS, that's a cop-out. As so many people are so fond of saying, you should use the right tool for the job. In this case, you're trying to use the hammer of Python to pound a screw, when you'd be better off with the screwdriver of SQL. Sure, you may be able to pound that screw into the wood, but it'd be quicker and easier to do it in SQL. Maybe you don't want to learn SQL, in which case you probably shouldn't be working with a database. You have a responsibility to learn how to use the tools you're going to apply to a problem.

I've intentionally ignored the problem of database portability, because a) you should be using stored procedures, which means you'll want to port them yourself anyway for maximum benefit, b) you should be using a proper DBI layer such that you just have to tell it, "I'm using Oracle now instead of Postgres, do the right thing", and c) because you're using stored procedures, you won't be switching to a DBMS that doesn't support the (*cough*MySQL*cough*) (and yes, I know MySQL "will" support them, but that's still some way off in non-beta form).

Re:SQLObject rocks!

[Osty]

Call me old fashioned, but I use databases to store information in, not to program in.

Okay, you're old fashioned. It sounds to me like you don't want a DBMS (a la Oracle, Postgres, SQL Server, MySQL *shudder*, etc), but something more along the lines of BDB -- some tuple storage system that stores your data and gives it back to you on demand. That's fine, but that has nothing to do with supposed relational (or not) database management systems. If you're not going to take advantage of the services they provide (and they do provide many, such as the previously-mentioned stored procedures and triggers, along with relational, check, default, and unique constraints for example), then you'd be better off even with just a flat file or XML-based storage system. Nothing wrong with that, but then you're not the target audience for a DBMS.

Re:SQLObject rocks!

[marcello_dl]

The SQL database abstraction layer is an important feature of SQLObject, that Ruby on Rails doesn't currently support -- you have to write database dependent SQL code mixed in with your Ruby code.

This doesn't look at all like database dependent SQL code... SQL queries are an option in Ruby on Rails, but active record tries to make you avoid them as much as possible.

Re:SQLObject rocks!

[moro_666]

  stored procedures and triggers *are* code - its a hack to follow the "do it all in the database" mentality.

no kidding, they are indeed code, but this code mostly is on the other side of the database pipe/socket/stream/ which is very important. for example if you update 500 000 records then a database side trigger is affordable ... but any hack in sqlobject or is just awfully slow and inefficient.

dont get me wrong here, i like the sql to object and vica versa mappings, the idea is cool ... but it always comes with a performance penalty, and if your project planning isnt provisional enough, you might end up using the cute toy in the very wrong space and after all the hussle you just rewrite the code to some sort of optimized stuff. and using the sql-object layer with an additional hack to get the same speed is actually much worse in terms of portability than either one of the pure implementations (either pure object-sql bridge or pure sql code without any bridge at all), and the most likely one to break too.

choose the right thing for the right job.

and no, i dont think we need another wiki ... maybe it's time to evolve to a next level and invent something totally new ?

Re:SQLObject rocks!

[Trejkaz]

ActiveRecord basically autogenerates methods to link classes together based somewhat on the data found in the SQL database.

What people are talking about here is the exact opposite... generating the SQL from the code.

Of course, Ruby also has at least one method to do this already... it just isn't part of Rails. Yet.

Re:SQLObject rocks!

[Trejkaz]

There's a reason why stored procedures are important, and it doesn't have anything to do with perceived performance increases from not having to re-parse the same SQL code every call (that's definitely a benefit, but it's minor). In this case, stored procedures shelter you from SQL injection attacks

Whereas this is true, it's also true that using "?" parameters for any user-entered values can solve the SQL injection problem just as well.

Re:SQLObject rocks!

[csirac]

The SQL database abstraction layer is an important feature of SQLObject, that Ruby on Rails doesn't currently support -- you have to write database dependent SQL code mixed in with your Ruby code.

Hmm, SQLObject sounds interesting, but I'm confused.

All the Ruby demos I've watched made a point of showing that you didn't have to write SQL statements. You wire Activerecord up to a database you've made (the "wiring" can be as simple as specifying the server/db/table - and it infers the rest so you can build what sound to be something similar to SQLObject).

Tell me - how and where does one start writing SQL when you're using Ruby's Activerecord (which sounds like a building block for making Ruby objects similar to Python's SQLObject but without writing a single line of SQL)?

I imagine the difference is in specifying the data relationships. I haven't got to that part of learning about rails yet, but IIRC you specify the one-to-many, one-to-one, etc. relationships using Activerecord. Does Python's SQLObject do all this for you via inheritence? If so, sounds nifty, but I'm still confused where you would start writing SQL code in Ruby...

I'm an expert at neither: I'm still working through Why's Poignant guide... I intend to start working with Ruby in the next few weeks. I'm curious what people have to say about one vs the other...

Re:SQLObject rocks!

[njyoder]

if it's mixing in dynamic SQL queries there's a good chance it also has SQL injection holes.

I'll take that "if" to mean you haven't read the documentation and didn't actually watch the video. Stop frothing at the mouth about SQL injection holes. If you had bothered to watch the fucking video (which you didn't), you'd notice it has specific mechanisms to deal with them.

Also, its main purpose isn't to "catch bugs", it's to make things easier on the programmer. Abstraction toolkits like this are good. God forbid someone make things easier.

I call BS. First off, SQL is a set-based language. Very rarely do you need to loop over a result set (if you find yourself looping in SQL code, you're not thinking hard enough). Whatever "loop and operate" action you'd take with Python can be done quicker and more efficiently with SQL code than with app code.

And I call BS on your BS, because you didn't actually read what you were responding to. I'm beginning to notice a pattern here, first you start frothing at the mouth based on speculation from not having RTFV (video), then you do it for not reading the comment you're responding to. They specifically used the descriptive words "powerful" and "easier" not "faster." The issue isn't effeciency, no one is claiming that TurboGears produces the most effecient code. The whole purpose of TurboGears is to make things easier on the programmer, which is what this does. Designing the logic within python, which actually does allow you to do quite a bit more in terms of 'business logic', makes things easier.

Finally, while it may be "easier" for a developer proficient in Python and not the SQL dialect used by your chosen DBMS, that's a cop-out. As so many people are so fond of saying, you should use the right tool for the job

No, even for someone very proficient in SQL it's still easier in Python due to these bindings. Of course, not having actually read anything about TurboGears (e.g. the documentation) and engaging and rampant speculation and all, you wouldn't know that.

And they ARE using the right tool for the right job, you should follow your own damn advice. TurboGears is designed for jobs where EASE is a priority over effeciency. GUESS WHAT, effeciency isn't always the #1 priority, genius.

You speak boldly, but you can't read worth shit, excercise critical thinking skills nor even follow your own advice.

I've intentionally ignored the problem of database portability, because a) you should be using stored procedures, which means you'll want to port them yourself anyway for maximum benefit, b) you should be using a proper DBI layer such that you just have to tell it, "I'm using Oracle now instead of Postgres, do the right thing", and c) because you're using stored procedures, you won't be switching to a DBMS that doesn't support the

A isn't even a "reason", it's just a circular statement. "You'll want to do it because you'll want to."
B isn't valid either, which leads me to believe that you're not actually a database programmer, since DBI layers don't just magically translate from one proprietarism to another. If you DO use an sql stored procedure, you're forced to stick to strict standards, otherwise the DBI layer becomes useless.
C is just a lame excuse to bash MySQL, it's not even a real reason. Keep on frothing there, buddy.

Re:SQLObject rocks!

[afd8856]

I'm programming "database applications" with python. I'm storing with zodb or simple pickle. Of course, they're not big apps, but that's the point. For small apps, why bother with an SQL engine/server?
About optimization. If you need to do lots of specific lookups, you can do your own indexes. Have an extra dict that points to a list with the ids for that record. {'kw1':[id1,id2],'kw2':[id3,id1],}. Or better, use ZODB + ZCatalog.

Re:SQLObject rocks!

[timmarhy]

i wouldn't call you old fashioned. stored proc's and triggers are one of the orginal extentions to db's, in no way following do it in code style. they are the exact oppersite. they implement things inside the database. when i write an app i make my function do all the work and just send it the data. it is a million times faster and much more trust worthy then hacking out some crap in php or python.

for example, by using functions it means i can write to one table, then write a pre culminated figure into another table. if you did that in code you'd have to do 2 seperate lots of formatting and 2 seperate database base inserts.

and before anyone brings out the old war horse of database portablity.. what a crock of shit. how often do you move large db's to totally new database packages? NEVER. unless you choose something shit in the first place like access or mysql, however if you do that from the start it's your bed so lie in it.

Re:SQLObject rocks!

[Anonymous Coward]

amen brother. I've been working on DB driven apps, web and non-web and never ever have I had to move a production system to another DB, even for another version. The closest I've ever gotten to this is in desktop development writing odbc code or a very generic DB layer for a product that supported almost every database out there, but never for a web app.

Re:SQLObject rocks!

[JamesOfTheDesert]

Of course, Ruby also has at least one method to do this already... it just isn't part of Rails. Yet.

Ruby already has Og/Nitro, which drives the SQL from the code. Developers can focus on thinking in terms of their object model and object relationships, and let Og, the ORM part, handle the database work.

Many developers find this way of working far more natural than spreading their object definitions over multiple locations (SQL definitions and Ruby code).

Good to hear, though, that Rails continues to pick up advanced ideas from Og/Nitro.

Re:SQLObject rocks!

[swimmar132]

Ruby has Migrations, which allows you to create database tables from pure Ruby code.

I'ts what I use. So I never have to manually create any database fields or tables.

Re:SQLObject rocks!

[fireboy1919]

In this case, stored procedures shelter you from SQL injection attacks

I'm not sure that they do. In fact, I'd go so far as to say that this doesn't make much sense at all. Not that its hard to protect against injection attacks, though.

Let's say you have a stored procedure called "getnumber" that takes two args.
Your goal is that this procedure is going to protect you from injection attacks; that you can put two untrusted args into it and you won't have to worry about someone using the procedure to do something you don't want them to do.

  So you run
"select * getnumber(arg1,arg2);"

Is this vulnerable to injection attacks? Yes.

If arg2="32); create user..."

You see the problem? Stored procedures didn't help. Actually, I'd like to know what stored procedures will do for you that does help.

On the other hand, having a single place that validates the input does help - pretty much every time, and you're invulnerable. Its not nearly as hard as making sure that there's no buffer overflows.

This is all that is necessary and sufficient to protect against SQL injection:
1) Any quote symbols in the user input are escaped so that they won't be considered quotes.
2) Each user-supplied value is quoted before being put into an SQL statment.
3) Users are allowed to enter values only. If users are allowed to enter column names, then the data must be validated against all allowed column names. But generally this is stupid. Best to let users enter data only, never metadata.
4) Users may not enter any SQL.

Pretty much all of these database converters have one or two points of entry that do 1-2 before doing any kind of optimization/fetching/whatever, and they don't really allow you to do 3-4. Of course, maybe I'm wrong and the GP knows about some mystical forms of SQL injection attacks that I don't. If so, I would certainly like to hear about any injection attempts that can fool this sort of thing. I would also like to hear about some big ORM (object-relational mapping) tools that prove that what the GP says isn't FUD.

Keep in mind that escaped values are escaped both when they're entered, and when they come out of a database. You pretty much always have to manually unescape them if you want to show them the original way.

Re:SQLObject rocks!

[ooze]

You seem to forget one basic thing. Arguments in Stored procedures are typed. And types can have a hell lot of restrictions, which being one of the major points of a database.

And you seem to forget another basic thing. Even if you too the unrestricted type string as the parameter type, simply using stored procedures pretects you from all SQL injection. Stored procedures are compiled. All the statements in the procedures need no parsing anympore, they are already executable binary. No matter what you write in those arguments, at runtime there is no parsing anymore and no ambiguities by cleverly chosen strings matter to the parse anymore...since it is simply not used.

While I'm no SQL expert by far (and I get shudders on the syntax explosion they call SQL and is insanely the industry standard for accessing relational databases, and I avoid having to work with SQL at any costs) I understand the basic underlying technologies and principles of databases and database implementation. And SQL injection isn't a problem unique to SQL interpreted at runtime. command injection is a weakness every interpreted language has. There was a reason text and data was split in computing and self modifying code is frowned upon. Every language that is parsed and interpreted at runtime (well, strictly speaking right before runtime) introduces all those cans of worms of self modyfying code again.

Script languages: great for prototyping, great for presentation, great for personal tools. But never ever use a language interpreted at runtime fore any project where more than 10 people can input data and the code as even the sligtest chance to modify data.

Re:SQLObject rocks!

[sammy baby]

Tell me - how and where does one start writing SQL when you're using Ruby's Activerecord (which sounds like a building block for making Ruby objects similar to Python's SQLObject but without writing a single line of SQL)?

In pretty much every Ruby tutorial I've seen (including Agile Web Development with Rails ), the sequence goes: write the SQL, write the class, let Rails infer what isn't worth specifying.

More specifically: Rails makes a vast number of assumptions about the manner in which you store your data, but for the most part the assumptions are 1) reasonable, and 2) overrideable. This goes along with their philosphy of "convention over configuration," something any J2EE developer can likely relate to.

For example: Rails expects that if I create a class called "LineItem," that it can find a table in the database called "LineItems" that has an autoincrementing primary key called "id." If I create a "ShoppingCart" class, and in my class definition I put "has_many :LineItems", it expects a field in the "ShoppingCarts" table called "LineItem_id," ostensibly a foreign key (although it will handle the standard key constraints itself, w/o intervention from the RDB.) And so forth. But after you've written your data definitions, you basically don't have to write any SQL again afterwards. And if you find the Rails conventions to onerous, you can always override them.

Active Record has a new feature called "Migrations" coming out RSN which abstracts away even this initial involvement in the SQL, but I haven't so much as touched it yet.

Re:SQLObject rocks!

[sammy baby]

The SQL database abstraction layer is an important feature of SQLObject, that Ruby on Rails doesn't currently support -- you have to write database dependent SQL code mixed in with your Ruby code. ...So you don't have to change any of your Python code to port it to a different database, and you don't have to mix together two different notations, or quote a bunch of SQL strings in your Python code. It's a much more "pythonic" way of database programming than raw SQL.

I'm going to assume you haven't actually done any Ruby on Rails.

The only raw SQL you're ever forced to write in a RoR app is the initial data definition where you create the tables in your database. After that, it's all object-relational mapping as handled by the Active Record component. No SQL to be found, no rewriting code to port from one database to another. (Well - unless you count recreating the tables, or changing the name of the database driver in the config file.)

Re:SQLObject rocks!

[Anonymous Coward]

The SQL database abstraction layer is an important feature of SQLObject, that Ruby on Rails doesn't currently support -- you have to write database dependent SQL code mixed in with your Ruby code.

Plain wrong. Rails indeed supports that feature through the ActiveRecord abstraction layer.

In fact, the majority of Rails' application don't contain a single line of SQL. Furthermore, if you insist, you may embed SQL code in your source. Rails allows this. But for the rare occassions that you would need to do so, you'd also need to do it in SQLObject. I can see no feature difference between those approaches.

Re:SQLObject rocks!

[fireboy1919]

Arguments in Stored procedures are typed. And types can have a hell lot of restrictions, which being one of the major points of a database.

Like that number I used in the injection was typed? It was clearly a number, and the stored procedure would have recognized it as such. Didn't matter, did it?

No matter what you write in those arguments, at runtime there is no parsing anymore and no ambiguities by cleverly chosen strings matter to the parse anymore...since it is simply not used.

That only works if you can guarantee that you're only sending a single command to your database at any given time, doesn't it? I mean, maybe you can't do anything nasty inside the stored procedure, but you can outside it, can't you? Like in the example I gave?

You still have to do the checks I talked about in order to prevent SQL injections.

Databases are interpreters. They interpret SQL, and the ones that understand stored procedures can interpret even more than that. If you have something against interpreters, then you should stop using databases.

As it stands for me, though, input validation really isn't that hard. Use OOP, and build validation into the top of your hierarchy and you never have to worry about it again. There are not many and varied ways to do it. Wierd combos don't come up that cause problems. Unlike the other security risks, this one only comes up from ignorance, since its so easy to prevent.

Re:SQLObject rocks!

[Anonymous Coward]

The example you used was misleading as it would only work in a stored procedure that executes dynamic SQL. If the parameters are instead parameters, bind variables, or whatever else you want to call them, they are resolved only as variables and never as code.

Much the same should be true of positional notation in JDBC - an efficient implementation should resolve down to a SQL statement with bind variable positions, then bind the positional variables on each call, rather than replacing each variable with a hard-coded value and re-parsing the SQL. The code injection problem is the same in both approaches - it only applies to SQL that is built on the fly.

I do like the idea of keeping the SQL in stored procedures though, as it allows you to seperate your D/B developers from your Java ones, which means the Java guys can quite literally say the D/B performance isn't their problem, and the D/B guys don't find the database being brought to it's knees by someone who doesn't have a clue how to write a query.

It's abstraction without believing a tool can do it for you.

Re:SQLObject rocks!

[Anonymous Coward]

"select * getnumber(arg1,arg2);"
Is this vulnerable to injection attacks? Yes.
If arg2="32); create user..."

What kind of moron would do something like that? Use bind variables for crying out loud. With bind variables SQL injection is a non-issue. This version of the select does not have the problem you mention:

"select * getnumber(:arg1,:arg2);"

or

"select * getnumber(?,?);"

Re:SQLObject rocks!

[WaterBreath]

So you run
"select * getnumber(arg1,arg2);"

Is this vulnerable to injection attacks?

Well yeah, if you're using dynamic SQL. If the select query is in a dynamic SQL block, or if the getnumber function runs a dynamic SQL block with arg2 as a bound parameter, then you might have a problem. But if you're not using dynamic SQL then the injection attack you specified is not possible, AFAIK.

Dynamic SQL should be avoided when possible for a number of reasons, including the possibility of SQL injection. It's certainly a useful and appropriate tool sometimes. But it is a specialized tool that should be used only when necessary.

Re:SQLObject rocks!

[Ian Bicking]

There's your problem -- you're mixing SQL code in with your app code. Bad developer! Bad! You're introducing SQL injection holes

Of all the bugs SQLObject has had, I don't think SQL injection has ever come up. Some languages are stupid (stupid stupid stupid) and encourage SQL injection. E.g., in PHP the "right" way to construct SQL is something like "select * from foo where name = '" . addslashes(stripslashes($name)) . "'". Or some monstrosity, depending on whether magic quotes is on, and what database you are using. This is insanely bad. If you forget addslashes everything will work until someone puts a ' into the input. (With separate weird problems if you forget stripslashes -- or include it when you shouldn't)

Python (and other sane languages) use something like query("select * from foo where name = %s", (name,)) (SQLObject gives this a different syntax, but same idea). With this, if you forget the substitution or do it incorrectly it will be Completely Broken -- because you'll be sending a query like select * from foo where name = Tom. This is a good poka yoke.

As far as putting SQL in the code, sometimes a query is best expressed at the time you will use it. If you want all customers in an area code, is it really best to make a customersInAreaCode(areaCode) method, or just use Customer.selectBy(areaCode=X)? They look pretty much equivalent to me -- except the latter comes for free, and is completely localized to the place you use it. That localization is a negative if it means you are duplicating code; however, creating an abstraction before any duplicate code would be created is also a negative.

As far as the other issues: SQLObject uses the Active Record model (which Rails' ActiveRecord happens to be named after) and is best suited to Application Databases. There are pluses and minuses to this. You can often use a mixture of both styles, if you find a need. But if you are an organization of web developers without dedicated database personel and with a tight schedule, you will be best served by the Application Database style of development, because (from a developer's point of view) it gives you the most control.

Re:SQLObject rocks!

[Anonymous Coward]

Actually, a wrapper around SQL can easily be used to guard against SQL injection, if one cared to. It would be trivial to 'escape' any SQL reserved words. The object/relational wrapper can have, as one of its primary jobs, the responsibility of guarding against such attacks, freeing the programmer from this.

In some Python web frameworks we have this functionality already guarding against cross site scripting attacks - a form of user-driven 'injection' of unsafe content. Anything that isn't explicit is assumed dangerous and is protected.

But all this is beside the real point. Why use SQL if you don't need it... drink from the object database koolaid cup, you'll like it ;-)

Re:SQLObject rocks!

[poot_rootbeer]

Whereas this is true, it's also true that using "?" parameters for any user-entered values can solve the SQL injection problem just as well.

Yeah basically.

When it comes down to it, somewhere along the line your application needs to understand what are valid parameters to a database call and what are not. There's nothing about DBMS-layer stored procedures that is intrinsically better at this than business-layer code, especially if you use a mature DB abstraction package.

The benefit of stored procedures in reducing useless traffic between the database and the business layer is harder to deny. Sometimes, especially in transactional operations, you can't write a single SQL query complex enough to provide just the data you need.

Re:SQLObject rocks!

[jaydonnell]

There's your problem -- you're mixing SQL code in with your app code. Bad developer! Bad! You're introducing SQL injection holes (I don't care how well you think you're filtering your input, chances are you missed something,

In this case, stored procedures shelter you from SQL injection attacks (assuming, of course, that your stored procedures aren't just dumb "take a string and 'exec' it as SQL code" queries).

I hope this makes the hole in your logic clear. Your saying that people will make mistakes in python that will create sql injection holes. You then say that stored procedures solve this problem as long as you don't make dumb mistakes. Um, what's the difference. Your simply pushing the problem into a layer that isn't as powerfull as a real programming language, and just generally is a pain to deal with because stored procedures aren't as easily integrated into your development processes (version control, unit testing, etc).

Re:SQLObject rocks!

[jguthrie]

The example you used was misleading as it would only work in a stored procedure that executes dynamic SQL. If the parameters are instead parameters, bind variables, or whatever else you want to call them, they are resolved only as variables and never as code.

While that may be true, it is also true that the call to the stored procedure is itself SQL, and that SQL is going to be dynamically interpreted because you won't know what the arguments. So, somewhere in the interface to the database, you're going to have a line that looks like this:

$query = "select * getnumber(".$first_arg.",".$second_arg.");";

That works great, as long as both $first_arg and $second_arg are numbers. However, depending on where they get their values, they could be anything. What that means is that unless you escape everything, the call to the stored procedure that is supposed to look like this:

select * getnumber(1, 32);

Might actually look like this:

select * getnumber(1, 32); create user foo with god_access;

In both cases, the call to the stored procedure is fine, it's the generation of the call to the stored procedure that causes the problem in the second case.

Re:SQLObject rocks!

[Hugo Graffiti]

You can and should avoid that cost by processing the data first at the SQL Server (hey, stored procedures again! Why return 100,000 rows that you're going to process down to 1,000 when you could just return the 1,000 processed rows in the first place?

But if you're using Python then chances are you're not going to be doing heavy duty stuff with hundreds of thousands of rows! This is small scale web apps we're talking about. I bet you can't write your stored procedures in Python can you, so it's just more Stuff You Have To Learn.

Re:SQLObject rocks!

[SimHacker]

Osty, you have poor reading comprehension. You're attacking exactly the opposite of what I was saying. Go back and read my message you're replying to more carefully, and while you're at it, read the article and watch the video so you'll be more qualified to participate in the discussion.

My point is that SQLObject does insulate you from SQL injection attacks and database dependencies, in ways that Ruby's ActiveRecord does not.

Of course you're ignoring the problem of database portability, because the hammer you want to attack all problems with seems to be stored procedures. If the goals are database portability and database application programming with a scripting language like Python or Ruby, then your hammer breaks the original goals, so it's the wrong hammer. I want to integrate my existing Python code with the database, not rewrite it all as stored procedures -- that would make no sense whatsoever. If I wanted somebody to tell me to write database dependent code, lock myself into the biggest database with the most features, and give up my scripting language of choice, then I'd call Oracle, bend over, and do exactly what they told me. But I'm not that massochistic.

Here is a typical snippet of Ruby/ActiveRecord code, and you can clearly see that it has SQL code written in as quoted strings (sorry about the indentation -- slashdot is now ignoring nbsp's as well as pre's, which makes it hard to discuss code):

def self.authenticate(login, pass)
find_first(["login = ? AND password = ?", login, sha1(pass)])
end

Here is the equivalent Python/SQLObject code, which you can see uses Python syntax to accomplish the same thing, without quoting any SQL code at all:

def authenticate(self, login, pass):
return User.select(AND(User.q.login == login, User.q.password == sha1(pass)))[0]

This is accomplished with the "SQLBuilder" utility, that lets you write python expressions like "AND(person.q.first_name == 'John', person.q.last_name == 'Doe')" instead of raw SQL expressions like "(person.first_name = 'John' AND person.last_name = 'Doe')".

SQLObject also supports methods called "startswith()" and "endswith()" and a function called "LIKE", which insulate you from the differences between SQL "LIKE" syntaxes, so you can portably write "User.q.firstName.startswith('X')" instead of "User.firstName LIKE 'X%'".

Of course the 15 minute Ruby on Rails demo didn't get deep enough into programming a complex web application, that they had to show you any of the SQL dependencies and other warts of Ruby on Rails. They're evangelizing, not airing dirty laundry -- what do you expect?

SQLObject and ActiveRecord are based on a lot of the same ideas (Martin Fowler's Active Record model), but SQLObject is also different than ActiveRecord in many ways. Ben Bangert's compares SQLObject and ActiveRecord in his talk, "Presentation on SQLObject & FormEncode", that he gave at the Bay Area Python users group.

SQLObject is a pure ORM (Object-Relational Mapper), while ActiveRecord is a full fledged Model (as in MVC). Both SQLObject and ActiveRecord let you automatically read in an existing database schema. But SQLObject also lets you define the schema in Python and automatically generates the tables (for any database) from the Python declarations -- it works either way! SQLObject also supports runtime column changes, so it can be used to implement dynamic database schema editors.

-Don

Re:SQLObject rocks!

[Trejkaz]

Yeah, Rails migrations are a fairly good solution. They market it as a migration framework, but really there isn't any difference between migration from some tables and migration from no tables.

Re:SQLObject rocks!

[Anonymous Coward]

You don't have to use stored procedures to stop SQL injection and compile your SQL. Prepared statements do the same thing and are as portable as the SQL they contain.

Re:SQLObject rocks!

[Anonymous Coward]

Here is a typical snippet of Ruby/ActiveRecord code, and you can clearly see that it has SQL code written in as quoted strings (sorry about the indentation -- slashdot is now ignoring nbsp's as well as pre's, which makes it hard to discuss code):

def self.authenticate(login, pass)
find_first(["login = ? AND password = ?", login, sha1(pass)])
end

You don't seem to know ActiveRecord that well. Two things to note:

(1) When you use the ? form, ActiveRecord performs stripping of the string; it doesn't merely insert the raw text. So it does protect you from SQL injections in that manner.

(2) ActiveRecord uses Ruby's reflection features to define all of its methods. Not only the basic field requests, but also searches and whatnot. So you could implement that method as the easier:

def self.authenticate(login, pass)
    find_by_login_and_password(login, sha1(pass))
end

Which requires no SQL and is more readable, to boot.

Re:SQLObject rocks!

[buttahead]

never build checking into high levels if it can be sanely built into low levels. that allows high levels to be flighty and ignorant, while still allowing security. imagine 10 high level interfaces... you want each one to be responsible for validaing security, or one low level to reject bad requests?

Re:Does it scale?

[Anonymous Coward]

Bullshit. J2EE is all about hype, the latest buzzword that clueless managers will use whenever a new project is discussed. People talking about the "power of Java" don't really know what they're talking about, or have never actually used it.

Re:Does it scale?

[Anonymous Coward]

Amazon don't use J2EE most of it is written in C/C++.

I'm sure there's a Google engineer chuckling at the thought of using J2EE for their entire system. If anything Google are quite varied in what languages they use I expect, like any sensible company. If your too tied to a single language, your screwed in the long term.

Java != Panacea.

Peon.

Scaling up, and scaling down

[DavidNWelton]

Ok, so some of these tools are not suitable for running Amazon. But guess what - most people are not running Amazon! A lot of people don't have the development resources that amazon has, either, so what they are really looking for is a sweet spot that lets them get going quickly, and will grow within reason.

I'm still mulling it over and working on it, but I talk some about "scaling down" in this article:

http://dedasys.com/articles/scalable_systems.html

You're right of course that you don't want stuff that falls over the first time traffic spikes a bit, but you absolutely must have something that you can use to produce a functional product. You can have the fanciest, most scalable system out there, but if you spend two months twiddling with XML config files, things just aren't going to work out.

torrent (just in case)

[Anonymous Coward]

torrent

OpenDoc Cookbook

[Graymalkin]

I'm definitely not an expert in Python, in fact I've only ever given it a cursory look. However that tutorial was damn impressive. Obviously he had some prior knowlege of Python and using TurboGears but it is really not all that difficult to build something like a Wiki using that framework. As far as web work I've slowly become disenchanted with PHP. It's a good language to be sure but it's simplicity is short-lived. As you want to do more complex things you end up having to work around PHP more than you get to benefit from it. A large web project in PHP ends up structured like a project of similar size in Perl or Python. Between TurboGears on Python and Ruby on Rails it looks like I have some reading to do.

Re:OpenDoc Cookbook

[Bogtha]

As far as web work I've slowly become disenchanted with PHP. It's a good language to be sure but it's simplicity is short-lived. As you want to do more complex things you end up having to work around PHP more than you get to benefit from it.

That's exactly what I found. I was already familiar with Python in non-web application domains, so I started looking around for decent web application platforms for Python. I settled on mod_python, but the templating was awful, and none of the alternatives seemed much better.

I'd actually gotten most of the way through implementing a PHP compiler for mod_python* before I discovered Kid, the templating system used in Turbogears. It really hits the sweet spot where it's not too rigid like other XML-based templates, but not too loose, unstructured (and just plain bizarre) as other Python-in-HTML systems.

I'd highly recommend trying out Python + Kid for any decent PHP coders, even if it's not specifically TurboGears (which I haven't tried out yet). After the first couple of days, your productivity will shoot up and you'll wonder how you ever managed to cope with PHP.

* Compiling to Python bytecode is simple, it's the irregularities of PHP that's the bitch. PHP is a fairly nice templating language, which is what it was originally - it's just the actual programming where it sucks.

Re:OpenDoc Cookbook

[Anonymous Coward]

Don't forget to check out CherryPy too. It's also used in TurboGears.

---------------

Slow Down Cowboy!

Slashdot requires you to wait between each successful posting of a comment to allow everyone a fair chance at posting a comment.

It's been 38 minutes since you last successfully posted a comment

Re:OpenDoc Cookbook

[freeplatypus]

It's a good language to be sure but it's simplicity is short-lived. As you want to do more complex things you end up having to work around PHP more than you get to benefit from it.

I dare to ask, what things You can't do in PHP, that You can do in Python or Perl?

Re:OpenDoc Cookbook

[Graymalkin]

It's not that PHP can't do things Python and Perl can, PHP in my experience tends to get more difficult to deal with as your code's complexity increases. PHP wants a bunch of display code embeded within business logic, this works fine when your needs are simple and your code is basically acting as a pretty database front end. This sucks when you want a single back end to drive multiple front ends. You need to go back and make everything nice and friendly and layered. By the time you get this done you could have rewritten the whole project against a friendly web app framework like TurboGears or Maypole/Catalyst.

Re:OpenDoc Cookbook

[freeplatypus]

Well this is true with most of the languages like Python, PHP and Perl. This is why You have frameworks. PHP also has frameworks, take for example Seagull.

Re:OpenDoc Cookbook

[teknomage1]

what things You can't do in PHP, that You can do in Python or Perl?

PHP's anonymous functions cannot contain state, so you cannot form lexical closures. Lexical closures are an important tool of abstraction.

PHP lacks any scope besides, global and function, which though not crippling is a bit annoying. All PHP functions are global in scope, making it hard to provide an interface to a code package, and it also lacks namespaces which compounds the problem. It makes it difficult to work on large projects, without incurring naming conflicts.

PHP's arrays are actually hashes, with an occasional numeric keys. There are no lists or other containers.

This is not to say I hate php, in fact I use it for many simple web projects just becaue it's nearly universally installed on web hosting providers. I do not however think the current generation of php4/php5 are well designed. But then again, perl was supposed to have been pretty poor off around version 2 so there's always hope that when the team goes on to begin php6, they'll have more experience, and fix some things.

Re:Does it scale?

[ashot]

Python is one language they actually use quite a bit

Re:Does it scale?

[Scarblac]

if you want to do something for your professional career, don't waste your time with those kind of frameworks.

If you want to do something for your professional career, get familiar with as varied a collection of tools as you can. Know the pros and cons of each. Actually test their performance, make toy projects, steal ideas and patterns. Be opinionated, but prepared to honestly choose the best tool for the job you're given, and to explain why it is the best, to suits and to techies. A few hours getting to know something new is never wasted.

Perl on rails?

[burbilog]

Anybody?

Re:Perl on rails?

[jcr]

God Forbid.

-jcr

Re:Perl on rails?

[holy zarquon's singi]

yeah, Maypole and Catalyst.

Re:Perl on rails?

[mooingyak]

My wife's grandmother made a delicious cake... and then put lemon-flavored icing on top, basically ruining it.

I wanted to ask her the same thing I'll ask you:

Who would do such a cruel thing?

Link to 20 min Quicktime Movie on ./ = Cruelty

[Anonymous Coward]

Linking to a site with a 20 min Quicktime Movie on ./, and then pointing it out so everyone downloads it is just cruel! Won't anyone think of the poor servers?

*yawn* old news

[nurmr]

nothing to see here, move along

Tried it, too bitty

[Boffy]

Tried TurboGears, but the fact that it's a glue was way too appparent. I then moved on to trying Django and fell in love. All the stuff TurboGears can do Django can too, but natively.

Re:Tried it, too bitty

[Anonymous Coward]

way to go boffy - django rocks

Re:Tried it, too bitty

[aleph]

Still downloading the movie from TurboGears (tried streaming, but well.. That wasn't happening too well :)), but I have been playing with Django a bit the past few days.

It does look kind of cool, the admin interface is good for _simple_ objects, but seems to start to fall down when you want to construct interesting relationships between the objects, and I haven't started to look into doing custom (or using generic) create/update templates yet.

That being said I'm still suffering a bit of paradigm lag since i've mostly been dealing with Zope/Plone for the past couple of years, and while that platform isn't without it's problems, it does have a lot of nice features you begin to take for granted.

I also came across areas where Django would silently fail to do things in the admin interface with no sign of an error anywhere (it wasn't even attempting to insert the new object in the db, nor raising an exception anywhere). That kind of behaviour makes me a bit nervous :p

Re:Tried it, too bitty

[ra1]

Yup, Django is definitely the way to go.

Re:Tried it, too bitty

[Anonymous Coward]

Yeah. OK. Django. But where's the Django video???

I've been trying and trying, with Django. Still struggling!

Re:Tried it, too bitty

[Anonymous Coward]

Pardon the ignorant question, but I'm looking to do some Web stuff with Python. As a beginner in the language, am I better off trying something like Zope or something like Django? For what it's worth, I use Java + Tapestry regularly, so I do have experience with component-based webapp frameworks.

Re:Tried it, too bitty

[freeplatypus]

Well, You maybe tried to be funny, but the problem with Django is an incomplete documentation. What a pity since it is promissing framework.

Yawn. Perl Catalyst had this same example

[tezza]

Catalyst Web Framework on Perl.com

The best bits about catalyst are:

1. built in webserver. Apache/Apache2 is in flux at the moment, and you're caught in shifting sands of documentation, libraries and online-help. A built in webserver means you can prototype fastest.
2. Eclipse. With EPIC you can code and debug your perl in Eclipse.

Re:Yawn. Perl Catalyst had this same example

[alphan]

Rails comes with a webserver (Webserver), and there is a standalone IDE (RadRails) for development (though it is obviously Eclipse powered).

Re:Yawn. Perl Catalyst had this same example

[alphan]

Webserver=Webrick

Re:Yawn. Perl Catalyst had this same example

[ThatDamnMurphyGuy]

Apache/Apache2 is in flux at the moment,

Bunk. I use Catalyst. It works fine in Apache/MP1 and Apache2/MP2 as well as CGI/FastCGI. MP2 has been released for quite some time now; although you wouldn't know it by some of the way out of date packages in some distros (like Ubuntu that is still on the 1.99dev version).

Re:Yawn. Perl Catalyst had this same example

[LordBodak]

In addition to the standalone RadRails, there are plugins for Eclipse to provide the functionality to a standard Eclipse installation.

Pales in comparison to Ruby On Rails

[the_pilif]

Hi,

I've seen both quicktime movies now: Ruby on Rails and now this one. And this video is not nearly as impressing as the rails one is - mostly, because the python solution here is violating the DRY (don't repeat yourself) principle at many places.

This makes the demonstration look dull and complicated.

While I like the general idea of the project, there's still a large way to go till it reaches the ease and simplicity of Rails.

Philip

Re:Pales in comparison to Ruby On Rails

[helifex]

Having jumped ship to Ruby/Rails quite a while back I'd have to say I didn't find it very impressive in comparison but if I was using Python I suspect I'd see it differently.

turbo-gear

[pedicabo]

Quote " as natural as writng a function". In other words, completely unnatural, formal in the extreme, a highly technical task. Is this why linux geeks think that one day all the world will use Linux/Debian/BSD.

10 seconds in [your language here]1

[Anonymous Coward]

new Weblog(); new Wiki(); tadaaah!

Rails everywhere.

[killjoe]

If imitation is the sincerest form of flattery then David Heinemeier Hansson must be blushing right now. His framework has insipired every other language proponent to make a rails like framework. There is trails in java, django in python (and now turbogears), there are at least two PHP frameworks and even a .NET one. I think somehow David touched a nerve someplace and produced something really beautiful.

Much of what makes rails great is the highly dynamic nature of Ruby itself. Do yourself a favor and read the activerecord source code. Even if you don't know ruby it's an interesting lesson on how to take advantage of dynamic scripting languages. I am sure python and PHP could also do something like this but the Java and .NET people are resorting to code generation to try and mimic some of the functionality of rails.

To me it seems like a silly exercise to replicate rails in python or what have you. Ruby is easy to pick up and a nice language to boot. Why bother really? Just learn ruby and get on board.

Re:Rails everywhere.

[hexene]

Although only recently open-sourced, Django has powered real-world sites for several years now. So whilst the project will borrow ideas from other frameworks where it makes sense, it's more of a case of parallel evolution than trying to ape Rails.

Re:Rails everywhere.

[Anonymous Coward]

To me it seems like a silly exercise to replicate rails in python or what have you. Ruby is easy to pick up and a nice language to boot. Why bother really? Just learn ruby and get on board.

Because Ruby programmers are few and far between? Because your existing team already knows Python? Because, like it or not, functional programming has been trying to hit the mainstream for decades without result because functional programming is not as easy and straightforward as imperative programming. Because you already have a lot of Python code already written and working.

For a thousand other reasons. Tell anybody in the real world "oh, just switch languages!" and watch yourself get laughed out of the building. Ruby is far from being a panacea, and even if it was, there's plenty of good reasons to stick with what you already have.

Re:Rails everywhere.

[arevos]

To me it seems like a silly exercise to replicate rails in python or what have you. Ruby is easy to pick up and a nice language to boot. Why bother really? Just learn ruby and get on board.

Whilst Rails is an excellent framework for web applications, I've found that it gets exponentially more difficult to work with when your database structures grow more complex than interconnected lists. I recently designed a double-booked accounts program in Rails, and whilst most of the code was simple to design, I quickly got bogged down in handling the accounts-tree and the double-booking of financial records. Here the documentation ran out, and I was forced to go through the Rails source to discover a solution to my problem, which turned out to be less than optimum.

Secondly, whilst I have done a fair bit of work in Ruby, I can't help but prefer Python. If there's little difference between Rails and, say, Turbogears, Django or Subway, then surely it comes down to personal preference. Python web frameworks appear to take a more piecemeal approach than Rail, which can provide a more flexible solution in certain situations.

Can't say I much like SQLObject's syntax, though; but CherryPy seems rather elegant.

Re:Rails everywhere.

[Anonymous Coward]

Have you even looked at Ruby? It is not a functional language by any stretch -- it is very similar to Python and Perl. It is by far the cleanest one of the three built with objects from the ground up.

Re:Rails everywhere.

[Anonymous Coward]

Python is also OO "from the ground up".

Re:Rails everywhere.

[Ian Bicking]

To me it seems like a silly exercise to replicate rails in python or what have you.

1. TurboGears is not a Rails clone.
2. Most parts of TurboGears existed before Rails: CherryPy, SQLObject, FormEncode (and Python of course).
3. Kid is most closely related to Zope Page Templates (from the Python world), not anything from Rails.
4. MochiKit has a certain relationship to Prototype (the Javascript library from Rails), and is compatible with it. However, it's not that the author particularly likes Prototype.

Rails has taught us some important lessons, but they aren't really technical lessons:

1. We shouldn't sit around and say "oh, those poor people using PHP/Java/etc, too bad they don't know about what you can do using X". Instead we should talk more loudly and insistently about the advantages of our platforms. If you do it right people will pay attention.
2. We haven't concentrated enough on full-stack integration. We've been overvaluing decoupled pick-and-choose components. Full-stack integration doesn't have to mean coupling -- it can just be a matter of presentation, and making sure tools are complimentary. Not all of the Python frameworks are coming at it from this direction, but TurboGears very much is.
3. Things like screencasts are nice.

After looking at various pieces of Rails, these lessons have stood out to me, but the particular technology in Rails has not. Sure, there are some good ideas, but nothing radical, and there's good ideas everywhere waiting to be mined. We're not beneath mining other people's ideas, but it does not follow that the result is merely a "replication" in part or in whole.

As for Ruby: I think the two languages are largely equivalent in terms of what you can do. I would not say the same about PHP or Java. As for Rails specifically, I think it is only ahead of Python options in the second derivative. With conscious players the second derivative doesn't mean a whole lot.

Re:Rails everywhere.

[Anonymous Coward]

Ruby is easy to pick up and a nice language to boot. Why bother really? Just learn ruby and get on board.

A few years ago, I wanted to use a higher-level language for my next project. Eventually I got down to Ruby or Python. I decided to use Ruby, because I liked the Smalltalk-like syntax. After about 2 days, I switched to Python. Yes, the Ruby language is really nice, but it had almost none of the libraries that I wanted to use. And whenever I checked, Python had bindings for every library I could think of.

And even you Ruby people must know this. That's why you're suggesting we use Ruby: for Rails! And once you can advocate using Ruby for Ruby on Rails, it's not a great leap to choose a language based on *all* the libraries you need, or may need soon, and that means Ruby might not be the best answer.

Re:Rails everywhere.

[larry bagina]

cache/objectscript was doing this long before ruby on rails. And much better, since the db is integrated into the language.

Re:Rails everywhere.

[killjoe]

The innovation in rails was to take the database as the authoritive source and have the code adapt dynamically to changes in the database. In most ORM layers the XML descriptor is the authoritive source.

Re:Rails everywhere.

[Ian Bicking]

I added this feature (using the database as the source of the model) to SQLObject a couple years ago, before I'd ever heard of Rails. But, after adding it, it never seemed like that neat of a feature, and it only gets moderate use. I don't claim SQLObject was the first... rather that it's not that great a feature compared to using the class itself as an authoritative representation (which no one uses in Rails because they don't have a choice, since ActiveRecord doesn't implement that).

No one familiar with Python development should have ever been wowed by anything Rails does from a technical point of view. All the surprise about these features is coming from Java and PHP people, who have lived in darkness and to whom all light is blinding and full of amazement. I don't have a problem with that -- you'll get no arguments from me that Rails is better than what PHP or Java offers. But it doesn't mean Rails is novel.

Re:Rails everywhere.

[zapadoo]

here's one SQLObject user who has always appreciated its ability to be driven by the Database schema -- a real benefit when working with legacy systems.

My first blush take on rails was "hey, we've seen active record before, in python: its called SQLObject"

I'm mostly impressed with the head of steam Rails has; lots of people working together can knock off more innovations and that's cool.

But, try as I might, I can't bring myself to enjoy programming in Ruby. I find python amply expressive, less cluttered to look at and less obtuse. If Rails offered some whizbang feature that the python world didn't have, that difference would not last for long.

now if ruby would only drop all those end statements ;-)

Small Wikis

[xihr]

If you're up for small Wikis, there's always HeyHeyWickie, a Python Wiki in under 4K lines of code (using EmPy and docutils).

Re:Small Wikis

[Ian Bicking]

(a) I think you mean 4k of code, not 4000 lines (which would be quite large for a wiki), and (b) if you want small, in Wiki terms, 23 (wrapped) lines is closer to what you should shoot for (at least in Python).

Some friendly competition

[vandan]

Interesting. It's good to see a number of people taking on the project of providing RAD tools with database access.

I have a competing project: http://entropy.homelinux.org/axis_not_evil

It's a collection of Perl modules:

- Gtk2::Ex::DBI ( forms )
- Gtk2::Ex::Datasheet ( datasheets )
- PDF::ReportWriter ( reports )

It's all open-source, cross-platform goodness. It of course uses Gtk2 as the widget toolkit ( which is now a push-over to install on Windows thanks to a number of people providing single-click installers ).

Admittedly my project isn't quite as polished ( in terms of point-and-click setup of objects ) as TurboGears, but in my defense, there's only one of me, and I've only been programming in Perl for just over a year.

Re:Some friendly competition

[Anonymous Coward]

And i'll take this moment to do my shameless plug of my RAD tools for GTK2 and the web.

Forms: http://www.treshna.com/bond
Database framework: http://www.treshna.com/bonddb
Reporting: http://papyrus.treshna.com/

You can choose to export your desktop apps the desktop via GTK2 widget set or to the web via HTML/Javascript/AJAX without any code changes.

Re:Some friendly competition

[jdgreen7]

Interesting project. We use Access extensively at my office, and I am the main "programmer", but that's really only about 1/3rd of my job. There are 3 of us in the IT staff, and we handle everything from remote access to printer issues to networking to phone system wiring to appliation development, so there is never a shortage of issues to be dealt with. We will be tied to the Windows platform for a very long time due to government reporting requirements, and software doesn't exist for other platforms that meet these requirements. They change far too often for us to develop in-house replacements, but I would like to move onto a license-free platform for more applications. A tool like yours accessed through a Linux Terminal Server might be the ticket for some of our smaller apps. Web-based is nice, but the tools just aren't there yet for small developers to create rich environments for the end-user. I really like the rails concept and the one in this article, but I'm just not familiar enough with web development to move forward with it.

Anyway, enough about that. Your site has some nice screenshots and looks fairly easy to use. I'll keep tabs on it and good luck with your development! Do you have a listserv or CVS-commit mailing list to subscribe to by chance?

Re:Does it scale?

[Qbertino]

Wrong.
If you're building Amazon or a simular service (million visits/day) everything scales. I don't know the exact budget the Amazon portals had, but buying Guido von Rossum or Larry Wall and ten of their favorite programmers for life would've probalby been less than a month of amazons electricity bill.

If I were to rebuild Amazon I'd actually consider Python. If the VM doesn't cut it, I'd hire 20 programmers to optimize it. Which, btw, I don't think would be neccesary.

Another scenario is even more realistic. MySQL sucks for certain purposes. But if a certain OSS CMS I like only runs with it, but the project is big enough to imperatively require Postgres or Firebird, then the projects budget should allow to patch in support for that DB.

Twisted

[jcr]

Also worth checking out: Twisted. I haven't had occasion to use it myself, but people I know swear by it.

-jcr

Re:Twisted

[Feign Ram]

You sure those people aren't twisted ?

Re:Twisted

[jcr]

Of course they are, but in a good way. ;-)

-jcr

Re:Twisted

[Ian Bicking]

If you are doing network programming -- stuff like VOIP, chat servers, etc -- then you should definitely look at Twisted. You can use it for web development too, but it's a bit challenging.

Ruby functional? Not.

[archeopterix]

Because, like it or not, functional programming has been trying to hit the mainstream for decades without result because functional programming is not as easy and straightforward as imperative programming.

Ruby is no more functional than Python, even by the most twisted definition of 'functional'. Yup, it uses closures (aka blocks) and that is how far it goes towards functional. For a language to be functional it is essential to restrict side effects (haskell with monads, ocaml with 'mutable'), or at least have some good support for no-side-effects programming: currying, higher-order functions, the "let x = expr1 in expr2" (aka non-destructive assignment) construct and so on. Both Python and Ruby make routine use of destructive assignment, which is the epithome of side effects.

Re:Ruby functional? Not.

[Anonymous Coward]

Sorry, was reading up on both Ruby and Haskell this morning - had a brain fart. But the rest of my points still stand though - Ruby is far from perfect and declaring that we should drop all development in other languages and immediately switch to it is just plain fucking stupid and lacking in any pragmatism whatsoever.

Re:Ruby functional? Not.

[csirac]

Ruby is far from perfect and declaring that we should drop all development in other languages and immediately switch to it is just plain fucking stupid and lacking in any pragmatism whatsoever.

Perhaps he's not saying Ruby should become a new standard, but that people like you should open your mind just a little, tiny bit, open the door and walk outside - and try a new language for a few hours sometime.

People like you like to get stuck in a rut and cling to what you know, forsaking everything else. For years, all I've ever known is C, 8/16-bit ASM, Perl, a few FPGA HDLs... even if I don't do websites or databases, RoR has inspired me to climb out of my comfort zone and take on something new. Guess what? I've discovered a language which looks to be a new favourite of mine. I'm going to use it now instead of where I would have used Perl.

You can't even distinguish it from Haskell and functional programming, but yet you seem to think you're qualified to say how "perfect" it isn't... no wonder you're posting AC.

Re:Ruby functional? Not.

[Anonymous Coward]

Perhaps he's not saying Ruby should become a new standard

When he posts things like "Why bother - just learn Ruby", it's quite clear that is exactly what he means.

people like you should open your mind just a little, tiny bit, open the door and walk outside - and try a new language for a few hours sometime.

What part of "I was reading up on Ruby and Haskell this morning" don't you understand? I assure you, I'm not a single language aficionado, unlike the OP seems to be.

People like you like to get stuck in a rut and cling to what you know, forsaking everything else.

Sorry, that doesn't describe me in the slightest. You've taken a couple of comments and decided you know everything about me. You're clueless.

even if I don't do websites or databases, RoR has inspired me to climb out of my comfort zone and take on something new. Guess what? I've discovered a language which looks to be a new favourite of mine.

Good for you! And I have done exactly the same thing a few times in my career. I'm certainly not close-minded. In fact, if you read my comments, you'll see that I haven't said anything that would conflict with your ideas on learning new things.

How about you step outside your comfort zone, and get some English comprehension skills? Nowhere did I say learning Ruby, or any language, for that matter was bad. I merely said that it wasn't perfect and that any attempt to paint it as such is the attitude of a fool. I'm not the one being extreme here. I'm just disagreeing strongly with the idea that everyone should drop Python development because Ruby exists.

You can't even distinguish it from Haskell and functional programming

No, I just got the names mixed up. I'm not a morning person and I have other stuff on my mind. Sorry, but your ad hominems aren't applicable in this case.

yet you seem to think you're qualified to say how "perfect" it isn't

Do you disagree? You think Ruby is perfect?

no wonder you're posting AC.

Says somebody known only as "csirac" with no homepage or email address listed. Hypocrite.

Re:Ruby functional? Not.

[csirac]

Perhaps he's not saying Ruby should become a new standard

When he posts things like "Why bother - just learn Ruby", it's quite clear that is exactly what he means.

I think I'll write this one off to the "not being a morning person" thing... I read it as: "Why bother to learn the imitation, have a go at the original". Or something along those lines.

People like you like to get stuck in a rut and cling to what you know, forsaking everything else.

Sorry, that doesn't describe me in the slightest. You've taken a couple of comments and decided you know everything about me. You're clueless.

Seems like we've got a bit road-rage happening here... but I'll keep it interesting: you wrote off Ruby quite visciously with what appeared to be little thought... you did a bit of reading about it, along with other languages this very morning, and now I'm clueless because I made an equally whimsical assertion about your learning habits?

How about you step outside your comfort zone, and get some English comprehension skills? Nowhere did I say learning Ruby, or any language, for that matter was bad. I merely said that it wasn't perfect and that any attempt to paint it as such is the attitude of a fool. I'm not the one being extreme here. I'm just disagreeing strongly with the idea that everyone should drop Python development because Ruby exists.

It seemed to me you and the OP were two sides of the same coin. The OP was a little "over-enthusiastic", I didn't sincerely believe that he intended what you think he meant, because it would indeed be a very stupid thing; I'm sorry you seem to have read differently.

For what it's worth, it's 11:30PM here in Australia now and I've been up since 6am. If your excuse for comprehension difficulties is because it's too early, then mine is because it's too late.

You can't even distinguish it from Haskell and functional programming

No, I just got the names mixed up. I'm not a morning person and I have other stuff on my mind. Sorry, but your ad hominems aren't applicable in this case.

Yet more evidence that...

yet you seem to think you're qualified to say how "perfect" it isn't

Do you disagree? You think Ruby is perfect?

... impaired comprehension and judgement due to sleep deprivation coupled with a desire to put the world into black and white terms has turned a discussion we may all have agreed upon under different circumstances --- into a flame war.

BTW: Perfect? No. Somewhat unique and worth learning before dismissing it? Yes.

no wonder you're posting AC.

Says somebody known only as "csirac" with no homepage or email address listed. Hypocrite.

Unlike an AC posting, I actually care to a certain degree about the reputation of this handle.

Re:Ruby functional? Not.

[Anonymous Coward]

you wrote off Ruby quite visciously with what appeared to be little thought...

Please point out where I wrote off Ruby viciously. I'm quite willing to accept that I wrote off the idea that we should "just learn Ruby" viciously, but I wasn't vicious to Ruby. I said it was far from being a panacea. Do you disagree? I said Ruby programmers were few and far between. Do you disagree? I made the mistake of lumping it in with functional languages, which I immediately retracted. So where's the viciousness towards Ruby?

I didn't sincerely believe that he intended what you think he meant, because it would indeed be a very stupid thing

Then in the grand scheme of things, all we really disagree on is our interpretation of his comment.

impaired comprehension and judgement due to sleep deprivation coupled with a desire to put the world into black and white terms has turned a discussion we may all have agreed upon under different circumstances --- into a flame war.

Well no. In that particular instance, you were jumping on my statement that Ruby was not perfect as an excuse to attack my qualifications. If you actually agree with me that Ruby isn't perfect, then you have no reason to do that beyond ad hominem attacks.

BTW: Perfect? No. Somewhat unique and worth learning before dismissing it? Yes.

And I have never said anything to contradict that. I too think Ruby is somewhat unique and worth learning - which is why I was reading up on it. But statements like "why bother - just learn Ruby" rile me up because I really do have no patience whatsoever for idiocy today. And the idea that you should learn a new language just to use a particular style of development when other languages can do it comparably well is just language bigotry/elitism, which counts as idiocy in my book.

Learn Ruby because it's a cool language? Fine by me. Learn Ruby because it just happened to be the language something was first implemented in? Get lost. Imagine if the idea of a web browser written in anything other than Objective C was taboo. A web browser? In C++? God, why don't you just learn Objective C already? Obviously Safari, Opera, etc, couldn't possibly be as good as WorldWideWeb, after all, the first web browser was written in Objective C, not C++. Why bother with imitators?

Unlike an AC posting, I actually care to a certain degree about the reputation of this handle.

It's still anonymity as far as I or anyone else is concerned. "csirac" and "Anonymous Coward" are both opaque tokens that aren't resolvable to a real-life person.

Re:Ruby functional? Not.

[Anonymous Coward]

You've taken a couple of comments and decided you know everything about me. You're clueless.

Oh the irony.

And the original post did not suggest forsaking all other languages. He was referring to the use of a specific type of framework for a specific type of problem.

Re:Ruby functional? Not.

[the_2nd_coming]

are you a fucking moron? RUBY is Imperative NOT Functional.

Re:Ruby functional? Not.

[csirac]

... which is why I said:

"... distinguish it FROM Haskell and functional programming".

Perhaps I should use parenthesis in my written english to make it clearer? Like so:

"... distinguish (it) FROM (Haskell and functional programming)".

Re:Ruby functional? Not.

[Anonymous Coward]

He was referring to the use of a specific type of framework for a specific type of problem.

Um, no. He was writing off one particular framework because it wasn't written in his language of choice. Huge difference. If he was referring to the use of a specific type of framework for a specific type of problem, then he wouldn't be eschewing TurboGears, because it's exactly that type of framework.

And if there was any truth to what you say, then it would be hypocrisy, not irony. Buy a dictionary.

I find it hard to believe ...

[Feign Ram]

this video, because Stephen Grady of RedMonks is not featured - he is not even quoted in the Video. I am forced to conclude that this software project is amateurish and not enterprise class.

Re:I find it hard to believe ...

[sog11]

many thanks for the support Feign Ram, though i don't know that i can claim any special insights into the viability of TurboGears. will certainly give it a look, however.

That's nothing!

[Anonymous Coward]

I'm using Perl on Poles!

Database portability

[Craig Ringer]

There's one issue with database portability, and that's that each database uses different languages for stored procedures. This is getting better in some areas, and some DBs now have a basic procedural language that's mostly compatible, but it's not really good from a DB portability point of view.

Of course, for the plain SQL procedures that many or most of your stored procedures will be that's no problem at all. Porting your procedural stored procs isn't going to be too bad so long as they're all well documented and isolated neatly.

Additionally, many DBs let you use your preferred language for stored procedures - for example, Java or Python . Sometimes this can be a very handy thing, though of course abuse of it is hideous.

I tend to agree regarding doing data operations in the app. The more SQL and good database use I learn, the simpler the code that's responsible for fetching and storing data gets as more and more of it moves into the database its self.

Proof is Slashdot itself

[arevos]

Slashdot itself uses Apache/mod_perl/MySQL, and there's little between mod_perl and mod_python in terms of performance. Slashdot handles 3 million pages per day without any trouble. Does Slashdot not count as a 'large site'?

The idea that J2EE is the only system that can handle high traffic sites is a myth.

Re:Proof is Slashdot itself

[larry bagina]

90% of what /. serves is static (cached). disappearing stories, not being able to post comments, comments appearing int the wrong stories, miscellaneous 403 and 501s, etc, are common occurrances.

Re:Proof is Slashdot itself

[jericho4.0]

That's not the tools, but what was built with them.

Re:Proof is Slashdot itself

[arevos]

The errors in /. are beside the point; they're obviously not caused by any bugs in the language, but in the code itself. As for caching, if it's transparent to the user, than what's the difference? I don't think that Amazon or a large site such as that is going to get by without some form of caching.

The performance of mod_* and J2EE aren't so different.

Re:Proof is Slashdot itself

[scheme]

The errors in /. are beside the point; they're obviously not caused by any bugs in the language, but in the code itself. As for caching, if it's transparent to the user, than what's the difference? I don't think that Amazon or a large site such as that is going to get by without some form of caching.

The errors are exactly the point. The errors appear because the slashdot code has to do a lot of caching and other tricks to reduce system load and handle the traffic in a reasonable manner.

Other frameworks that can scale better avoid this by either having better integration/performance so you can add servers and handle the load without resorting to dangerous/complex optimizations or by implementing these optimizations in the framework so they they well tested and debugged as opposed to an ad-hoc solution.

Re:Proof is Slashdot itself

[arevos]

The errors are exactly the point. The errors appear because the slashdot code has to do a lot of caching and other tricks to reduce system load and handle the traffic in a reasonable manner.

If I understand your argument correctly, you're saying that even if an unstable piece of software can serve up 30 pages per second, a stable piece of software cannot do the same. I must confess to being puzzled by this; why does speed imply a greater risk of instability? I cannot think of a single 'trick' related to scalability that would inherently increase instability of the overall system.

Re:Proof is Slashdot itself

[scheme]

If I understand your argument correctly, you're saying that even if an unstable piece of software can serve up 30 pages per second, a stable piece of software cannot do the same. I must confess to being puzzled by this; why does speed imply a greater risk of instability? I cannot think of a single 'trick' related to scalability that would inherently increase instability of the overall system.

No, my argument is that the errors in slashdot illustrate that some frameworks aren't suitable for high volumne usage. The mod_perl and software framework used by slashdot doesn't offer much scalability. Therefore, the developers were forced to implement an ad-hoc caching system that doesn't work well. A better software framework would have better scalability and builtin caching that gives the same performance without the bugs.

In regards to your question of speed vs. stability, there's often a tradeoff between the two. Take the simple example of mutexes in multithreaded software. Leaving out the mutexes and their associated controls would let the software run more quickly but leaves the possibility that every once in a while threads interfere with each other. With caching, it's easy to get a little too aggressive and end up caching stale information or writing that stale information to persistent storage.

Re:Proof is Slashdot itself

[arevos]

The mod_perl and software framework used by slashdot doesn't offer much scalability. Therefore, the developers were forced to implement an ad-hoc caching system that doesn't work well.

With caching, it's easy to get a little too aggressive and end up caching stale information or writing that stale information to persistent storage.

So mod_perl can only be used for high volume traffic if it uses an unstable caching system. If we follow this strange premise to its inevitable end, we must conclude that there exists an unstable caching algorithm, that can be programmed in Perl, that is exponencially faster than the fastest known stable caching algorithm. After all, this theoretical unstable algorithm cannot just be, say, twice as fast - as one could just double or triple the hardware - for a stable caching algorithm to be realistically impractical, our unstable caching algorithm needs to be exponentially faster.

Let me outline a stable caching algorithm for comment posting:

Get POST data from user
INSERT user data as row in comments table
Delete cache for comments index
For each parent of comment:
____ Delete cache of comment

Now, one could remove the cache deletions and use a cron task to remove them at leisure, which would improve performance. But whilst this would result in a delay between when a user posts to when the post appears, it would not result in destruction or alteration of data, because comments are immutable.

Perhaps you could outline an algorithm that is exponentially faster, but more unstable, than the one I've given above? Extraordinary claims require extraordinary evidence.

In regards to your question of speed vs. stability, there's often a tradeoff between the two.

In several years of programming, I don't think I've ever come across a situation where stability has to be sacrificed for speed.

Re:Proof is Slashdot itself

[scheme]

So mod_perl can only be used for high volume traffic if it uses an unstable caching system. If we follow this strange premise to its inevitable end, we must conclude that there exists an unstable caching algorithm, that can be programmed in Perl, that is exponencially faster than the fastest known stable caching algorithm. After all, this theoretical unstable algorithm cannot just be, say, twice as fast - as one could just double or triple the hardware - for a stable caching algorithm to be realistically impractical, our unstable caching algorithm needs to be exponentially faster.

Okay, now you're just making up a strawman arguments to knock down. Where did I say that mod_perl can only handle high volume traffic if it uses an unstable caching system? I made an argument that the missing features in the software framework that slashdot uses may have led to it's developers getting too aggressive with caching and optimization and thus resulting in an errors in it's functioning. How you get from there to requiring an exponentially faster caching doesn't make sense.

Let's take your requirement for the unstable exponentially faster caching algorithm. From looking at queues, I'm sure you will agree that a linear increase in rate of requests can result in a condition were the queue will keep increasing in size. Now, you assert that you can simply double or triple the hardware speed to handle this.

Although this is theoretically true, in real life this is not. Consider the situation where you have a 64 processor x86 machine, you can't simply double this to get a 128 processor system since such a system doesn't exist aside from possibly in a lab. Although the web portion can be parallelizable, the database isn't easily so. Establishing and maintaining a coherent database over a cluster isn't easy and can't just be handwaved away. More significantly, anyone has budgetary constraints. Slashdot couldn't simply plunk down more cash to purchase newer hardware at will during most of its existence earlier existence and that is probably also true to some extent now.

In several years of programming, I don't think I've ever come across a situation where stability has to be sacrificed for speed.

Such a scenario is certainly possible. Imagine a race condition that occurs once every few months on average and doesn't do much harm. However guarding against this race condition requires putting a mutex around a variable that is used everywhere. Given a situation where you are trying to get all the speed you can, I can see developers deciding to live with the race condition.

Re:Proof is Slashdot itself

[arevos]

Okay, now you're just making up a strawman arguments to knock down.

Apologies; this was unintentional, and I appear to have misunderstood your argument, and you mine.

My original assertion was that mod_python is similar in performance to mod_perl, and mod_perl can scale successfully to large loads. Evidence of this can be seen in Slashdot, which uses mod_perl and handles around 3 million hits daily.

Others in the thread alleged that Slashdot was sometimes unstable, and that this demonstrated mod_perl was inherently unscalable to large loads. I disagreed, pointing out that any instabilities were due to bugs in Slashcode, rather than mod_perl's inability to handle high traffic sites.

If you are not arguing that mod_perl (or mod_python) cannot handle high loads without becoming unstable, then I have no issue. I'm not defending Slashcode, but I do take exception to the tenuous assertion that mod_perl and mod_python do not scale, as some Slashdot posters seemed to suggest. Especially when this assertion is not backed up with even the most circumstancial of evidence.

What codec?

[Cesaro]

Got Apple QT installed on an XP box and I can't play this video. I don't know that I've ever seen QuickTime tell me I didn't have a codec on a .mov file.

I'm a loss...gSpot doesn't understand the .mov file so what do I need to play this? Oh and they're calling it a compressor in my error message...

Sound works great though.

Re:What codec?

[Anonymous Coward]

There is something wrong with your setup or it is in a newer version of QT than you have. I am on XP and had no problem but I am using QT7.

RUBY || PYTHON || .NET || ... why?

[Elixon]

I think that the biggist problem of all the "platforms" is incompatibility. Do you want to use Python on Rails and you don't know Python? Do you want to use "*" and you don't know "*"? Did any "platfrom" developer thought about this problem? Or is it just a marketing how to bind the developers just to your platform?

I think it is not so difficult. My working prototype of an extension for my project will allow you to write modules for my CMS in ruby, python, php, .net... generally whatever language you know... I generally don't like tons of documentation/tutorials and so on to get integrated with other platform. I'm sure that the platform developers and I always share at least one technology that I can use without reading "How to integrate with *" tutorials...

I don't like to switch my "mother" programming language for other language just to take advantage of some great functions in some third-party platform...

Hey! Why do we have so great _standards_ and widely supperted technologies...?

Regular expressions

[Tidal Flame]

So, do you think he actually wrote a regular expression off the top of his head, or do you think he had it memorized or written down somewhere? I didn't think any human being could write regular expressions that quickly. =P

Twisted/Nevow/Axiom is better!

[Hylander]

If you want similar functionality with some more capable tools, have a look at Twisted with Divmod's Nevow & Axiom. Similar functionality but with a a whole lot more to boot.

Re:Does it scale?

[m50d]

Google uses python all the time.

Yeah, why use SQL halfway?

[TheLink]

Yeah, that's the part that gets me wondering. People keep talking about hiding away the SQL. But if they don't want SQL, then why are they using an SQL RDBMS in the first place?

Shouldn't they be using something else then? Otherwise they'll get the drawbacks of using an SQL database but fewer of the advantages. What happens if performance in a particular area is not good enough?

Say you want to store a session in a database and you want it to expire after X seconds of inactivity.

A simplistic method would be to update the session row each time there's activity. But this would cause lots of writes which would be slower in most proper databases (those that actually write to disk for writes). An alternative would be the databases equivalent of "update sessiontable set lastactive=now() where sessionid='$sessionid' and lastactive+'1 second' <= now()".

With this, you could have thousands of hits per second but only 1 forced write to disk per second due to that query.

How would you achieve this when you're so abstracted away from the SQL database? And it might look strange to others when you try to do the same thing N layers above the database. I'm sure there are better examples.

Those sort of queries are likely to look different on different RDBMSes. You could make a function that looks the same, but someone still is going to have to write the SQL for portability (and sometimes bad luck, it's not possible - DB doesn't support functions or that sort of function). Also, if only the program's session module does that stuff, then what's so bad about leaving the SQL in there? At least then there'll be some context to understand the SQL (and whether it's wrong or right ;) ), rather than it being neatly stored with 1000 other nonrelated SQL statements (as functions or otherwise).

Sure it's ugly. But if people want it all so elegant and clean maybe they should write _everything_ in some version of Lisp, and not interface with the rest of the ugly real world.

Re:Yeah, why use SQL halfway?

[poot_rootbeer]

But if they don't want SQL, then why are they using an SQL RDBMS in the first place?

How many non-SQL RDBMS's do you know of? How many of them compare to MySQL in terms of enterprise-readiness, much less Oracle or Sybase?

SQL is the lingua franca of relational database management. Any advantage in performance you might get from being able to bypass SQL's lexical parsing and query the data model directly is eclipsed by the lack of support, scalability, portability, and developer knowledge that come with a non-SQL database.

Re:Yeah, why use SQL halfway?

[arevos]

Yeah, that's the part that gets me wondering. People keep talking about hiding away the SQL. But if they don't want SQL, then why are they using an SQL RDBMS in the first place?

In one word: abstraction.

With most web applications, one often finds oneself executing many similar SQL queries. The Rails and SQLObject frameworks attempt to take some of the work out of this. For instance, using Python's standard MySQL interface, one might write something like:

cursor.execute("SELECT p.id, p.subject FROM users u LEFT JOIN posts p ON (u.id = p.user_id) WHERE u.name = %s", (username))
for i in range(cursor.rowcount):
#### do something

Whilst using a database abstraction layer, your code could look something like:

for post in User.selectBy(name = username).posts:
#### do something

Which is rather neater. Generally, you can also use inheritance to customise the database classes. However, if your program relies on a lot of complex SQL code, and very little formulaic SQL, a jack-of-all-trades database abstraction layer can just mean more work.

Re:Yeah, why use SQL halfway?

[spagetti_code]

This is a very good question - and as always it comes down to the best tool for the job.

IAADBD (I am a DB designer) who has designed and implemented DBs for large apps (millions of rows, hundreds of entities). If your data is important, you need to look at this from the data's point of view.

• Ensure the data stays clean
• Ensure the data is well designed for growth (normalization)
• Execute business rules inside the DB

I achieve these by permitting access only via stored procedures (with rare, simple exceptions), and having complete coverage with DRI (declaritive referential integrity), triggers, defaults, rules, timestamp checking and so on. I control what access people/processes have to the data, and am able to encapsulate people away from the underlying structures.

I can therefore be sure the data stays clean and well formed. I can also be sure that people are not pulling large gobs of data out of the database and across the network to implement business logic that should be done inside the DB.

If your data can be adequately protected using DRI alone, and you are not so concerned with encapsulation and performance then these tools are brilliant.

Horses for courses, as always.

Object/relational mapping

[amightywind]

It shows a high level of abstraction when you access the DB by simply loading/persisting objects

An idea that is at least 15 years old, and has never worked very well either. The problem is most relational database schemas (structure, types) do not map cleanly C++ or Java to objects. From what I've seen in corporate applications the design of database and software are typically done by separate teams with different modelling skills and design priorities. As a software developer I want speed and simplicity in the RDB and to avoid deadlock at all cost. The RDB person wanted complete problem representation in data. If the requirement is really for the database just to provide for serialization of objects it is surprising that object oriented databases have not become more popular. They've also been around 15 years. Corporations are too hooked on the report and ad hoc query capability of RDBs I guess.

Re:Object/relational mapping

[sammy baby]

David Heinemeier Hansson (one of the RoR progenitors and its most recognizable evangelist) recently addressed this dichotomy on his blog.

Active Record is opinionated software, just like the rest of Rails... And the opinion goes as follows: I don't want my database to be clever!

In other words, I want a single layer of cleverness: My domain model. Object-orientation is all about encapsulating clever. Letting it sieve half ways through to the database is a terrible violation of those fine intentions. And I want no part of it.

Re:Object/relational mapping

[mikkom]

In other words, I want a single layer of cleverness: My domain model. Object-orientation is all about encapsulating clever. Letting it sieve half ways through to the database is a terrible violation of those fine intentions. And I want no part of it.

This works fine if you only have one program accessing the database but is terrible if you have multiple clients for example. Even worse if you have real "enterprise" environment with legacy apps and such.

Re:Object/relational mapping

[Anonymous Coward]

That post was, quite possibly, the most idiotic thing DHH has written. The post he links to at the end is amazingly incoherent as well, but unfortunately quite typical. Although I still think Rails is the best for *application design*, I have to preface my recommendations to others with "but don't listen to a word they say about data management, it's a step back to 1960's-era ad hoc application-centric crap". The model in the DBMS *is the model of the business rules*. Not what's in Rails. What's in Rails is ad-hoc.

Replace "clever" with "correct" and see if you still like that excerpt. That's what we're talking about: data integrity, correctness, correctly modeling the problem domain. DHH's "advice" shows a clear ignorance of basic relational theory (a DBMS is a complete general-purpose system for maintaining data integrity) as well as a misunderstanding of how his own creation operates (ActiveRecord *already* splits business logic between the DB and the application.. hint.. where are the column names and types declared?).

Instead of calling for better-designed, Ruby-friendly databases that follow the relational model (for example, updateable views is how RM implements the lovely *encapsulation* that DHH wants), which would make Rails apps even easier to write (a truly relational database wouldn't even *need* ActiveRecord), DHH simply repeats the same old nonsense and muddled thinking from the days of CODASYL.

I'm not saying DHH is stupid. I'm sure if he pulled his nose out of Fowler's enterprise patterns book (full of awful data management anti-patterns) for a couple seconds, he'd figure it out. After all, he used to think Java was great (see some of his old blog postings), and then he saw the light of a dynamic, lisp-like language.

I'm still waiting for the visionary that will guide us into a world of great *application design* coupled with great *data management*, not just one or the other. DHH does not appear to be it.

Re:Object/relational mapping

[swimmar132]

(ActiveRecord *already* splits business logic between the DB and the application.. hint.. where are the column names and types declared?)

In the Ruby Migration script, in my projects at least.

See http://api.rubyonrails.com/classes/ActiveRecord/Mi gration.html

Re:Object/relational mapping

[kimanaw]

Corporations are too hooked on the report and ad hoc query capability of RDBs I guess.

Yes, those damn corporations (and governments, too!). Actually trying to distill useful (often legally required ) information from terabytes of data using a simple declarative language! How dare they! Don't they realize there is an army of unemployed Java/Python/Ruby developers out there that can process each of those 5 billion records one at a time after transfering them across the wire without any transactional controls ? Thats certainly much better than running the query inside the database

• under transactional control
• without pushing any data across the wire except for the final results
• using fine grained parallelism
• using state of the art data management algorithms, including advanced statistical OLAP functions
• in a centralized, manageable datastore
• within a coherent data model

My personal experience dealing with critical information from very large datawarehouses populated by the sort of O-R nonsense that seems to be "best practice" is that the O-R folks are more concerned about the user's browser than the data model. Here's a free clue to the O-R advocates: the information you're storing isn't being collected just for the convenience of persisting web forms in an indexed journaled filesystem. A personal anecdote: After struggling to derive useable information from an O-R generated database, consisting of 3 tables with about 100 columns each, I arrived at the office one day to discover there were now 5 tables with about 75 columns each. Thinking to myself, "Maybe they've finally figured out what a data model is...", I inquired about the change, and was told "because the tables were getting more than 100 columns each, so we thought we'd better break them up.".

There are people (many of whom work in corner offices, and some in legal enforcement offices) that expect to be able to query that data and quickly get rollup data to tell them how the business is doing, project how the business will do next month/year; or to find fraud; or to find customer patterns. Data models exist for a reason, and ad hoc queries exist for a reason.

Re:Object/relational mapping

[sammy baby]

Of course, there are going to be situations where RoR isn't the ideal solution. But integration with legacy apps prejudices the field against almost anything that isn't "the thing that I was last working with."

As for workign with a real "enterprise" environment: if SAP doesn't count, I'm not sure what does. (warning: links to page which has horrible, non-wrapping formatting in my browser. YMMV.)

Re:Object/relational mapping

[amightywind]

Yes, those damn corporations (and governments, too!). Actually trying to distill useful (often legally required [wikipedia.org]) information from terabytes of data using a simple declarative language! How dare they! Don't they realize there is an army of unemployed Java/Python/Ruby developers out there...

Now don't get you color up, database guy. I think we both agree companies need data stores of some form. I just think the relational database designs that predominate in business systems stink from a software perspective. Do think so? Try to adapt one large RDB solution to another. You overpaid genius database designers produce relational works of art that work like shit in practice, then call your job done. Then it us poor armies of programmers to glue together systems that cannot work because they have been designed by people who cannot understand complete systems. Remember, Edsgar Dykstra once said (I paraphrase), "programs equal datastructures and algorithms." I wonder if you database guys have that firmly mind.

Using a bulldozer to kill weeds

[porkThreeWays]

J2EE based web services are 900 pound gorillas. Sure, amazon etc use it, but I don't spend my day making amazon.com's. I spend it getting requests from various departments for single purpose programs that are web based (so we don't have to maintain yet another fat client on the desktop). These simplier frameworks make it possible to make quality web applications quickly. Most of these programs are just a varation on a theme. 2 or 3 tables with maybe a foreign key and 3 or 4 web pages. These frameworks make it possible for me to give them a finished application in a couple of hours so I can spend my time doing more important stuff.

Podcast about TurboGears

[Anonymous Coward]

Hello to all. Interestingly enough I just installed TurboGears for the first time this weekend. To test how versatile it is, I installed it three times: on my main Mac OS X box, on a Windows laptop and on an old Linux box.

All three installs went flawlessly and I was up and running in less than half an hour. I must say that the installation of TurboGears is as smooth as I have ever seen, and the documentation is superb.

I went ahead and made a podcast recording my initial experiences, and I will continue to podcast about TurboGears as I go forward, as part of my Python411 podcast series that can be found here: http://www.awaretek.com/python/index.html or subscribed to here: http://www.awaretek.com/python/index.xml

Ron Stephens

Why Mirrors?

[Anonymous Coward]

Why does this person insist on mirrors of these sites on his own network? (http://www.rubyonrails.org.nyud.net8090/ http://www.turbogears.org.nyud.net8090/ The first was slow and the second won't pull up for me. Please, just put a link to the mirrors site or link to the main page, especially for projects like rubyonrails.org.

Thank you, and good night.

Re:Why Mirrors?

[PReDiToR]

You forgot to put the colons into those URLs:
http://www.turbogears.org.nyud.net:8090/
http://www.rubyonrails.org.nyud.net:8090/

Weird

[jolande]

And I thought Rails was some type of new animal.

Re:Does it scale?

[swimmar132]

How the hell do you know that they don't scale? Have you tried it? It's called the 'share nothing' approach. Livejounal and yahoo both use it.

http://www.loudthinking.com/arc/000479.html

Re:Does it scale?

[llamaguy]

Yep. Take a look at Gmail - there are a LOT of .py files running around in there. Not to mention that the original web crawler/indexer (the one that started their meteoric rise) was written in Python...

Visual Programming

[youval]

My project, Tersus, uses an alternative approache to web application development. Take a look at http://www.tersus.org/

Well, they didn't get Slashdotted

[shis-ka-bob]

I was pleasantly surprised that the TurboGears site not only came up, it came up fast. Quite unusual for a new developer framework appearing for the frist time on Slashdot. Especically since 'Python', 'Ruby on Rails' and 'AJAX' were mentioned, I'll bet they got hammered pretty hard in the last few minutes. So at least we can see that the server scales pretty well with the number of users. This could mean nothing more than the site uses a cached home page and we are seeing the value of (something like) Squid. No matter what the cause, it is encouraging to see thay they didn't get swamped by all of us checking out the new tool.

Re:Well, they didn't get Slashdotted

[rsax]

That's because the article submitter was smart and used Coral CDN links for all the URLs. Like this one..... http://www.turbogears.org.nyud.net:8090/

So it isn't unusual at all that their site did not get Slashdotted. Even if they were using Squid it wouldn't be of much help since Slashdotting mainly sucks up your upstream provider's bandwidth, dynamic content or not.

Amateurs Versus The Industry

[Greyfox]

I've tinkered with rails a bit and I think it's a step in the right direction. What I want to know is, why doesn't anyone in the industry make it this easy? All the commercial solutions seem to be going in the opposite direction, constantly adding more XML files and more layers of API to try to cope with HTTP's deficiencies as an application development protocol. Meanwhile a bunch of amateurs with a semi-obscure scripting language manage to upstage all of them. And if I'm not mistaken, you have to do any xml httprequest stuff in the commercial packages by hand, while rails seems to do it all more or less automatically.

Rails is another example of innovation in the open source community, and will be conveninetly forgotten by the people who say there's no innovation in the open source community.

Re:Amateurs Versus The Industry

[swimmar132]

Why do you think the Rails developers are amateurs?

Anyways, the answer is probably something along the lines of "more complexity = more enterprisey = more money". Corporate execs are big fans of "enterprise" software, cuz enterprise software is adventureous and complex and hard and stuff.

Re:Amateurs Versus The Industry

[jericho4.0]

Since when has industry been the ones delivering new and exiting?

Re:Amateurs Versus The Industry

[Greyfox]

Well I haven't seen any company claim credit for it, so...

Although I suppose a lot of them technically aren't, since they probably also make a living doing that sort of thing. But the big software companies have this attitude that you have to be blessed with corporate funding to make truly useful software. Like when you get hired you suddenly have this magical software engineering powers that let you develop operating systems and other useful software. When in fact we all know that all you need is a wiseass kid from Finnland...

Re:Amateurs Versus The Industry

[binarytoaster]

Well I haven't seen any company claim credit for it, so...
So 37signals isn't a company? Rails was extracted from Basecamp and open-sourced.

So, all the arguments about RoR being untested are untrue as well because 37s is making quite enough money off products built on Rails.

Question

[g2devi]

> For anything substantial, though, it always seem to wind up
> being more work as I figure out how to configure & trick the
> persistance layer into giving me my data in the most efficient
> way. This can be frustrating when you know how to accomplish
> the same thing in 5 seconds using plain SQL. Maybe it's just me?

That raises a question about these persistence layers. Most of the tables we create use an "always insert, never update or delete" model so we can keep track of who made the change, when the change was made, and by whom.

You have to code, normalize, index, and select from these tables in a particular way otherwise you'd end up with extremely inefficient code. I just don't see how a persistance layer can handle this.

Another thing, one thing I've learned is that for enterprise applications, data outlives the application and in many cases, it predates the application. In essense, the persistence layer will need to be retrofitted into the application. Top-down GUIs like the ones presented for Rails appear to be of no use. We already know the exact representation of the data so Rails will need to work from the bottom up by default.

As a side note, I don't see why people are so afraid of a little SQL. Sure it takes time to learn how to master efficient SQL coding, but it's well worth it. It's the most efficient way to deal with most data, and because it's a declarative language, it's often more intuitive than the imperative language you're working in. It's also supported by most languages, so it's universal. If you're using a strict model-view-controller paradigm, database access will be restricted to the model and recast into data objects that can be used by the view and controller. If done correctly, you can rewrite the data code of several applications every day of the week without affecting the view and the controller (i.e. the bulk of your code). From what I've seen, that clarity of what exactly is happening and how complex things are mapped really helps with maintenance and optimization. From trying to maintain persistense layers, I'm left trying to figure out "what on earth is the persistense layer trying to do that is causing so much trouble".

Alternate to MOV?

[mike6496]

Sigh.. the only way to watch it is via MOV?? And they're 50-80Mb!!? People please be kind and give options.. I'd prefer a format like this 5 minutes to build an app that retrieves images from Flickr(presentation from web 2.0 conference in Flash8) or better yet one of these Which is a tiny file.. or even a WMV file..

I really need to watch the videos referenced in this article, but I have to install software on my system to do it and would rather not have Qucktime on my system.

Re:Alternate to MOV?

[bhaskin]

It's also in written form on the site.

http://www.turbogears.org/docs/wiki20/index.html

Brian

Re:Alternate to MOV?

[Anonymous Coward]

Complaining about Quicktime (openly documented and available for anyone to use royalty-free) and then asking for proprietary Flash 8 or WMV is idiotic. Perhaps you have strayed too far from your Windows fanboy sites.

Re:Alternate to MOV?

[sanjed]

this worked for me http://www.free-codecs.com/download/QuickTime_Alte rnative.htm you can view your .mov's in windows media player classic.

Re:Alternate to MOV?

[mike6496]

spoken like a true anonymous coward...
No, it's all about choice. I choose what applications/plugins, etc are on my system, and if someone is going to create information like these tutorials for the public in general then they should offer a choice of how they can be viewed.

I'm not a fan of windows btw.. thanks for the kind thoughts..

Re:Alternate to MOV?

[/dev/trash]

fire up MPlayer

Re:Alternate to MOV?

[TheInternet]

and would rather not have Qucktime on my system

Um, like, why?

      - Scott

I've got one major problem with Django

[jocknerd]

I was very interested in Django since I love Python. It looked simple enough and I was able to get through the tutorials and thought that Django was my holy grail. But then I tried to adapt it to an existing database system I had. And here is where the project failed for me.

The problem I have with Django is that it wants to hijack your database. Sure, they make it real nice to start from scratch. You write classes, run some scripts, and next thing you know, its created a database and all the necessary tables and indexes. But what if you've already got a database? Well, they provide a way to use your existing database too. But unfortunately, Django wants to add its own stuff to your database. I find this unacceptable. If Django needs a database for its internal administration stuff, fine. Create a separate database for that. But don't throw your tables inside a database that may already be in use for other projects.

Plus, what if I want this database used for multiple projects. I don't think Django makes that possible.

too much

[l3v1]

So it's simple, it only uses
- Python,
- SQLObject,
- CherryPy,
- Kid,
- MochiKit,
- some extra pythonic glue.
I know you don't really have to know these to trash together some wiki. Still, I would never ever in this life give the task to someone who doesn't know them just uses them (by some higher level lib or toolkit whatever).

All in all, I find it nice that there are quite some number of ways for mostly average skilled users to put together stuff like that wiki. Still, for a real task, I'd prefer someone who can make it all happen with one or max two languages and some db backend.

catalyst

[Hohlraum]

catalyst gave me more of a programming woody than rubyonrails did :)

Looks like Django!

[fernique]

This framework looks very-vey similar to Django. Unlike TG, Django is all-in-one solution: built-in admin interface, built-in extensible template system, and so on.

If someone is interested: http://it.slashdot.org/article.pl?sid=05/08/02/005 1258

mod_python support?

[MORTAR_COMBAT!]

So... do I have to run it on the "built in" webserver or will it run under mod_python? Doesn't appear to do so.

Re:mod_python support?

[Gunfighter]

That's one of the first things I checked. According to the CherryPy website, the 2.1 version (currently in release candidate stage) will integrate directly with mod_python.

Re:mod_python support?

[fernique]

It seems they recommend just a standalone process behind apache for production setup.

Re:mod_python support?

[MORTAR_COMBAT!]

Well, it's good enough for WebSphere, right?

Float: r;ght

[abxpacketloss]

whoops...

A necessary evil

[Kaseijin]

But if they don't want SQL, then why are they using an SQL RDBMS in the first place?

It's a robust data store that isn't tied to any one application language or object model. Often, SQL per se isn't the problem, but differing dialects are.

How would you achieve this when you're so abstracted away from the SQL database?

Use a mapper with the required feature or drop down a level. Abstraction isn't all-or-nothing.

Those sort of queries are likely to look different on different RDBMSes. You could make a function that looks the same, but someone still is going to have to write the SQL for portability....

If someone already has, why should I do it again?

FYI

[SamSim]

20/15 is 133% :)

if you refuse to use Ruby, I guess this is okay

[Anonymous Coward]

But, man, it ain't rails. Look at this:

  class Page(SQLObject):
    _connection = hub
    pagename=StringCol(alternateID=True, length=30)
    data=StringCol()

To me, that looks too much like your typical Python code. Quite *readable* yes, but when you read it, you have no idea *why* all that stuff is there.  StringCol? data=Stringcol()? And yes, a leading underscore thrown in for good measure (every Python program must have a few underscores thrown in.. I guess python programmers long for the dollar signs and punctuation of Perl).

Now look at the Rails (ActiveRecord) equivalent:

  class Page < ActiveRecord::Base
  end

Hmm, no boilerplate.

Now, let's have a laugh:

  @turbogears.expose(html=  "wiki20.templates.page"  )
  def index(self, pagename="FrontPage"):
    page = Page.byPagename(pagename)
    content = publish_parts(page.data, writer_name=  "html"  )[ "html_body"  ]
    content = content.encode(  "utf8"  )
    return dict (data=content, pagename=page.pagename)

What is all that *STUFF* for? He even goes on to call that "very readable" !!! Here's the ruby version (as far as I can make it out, anyway):

  def index
  end

Do you see the difference? In a Rails app, most of the code is at the level of your problem domain, NOT the level of pushing bits around from the DB to the HTML templates. All the knock-offs are full of *noise*.

That's what makes Rails so lovely.

But I guess if you just refuse to use Ruby, you'll have to settle for what you can get.

Re:if you refuse to use Ruby, I guess this is okay

[lcracker]

You're comparing apples to tacos here. The SQLObject example contains the definition of the table, so that it can be automatically created if it does not already exist, where the ActiveRecord example depends on the existence of the table in the database.

As for the page example, clearly you have no idea what you're talking about. It says:

1. Expose an "index" URL as this method with the "wiki20/templates/page.html" template
2. Look up the content of the page in the database
3. Use docutils to parse the content of the page as a reStructuredText document, and get the result as an HTML fragment.
4. Encode the HTML fragment as UTF-8
5. Return a dictionary containing the page name and the HTML fragment for usage by the template (or a direct JSON request)

Re:if you refuse to use Ruby, I guess this is okay

[badmammajamma]

No amount of eligance saves you from the fact that Rails is as slow as a two-legged dog. It simply can't compete with anything else performance-wise.

Re:if you refuse to use Ruby, I guess this is okay

[downward dog]

Hmm - if your priority is raw speed, then Rails might not be the best framework for you. But how many applications are like this?

Let's assume (and this is my experience, and the experience of countless others) that a Rails project can be developed much faster than a Java project or a PHP project. (I come from the PHP world and can back this up; others have seen the same advantages with Java.) And development is typically a much bigger cost than hosting when it comes to most web applications.

We can immediately cross off any website that is hosted on a shared server (99%?), since these sites don't even need a dedicated server (which can be had for a fairly trivial cost of $99-$299/month). And if a single server is too slow, then scaling across 2-4 machines a fairly affordable option. So what are we looking at? Banking software? Amazon.com?

The only sites I can imagine that "need" the speed avantage of PHP or the database scaling of Java are (1) sites that are ridiculously large, or (2) sites that are very easy to program and get a huge amount of traffic. If a site can be built in a day or two in PHP and has hundreds or thousands of concurrant users, then the development advantages of Rails are less important. And if a site is going to be built by 10+ developers, and it needs to scale across a dozen servers, then maybe Rails isn't right. But I would suggest that for 99.9% of projects, application speed should be one of the least important criteria used when deciding on a language or framework.

Re:if you refuse to use Ruby, I guess this is okay

[badmammajamma]

Spin it any way you like but I've never been on a project, web-based or otherwise, where they didn't care about performance. Until someone shows me something other than some annecdotal evidence that the development of Rails is SO much faster than Java or PHP that it makes up for the massively beefed up server you will need to run it on, I call bullshit. For that matter, most companies aren't willing to simply toss servers that they have that work perfectly well currently. Sometimes you just can't upgrade them enough to make the difference and since Rails apps are an ORDER OF MAGNITUDE slower than Java and PHP apps you will definitely have to make a huge upgarde to keep the equivalent performance level.

It's funny because I remember the time when people use to make all kinds of jokes about the old "throw more hardware at the problem" solution to problem solving but now that we have Rails it's simply Ok!

The other problem with Rails is that if you have an existing data model you need to code against, it suddenly becomes tons less attractive and less quick. Sorry but there's just too many limiting issues with it for me to call it ready for primetime for anything but the most low capacity, non-mission critical sites imaginable.

Re:if you refuse to use Ruby, I guess this is okay

[downward dog]

I come to Rails from PHP. I was a pretty good PHP programmer, but no expert. I wrote object-oriented code, tried to keep logic out of my presentation, etc. I switched to Rails three months ago, and I would estimate that I already program at least 3x-5x faster in Rails. I recently recreated a project that took me 3 weeks in 2 days. Granted, having built the site once, I had worked through the planning issues, but these were not that complex. My experience with PHP was that it is great for small projects (a few pages), but time and difficulty grew really fast as the project got bigger. It was like going from 5 major functional items to 6 major functional items would double the project.

A better PHP programmer could have probably avoided some of the problems I ran into. But after a few months on Rails, I am developing faster, making more money, and writing far better code. It is scary to think of where I will be in a year.

This is anecdotal, like you said, but it is true. People have come from the other direction (Java) and reported similar things.

Rails may not be for everyone. If I worked for a team 10+ programmers, I might not use it. If I were a designer who just needed to add basic functionality to pages (like a random image gallery), I might go quick-and-dirty with PHP. But for my own purposes--working with one or two other programmers on a project; building $8k-$25k web applications for small businesses--Rails is perfect.

The reason performance isn't a big deal is simple, for myself and for many others.
1. Most of my projects are low traffic/high complexity.
2. A dedicated server can handle a lot of traffic, even with Rails, and can be had for $1200-$3000/year.
3. Most of my projects don't even need a dedicated server to begin with, even using Rails.
4. Development and maintenance savings far exceed any extra hosting costs.

Like I said earlier, a low complexity/high traffic site might benefit from PHP. But in my world, these sites are rare.

Finally, in my experience the low performance of Rails is a lot of hype. Use caching, use fastcgi, optimize your queries, and it is definitely _not_ "an order of magnitude" slower than other technologies. If I am wrong, fine... Just show me some solid data.

Re:Does it scale?

[Anonymous Coward]

And their WHOLE support system.

Wilst?

[Anonymous Coward]

Any post that starts with "Wilst" gets auto-ignored by me.

Learn to speak Americun English, dude!

Comment removed

[account_deleted]

Comment removed based on user account deletion

Re:Confusing

[swimmar132]

Because your last example doesn't close the HTTP connection. And it's not exception safe.


File.open("my_file") do |f|
    f.read # do stuff with it
    # if exceptions are thrown, the file will be closed
end

Not relevant?

[PCM2]

And abstraction (in this case) is good, as it helps the developer concentrate on the relevant parts of the program.

OK, I understand what you're getting at, but I concur with some of the earlier posters to this topic. Here we are, implementing a Wiki, the very definition of which is a kind of message board that collects comments from participants and keeps them in a data store -- and you're saying the data storage part is not relevant? Abstraction is very useful in early phases of development, but I'd hope it was possible to crack open that abstraction and muck with the SQL etc. when it came time to optimize your app.

Re:Not relevant?

[swimmar132]

You can easily do custom SQL with Rails.

I'm ready... but I can't get it working

[MrBlic]

I've been following python web development for a while, and currently have a few sites running with Zope + Zope Page Templates + ZSQL Objects + MySQL and they work great. The only problem is that I want a more lightweight faster server.

I am all ready to try TurboGears, but I have not been able to get mod_python + apache2 running on my mac mini. Does anyone know of a howto that will make my TurboGears web app start when the mac starts and mix TurboGears with static content? I really want to follow this example http://www.jamwt.com/mpcp.py but don't quite know how to get past some compilation errors with mod_python on my mac (OSX 10.3) and convert this to be TurboGears-aware instead of just cherrypy aware.

The Kid templates are a great alternative / improvement over the Zope Page templates. The pages are cleaner and I don't have to look up how to do tal:defines as often. I would probably not use SQLObject, but instead start with Durus.

I'm just waiting a few months for it to become even more of a no-brainer for me.

-Jim

Comment removed

[account_deleted]

Comment removed based on user account deletion

Re:Does it scale?

[Ian Bicking]

From what I understand Google uses C++, Python, and Java. Java is used mostly for internal accounting projects. C++ is used for all the stuff that has to be Really Fast, and I suspect lots of code gets migrated from Python to C++ during the development cycle. They use SWIG to interface between Python and C++. I haven't heard about them using many other languages, except XSLT and other special-purpose languages.

Re:Does it scale?

[Jack9]

There's too many new/emerging techs integrated into one monolithic framework as if it will work next year...Here's something new it will teach you, what works one year for a complex framework (made up of independent vendors) wont work next year when you install the latest releases. If someone is specifically maintaining the framework for $$$ you will probably be able to get it to work with a lot of tweaking the Pythonic glue... I think they meant pythonic scripts. Right. I will reiterate the parent, this is a waste of time.

Re:Confusing

[swimmar132]

do || .. end is basically equivalent to { || ... }, so I'm not sure where you're going with that.

Personally, I find the block approach to be more clear. The other method is what C / C++ / Perl / Python probably do, but blocks are a big reason of why I like Ruby.

Using blocks makes it really clear when the HTTP connection is open and when it is closed. Same with doing a database transaction or a File operation.

Re:Does it scale?

[swillden]

A few hours getting to know something new is never wasted.

I once spent a couple of hours looking at VBScript.

I think I came away knowing less about good programming than before, *and* I was out two hours.

You're not getting "functional programming" right.

[Estanislao Mart�nez]

For a language to be functional it is essential to restrict side effects (haskell with monads, ocaml with 'mutable'), or at least have some good support for no-side-effects programming: currying, higher-order functions, the "let x = expr1 in expr2" (aka non-destructive assignment) construct and so on.

I don't think any of these features, other than higher-order functions, is really essential. Though if you rework the "let x = expr1 in expr2" as "lexical scoping," I might accept that (though I'd still point out that let is "really" just lambda: "((lambda (x) expr2) expr1)"). Scheme is routinely labeled as a functional language, and it has side effects. Cons pairs in Lisp in general are mutable.

Anyway, I think asking whether a language is "functional" is not as good of an exercise as seeing functional programming as a programming style or set of techniques, and then asking how much support a language provides for using that style, and how natural it is to use it in that language.

Ruby is no more functional than Python, even by the most twisted definition of 'functional'. Yup, it uses closures (aka blocks) and that is how far it goes towards functional. [...] Both Python and Ruby make routine use of destructive assignment, which is the epithome of side effects.

Here I'd respond that Ruby provides much more natural support for higher-order functions (because of blocks), its libraries are designed around them. It's actualy hard to avoid using them any time you program in Ruby.

This is not so with Python, whose design is in fact evolving to discourage a functional programming style (Guido has repeatedly rejected adding anonymous closures to the language, and is planning to get rid of built-in functional combinators like map and reduce).

What's non-functional about Ruby? I'd say the biggest thing is the fact that its default sequence datatype is a flexible array type, and not a singly-linked list type.

How many times does it have to be said

[Anonymous Coward]

DB interfaces offer you a way to bind parameters

so you can create queries like

qry = new Query("select * from foo where bar = ?");
qry.setParameter(1, "xxx");

or execute stored procedures like

qry = new Query("exec foo(?,?)");
qry.setParameter(1, "zzz'x'");
qry.setParameter(2, "yyyy'q'");

and the interface ensures that no sql injection attacks are possible

You're making the wrong comparison

[Estanislao Mart�nez]

You're saying that a statement that does *less* than the other one is easier to understand.  Way to go, Sherlock.

Now let's try a comparison between statements that do the *same* thing.  First, the one using the block to manage resource allocation and deallocation:

    Net::HTTP.start( 'www.ruby-lang.org', 80 ) do |http|
      print( http.get( '/en/LICENSE.txt' ).body )
    end

...and second, the *real* equivalent without blocks:

    begin
      http = Net::HTTP.start( 'www.ruby-lang.org', 80 )
      print( http.get( '/en/LICENSE.txt' ).body )
    ensure
      http.close
    end

Needless to say, the block version is shorter.

Anyway, this pattern you see here is one of the most common Ruby idioms, which you should get the hang of if you're learning the language: using blocks to decouple code that manages a resource *through its whole lifecycle* from code that use the resource.  It's always like this:

    Resource.acquire {|resource|
      # do stuff that requires resource
    }
    # resource has been released, without you having to say so,
    # even if the there was an exception

The same pattern applies to files, network connections, database result sets, thread locks, anything.  Sure, it's unfamiliar to *you*, but you only need to learn the pattern *once*, and you reuse it all the time for different kinds of resources.

And to top it off, you can write your own methods like acquire() above in Ruby itself, just by using yield.  This is not some special syntax sugar for resource management--this is just blocks.  Here's the pattern for writing a method like acquire():

    def acquire
      begin
        resource = low_level_acquire()
        yield resource
      ensure
        resource.low_level_release()
      end
    end

Re:You're making the wrong comparison

[tdelaney]

Fortunately, Python 2.5 is going to have context managers with a much nicer syntax than the Ruby syntax:
http://www.python.org/peps/pep-0343.html (note: that's not quite up to date - there's been a slight change recently on python-dev).

with opening("filename"):
        do stuff

which will open the file, do stuff with it, and ensure that it is closed. Any generator can be changed into a context manager, and any class can likewise be a context manager (or a manageable context - that's the new bit).

Re:Confusing

[Anonymous Coward]

It's "much easier to understand" if you're a Python or Java programmer, maybe.

The idea of explicitly binding locals for only a specified scope isn't new -- if you're a Lisp or Smalltalk programmer the other way seems much more natural.

I find that lots of programmers who grew up with low-level languages are afraid of high-level features -- for example, they'll find that passing closures to map/reduce isn't as "easy to understand" as doing a for(i=0;ilt;n;i++) {...} loop. There are good reasons for using high-level features, though: you can't really have an off-by-one error when using map and reduce, and it's much easier to identify the invariants. (Everybody remembers how to prove correctness, right?) Avoiding assignments makes things clearer.

I'm a Python programmer, and I've never written more than 50 lines of Ruby, but I think the {|x| ...} syntax is much more natural.

Re:You're not getting "functional programming" rig

[Anonymous Coward]

This is not so with Python, whose design is in fact evolving to discourage a functional programming style (Guido has repeatedly rejected adding anonymous closures to the language, and is planning to get rid of built-in functional combinators like map and reduce).

Point the first: nothing that you mentioned has yet been removed from python.

Point the second: map and reduce are on the suggested drop list in favor of other functional programming constructs like list comprehensions and generators that do the exact same thing with cleaner syntax. Because that's what python is about ... reduction and simplicity without losing functionality.

Point the third: please explain how anonymous closures are superior to lambda constructs, list comprehensions, or generators each used in the appropriate context.

Re:Does it scale?

[ucblockhead]

When I interviewed at Amazon last year, they told me that they still used C++ for a lot of stuff.

Re:You're not getting "functional programming" rig

[arevos]

Here I'd respond that Ruby provides much more natural support for higher-order functions (because of blocks), its libraries are designed around them. It's actualy hard to avoid using them any time you program in Ruby.

I think you'll find there's not too much difference between Python and Ruby in this respect. In Ruby, code such as this:

method(method_args) { |yield_args| code }

is just about equivalent to this in Python:

for yield_args in method(method_args): code

Ruby and Python take different approaches to the same problem, but I'd hesitate in saying one was clearly superior to another.

This is not so with Python, whose design is in fact evolving to discourage a functional programming style (Guido has repeatedly rejected adding anonymous closures to the language, and is planning to get rid of built-in functional combinators like map and reduce).

That's because list comprehensions, generators, and inner functions do the same thing in a more Pythonic way. No functionality will be lost with the disappearance of lambda, map, filter, etc.

Bah!

[Anonymous Coward]

I'm waiting for Perl's Acme::Rails module, myself.
I'm sure it will be MUCH faster than these.

Unless someone compares it to Roadrunner, at least. No beating that thing.

Re:Bah!

[Anonymous Coward]

Bah! Perl is dead. You Perl morons should really wake up. Perl 6 is going to come out in 10 years. So what? You'll all be dinosaurs by then. Nobody cares anymore.

Re:You're not getting "functional programming" rig

[Ian Bicking]

Point the third: please explain how anonymous closures are superior to lambda constructs, list comprehensions, or generators each used in the appropriate context.

There are two things I can think of (and I'm saying this as a Python advocate): finalization and some kinds of callbacks (some callbacks are best presented as functions, so they are already fine in Python). Finalization currently is done in python with try:finally:, and it's easy to forget the proper wrapping. Python 2.5 will add a new construct for this, described in PEP 343 (with some acknowledged heritage from Ruby). The second, callbacks, I think is largely addressed with coroutines, described in PEP 342.

Some people would say that Ruby's one feature -- anonymous closures -- is superior because it is responsible for all sorts of functionality that is implemented via several constructs in Python. That I would disagree with -- the features (things like looping) are such fundamental patterns in a language that their use will always be idiomatic, no matter how elegantly they fit into some underlying syntax or functionality.

Oh, great.

[Estanislao Mart�nez]

Fortunately, Python 2.5 is going to have context managers with a much nicer syntax than the Ruby syntax [...]

First, by "much nicer syntax" you must mean "with a syntax that I'm already used to." (OTOH, I spend all day programming Scheme, so you probably think I'm a loony anyway when it comes to syntax.)

Anyway, yeah. The Python language is going to be revised to add yet another special-purpose feature and syntax to do yet another specific thing that any language with closures can do trivially with a higher-order function, without any revisions to the language.

And then it's going to take a while for the revisions to be available in all of the implementations of Python, and for third-party libraries to catch up with the language and implement APIs that use this feature. Sure. Sounds great.

Same story with list comprehensions, generators and generator expressions. Ruby, Scheme, and other such languages don't have any of those. Instead, they've had anonymous lexical closures since the start. For example, Python required a new language version to implement generators; in Ruby, the same functionality was implemented as a standard library, and to boot, it was implemented in Ruby itself.

Or, in other words...

[Estanislao Mart�nez]

This is not so with Python, whose design is in fact evolving to discourage a functional programming style (Guido has repeatedly rejected adding anonymous closures to the language, and is planning to get rid of built-in functional combinators like map and reduce).

That's because list comprehensions, generators, and inner functions do the same thing in a more Pythonic way.

Or, in other words, list comprehensions, generators and inner functions are designed to discourage a functional programming style.

Re:Or, in other words...

[arevos]

lambda's and maps != functional programming The main aspect of functional programming is functions without side-effects. Lambdas are one way of achieving this, but not the only way.

Hornet (was Re:no sql?)

[SlightOverdose]

Archetypes does not handle Object-Relational mapping well, and shouldn't be used to store relational data regardless- The ZODB was not designed for large numbers of objects, and most certainly doesn't handle relational data well.

Try Hornet (http://www.plone.org/products/hornet). It stores everything directly in a relational database, works with ZODB transaction (So it commits and rolls back at the same time, keeping ACID complience), and uses lazy iteration to handle millions of rows without a problem. Plus, As far as any regular zope app is concerned, you can treat it as a normal folder/object in Zope/Plone whatever.

Theres no excuse to ever store relational data in the ZODB again :)

New Language?

[TheInternet]

Come on people, there's no need to make a revolution every time a new language is released!

Ruby's been around for ten years.

      - Scott

new == "True"if new == True

[andrzejl]

on http://www.turbogears.org.nyud.net:8090/docs/wiki2 0/page4.html

should not it be:

2 def save(self, pagename, data, submit, new):
3 hub.begin()
4 if new == True:
5 page = Page(pagename=pagename, data=data)
6 else:
7 page = Page.byPagename(pagename)
8 page.data = data

Re:new == "True"if new == True

[Anonymous Coward]

1. No it shouldn't.

2. For a better way than new == "True", see, um, the very next page in the tutorial!

Comment removed

[account_deleted]

Comment removed based on user account deletion