Canonical Begins To Open-Source Launchpad

kripkenstein writes "Canonical, the corporation behind Ubuntu, has begun to open-source Launchpad. Canonical has been criticized for not doing so earlier. The first component of Launchpad to be open-sourced is Storm, described as an 'object-relational mapper for Python.' A tutorial with many examples is available. The license for Storm is the LGPL 2.1. Inspection of the source files shows they contain the common phrase, 'either version 2.1 of the License, or (at your option) any later version,' meaning that Storm is LGPLv3-compatible."

Comments

It's Always a Choice

[eldavojohn]

Canonical, the corporation behind Ubuntu, has begun to open-source Launchpad. Canonical has been criticized for not doing so earlier. I wish people would relax and remember that everyone has the choice to release their software under whatever license they deem acceptable.

Now, you might be wondering why anyone would open source anything at all. And my simple answer to that is this: Prior to today, if someone said 'Launchpad' to me, it would be the Ducktales character. As soon as this application is open source, it's going to become something I installed on my box and played around with for a while. And that's the difference, if it's proprietary, you better be prepared to sell it or it's going to die a death of obscurity without anyone even hearing about it.

People have the choice not to open source software and oftentimes, it's for very good reasons. I don't think this case is any different as the Wikipedia article states:

Mark Shuttleworth responded personally to this criticism stating that Launchpad needs paid-programmers to continue the development of the Launchpad platform and that there would be no point in developing multiple versions of Launchpad due to the probable incomparability of the forks [2]. Mod me as flamebait if you want but the original creators deciding that they value the quality or single source of code is just as valid as any other for delaying a release under an open source license.

Canonical isn't stupid and, yes, they're making money. According to Wikipedia, their 50 person company has an annual revenue of $10 million. Which isn't too shabby. I think these guys are genuinely interested in being both an active member of the open source (Ubunutu) and commercial (Project Landscape) worlds. Isn't it obvious who their thinking of when they put:

'either version 2.1 of the License, or (at your option) any later version,' which is pretty much proof that they have companies in mind who refuse to proceed to GPLv3 (like Microsoft and the cadre of companies that have paid them for software patent protection). You can paint this move as evil, brilliant, successful, intelligent or any of all of them. To me, I trust these guys as Ubuntu has made open source a little more accessible to the world and I really believe that not only do they know what they're doing but they're going to be around for a while. That's good news for software and (at least in my opinion) therefor good news for everyone.

Re:It's Always a Choice

[physicsnick]

As soon as this application is open source, it's going to become something I installed on my box and played around with for a while. I don't think you understand what Launchpad is.

Part of the reason Canonical was in no hurry to ready Launchpad for open source was that it wasn't really meant to be hosted on a variety of different servers or instanced for each project, like Trac; it's a centralized system designed to host many projects concurrently, like SourceForge. In short, there was only supposed to be one Launchpad.

The fact that people wanted Launchpad open source to host their own projects meant that Canonical first had to design standards for communication between different instances of the software. This should allow you to, say, host your own project on your own Launchpad, but still have bug reports communicated automatically with Ubuntu.

Re:It's Always a Choice

[morgan_greywolf]

which is pretty much proof that they have companies in mind who refuse to proceed to GPLv3

Or, maybe the component contains code that's only available under a license that's not GPL 3.0 compatible.

Re:It's Always a Choice

[semiotec]

their 50 person company has an annual revenue of $10 million.

revenue isn't profit. Sure they are generating quite a bit of revenue for the relatively small staff, but there's no indication of what their operational cost is. Considering they do stuff like mailing out free CDs to anyone that requests it and likely many other nice gestures for the community, their operational cost is probably also high.

Having said that, I do sincerely hope they are and can remain in the black.

Re:It's Always a Choice

[mabhatter654]

I think they wanted Launchpad as their "money maker" the end product would be Open Source, but they would own the project building system as it represents THEIR work structure. Like how Sourceforge does things only more focused. They've been squeezed by the FSF to open it up.. the FSF even created a non-launchpad based distro GNUsense. Over all, I think they didn't really make as much money as they thought they would being the sole gatekeeper and to leave it competing against the other project tools that are all the way Open Source is silly. Perhaps if they donate the code projects like Debian proper or other independant projects will plug in. Cannonical wants to stop dealing with all the source themselves. They need to move the bugs and requests to the actual program devs so they don't get bogged down. Having the author's version would give them a leg up over companies like Red Hat that rely on maintaining their own packages for everything. I'd expect the Dell deal opened up some cash. Linus may be shopping for a better project tracking system too for the actual Kernel. Nobody in OSS wants to "owe" another company to keep its project alive because it's caused problems in the past.
Launchpad does do something unique, better than others. It combines CVS, groups, collaboration, bug tracking, discussion, and project management in one place. It has a successful distro with a few thousand packages pumping out of it, so it's not a hobby project.

Re:It's Always a Choice

[marcosdumay]

One have the option of relicensing it into GPLv3.

Re:It's Always a Choice

[Anonymous Coward]

And that's also an unsourced statement in that wikipedia page... have a look at the source of the page, there's a comment basically saying "I think I read this in an interview somewhere that this is how much it costs to run", which is a rather shaky basis to assert revenue from!

Re:It's Always a Choice

[H3g3m0n]

Having the choice of GPLv3 is fairly close to being pointless, the GPLv3 is designed to plug loopholes but if a company wants that can just go with the GPLv2 without the restrictions. With that said I doubt Launchpad is going to be running on a TIVO any time soon. The possibility of patents does exist, since a company could take launchpad and code patented stuff into their fork of launchpad, then charge for it and sue anyone who uses the opensource product without a licence. Having the GPLv3 option does allow someone to fork it and make a GPLv3 only version, but then this version would have no way to share code between the 2, its also unlikely to become widely used unless it has some killer feature (that isn't just reimplemented on the official branch). It does provide the option of allowing the product to change license if the rest of the opensource community goes GPLv3.

ORM == good

[wawannem]

After having used Object-Relational Mapping in Java for a little bit, I really think that it should be adopted more widely. Prior to using Hibernate in Java, I stood by the notion that CUD operations in the database should strictly be done in stored procedures. This notion seems to be going away, not just for myself but in the industry as a whole. -Wes

Re:ORM == good

[cromar]

Can I get a hell yeah?

Seriously. ORM is sooooooo niiiiiiiice.

Re:ORM == good

[bockelboy]

ORM is great! ... until you you have a couple hundred thousand rows. Then it's slow. ... until you have a couple million rows. Then it's unbearable.

I love ORM for smaller applications, but there's always a point where heading down the hall to say "hi" to the local DBA is a good idea. And beware, redesigning the DB from the ORM to your own schema can be extremely painful. How close the ORM schema is to "pleasant" depends highly upon the package you use.

This is from someone who is trying to perform queries on someone else's database designed with Hibernate. One that has 12 million rows (average row size, 9KB). Which has been running my simple query for 40 minutes.

Re:ORM == good

[Shados]

terrabytes of data here. Works pretty good. Where's the problem exactly? In most scenarios, it will be doing the exact same query anyway (not a billion ways of doing CRUD for example, and most data mining operations should be done on an OLAP system anyway, so....)

Re:ORM == good

[wawannem]

> ORM is great! ... until you you have a couple hundred thousand rows. Then it's slow. ... until you have a couple million rows. Then it's unbearable.

> This is from someone who is trying to perform queries on someone else's database designed with Hibernate. One that has 12 million rows (average row size, 9KB). Which has been running my simple query for 40 minutes.

Don't get me wrong, bad database design is bad database design whether it is ORM or any other technology. Something tells me that based on the average row size of 9KB, I'm thinking this table does not really follow good relational design principles (1st normal form, I'm guessing). I've had a project where the data grew quite quickly, quicker than we had anticipated and the database needed optimized. Fortunately, since we were using Spring backed with Hibernate, we had interfaces defined for all of our persistent objects. Our solution was to use iBATIS to get some more granularity with our database queries, build some stored procedures (so that the query plan would be compiled) and add an index or two. The iBATIS classes were retro-fitted to implement the interfaces for the hibernate objects and then all we had to do was tell Spring to use the iBATIS objects rather than the Hibernate objects. Surprisingly, all of the unit tests passed and post-install, we increased our performance quite drastically. The whole project went as smooth as you could hope and I was pleasantly surprised because I really didn't think it would work out. I've been a fan-boy ever since. I still start with hibernate because it does help cut-down the design time, and I believe that "premature optimization is the root of all evil" (Knuth) -Wes

Re:ORM == good

[Cyberax]

Your statement is just plain stupid.

Decent ORMs do nothing but map object operations into SQL statements. SQL from an ORM tool is not going to magically work more faster or slower than a hand-written one.

Again, decent DB schemas (i.e. fairly normalized ones) map nicely into object models (hell, ER-diagrams used to model relational tables map directly into object diagrams). It's the databases with weird tables without PKs and strange stored procs which do not map well.

I've worked with Hibernate application handling OLAP operations on 10 terabytes of data without any problems.

Re:ORM == good

[nuzak]

> After having used Object-Relational Mapping in Java for a little bit, I really think that it should be adopted more widely.

In Python, it already is. I'll be amazed if Storm is even nearly as good as SQLAlchemy. My guess is it's closer to SQLObject, and even SQLObject has had many years to shake out any issues it has. Unless Storm is radically different, it's just another Not Invented Here framework of dubious utility.

ORMs are all right, but I wish I could express relations more succinctly in the language itself. Right now I can't even get declarative constraints in most languages that ORMs are pitched for.

Re:ORM == good

[iluvcapra]

Everybody's so quick to throw that 'stupid' word around.

I have on more than one occasion run into code that looks like this (pseudo-code):

Array myResults = Array();
Array products = MY_FAVORITE_ORM.all_rows_for_entity('products');
for product in products {
if (product.price < 2.99) {
myResults.push(product);
}
}

This works a hell of a lot slower than a SELECT, and this is just a stupid example. You're much more likely to run into it when people are building big honking summaries: they run a ton faster if you know how to build nested SELECTS and how to GROUP by one thing and ORDER by another while further knowing the difference between an INNER JOIN and a LEFT OUTER JOIN. But a lot of people don't, so they just use the language to do the summarizing.

There are other nastier things that can happen, particularly when people try to use ORMs to do what TRIGGERS usually do, and so on.

Re:ORM == good

[DragonWriter]

Decent ORMs do nothing but map object operations into SQL statements. SQL from an ORM tool is not going to magically work more faster or slower than a hand-written one.

As I understand it, a procedure in an application making a series of SQL calls to a database to achieve some result may in certain circumstances be much slower than a a single SQL call to the database which executes a stored procedure which does all the work.

But I don't think the size of the database (or table, which seems to be the real issue GP is getting at with row counts) should normally be the issue, though I could be wrong, I'm not a database expert.

Re:ORM == good

[bockelboy]

Correct Usage: Human writes SQL schema, then integrates the ORM. As long as human doesn't do silly things in the programming language, we have success. Database objects become much easier to use, and speed is fast.

Incorrect Usage: Human writes an object spec. ORM auto-generates SQL schema. Human blindly uses machine-generated ORM bindings without understanding underlying SQL. Database gets mildly large, then human complains "the stupid thing is slow".

If ORM is done for convenience, great! That's what I use it for.
If ORM is used in lieu of understanding how SQL works, you could be headed for trouble.

Re:ORM == good

[Anonymous Coward]

Ok, maybe it's because I don't understand the concept fully. Being new to this job has been tough and none of this open source stuff was covered on the MCSE exam.

My apologies for the harsh post earlier; sometimes I say/post things before thinking and they end up making no sense.

Re:ORM == good

[afd8856]

I'd be interested more in if they use it as storage for zope 3 objects, and if yes, when they'll release the component that binds them together.

Re:ORM == good

[Anonymous Coward]

So if you have stupid or ignorant people misusing an ORM product, the resulting software can be really slow? Wow, what a surprise.

Re:ORM == good

[imbaczek]

Face it: the code you posted _is_ stupid. Hopefully you know that.

That aside, ORM alone won't make anyone a great programmer. It's a tool that does a lot of boring stuff so the man that operates the keyboard doesn't have to; it won't cure cancer and bring world peace just because you .select('world_peace') instead of SELECT world_peace.

Re:ORM == good

[Anonymous Coward]

You might find

http://sourceforge.net/projects/dee-python/

interesting. This is an attempt to make
a 3GL (Python) work seamlessly with relational
data. As a relational language, it is not SQL, and its authors argue
that the Dee approach is better than SQL.

Re:ORM == good

[ForumTroll]

If you work with complete idiots using an ORM might cause problems similar to your example, but I wouldn't count on those same idiots to write efficient SQL and manage caching and connection pooling either.

Re:ORM == good

[mixmasterjake]

My personal feelings is that an ORM makes the easy stuff easier and the difficult more difficult. Dealing with aggregate queries, complex joins and huge datasets requires a good knowledge of the ORM and how it generates SQL. If you're used to having direct access to the DB, getting the ORM to do the proper magic can be frustrating.

This code you posted, though, would be bad regardless of using an ORM or not. I've seen the same thing using regular SQL to grab all the rows and then filtering them out in a code loop. This is just bad, un-scalable code in general. The right way to do this with an ORM is to just use it's expression or criteria functionality to do the filtering instead of grabbing all the records.

Re:ORM == good

[Spiv]

Canonical has used both SQLAlchemy and SQLObject on large projects. Storm was written because neither of those did what we needed. There's a little bit of discussion here: http://programming.reddit.com/info/24oo3/comments.

Re:ORM == good

[Shados]

I don't know about these, but the more advanced ORMs out there will take multiple operations in a batch, parse them, optimize them, then make a single SQL query (or at least, as few as possible), and send that in one shot. Or at least, have the option to do so, so its rarely a problem. The one thing that tends to cause issues, are very specific things like query planner hints, but some ORMs even support those. But for the 3 queries in a 5000 tables with replication system you have that require such things, again, most good ORMs will also support mapping stored procedures to objects, so you can have the best of both worlds, no issues :)

Re:ORM == good

[Cyberax]

I generally use autogenerated schemas (with manually-added indexes and advanced integrity checks) when I'm using Hibernate. I don't remember the time when Hibernate offered bad schema. Usually mapping from object structure to SQL DDL is very simple (you map objects as tables, one-to-many relations as backpointers, etc.).

Re:ORM == good

[Cyberax]

You can have equally stupid example with SQL:

RecordSet r=db.query("SELECT * FROM product");
while(r.hasMoreRows())
{
      Row row=r.nextRow();
      if (row.get(0)<2.99)
          myResults.push(row[0],row[1],row[2]);
}

I've seen enough of such WTFs, usually in PHP programs.

Re:ORM == good

[Cyberax]

In all ORMs that I know, you can use plain SQL for complex queries. And it's usually much easier to use plain SQL than ORM's built-in query language.

But in my experience, 90% of queries are simple "SELECT blah.blah,... FROM blah WHERE blah.blah=?" and ORMs allow to forget about this simple CRUD.

Re:ORM == good

[rgravina]

Do you know of Axiom? Storm seems to have an API very much like it. Storm looks great, well done!
http://divmod.org/trac/wiki/DivmodAxiom

Storm?

[Klowner]

Anyone know how Storm compares to SQLObject? They appear to achieve the same goal.

Oh, wow!

[TobyRush]

Canonical Begins to Open-Source Launchpad

Hey, that's great! I always get excited when advectives begin to noun things.

Re:Oh, wow!

[evanbd]

Words are more fun when you verb them.

Re:Oh, wow!

[nuzak]

"Verbing weirds language." -- Calvin

Re:Oh, wow!

[thomasj]

"Verbing wirds how we language wordings" -- The Thomased being

Re:Oh, wow!

[qualidafial]

"Verbing weirds language." -- Calvin
In Soviet Russia, verbing weirds you!

One Launchpad to rule them all..

[rustalot42684]

The reason they don't opensource it because there should only be one Launchpad, or you get all kinds of complex problems that are totally unnecessary. The point of Launchpad is to have a central system to manage things. Why do you need your own Launchpad, when it's better for it to be centralized?

Re:One Launchpad to rule them all..

[computersareevil]

Enlighten me (because I really don't know): Launchpad sounds like another version of Sourceforge. If it is, then why would they start another SF and say it has the be the "central" system to manage things?

Re:One Launchpad to rule them all..

[Anonymous Coward]

What is 'Launchpad' and how does it relate to the end user? Sorry, never heard of it before.

Re:One Launchpad to rule them all..

[nuzak]

Launchpad is a bug tracker, with some project management features. The bug tracker in Sourceforge is easily its worst feature, and it has zero project management facilities (at least not in the free version).

Re:One Launchpad to rule them all..

[Knuckles]

Is google down or what? https://launchpad.net/+about

Re:One Launchpad to rule them all..

[computersareevil]

Thanks for the trollish response.
 
I've read all that before and it sounds just like Sourceforge. So again, it sounds like they are worried about redundant versions of their redundant project management system.
 
Not that I have a problem with them creating another management system with better/different features, I'm just perplexed by their explanation for keeping it closed.

Re:One Launchpad to rule them all..

[Phroggy]

What is 'Launchpad' and how does it relate to the end user? Sorry, never heard of it before. Perhaps it doesn't, and that's why they haven't bothered to open-source it before?

Re:One Launchpad to rule them all..

[Knuckles]

Seems as if you did _not read the launchpad about page. Launchpad tracks bugs (and specs and similar) with the ability to sync between all projects that joined launchpad. So if I find a bug in some package in Ubuntu, and the upstream of this package also uses launchpad, the upstream is notified directly. For example. It also provides translations services ... ah, read for yourself.

Re:One Launchpad to rule them all..

[shura57]

This is the dream. The reality is that upstream never uses launchpad. So all this beatiful integration is *just another bug racker*.

Check this out: https://bugs.launchpad.net/ubuntu/+source/gramps/+ bug/120569
It will stay open forever, until upstream of gtkspell decides to fix it.

Re:One Launchpad to rule them all..

[Knuckles]

That's a different discussion.

Re:One Launchpad to rule them all..

[khanyisa]

And therefore in their minds it gives Ubuntu as a community an advantage over the other distros. Of course, the basic response is that to make a distro genuine free software the software used to produced it should be free as well...

Cudoes to Canonical, better late than never.

[deweycheetham]

We always want stuff faster, but we get epically testy about Open Source. Welcome to /.

What about the content?

[Anonymous Coward]

I never thought that Canonical was obliged to open source the engine of their web portal. IMHO what matters more is the content of this portal. As a user of Ubuntu I contribute bug reports for free, extend the wiki and so on. I think the contributors simply retain their copyright and don't license it specifically (couldn't find anything on the site in this respect).

This means effectively that only Canonical is allowed to distribute the collected content of the whole portal (but only because the contributors gave implicit permission to do so, without explicit license). So if Canonical turns evil, nobody can start a forked project website easily. They would have to get permission from each contributor individually.

We shall remember rosetta bug #44

[JucaBlues]

I am waiting to see rosetta component (software translation tool) opensourced. There is a 'most wanted' feature that I guess will be implemented quickly as soon as rosetta code gets released:

Rosetta Bug #44,
"Translations should be searchable"
first reported on 2005-01-10

Today it completes exactly 2,5 years of waiting and nothing!

https://bugs.launchpad.net/rosetta/+bug/44

begins?

[alfino]

Uh, releasing a component used by Launchpad does not mean releasing the whole thing, now does it? I doubt Launchpad will become OSS, it would go completely against what I understand Canonical's mission to be.

Re:begins?

[Stemp]

Mark Shuttleworth says: We are steadily releasing pieces of Launchpad, but the whole thing still has proprietary elements. We offer services to people who make Ubuntu derivatives using it, and that revenue stream is important. You can get all your data out easily (and this will soon get even easier to automate) so you're not locked in. In time, LP will all be published under a free software license, but for now it is not.

http://www.markshuttleworth.com/archives/103

Lead by example

[TheDarkener]

It's difficult to see, in this day in age, ones who lead by example. This should be considered a rare (yet inspirational) occurrence of true leaders - ones who practice what they preach.

Thank you, Canonical, for doing what you do.

Sincerely,
A proud GNU/Linux user.

Re:Lead by example

[Afecks]

I'm touched, no wait, that's just my hand. Carry on.

Re:Lead by example

[Anonymous Coward]

Mmmmmhh... Canonical is like ... making sweet sweet love to a gnu in a Soyuz capsule in outer space. Downing BSD or fondling slippery linuxes doesn't even come close to this.

Storm vs. SQLAlchemy

[Anonymous Coward]

How does Storm compare to SQLAlchemy in terms of features?

Re:Storm vs. SQLAlchemy

[MtHuurne]

Reading the Storm tutorial, I wonder why SQL statements are needed to create tables, while the Python data classes already contain all the needed information. In SQLAlchemy, you define the schema in Python, at which point the SQL create statements are generated for you and the data classes automatically have attributes corresponding to the columns of the table.

While you could argue about what the best place is to define the structure of the data, I think it is always desirable to have the structure defined in just one place. The advantages of SQLAlchemy's approach are that ORM is optional (unlike defining the structure in the data classes) and that it can abstract from some of the differences between databases (unlike defining the structure in the database).

SQLAlchemy is a relatively new toolkit though (the first release I could find on sf.net is from 2006), so maybe it didn't exist or wasn't usable yet when Canonical started work on Launchpad.

I'm a fan of launchpad

[Skeith]

It gives an easy way for computer idiots like myself to contribute to Ubuntu in a small way

Re:I'm a fan of launchpad

[dvice_null]

That is true. It is really easy for the newbies, because you can really just go there and submit a bug report and "they" will find out who is responsible for the problem and provide them a detailed bug report so the problem can be fixed.

So?

[LWATCDR]

"if it's proprietary, you better be prepared to sell it or it's going to die a death of obscurity without anyone even hearing about it."
Eventually yes but making it open source will not make insure that will not happen.
There are probably billions of lines of proprietary software out there. A lot of it was written to do one specific job. Not everybody is going to take the time to release code as FOSS because it takes a lot of work to release code as FOSS and to manage the project.

Re:Don't forget...

[goarilla]

at last where were you i mean ... it's kinda hard but i
missed you trill