Slashdot Mirror

An anonymous reader writes: On April 6, a hacking outfit going by the name of Cyber Justice Team leaked data from multiple Syrian government and private websites. The leak includes the password file from the breached server, along with MySQL host permissions, admin passwords, and a link to the 10GB compressed file, uploaded to the file sharing site MEGA. While some of the data seems to be from older data breaches, some of it is also new. This is one of the biggest leaks of Syrian government data, a regime that has remained protected against such threats due to an aggressive cyber-policy. The government has been known to secretly back the Syrian Electronic Army hacker group, who the US government recently indicted (3 members at least).

New submitter Archie Cobbs writes "Last May I encountered a relatively obscure performance bug present in both MySQL 5.5.x and MariaDB 5.5.x (not surprising since they share the same codebase). This turned out to be a great opportunity to see whether Oracle or the MariaDB project is more responsive to bug reports. On May 31 Oracle got their bug report; within 24 hours they had confirmed the bug — pretty impressive. But since then, it's been radio silence for 3 months and counting. On July 25, MariaDB got their own copy. Within a week, a MariaDB developer had analyzed the bug and committed a patch. The resulting fix will be included in the next release, MariaDB 5.5.33."

An anonymous reader writes "As reported earlier on Slashdot it appeared the license covering the MySQL man pages was changed from the GPL to something less good. However, as speculated, this appears to be a bug." The build system was grabbing the wrong files, oops. The fix should be coming shortly: "Once the fixes have been made to the build system, we will rebuild the latest 5.1, 5.5, 5.6 releases plus the latest 5.7 milestone and make those available publicly asap."

First time accepted submitter sagecreek writes "Hadoop is an open-source, Java-based framework for large-scale data processing. Typically, it runs on big clusters of computers working together to crunch large chunks of data. You also can run Hadoop in "single-cluster mode" on a Linux machine, Windows PC or Mac, to learn the technology or do testing and debugging. The Hadoop framework, however, is not quickly mastered. Apache's Hadoop wiki cautions: "If you do not know about classpaths, how to compile and debug Java code, step back from Hadoop and learn a bit more about Java before proceeding." But if you are reasonably comfortable with Java, the well-written Hadoop Beginner's Guide by Garry Turkington can help you start mastering this rising star in the Big Data constellation." Read below for the rest of Si's review. Hadoop Beginner's Guide author Garry Turkington pages 374 publisher Packt Publishing rating 9/10 reviewer Si Dunn ISBN 9781849517300 summary Explains and shows how to use Hadoop software in Big Data settings. Dr. Turkington is vice president of data engineering and lead architect for London-based Improve Digital. He holds a doctorate in computer science from Queens University of Belfast in Northern Ireland. His Hadoop Beginner's Guide provides an effective overview of Hadoop and hands-on guidance in how to use it locally, in distributed hardware clusters, and out in the cloud.

Packt Publishing provided a review copy of the book. I have reviewed one other Packt book previously.

Much of the first chapter is devoted to "exploring the trends that led to Hadoop's creation and its enormous success." This includes brief discussions of Big Data, cloud computing, Amazon Web Services, and the differences between "scale-up" (using increasingly larger computers as data needs grow) and "scale-out" (spreading the data processing onto more and more machines as demand expands).

Dr. Turkington writes, "One of the most confusing aspects of Hadoop to a newcomer is its various components, projects, sub-projects, and their interrelationships."

His 374-page book emphasizes three major aspects of Hadoop: (1) its common projects; (2) the Hadoop Distributed File System (HDFS); and (3) MapReduce.

He explains, "Common projects comprise a set of libraries and tools that help the Hadoop product work in the real world."

The HDFS, meanwhile, "is a filesystem unlike most you may have encountered before." As a distributed filesystem, it can spread data storage across many nodes. "[I]t stores files in blocks typically at least 64 MB in size, much larger than the 4-32 KB seen in most filesystems." The book briefly describes several features, strengths, weaknesses, and other aspects of HDFS.

Finally, MapReduce is a well-known programming model for processing large data sets. Typically, MapReduce is used with clusters of computers that perform distributed computing. In the "Map" portion of the process, a single problem is split into many subtasks that are then assigned by a master computer to individual computers known as nodes (and there can be sub-nodes). During the "Reduce" part of the task, the master computer gathers up the processed data from the nodes, combines it and outputs a response to the problem that was posed to be solved. (MapReduce libraries are now available for many different computer languages, including Hadoop.)

"The developer focuses on expressing the transformation between source and result data sets, and the Hadoop framework manages all aspects of job execution, parallelization, and coordination," Dr. Turkington notes. He calls this "possibly the most important aspect of Hadoop. The platform takes responsibility for every aspect of executing the processing across the data. After the user defines the key criteria for the job, everything else becomes the responsibility of the system."

In this 11-chapter book, the first two chapters introduce Hadoop and explain how to install and run the software.

Three chapters are devoted to learning to work with MapReduce, from beginner to advanced levels. And the author stresses: "In the book, we will be learning how to write MapReduce programs to do some serious data crunching and how to run them on both locally managed and AWS-hosted Hadoop clusters." ["AWS" is "Amazon Web Services."]

Chapter 6, titled "When Things Break" zeroes in on Hadoop's "resilience to failure and an ability to survive failures when they do happen.much of the architecture and design of Hadoop is predicated on executing in an environment where failures are both frequent and expected." But node failures and numerous other problems still can arise, so the reader is given an overview of potential difficulties and how to handle them.

The next chapter, "Keeping Things Running," lays out what must be done to properly maintain a Hadoop cluster and keep it tuned and ready to crunch data.

Three of the remaining chapters show how Hadoop can be used elsewhere within an organization's systems and infrastructure, by personnel who are not trained to write MapReduce programs.

Chapter 8, for example, provides "A Relational View on Data with Hive." What Hive provides is "a data warehouse that uses MapReduce to analyze data stored on HDFS," Dr. Turkington notes. "In particular, it provides a query language called HiveQL that closely resembles the common Structured Query Language (SQL) standard."

Using Hive as an interface to Hadoop "not only accelerates the time required to produce results from data analysis, it significantly broadens who can use Hadoop and MapReduce. Instead of requiring software development skills, anyone with a familiarity with SQL can use Hive," the author states.

But, as Chapter 9 makes clear, Hive is not a relational database, and it doesn't fully implement SQL. So the text and code examples in Chapter 9 illustrate (1) how to set up MySQL to work with Hadoop and (2) how to use Sqoop to transfer bulk data between Hadoop and MySQL.

Chapter 10 shows how to set up and run Flume NG. This is a distributed service that collects, aggregates, and moves large amounts of log data from applications to Hadoop's HDFS.

The book's final chapter, "Where to Go Next," helps the newcomer see what else is available beyond the Hadoop core product. "There are," Dr. Turkington emphasizes, "a plethora of related projects and tools that build upon Hadoop and provide specific functionality or alternative approaches to existing ideas." He provides a quick tour of several of the projects and tools.

A key strength of this beginner's guide is in how its contents are structured and delivered. Four important headings appear repeatedly in most chapters. The "Time for action" heading singles out step-by-step instructions for performing a particular action. The "What just happened?" heading highlights explanations of "the working of tasks or instructions that you have just completed." The "Pop quiz" heading, meanwhile, is followed by short, multiple-choice questions that help you gauge your understanding. And the "Have a go hero" heading introduces paragraphs that "set practical challenges and give you ideas for experimenting with what you have learned."

Hadoop can be downloaded free from the Apache Software Foundation's Hadoop website.

Dr. Turkington's book does a good job of describing how to get Hadoop running on Ubuntu and other Linux distributions. But while he assures that "Hadoop does run well on other systems," he notes in his text: "Windows is supported only as a development platform, and Mac OS X is not formally supported at all." He refers users to Apache's Hadoop FAQ wiki for more information. Unfortunately, few details are offered there. So web searches become the best option for finding how-to instructions for Windows and Macs.

Running Hadoop on a Windows PC typically involves installing Cygwin and openSSH, so you can simulate using a Linux PC. But other choices can be found via sites such as Hadoop Wizard and Hadoop on Windows with Eclipse".

To install Hadoop on a Mac running OS X Mountain Lion, you will need to search for websites that offer how-to tips. Here is one example.

There are other ways get access to Hadoop on a single computer, using other operating systems or virtual machines. Again, web searches are necessary. The Cloudera Enterprise Free product is one virtual-machine option to consider.

Once you get past the hurdle of installing and running Hadoop, Garry Turkington's well-written, well-structured Hadoop Beginner's Guide can start you moving down the lengthy path to becoming an expert user.

You will have the opportunity, the book's tagline states, to "[l]earn how to crunch big data to extract meaning from the data avalanche."

Si Dunn is an author, screenwriter, and technology book reviewer.

You can purchase Hadoop Beginner's Guide from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

First time accepted submitter jsmyth writes "MySQL 5.6.10 has been released, marking the General Availability of version 5.6 for production." Here's more on the features of 5.6. Of possible interest to MySQL users, too, is this look at how MySQL spinoff MariaDB (from Monty, one of the three creators of MySQL) is making inroads into the MySQL market, including (as we've mentioned before) as default database system in some Linux distributions.

First time accepted submitter jsmyth writes "MySQL 5.6.10 has been released, marking the General Availability of version 5.6 for production." Here's more on the features of 5.6. Of possible interest to MySQL users, too, is this look at how MySQL spinoff MariaDB (from Monty, one of the three creators of MySQL) is making inroads into the MySQL market, including (as we've mentioned before) as default database system in some Linux distributions.

mpol writes "Sergei from MariaDB speculated on some changes within MySQL 5.5.27. It seems new testcases aren't included with MySQL any more, which leaves developers depending on it in the cold. 'Does this mean that test cases are no longer open source? Oracle did not reply to my question. But indeed, there is evidence that this guess is true. For example, this commit mail shows that new test cases, indeed, go in this "internal" directory, which is not included in the MySQL source distribution.' On a similar note, updates for the version history on Launchpad are not being updated anymore. What is Oracle's plan here? And is alienating the developer community just not seen as a problem at Oracle?"

New submitter simula67 writes "Oracle wins back some karma from the open source community by releasing MySQL cluster 7.2 with ambitious claims of 70x performance gains. The new release is GPL and claims to have processed over 1 billion queries per minute. Readers may remember the story about Oracle adding commercial extensions to MySQL."

New submitter simula67 writes "Oracle wins back some karma from the open source community by releasing MySQL cluster 7.2 with ambitious claims of 70x performance gains. The new release is GPL and claims to have processed over 1 billion queries per minute. Readers may remember the story about Oracle adding commercial extensions to MySQL."

Michael J. Ross writes "If you were to ask some database developers to cite their favorite strategies for expanding the functionality of the relational database management systems with which they work, you would probably hear a variety of answers. One individual might recommend the use of an alternate database engine optimized for the given application. Another might explain the many advantages of using stored procedures to replace SQL queries embedded in the source code of any programs that connect to databases. But one answer you likely would not receive involves changing the internals of the database engine itself. With the latest major release of MySQL, developers using that particular RDBMS are now able to extend the capabilities of the built-in database engines, by creating plug-ins. This is the topic of a book by Sergei Golubchik and Andrew Hutchings: MySQL 5.1 Plugin Development." Read on for the rest of Michael's review. MySQL 5.1 Plugin Development author Sergei Golubchik and Andrew Hutchings pages 288 pages publisher Packt Publishing rating 9/10 reviewer Michael J. Ross ISBN 978-1849510608 summary A practical and example-rich introduction to developing MySQL plug-ins. This title was released by Packt Publishing on 26 August 2010, under the ISBN 978-1849510608. On the publisher's page for the book, visitors will find a detailed description, the table of contents, a sample chapter (the fourth one in the book, "Information Schema Plugins," as a PDF file), and links to purchase the print and/or electronic versions of the book. There is also a link for downloading all of the sample code used in the book, except for the commands found in the first chapter. The sample code is sufficient to demonstrate the complexity of the subject area, and thus it is good that both authors possess a lot of experience in all of the primary technologies discussed in the book — particularly MySQL. In fact, Sergei Golubchik was one of the principal architects of the MySQL Plug-in application programming interface (API). Incidentally, in the "About the Authors" section, we read that he "has continued as a MySQL AB employee since 2000," which makes it sound as though he is still employed there (now Oracle); but then we are told that he resigned to join a startup firm, which is a bit confusing.

The book spans 288 pages, most of which are organized into ten chapters, followed by an appendix and an index. The first chapter explains the details of how to compile, link, and install MySQL User Defined Functions (UDFs), as well as proper MySQL plug-ins. For those people working on Windows platforms, numerous screenshots are provided, showing how to work with Microsoft Visual Studio (which is freely available). Readers learn how to use MySQL command-line utilities for building and packaging plug-ins, and the options needed to do so. The subsequent chapter focuses on UDFs — both normal and aggregate ones — which are technically not part of the MySQL Plugin API, although they may be in the future. However, they can be thought of as precursors to true plug-ins, because they are written in C/C++, loaded dynamically at runtime, and extend the capabilities of the MySQL server — in this case, by being callable from within SQL statements. The authors explicate how to install and utilize UDFs, and provide several examples.

The book's remaining chapters explore different types of plug-ins, starting with the most basic kind of all, Daemon plug-ins, which can run code utilizing a dedicated thread in the mysqld server process. Readers are shown how Daemon plug-ins are structured — including their declarations, types, status variables, and configuration system variables. To demonstrate these components, the authors dissect four separate sample plug-ins, line by line, with a great deal of helpful commentary. The next two chapters, 4 and 5, delve into schema-related plug-ins, starting with those that create tables, and ending with more advanced topics, such as how to access and output information about a server's internal data structures. These two chapters present almost half a dozen examples, as equally detailed as those of the earlier material.

During the past several years, all relational database systems are seeing increased use of full-text parsing, for various purposes. Chapters 6 and 7 show the reader how to create plug-ins that supplement the full-text search capabilities already baked into MySQL. The first sample plug-in presented by the authors could be used by PHP programmers for parsing their scripts, while another sample could be used by developers who need to match user input (which may include typos), using a Soundex algorithm. The final three chapters cover many aspects of storage engines, ranging from a basic read-only engine to a more complex one that supports indexes. The book concludes with an appendix that surveys the primary enhancements to the Plug-in API that database developers may see in versions of MySQL after 5.1.

On the publisher's site, there are no reported errata, but here are some that I found in just the first couple pages, to get the list started: "class [a] to" (page 1), "on [a] MySQL fork" (page 2), and "ask [the] questions" (page 2). Also, countless phrases and sentences in the book are oddly constructed, with multi-word adjectives missing hyphens, commas used where semicolons are called for, and sometimes both mistakes committed in the same sentence, such as the very first sentence of the preface. In fact, the lead author admits that he prefers reading fiction to a dictionary. Regardless, the information and instruction provided by the authors are generally clear to the reader, and ably illustrated with the sample code.

The book and thus the reader benefit greatly from the extensive MySQL experience and knowledge of the authors, reflected in the depth of coverage of the various topics. MySQL 5.1 Plugin Development brings together valuable information that is otherwise tedious to find — scattered throughout the API source code, official documentation, and online forum threads. For any programmer interested in unleashing the full potential of their MySQL servers through the creation and use of plug-ins, this book is an essential resource.

Michael J. Ross is a freelance website developer and writer.

You can purchase MySQL 5.1 Plugin Development from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

joabj writes "While MySQL is the subject of much high-profile wrangling between the EU and Oracle (and the MySQL creator himself), the MySQL developers have been quietly moving the widely-used database software forward. The new beta version of MySQL, the first publicly available, features such improvements as near-asynchronous replication and more options for partitioning. A new release model has been enacted as well, bequeathing this version the title of 'MySQL Server 5.5.0-m2.' Downloads here."

joabj writes "While MySQL is the subject of much high-profile wrangling between the EU and Oracle (and the MySQL creator himself), the MySQL developers have been quietly moving the widely-used database software forward. The new beta version of MySQL, the first publicly available, features such improvements as near-asynchronous replication and more options for partitioning. A new release model has been enacted as well, bequeathing this version the title of 'MySQL Server 5.5.0-m2.' Downloads here."

An anonymous reader writes "Brian Aker, one of the core developers of MySQL, has written up a lengthy blog on how the Drizzle fork is handling both its code contributions and its testing. He has listed the tools they use and how they work with their processes. He also makes an interesting statement about the signing of corporate code-contribution agreements and how there are some, including Rasmus (creator of PHP), who refuse to sign them."

Gary Pendergast writes "Monty Widenius, the 'father' of MySQL, has created the the Open Database Alliance, with the aim of becoming the industry hub for the MySQL open source database. He wants to unify all MySQL-related development and services, providing a potential solution to the fragmentation and uncertainty facing the communities, businesses and technical experts involved with MySQL, following the news of the Oracle acquisition of Sun." Related to this, an anonymous reader writes that "MySQL has announced a project to refactor MySQL to be a more Drizzle-like database." Update: 05/14 20:50 GMT by T : Original headline implied that this was a project of Sun, but (thanks to the open source nature of MySQL) it's actually Monty Widenius — no longer with Sun — leading this effort.

Gary Pendergast writes "Monty Widenius, the 'father' of MySQL, has created the the Open Database Alliance, with the aim of becoming the industry hub for the MySQL open source database. He wants to unify all MySQL-related development and services, providing a potential solution to the fragmentation and uncertainty facing the communities, businesses and technical experts involved with MySQL, following the news of the Oracle acquisition of Sun." Related to this, an anonymous reader writes that "MySQL has announced a project to refactor MySQL to be a more Drizzle-like database." Update: 05/14 20:50 GMT by T : Original headline implied that this was a project of Sun, but (thanks to the open source nature of MySQL) it's actually Monty Widenius — no longer with Sun — leading this effort.

Mad Merlin writes "It's no secret that MySQL 5.1 has been a long time in the making, with the first beta release being in Nov 2005, but MySQL 5.1.30 has finally been released as GA. MySQL users can expect new features such as table/index partitioning, row based replication, a new plugin architecture, an event scheduler and a host of performance improvements from 5.1." Monty also had a blog post outlining some of the challenges faced in 5.1, including crashing bugs and a beta quality to most new features.

Mad Merlin writes "It's no secret that MySQL 5.1 has been a long time in the making, with the first beta release being in Nov 2005, but MySQL 5.1.30 has finally been released as GA. MySQL users can expect new features such as table/index partitioning, row based replication, a new plugin architecture, an event scheduler and a host of performance improvements from 5.1." Monty also had a blog post outlining some of the challenges faced in 5.1, including crashing bugs and a beta quality to most new features.

An anonymous reader writes "From Kay Arno's blog we see that David Axmark, MySQL's Co-Founder, has resigned. This comes on top of the maybe, maybe not, resignation of Monty. We saw earlier this year that Brian Aker, the Director of Architecture, has forked the server to create a web-focused database from MySQL called Drizzle. The MySQL server has been 'RC' now for a year with hundreds of bugs still listed as being active in the 5.1 version. What is going on with MySQL?"

An anonymous reader writes "From Kay Arno's blog we see that David Axmark, MySQL's Co-Founder, has resigned. This comes on top of the maybe, maybe not, resignation of Monty. We saw earlier this year that Brian Aker, the Director of Architecture, has forked the server to create a web-focused database from MySQL called Drizzle. The MySQL server has been 'RC' now for a year with hundreds of bugs still listed as being active in the 5.1 version. What is going on with MySQL?"

Paul Boutin writes "A reliable source tells Valleywag that MySQL inventor Michael Widenius, better known as Monty, has resigned from Sun. Sun bought Monty's MySQL company in a billion-dollar deal last January. Brian Aker, who forked the Web 2.0-friendly Drizzle SQL database (and former Slashdot engineer!), remains at Sun." Kaj Arnö and Sheeri Cabral share their thoughts.

Anonymous Dolphin writes "MySQL has released plans for a final RC for the MySQL 5.1 server. Monty Widenius, the CTO and founder of MySQL, has put up a request for more feedback from the community. You can get the latest RC here. Please help with the testing of 5.1 and report your bugs here."

Anonymous Dolphin writes "MySQL has released plans for a final RC for the MySQL 5.1 server. Monty Widenius, the CTO and founder of MySQL, has put up a request for more feedback from the community. You can get the latest RC here. Please help with the testing of 5.1 and report your bugs here."

Anonymous Dolphin writes "MySQL has released plans for a final RC for the MySQL 5.1 server. Monty Widenius, the CTO and founder of MySQL, has put up a request for more feedback from the community. You can get the latest RC here. Please help with the testing of 5.1 and report your bugs here."

krow writes "I am very happy to be announcing that MySQL will be forgoing close sourcing portions of the MySQL Server. Kaj has the official statement in his blog. No portion of the server will be closed source including backup, encryption, or any storage engines we ship. To quote Kaj 'The encryption and compression backup features will be open source.' This is a change from what was previously posted here on Slashdot. I've posted some additional thoughts on my own blog concerning how we keep open source from becoming crippleware. Word has it that we will also have a panel at this year's OSCON discussing this topic. Contrary to the previous Slashdot discussion, this shows Sun's continued commitment to Open Source."

Krow alerted me that MySQL has been bought by Sun. Right now there is only a brief announcement but it discusses what the acquisition will mean for the core developers, community etc.

Brian Aker is Director of Architecture for MySQL AB. He has also worked on the code (and database) that runs Slashdot, and is well-known in both Apache and Perl circles. Outside of the arcane world of open source "back-end" programming, though, hardly anyone has heard of him. This is your chance to ask Brian (hopefully after looking at his blog and Wikipedia listing) about anything you like, from Perl to database architecture to open source philosophy to upcoming events in Seattle. We'll send Brian 10 of the highest-moderated questions approximately 24 hours after this post appears. His (verbatim) answers will appear late this week or early next week.

Today we have Part 2 in our exciting 2 part series about the infrastructure that powers Slashdot. Last week Uriah told us all about the hardware powering the system. This week, Jamie McCarthy picks up the story and tells us about the software... from pound to memcached to mysql and more. Hit that link and read on.

The software side of Slashdot takes over at the point where our load balancers -- described in Friday's hardware story -- hand off your incoming HTTP request to our pound servers.

Pound is a reverse proxy, which means it doesn't service the request itself, it just chooses which web server to hand it off to. We run 6 pounds, one for HTTPS traffic and the other 5 for regular HTTP. (Didn't know we support HTTPS, did ya? It's one of the perks for subscribers: you get to read Slashdot on the same webhead that admins use, which is always going to be responsive even during a crush of traffic -- because if it isn't, Rob's going to breathe down our necks!)

The pounds send traffic to one of the 16 apaches on our 16 webheads -- 15 regular, and the 1 HTTPS. Now, pound itself is so undemanding that we run it side-by-side with the apaches. The HTTPS pound handles SSL itself, handing off a plaintext HTTP request to its machine's apache, so the apache it redirects traffic to doesn't need mod_ssl compiled in. One less headache! Of our other 15 webheads, 5 also run a pound, not to distribute load but just for redundancy.

(Trivia: pound normally adds an X-Forwarded-For header, which Slash::Apache substitutes for the (internal) IP of pound itself. But sometimes if you use a proxy on the internet to do something bad, it will send us an X-Forwarded-For header too, which we use to try to track abuse. So we patched pound to insert a special X-Forward-Pound header, so it doesn't overwrite what may come from an abuser's proxy.)

The other 15 webheads are segregated by type. This segregation is mostly what pound is for. We have 2 webheads for static (.shtml) requests, 4 for the dynamic homepage, 6 for dynamic comment-delivery pages (comments, article, pollBooth.pl), and 3 for all other dynamic scripts (ajax, tags, bookmarks, firehose). We segregate partly so that if there's a performance problem or a DDoS on a specific page, the rest of the site will remain functional. We're constantly changing the code and this sets up "performance firewalls" for when us silly coders decide to write infinite loops.

But we also segregate for efficiency reasons like httpd-level caching, and MaxClients tuning. Our webhead bottleneck is CPU, not RAM. We run MaxClients that might seem absurdly low (5-15 for dynamic webheads, 25 for static) but our philosophy is if we're not turning over requests quickly anyway, something's wrong, and stacking up more requests won't help the CPU chew through them any faster.

All the webheads run the same software, which they mount from a /usr/local exported by a read-only NFS machine. Everyone I've ever met outside of this company gives an involuntary shudder when NFS is mentioned, and yet we haven't had any problems since shortly after it was set up (2002-ish). I attribute this to a combination of our brilliant sysadmins and the fact that we only export read-only. The backend task that writes to /usr/local (to update index.shtml every minute, for example) runs on the NFS server itself.

The apaches are versions 1.3, because there's never been a reason for us to switch to 2.0. We compile in mod_perl, and lingerd to free up RAM during delivery, but the only other nonstandard module we use is mod_auth_useragent to keep unfriendly bots away. Slash does make extensive use of each phase of the request loop (largely so we can send our 403's to out-of-control bots using a minimum of resources, and so your page is fully on its way while we write to the logging DB).

Slash, of course, is the open-source perl code that runs Slashdot. If you're thinking of playing around with it, grab a recent copy from CVS: it's been years since we got around to a tarball release. The various scripts that handle web requests access the database through Slash's SQL API, implemented on top of DBD::mysql (now maintained, incidentally, by one of the original Slash 1.0 coders) and of course DBI.pm. The most interesting parts of this layer might be:

(a) We don't use Apache::DBI. We use connect_cached, but actually our main connection cache is the global objects that hold the connections. Some small chunks of data are so frequently used that we keep them around in those objects.

(b) We almost never use statement handles. We have eleven ways of doing a SELECT and the differences are mostly how we massage the results into the perl data structure they return.

(c) We don't use placeholders. Originally because DBD::mysql didn't take advantage of them, and now because we think any speed increase in a reasonably-optimized web app should be a trivial payoff for non-self-documenting argument order. Discuss!

(d) We built in replication support. A database object requested as a reader picks a random slave to read from for the duration of your HTTP request (or the backend task). We can weight them manually, and we have a task that reweights them automatically. (If we do something stupid and wedge a slave's replication thread, every Slash process, across 17 machines, starts throttling back its connections to that machine within 10 seconds. This was originally written to handle slave DBs getting bogged down by load, but with our new faster DBs, that just never happens, so if a slave falls behind, one of us probably typed something dumb at the mysql> prompt.)

(e) We bolted on memcached support. Why bolted-on? Because back when we first tried memcached, we got a huge performance boost by caching our three big data types (users, stories, comment text) and we're pretty sure additional caching would provide minimal benefit at this point. Memcached's main use is to get and set data objects, and Slash doesn't really bottleneck that way.

Slash 1.0 was written way back in early 2000 with decent support for get and set methods to abstract objects out of a database (getDescriptions, subclassed _wheresql) -- but over the years we've only used them a few times. Most data types that are candidates to be objectified either are processed in large numbers (like tags and comments), in ways that would be difficult to do efficiently by subclassing, or have complicated table structures and pre- and post-processing (like users) that would make any generic objectification code pretty complicated. So most data access is done through get and set methods written custom for each data type, or, just as often, through methods that perform one specific update or select.

Overall, we're pretty happy with the database side of things. Most tables are fairly well normalized, not fully but mostly, and we've found this improves performance in most cases. Even on a fairly large site like Slashdot, with modern hardware and a little thinking ahead, we're able to push code and schema changes live quickly. Thanks to running multiple-master replication, we can keep the site fully live even during blocking queries like ALTER TABLE. After changes go live, we can find performance problem spots and optimize (which usually means caching, caching, caching, and occasionally multi-pass log processing for things like detecting abuse and picking users out of a hat who get mod points).

In fact, I'll go further than "pretty happy." Writing a database-backed web site has changed dramatically over the past seven years. The database used to be the bottleneck: centralized, hard to expand, slow. Now even a cheap DB server can run a pretty big site if you code defensively, and thanks to Moore's Law, memcached, and improvements in open-source database software, that part of the scaling issue isn't really a problem until you're practically the size of eBay. It's an exciting time to be coding web applications.

SlashRating^© 11.9 slashdottit! ^tm krow writes "One of the unique qualities of the MySQL server is its ability to have multiple storage engine operate concurrently. Companies like Oracle and Solid have contributed their own storage engines to the open source project. With 5.1 MySQL has added the ability to now do this in a loadable fashion, allowing dynamic engines in the same manner as Apache with its modules. Now PostgreSQL can add its self to the list of databases who have contributed a storage engine to MySQL. I'm releasing today a plugin so that you can now plugin the Postgres database engine into MySQL and have it work natively along side other engines."

volts writes "MySQL quietly deprecated support for most Linux distributions on October 16, when its 'MySQL Network' support plan was replaced by 'MySQL Enterprise.' MySQL now supports only two Linux distributions — Red Hat Enterprise Linux and SUSE Linux Enterprise Server. We learned of this when MySQL declined to sell us support for some new Debian-based servers. Our sales rep 'found out from engineering that the current Enterprise offering is no longer supported on Debian OS.' We were told that 'Generic Linux' in MySQL's list of supported platforms means 'generic versions of the implementations listed above'; not support for Linux in general." Update: 12/13 20:52 GMT by J : MySQL AB's Director of Architecture (and former Slash programmer) Brian Aker corrects an apparent miscommunication in a blog post: "we are just starting to roll out [Enterprise] binaries... We don't build binaries for Debian in part because the Debian community does a good job themselves... If you call MySQL and you have support we support you if you are running Debian (the same with Suse, RHEL, Fedora, Ubuntu and others)... someone in Sales was left with the wrong information"

You asked. Mårten answered. He even added (and answered) a question he wished had been asked, but wasn't. If you have a comment or follow-up question, please post it. Mårten will spend as much time responding to your comments as his schedule permits. 1) Re:Biggest Problem?
by wild_berry

In light of comments here about PostgreSQL being superior (and F/LOSS) technology but MySQL being used because of "MCSE weenie mentality", are you, Marten Mickos, concerned about the complacency of 'good enough' technological solutions or that Free/Libre Open Source Software may never remove the entrenched market leaders?

Mårten: Not the least. There will always be fanatical supporters of various products and projects, but I would look at what real users are saying. As an example, Nortel, Alcatel and Nokia are building new mobile phone network elements that run on MySQL Cluster. That's leading edge open source technology in some of the most advanced high-availability use. Or take YouTube or some of the other new web successes - they are scaling incredibly fast and again it is open source that powers them (MySQL, to be specific).

It is a common misperception that advanced technology cannot be easy to use. But it can! Driving a Tesla is fairly straightforward, but building one is not. Flying the new Eclipse jet is comparatively easy, but designing and constructing the aircraft is extremely difficult. And although it was pretty difficult to fly Spaceship One out of the atmosphere twice in two weeks, it was nothing compared to the difficulty of designing such a modern spaceship.

2) Perception of low quality for 'free'
by OakDragon

How do you fight the perception that MySQL is not suitable for 'the real world' because it is free?

Mårten: That's a challenge we share with all open source products, and I believe that such misperceptions will be removed over time by the overwhelming momentum of open source at large. So it is not so much a product-specific issue. But we need everyone's help to set records straight.

3) R&D Directions?
by eldavojohn

In a market where people are just looking for stability, simplicity & scalability, where do you turn for innovation in your products? Is there a lot of research and development towards new features and completely new products in MySQL's community or do you aim primarily to do one thing well? How do you influence the direction of this research in such a large open source project? Do you attempt to add direction at all?

Mårten: I am very proud of the innovative power inside our company, but even more important is the notion that "innovation happens elsewhere" (that's a great book that I recommend, as is "Democratizing Innovation"). We just announced a new monitoring and advisory service which we have innovated inside the company and with the help of customers.

But more massive is the innovation that happens in the MySQL ecosystem - in the user space. Look at the XML wrappers, RSS converters, full-text search modules or any of the toolkits that have been developed for MySQL. Or take PBXT - a new transactional storage engine, or the custom-built storage engines that Google, Friendster and others have developed. It is in the interaction with these advanced users that we learn what's useful and what isn't.

It takes a degree of humility and openness (and eradicating any Not Invented Here feelings) to be able to make good use of innovations from the ecosystem. But in return it is a much more productive way. I see it as a Darwinian system where the best ideas survive. Nobody can know for sure exactly where the development is heading, but if millions of MySQL users try out various new ideas, all the right things get developed.

We still need to improve our ability to receive innovations from the community, but I am proud of how far we have come already. Look at the MySQL Forge if you are interested: fforge.mysql.com

4) Re:R&D Directions?
by bzipitidoo

A few questions, only one about the biz. For those of us who like designing and researching and wish to spend our time on that, and don't like the thought of spending time on sales and promotion, or begging for money from VC vultures, or figuring out tax forms, stock options, and such, or sifting through thousands of resumes trying to find a few good people to hire, or knowing when a deal is a bad one to be avoided at all costs, or all the other aspects of business, what are wannabe independent software developers to do? If I try to start a business, it'd be so I can make some money doing what I like, not sink time into the business of business. How'd MySQL handle that when it started?

The parent's R&D question reminded me of another question. A goal of future versions of ReiserFS is to make database programs unnecessary and obsolete. Reiser asserts the only reason you even need a database program is because current file systems cannot do things like efficiently store, access, and query thousands of small records. File systems should be able to do everything a database can do. Apart from the bit about being in jail, is Reiser crazy, or is he on to something?

Anyway, do you recommend or favor some file systems over others for best performance from MySQL?

Mårten: I think you had a total of three questions.

First, when MySQL got going, it initially was run day-to-day by the three founders Monty, David and Allan. In the year 2000 they realised it was growing beyond what they could handle, so they reached out to get a CEO, a board of directors, and VC capital. This is a great model for those who want the company to grow and who are ready to share decision-making with others.

Your other question is about what a software developer of today could do. I believe it is easier to start a business today than 10 years ago, and I believe there are some great avenues to follow. For instance, you can develop open source software that wins the hearts and minds of users, and then you can license or sell the software to a bigger vendor if you want to avoid the hassles you mention.

Mark Matthews is a great example. He had developed the best JDBC driver for MySQL, and then we acquired his software and hired him full-time. JBoss has done similar "acquisitions". Another alternative is to build a website that provides the intended functionality as a service. The benefit there is that you don't have to build a distribution channel, because the web is the channel. And then there can be some great outcomes if you don't want to run your own company. Just look at Flickr or YouTube, for instance, who got acquired by web giants for good sums of money.

Your third question is about filesystems versus databases. If I were a filesystem developer, I might claim that databases will become obsolete. But as I am a database developer, I will claim that filesystems will become obsolete! Humour aside, I don't have a specific recommendation, but I do believe that we will always have heterogeneity in the software world. There will always be several alternative technologies for every single problem.

5) Conflict of Interest
by 99BottlesOfBeerInMyF

One of the most common complaints I've heard about the business model of profiting on support for a product, is that it provides motivation to keep the product from becoming very user friendly. After all, if the product is too easy to use, who will pay for support? In my own experience, I've seen a lot of companies that consider support to be insurance, and don't use it for help with installation, configuration, or to overcome usability issues so much as a way to cover their asses in case something goes very wrong. Do a lot of your customers use support to overcome usability problems and if so, does this de-motivate you to solve other usability issues?

Mårten: This is a great question. First, I think closed source vendors are proving the hypothesis incorrect, because they are the ones who have un-friendly products although they have a licence revenue stream. And open source products, which typically lack a licence fee, are the ones with the best user friendliness. Why is that? I think the reason is that popularity is worth more than the marginally improved fees you could get for a user-unfriendly product. Sure, we lose some purchase orders because customers simply don't need our support. But at the same time we gain millions of new users, and they in turn help us develop the product and drive our marketing.

I think one of the reasons MySQL is so popular is that we continue to take user-friendliness seriously. We may joke about leaving some bugs in the product in order to have more support fees, but in reality we take the bugs very seriously. And when we at times have more bugs than we would like to have, the whole organisation is in pain until we get the upper hand and kill more bugs than we produce. Version 5.0 was such an example - when it first came out as GA it had more bugs than earlier versions. It didn't feel good to see the list of incoming bugs, so we rearranged tasks and expanded our QA test coverage in order to fix the situation. I hope you who have used the latest versions of MySQL 5.0.26 have seen the significant improvements.

6) Defects per KLOC
by eldavojohn

Your website touts you as having the lowest defects per KLOC by up to 12 times the industry standard, what do you attribute as the leading factor to your success in this respect? Since cold cash is the traditional method, how do you incentivise code quality in an open source product?

Mårten: I think open source has a wonderful benefit over closed source that often is forgotten: When you know that your work may be scrutinised in public by others, you simply do a better job. It is some magical combination of pride and (fear of) shame.

So I am claiming that the same developers will do a better job of producing open source than closed source software. The analogy is your backyard: why is the backyard never as tidy as the frontyard of a house? You spend more time in the backyard, don't you? And it is you taking care of both the front and the back. The only difference is that your neighbours see your frontyard but not your backyard. It's as simple as that.

Additionally, the reality of the scrutiny, i.e. the peer reviews, ensure that a lot of bugs and defects are detected and removed early. So I am not the least surprised that open source is 12 times cleaner than closed source. The really scary thing is all the stuff that is buried deep inside closed source software, out of the sight of everyone and without any proper stewardship.

7) MySQL business model for niche products?
by MarkWatson

Hello Marten,

First, congratulations on MySQL's market capitalization! My question is:

I have been working part time for about 6 years on software for text/data mining and general semantic information extraction. Almost all of my development is in Common Lisp, but I have ported little bits to Java and released that under the GPL in the past. I view this as a small, niche market, not like MySQL. What do you think that chances are for making money on GPLing a niche product?

MySQL is very widely used so if you capture commercial use icensing costs for a small percent of users, you do very well. For my software, with luck perhaps a few hundred companies a year might start adopting my product. Does it seem like wishful thinking for me to use a GPL based business model like MySQL's?

I want my customers to have my source code for a lot of reasons, but I would also like to capture revenue. I might just end up going to market as a proprietary product that incidently includes source code, with licensing that prohibits redistribution to non-customers.

Thanks for your help,

Mark

Mårten: Good question. I used to state in presentations that open source probably is best suited for products with huge user bases. But I was always proved wrong. Somebody would come up to me and tell about a successful open source niche product. So I have learned that you can be successful with open source in any environment.

I think the key is to provide real value to your users or customers - initially free of charge, but for continuous support against a fee. In that way you can build credibility in your market and gain important insights in what customers need, and you can also build a business. But don't think it is easy to find a business model that fits with free or open source software. It is difficult. But it is possible. You have to take some risks before hitting on the right answer.

8) Commercial vs free - where to draw the line
by Internet Ninja
I work for a good sized business and looked at using the Cluster Jumpstart but when I told my boss the cost and that we'd have to pay for flights he laughed at me, even though we're starting to really use mySQL pretty seriously now for some stuff.

With costs for things like this and gold/platinum support also relatively high on a per server basis it seems there's a wide gap between community based support which costs nothing and enterprise support which appears somewhat pricey.

How do you draw the line for paid vs free support particularly since a lot of SME's are using mySQL and may be unable to afford it? Was it a conscious decision to pitch it high to display value in the product?

Mårten: This is essentially a question of price elasticity. If we lower the unit prices, we may sell more, but will we sell that much more? If we increase the prices, we may have fewer customers but they may be more profitable.

We have an ambition to be affordable and available for all, and therefore our prices are typically a tenth of what you would pay for closed source databases. But MySQL Cluster is a state-of-the art solution that in its own market segment beats all other databases hands-down. The price may sound high to someone from a smaller company, but it is carefully weighed against market conditions. Many customers find that it is not only the best solution technically, it is also the least expensive one. If the price is prohibitively high for someone, then they can always use the community edition and do the work themselves.

9) Achievements & Fallout
by eldavojohn

In your five years as MySQL CEO, what has been your proudest moment? Do you find it difficult to lead a company based on a product that belongs to a community? Do you ever experience any fallout/backfire from running your company on such a business model?

Mårten: Thanks for the question! I felt very proud when we visited Google for the first time and people would come out of their rooms just to see the "MySQL folks". I had scheduled a CEO-to-CEO meeting with Eric Schmidt, but in addition to him we had a dozen developers who all wanted to know about MySQL internals. Fortunately, I had David Axmark (co-founder) with me and he did the technical presentation! Another proud moment was our users conference this year with a thousand or so in the audience, all gathered just to listen to the State of MySQL. Those situations make us realise that we are changing the world - we are making a difference.

The great thing with our business is that we get to pursue goals that we are passionate about. We get to work with the smartest people in the world on the newest and coolest database technologies. We get to stand up to defend the freedom of software and to fight software patents. We contribute to the wealth and happiness of what is called the bottom of the pyramid - people who may not have the financial means to buy software but who have dreams and goals they want to pursue. Of course it all needs to make sense from a financial perspective for everyone involved, so it all needs to boil down to a good business model. But, as I noted, that is exactly the great thing with our model.

But we also need to know that it is very difficult to walk the fine line between freedom of software and freedom to pursue profits. The two are not at odds with each other, but there are overlapping areas where you need to have all details right. We have made our mistakes over the years, and I hope we have fixed them. If we haven't, let me know and we'll get onto it. I am not expecting everyone to agree with our business model, and every now and then we get flamed by someone, but I do hope that people generally respect our business decisions. And so far that has clearly been the case.

A friend of mine just wrote to me: "If you can find a new way for folks to make money from FOSS without compromising fundamentals, good for you. Others can follow your lead. Just don't compromise the fundamentals." That captures the essence of the question very well.

10) Appliance possibility?
by Roblimo (added editorially
because of this discussion)

Now and then we hear rumors that Oracle is going to either come out with its own Linux distribution, sell a single "stack" with Oracle products running on Linux or else sell a complete hardware appliance that runs Oracle on Linux.

Back in 2003 there was a significant PR splash touting database server appliances made in partnership by Pogo Linux and MySQL, but we haven't heard much about that idea since. We know you now have many channel partners, including hardware vendors, but this is not quite the same.

Do you have any plans to come out with either a Linux/MySQL "single stack" software product or a preloaded Linux/MySQL+hardware appliance? And if not, would you change your mind if Oracle started selling either of these products?

Mårten: First, if Oracle launches their own Linux distro, then we take that as a great victory for open source. (And we ask - is the database next?) The most successful distros include JBoss and MySQL and many other products, so we would naturally make sure that Oracle gets all the help they need in putting our product on their operating system.

As for appliances and stacks, I am not sure that we would choose to compile our own stack. There are so many tempting business models and products, but to really grow fast there are just three main rules: Focus, Focus, and Focus. In our ecosystem we have many partners who could provide a stack, and we are very supportive of them. HP has the Linux reference architecture, Pogo their appliance, and Sourcelabs and Spikesource came out with software stacks. The power of open source and the power of LAMP is in the collaboration between the companies, not so much in what one single company can accomplish.

11) What about community?
(inserted by martenmickos himself)

MySQL is enormously popular and has a vibrant ecosystem, but it seems that the core product is primarily developed in-house. Are you at all interested in contributions, and what are you doing to stimulate community engagement?

Mårten (answering his own question): Initially, although open source, MySQL was developed by just one person - our founder Monty. Over the years we have expanded the group of developers both in-house and in the community. But we have more work to do. We need to make the product more modular so that it is easier to contribute distinct modules (such as storage engines or full-text search modules). We need to document the source code better (we are doing it with Doxygen). We need to make the contribution process as easy as possible.

We have very strict requirements on quality and ownership of intellectual property before we put something in the commercial product, but in the MySQL Community Server we are becoming much more open to contributions. We also need to set up websites and systems for broad collaboration in our community.

Planet MySQL has become a great aggregator of MySQL blogs, and the MySQL Meetups are popular local events. The MySQL Forge is a place for contributions, and soon we will launch the Winter of Code program with coding contests. For the most passionate MySQL users and developers there is the MySQL Camp in November (mysqlcamp.org). And of course all of us will get together in Silicon Valley in April for the annual users conference.

But no matter what we organise on behalf of MySQL AB (by the way, AB means "Aktiebolaget" i.e. "Incorporated"), the true value of the community will always be generated by the community itself. We are still on a learning curve in terms of all the possibilities that this offers. So please share your ideas with us. What should we do more of and what less? How could we bring together MySQL users from all over the world to accomplish bigger things? How could we together change the world for the better?

It is amazing what open source has accomplished in the last 15 years, and I am even more excited about what it can do in the next 15. But my excitement is anchored in our community. Alone I can accomplish nothing, but with the help of millions of users worldwide, we can accomplish just about anything.

I hope I answered all questions clearly and conclusively. Let me know if I missed anything.

Marten

------

You asked. Mårten answered. He even added (and answered) a question he wished had been asked, but wasn't. If you have a comment or follow-up question, please post it. Mårten will spend as much time responding to your comments as his schedule permits. 1) Re:Biggest Problem?
by wild_berry

In light of comments here about PostgreSQL being superior (and F/LOSS) technology but MySQL being used because of "MCSE weenie mentality", are you, Marten Mickos, concerned about the complacency of 'good enough' technological solutions or that Free/Libre Open Source Software may never remove the entrenched market leaders?

Mårten: Not the least. There will always be fanatical supporters of various products and projects, but I would look at what real users are saying. As an example, Nortel, Alcatel and Nokia are building new mobile phone network elements that run on MySQL Cluster. That's leading edge open source technology in some of the most advanced high-availability use. Or take YouTube or some of the other new web successes - they are scaling incredibly fast and again it is open source that powers them (MySQL, to be specific).

It is a common misperception that advanced technology cannot be easy to use. But it can! Driving a Tesla is fairly straightforward, but building one is not. Flying the new Eclipse jet is comparatively easy, but designing and constructing the aircraft is extremely difficult. And although it was pretty difficult to fly Spaceship One out of the atmosphere twice in two weeks, it was nothing compared to the difficulty of designing such a modern spaceship.

2) Perception of low quality for 'free'
by OakDragon

How do you fight the perception that MySQL is not suitable for 'the real world' because it is free?

Mårten: That's a challenge we share with all open source products, and I believe that such misperceptions will be removed over time by the overwhelming momentum of open source at large. So it is not so much a product-specific issue. But we need everyone's help to set records straight.

3) R&D Directions?
by eldavojohn

In a market where people are just looking for stability, simplicity & scalability, where do you turn for innovation in your products? Is there a lot of research and development towards new features and completely new products in MySQL's community or do you aim primarily to do one thing well? How do you influence the direction of this research in such a large open source project? Do you attempt to add direction at all?

Mårten: I am very proud of the innovative power inside our company, but even more important is the notion that "innovation happens elsewhere" (that's a great book that I recommend, as is "Democratizing Innovation"). We just announced a new monitoring and advisory service which we have innovated inside the company and with the help of customers.

But more massive is the innovation that happens in the MySQL ecosystem - in the user space. Look at the XML wrappers, RSS converters, full-text search modules or any of the toolkits that have been developed for MySQL. Or take PBXT - a new transactional storage engine, or the custom-built storage engines that Google, Friendster and others have developed. It is in the interaction with these advanced users that we learn what's useful and what isn't.

It takes a degree of humility and openness (and eradicating any Not Invented Here feelings) to be able to make good use of innovations from the ecosystem. But in return it is a much more productive way. I see it as a Darwinian system where the best ideas survive. Nobody can know for sure exactly where the development is heading, but if millions of MySQL users try out various new ideas, all the right things get developed.

We still need to improve our ability to receive innovations from the community, but I am proud of how far we have come already. Look at the MySQL Forge if you are interested: fforge.mysql.com

4) Re:R&D Directions?
by bzipitidoo

A few questions, only one about the biz. For those of us who like designing and researching and wish to spend our time on that, and don't like the thought of spending time on sales and promotion, or begging for money from VC vultures, or figuring out tax forms, stock options, and such, or sifting through thousands of resumes trying to find a few good people to hire, or knowing when a deal is a bad one to be avoided at all costs, or all the other aspects of business, what are wannabe independent software developers to do? If I try to start a business, it'd be so I can make some money doing what I like, not sink time into the business of business. How'd MySQL handle that when it started?

The parent's R&D question reminded me of another question. A goal of future versions of ReiserFS is to make database programs unnecessary and obsolete. Reiser asserts the only reason you even need a database program is because current file systems cannot do things like efficiently store, access, and query thousands of small records. File systems should be able to do everything a database can do. Apart from the bit about being in jail, is Reiser crazy, or is he on to something?

Anyway, do you recommend or favor some file systems over others for best performance from MySQL?

Mårten: I think you had a total of three questions.

First, when MySQL got going, it initially was run day-to-day by the three founders Monty, David and Allan. In the year 2000 they realised it was growing beyond what they could handle, so they reached out to get a CEO, a board of directors, and VC capital. This is a great model for those who want the company to grow and who are ready to share decision-making with others.

Your other question is about what a software developer of today could do. I believe it is easier to start a business today than 10 years ago, and I believe there are some great avenues to follow. For instance, you can develop open source software that wins the hearts and minds of users, and then you can license or sell the software to a bigger vendor if you want to avoid the hassles you mention.

Mark Matthews is a great example. He had developed the best JDBC driver for MySQL, and then we acquired his software and hired him full-time. JBoss has done similar "acquisitions". Another alternative is to build a website that provides the intended functionality as a service. The benefit there is that you don't have to build a distribution channel, because the web is the channel. And then there can be some great outcomes if you don't want to run your own company. Just look at Flickr or YouTube, for instance, who got acquired by web giants for good sums of money.

Your third question is about filesystems versus databases. If I were a filesystem developer, I might claim that databases will become obsolete. But as I am a database developer, I will claim that filesystems will become obsolete! Humour aside, I don't have a specific recommendation, but I do believe that we will always have heterogeneity in the software world. There will always be several alternative technologies for every single problem.

5) Conflict of Interest
by 99BottlesOfBeerInMyF

One of the most common complaints I've heard about the business model of profiting on support for a product, is that it provides motivation to keep the product from becoming very user friendly. After all, if the product is too easy to use, who will pay for support? In my own experience, I've seen a lot of companies that consider support to be insurance, and don't use it for help with installation, configuration, or to overcome usability issues so much as a way to cover their asses in case something goes very wrong. Do a lot of your customers use support to overcome usability problems and if so, does this de-motivate you to solve other usability issues?

Mårten: This is a great question. First, I think closed source vendors are proving the hypothesis incorrect, because they are the ones who have un-friendly products although they have a licence revenue stream. And open source products, which typically lack a licence fee, are the ones with the best user friendliness. Why is that? I think the reason is that popularity is worth more than the marginally improved fees you could get for a user-unfriendly product. Sure, we lose some purchase orders because customers simply don't need our support. But at the same time we gain millions of new users, and they in turn help us develop the product and drive our marketing.

I think one of the reasons MySQL is so popular is that we continue to take user-friendliness seriously. We may joke about leaving some bugs in the product in order to have more support fees, but in reality we take the bugs very seriously. And when we at times have more bugs than we would like to have, the whole organisation is in pain until we get the upper hand and kill more bugs than we produce. Version 5.0 was such an example - when it first came out as GA it had more bugs than earlier versions. It didn't feel good to see the list of incoming bugs, so we rearranged tasks and expanded our QA test coverage in order to fix the situation. I hope you who have used the latest versions of MySQL 5.0.26 have seen the significant improvements.

6) Defects per KLOC
by eldavojohn

Your website touts you as having the lowest defects per KLOC by up to 12 times the industry standard, what do you attribute as the leading factor to your success in this respect? Since cold cash is the traditional method, how do you incentivise code quality in an open source product?

Mårten: I think open source has a wonderful benefit over closed source that often is forgotten: When you know that your work may be scrutinised in public by others, you simply do a better job. It is some magical combination of pride and (fear of) shame.

So I am claiming that the same developers will do a better job of producing open source than closed source software. The analogy is your backyard: why is the backyard never as tidy as the frontyard of a house? You spend more time in the backyard, don't you? And it is you taking care of both the front and the back. The only difference is that your neighbours see your frontyard but not your backyard. It's as simple as that.

Additionally, the reality of the scrutiny, i.e. the peer reviews, ensure that a lot of bugs and defects are detected and removed early. So I am not the least surprised that open source is 12 times cleaner than closed source. The really scary thing is all the stuff that is buried deep inside closed source software, out of the sight of everyone and without any proper stewardship.

7) MySQL business model for niche products?
by MarkWatson

Hello Marten,

First, congratulations on MySQL's market capitalization! My question is:

I have been working part time for about 6 years on software for text/data mining and general semantic information extraction. Almost all of my development is in Common Lisp, but I have ported little bits to Java and released that under the GPL in the past. I view this as a small, niche market, not like MySQL. What do you think that chances are for making money on GPLing a niche product?

MySQL is very widely used so if you capture commercial use icensing costs for a small percent of users, you do very well. For my software, with luck perhaps a few hundred companies a year might start adopting my product. Does it seem like wishful thinking for me to use a GPL based business model like MySQL's?

I want my customers to have my source code for a lot of reasons, but I would also like to capture revenue. I might just end up going to market as a proprietary product that incidently includes source code, with licensing that prohibits redistribution to non-customers.

Thanks for your help,

Mark

Mårten: Good question. I used to state in presentations that open source probably is best suited for products with huge user bases. But I was always proved wrong. Somebody would come up to me and tell about a successful open source niche product. So I have learned that you can be successful with open source in any environment.

I think the key is to provide real value to your users or customers - initially free of charge, but for continuous support against a fee. In that way you can build credibility in your market and gain important insights in what customers need, and you can also build a business. But don't think it is easy to find a business model that fits with free or open source software. It is difficult. But it is possible. You have to take some risks before hitting on the right answer.

8) Commercial vs free - where to draw the line
by Internet Ninja
I work for a good sized business and looked at using the Cluster Jumpstart but when I told my boss the cost and that we'd have to pay for flights he laughed at me, even though we're starting to really use mySQL pretty seriously now for some stuff.

With costs for things like this and gold/platinum support also relatively high on a per server basis it seems there's a wide gap between community based support which costs nothing and enterprise support which appears somewhat pricey.

How do you draw the line for paid vs free support particularly since a lot of SME's are using mySQL and may be unable to afford it? Was it a conscious decision to pitch it high to display value in the product?

Mårten: This is essentially a question of price elasticity. If we lower the unit prices, we may sell more, but will we sell that much more? If we increase the prices, we may have fewer customers but they may be more profitable.

We have an ambition to be affordable and available for all, and therefore our prices are typically a tenth of what you would pay for closed source databases. But MySQL Cluster is a state-of-the art solution that in its own market segment beats all other databases hands-down. The price may sound high to someone from a smaller company, but it is carefully weighed against market conditions. Many customers find that it is not only the best solution technically, it is also the least expensive one. If the price is prohibitively high for someone, then they can always use the community edition and do the work themselves.

9) Achievements & Fallout
by eldavojohn

In your five years as MySQL CEO, what has been your proudest moment? Do you find it difficult to lead a company based on a product that belongs to a community? Do you ever experience any fallout/backfire from running your company on such a business model?

Mårten: Thanks for the question! I felt very proud when we visited Google for the first time and people would come out of their rooms just to see the "MySQL folks". I had scheduled a CEO-to-CEO meeting with Eric Schmidt, but in addition to him we had a dozen developers who all wanted to know about MySQL internals. Fortunately, I had David Axmark (co-founder) with me and he did the technical presentation! Another proud moment was our users conference this year with a thousand or so in the audience, all gathered just to listen to the State of MySQL. Those situations make us realise that we are changing the world - we are making a difference.

The great thing with our business is that we get to pursue goals that we are passionate about. We get to work with the smartest people in the world on the newest and coolest database technologies. We get to stand up to defend the freedom of software and to fight software patents. We contribute to the wealth and happiness of what is called the bottom of the pyramid - people who may not have the financial means to buy software but who have dreams and goals they want to pursue. Of course it all needs to make sense from a financial perspective for everyone involved, so it all needs to boil down to a good business model. But, as I noted, that is exactly the great thing with our model.

But we also need to know that it is very difficult to walk the fine line between freedom of software and freedom to pursue profits. The two are not at odds with each other, but there are overlapping areas where you need to have all details right. We have made our mistakes over the years, and I hope we have fixed them. If we haven't, let me know and we'll get onto it. I am not expecting everyone to agree with our business model, and every now and then we get flamed by someone, but I do hope that people generally respect our business decisions. And so far that has clearly been the case.

A friend of mine just wrote to me: "If you can find a new way for folks to make money from FOSS without compromising fundamentals, good for you. Others can follow your lead. Just don't compromise the fundamentals." That captures the essence of the question very well.

10) Appliance possibility?
by Roblimo (added editorially
because of this discussion)

Now and then we hear rumors that Oracle is going to either come out with its own Linux distribution, sell a single "stack" with Oracle products running on Linux or else sell a complete hardware appliance that runs Oracle on Linux.

Back in 2003 there was a significant PR splash touting database server appliances made in partnership by Pogo Linux and MySQL, but we haven't heard much about that idea since. We know you now have many channel partners, including hardware vendors, but this is not quite the same.

Do you have any plans to come out with either a Linux/MySQL "single stack" software product or a preloaded Linux/MySQL+hardware appliance? And if not, would you change your mind if Oracle started selling either of these products?

Mårten: First, if Oracle launches their own Linux distro, then we take that as a great victory for open source. (And we ask - is the database next?) The most successful distros include JBoss and MySQL and many other products, so we would naturally make sure that Oracle gets all the help they need in putting our product on their operating system.

As for appliances and stacks, I am not sure that we would choose to compile our own stack. There are so many tempting business models and products, but to really grow fast there are just three main rules: Focus, Focus, and Focus. In our ecosystem we have many partners who could provide a stack, and we are very supportive of them. HP has the Linux reference architecture, Pogo their appliance, and Sourcelabs and Spikesource came out with software stacks. The power of open source and the power of LAMP is in the collaboration between the companies, not so much in what one single company can accomplish.

11) What about community?
(inserted by martenmickos himself)

MySQL is enormously popular and has a vibrant ecosystem, but it seems that the core product is primarily developed in-house. Are you at all interested in contributions, and what are you doing to stimulate community engagement?

Mårten (answering his own question): Initially, although open source, MySQL was developed by just one person - our founder Monty. Over the years we have expanded the group of developers both in-house and in the community. But we have more work to do. We need to make the product more modular so that it is easier to contribute distinct modules (such as storage engines or full-text search modules). We need to document the source code better (we are doing it with Doxygen). We need to make the contribution process as easy as possible.

We have very strict requirements on quality and ownership of intellectual property before we put something in the commercial product, but in the MySQL Community Server we are becoming much more open to contributions. We also need to set up websites and systems for broad collaboration in our community.

Planet MySQL has become a great aggregator of MySQL blogs, and the MySQL Meetups are popular local events. The MySQL Forge is a place for contributions, and soon we will launch the Winter of Code program with coding contests. For the most passionate MySQL users and developers there is the MySQL Camp in November (mysqlcamp.org). And of course all of us will get together in Silicon Valley in April for the annual users conference.

But no matter what we organise on behalf of MySQL AB (by the way, AB means "Aktiebolaget" i.e. "Incorporated"), the true value of the community will always be generated by the community itself. We are still on a learning curve in terms of all the possibilities that this offers. So please share your ideas with us. What should we do more of and what less? How could we bring together MySQL users from all over the world to accomplish bigger things? How could we together change the world for the better?

It is amazing what open source has accomplished in the last 15 years, and I am even more excited about what it can do in the next 15. But my excitement is anchored in our community. Alone I can accomplish nothing, but with the help of millions of users worldwide, we can accomplish just about anything.

I hope I answered all questions clearly and conclusively. Let me know if I missed anything.

Marten

------

You asked. Mårten answered. He even added (and answered) a question he wished had been asked, but wasn't. If you have a comment or follow-up question, please post it. Mårten will spend as much time responding to your comments as his schedule permits. 1) Re:Biggest Problem?
by wild_berry

In light of comments here about PostgreSQL being superior (and F/LOSS) technology but MySQL being used because of "MCSE weenie mentality", are you, Marten Mickos, concerned about the complacency of 'good enough' technological solutions or that Free/Libre Open Source Software may never remove the entrenched market leaders?

Mårten: Not the least. There will always be fanatical supporters of various products and projects, but I would look at what real users are saying. As an example, Nortel, Alcatel and Nokia are building new mobile phone network elements that run on MySQL Cluster. That's leading edge open source technology in some of the most advanced high-availability use. Or take YouTube or some of the other new web successes - they are scaling incredibly fast and again it is open source that powers them (MySQL, to be specific).

It is a common misperception that advanced technology cannot be easy to use. But it can! Driving a Tesla is fairly straightforward, but building one is not. Flying the new Eclipse jet is comparatively easy, but designing and constructing the aircraft is extremely difficult. And although it was pretty difficult to fly Spaceship One out of the atmosphere twice in two weeks, it was nothing compared to the difficulty of designing such a modern spaceship.

2) Perception of low quality for 'free'
by OakDragon

How do you fight the perception that MySQL is not suitable for 'the real world' because it is free?

Mårten: That's a challenge we share with all open source products, and I believe that such misperceptions will be removed over time by the overwhelming momentum of open source at large. So it is not so much a product-specific issue. But we need everyone's help to set records straight.

3) R&D Directions?
by eldavojohn

In a market where people are just looking for stability, simplicity & scalability, where do you turn for innovation in your products? Is there a lot of research and development towards new features and completely new products in MySQL's community or do you aim primarily to do one thing well? How do you influence the direction of this research in such a large open source project? Do you attempt to add direction at all?

Mårten: I am very proud of the innovative power inside our company, but even more important is the notion that "innovation happens elsewhere" (that's a great book that I recommend, as is "Democratizing Innovation"). We just announced a new monitoring and advisory service which we have innovated inside the company and with the help of customers.

But more massive is the innovation that happens in the MySQL ecosystem - in the user space. Look at the XML wrappers, RSS converters, full-text search modules or any of the toolkits that have been developed for MySQL. Or take PBXT - a new transactional storage engine, or the custom-built storage engines that Google, Friendster and others have developed. It is in the interaction with these advanced users that we learn what's useful and what isn't.

It takes a degree of humility and openness (and eradicating any Not Invented Here feelings) to be able to make good use of innovations from the ecosystem. But in return it is a much more productive way. I see it as a Darwinian system where the best ideas survive. Nobody can know for sure exactly where the development is heading, but if millions of MySQL users try out various new ideas, all the right things get developed.

We still need to improve our ability to receive innovations from the community, but I am proud of how far we have come already. Look at the MySQL Forge if you are interested: fforge.mysql.com

4) Re:R&D Directions?
by bzipitidoo

A few questions, only one about the biz. For those of us who like designing and researching and wish to spend our time on that, and don't like the thought of spending time on sales and promotion, or begging for money from VC vultures, or figuring out tax forms, stock options, and such, or sifting through thousands of resumes trying to find a few good people to hire, or knowing when a deal is a bad one to be avoided at all costs, or all the other aspects of business, what are wannabe independent software developers to do? If I try to start a business, it'd be so I can make some money doing what I like, not sink time into the business of business. How'd MySQL handle that when it started?

The parent's R&D question reminded me of another question. A goal of future versions of ReiserFS is to make database programs unnecessary and obsolete. Reiser asserts the only reason you even need a database program is because current file systems cannot do things like efficiently store, access, and query thousands of small records. File systems should be able to do everything a database can do. Apart from the bit about being in jail, is Reiser crazy, or is he on to something?

Anyway, do you recommend or favor some file systems over others for best performance from MySQL?

Mårten: I think you had a total of three questions.

First, when MySQL got going, it initially was run day-to-day by the three founders Monty, David and Allan. In the year 2000 they realised it was growing beyond what they could handle, so they reached out to get a CEO, a board of directors, and VC capital. This is a great model for those who want the company to grow and who are ready to share decision-making with others.

Your other question is about what a software developer of today could do. I believe it is easier to start a business today than 10 years ago, and I believe there are some great avenues to follow. For instance, you can develop open source software that wins the hearts and minds of users, and then you can license or sell the software to a bigger vendor if you want to avoid the hassles you mention.

Mark Matthews is a great example. He had developed the best JDBC driver for MySQL, and then we acquired his software and hired him full-time. JBoss has done similar "acquisitions". Another alternative is to build a website that provides the intended functionality as a service. The benefit there is that you don't have to build a distribution channel, because the web is the channel. And then there can be some great outcomes if you don't want to run your own company. Just look at Flickr or YouTube, for instance, who got acquired by web giants for good sums of money.

Your third question is about filesystems versus databases. If I were a filesystem developer, I might claim that databases will become obsolete. But as I am a database developer, I will claim that filesystems will become obsolete! Humour aside, I don't have a specific recommendation, but I do believe that we will always have heterogeneity in the software world. There will always be several alternative technologies for every single problem.

5) Conflict of Interest
by 99BottlesOfBeerInMyF

One of the most common complaints I've heard about the business model of profiting on support for a product, is that it provides motivation to keep the product from becoming very user friendly. After all, if the product is too easy to use, who will pay for support? In my own experience, I've seen a lot of companies that consider support to be insurance, and don't use it for help with installation, configuration, or to overcome usability issues so much as a way to cover their asses in case something goes very wrong. Do a lot of your customers use support to overcome usability problems and if so, does this de-motivate you to solve other usability issues?

Mårten: This is a great question. First, I think closed source vendors are proving the hypothesis incorrect, because they are the ones who have un-friendly products although they have a licence revenue stream. And open source products, which typically lack a licence fee, are the ones with the best user friendliness. Why is that? I think the reason is that popularity is worth more than the marginally improved fees you could get for a user-unfriendly product. Sure, we lose some purchase orders because customers simply don't need our support. But at the same time we gain millions of new users, and they in turn help us develop the product and drive our marketing.

I think one of the reasons MySQL is so popular is that we continue to take user-friendliness seriously. We may joke about leaving some bugs in the product in order to have more support fees, but in reality we take the bugs very seriously. And when we at times have more bugs than we would like to have, the whole organisation is in pain until we get the upper hand and kill more bugs than we produce. Version 5.0 was such an example - when it first came out as GA it had more bugs than earlier versions. It didn't feel good to see the list of incoming bugs, so we rearranged tasks and expanded our QA test coverage in order to fix the situation. I hope you who have used the latest versions of MySQL 5.0.26 have seen the significant improvements.

6) Defects per KLOC
by eldavojohn

Your website touts you as having the lowest defects per KLOC by up to 12 times the industry standard, what do you attribute as the leading factor to your success in this respect? Since cold cash is the traditional method, how do you incentivise code quality in an open source product?

Mårten: I think open source has a wonderful benefit over closed source that often is forgotten: When you know that your work may be scrutinised in public by others, you simply do a better job. It is some magical combination of pride and (fear of) shame.

So I am claiming that the same developers will do a better job of producing open source than closed source software. The analogy is your backyard: why is the backyard never as tidy as the frontyard of a house? You spend more time in the backyard, don't you? And it is you taking care of both the front and the back. The only difference is that your neighbours see your frontyard but not your backyard. It's as simple as that.

Additionally, the reality of the scrutiny, i.e. the peer reviews, ensure that a lot of bugs and defects are detected and removed early. So I am not the least surprised that open source is 12 times cleaner than closed source. The really scary thing is all the stuff that is buried deep inside closed source software, out of the sight of everyone and without any proper stewardship.

7) MySQL business model for niche products?
by MarkWatson

Hello Marten,

First, congratulations on MySQL's market capitalization! My question is:

I have been working part time for about 6 years on software for text/data mining and general semantic information extraction. Almost all of my development is in Common Lisp, but I have ported little bits to Java and released that under the GPL in the past. I view this as a small, niche market, not like MySQL. What do you think that chances are for making money on GPLing a niche product?

MySQL is very widely used so if you capture commercial use icensing costs for a small percent of users, you do very well. For my software, with luck perhaps a few hundred companies a year might start adopting my product. Does it seem like wishful thinking for me to use a GPL based business model like MySQL's?

I want my customers to have my source code for a lot of reasons, but I would also like to capture revenue. I might just end up going to market as a proprietary product that incidently includes source code, with licensing that prohibits redistribution to non-customers.

Thanks for your help,

Mark

Mårten: Good question. I used to state in presentations that open source probably is best suited for products with huge user bases. But I was always proved wrong. Somebody would come up to me and tell about a successful open source niche product. So I have learned that you can be successful with open source in any environment.

I think the key is to provide real value to your users or customers - initially free of charge, but for continuous support against a fee. In that way you can build credibility in your market and gain important insights in what customers need, and you can also build a business. But don't think it is easy to find a business model that fits with free or open source software. It is difficult. But it is possible. You have to take some risks before hitting on the right answer.

8) Commercial vs free - where to draw the line
by Internet Ninja
I work for a good sized business and looked at using the Cluster Jumpstart but when I told my boss the cost and that we'd have to pay for flights he laughed at me, even though we're starting to really use mySQL pretty seriously now for some stuff.

With costs for things like this and gold/platinum support also relatively high on a per server basis it seems there's a wide gap between community based support which costs nothing and enterprise support which appears somewhat pricey.

How do you draw the line for paid vs free support particularly since a lot of SME's are using mySQL and may be unable to afford it? Was it a conscious decision to pitch it high to display value in the product?

Mårten: This is essentially a question of price elasticity. If we lower the unit prices, we may sell more, but will we sell that much more? If we increase the prices, we may have fewer customers but they may be more profitable.

We have an ambition to be affordable and available for all, and therefore our prices are typically a tenth of what you would pay for closed source databases. But MySQL Cluster is a state-of-the art solution that in its own market segment beats all other databases hands-down. The price may sound high to someone from a smaller company, but it is carefully weighed against market conditions. Many customers find that it is not only the best solution technically, it is also the least expensive one. If the price is prohibitively high for someone, then they can always use the community edition and do the work themselves.

9) Achievements & Fallout
by eldavojohn

In your five years as MySQL CEO, what has been your proudest moment? Do you find it difficult to lead a company based on a product that belongs to a community? Do you ever experience any fallout/backfire from running your company on such a business model?

Mårten: Thanks for the question! I felt very proud when we visited Google for the first time and people would come out of their rooms just to see the "MySQL folks". I had scheduled a CEO-to-CEO meeting with Eric Schmidt, but in addition to him we had a dozen developers who all wanted to know about MySQL internals. Fortunately, I had David Axmark (co-founder) with me and he did the technical presentation! Another proud moment was our users conference this year with a thousand or so in the audience, all gathered just to listen to the State of MySQL. Those situations make us realise that we are changing the world - we are making a difference.

The great thing with our business is that we get to pursue goals that we are passionate about. We get to work with the smartest people in the world on the newest and coolest database technologies. We get to stand up to defend the freedom of software and to fight software patents. We contribute to the wealth and happiness of what is called the bottom of the pyramid - people who may not have the financial means to buy software but who have dreams and goals they want to pursue. Of course it all needs to make sense from a financial perspective for everyone involved, so it all needs to boil down to a good business model. But, as I noted, that is exactly the great thing with our model.

But we also need to know that it is very difficult to walk the fine line between freedom of software and freedom to pursue profits. The two are not at odds with each other, but there are overlapping areas where you need to have all details right. We have made our mistakes over the years, and I hope we have fixed them. If we haven't, let me know and we'll get onto it. I am not expecting everyone to agree with our business model, and every now and then we get flamed by someone, but I do hope that people generally respect our business decisions. And so far that has clearly been the case.

A friend of mine just wrote to me: "If you can find a new way for folks to make money from FOSS without compromising fundamentals, good for you. Others can follow your lead. Just don't compromise the fundamentals." That captures the essence of the question very well.

10) Appliance possibility?
by Roblimo (added editorially
because of this discussion)

Now and then we hear rumors that Oracle is going to either come out with its own Linux distribution, sell a single "stack" with Oracle products running on Linux or else sell a complete hardware appliance that runs Oracle on Linux.

Back in 2003 there was a significant PR splash touting database server appliances made in partnership by Pogo Linux and MySQL, but we haven't heard much about that idea since. We know you now have many channel partners, including hardware vendors, but this is not quite the same.

Do you have any plans to come out with either a Linux/MySQL "single stack" software product or a preloaded Linux/MySQL+hardware appliance? And if not, would you change your mind if Oracle started selling either of these products?

Mårten: First, if Oracle launches their own Linux distro, then we take that as a great victory for open source. (And we ask - is the database next?) The most successful distros include JBoss and MySQL and many other products, so we would naturally make sure that Oracle gets all the help they need in putting our product on their operating system.

As for appliances and stacks, I am not sure that we would choose to compile our own stack. There are so many tempting business models and products, but to really grow fast there are just three main rules: Focus, Focus, and Focus. In our ecosystem we have many partners who could provide a stack, and we are very supportive of them. HP has the Linux reference architecture, Pogo their appliance, and Sourcelabs and Spikesource came out with software stacks. The power of open source and the power of LAMP is in the collaboration between the companies, not so much in what one single company can accomplish.

11) What about community?
(inserted by martenmickos himself)

MySQL is enormously popular and has a vibrant ecosystem, but it seems that the core product is primarily developed in-house. Are you at all interested in contributions, and what are you doing to stimulate community engagement?

Mårten (answering his own question): Initially, although open source, MySQL was developed by just one person - our founder Monty. Over the years we have expanded the group of developers both in-house and in the community. But we have more work to do. We need to make the product more modular so that it is easier to contribute distinct modules (such as storage engines or full-text search modules). We need to document the source code better (we are doing it with Doxygen). We need to make the contribution process as easy as possible.

We have very strict requirements on quality and ownership of intellectual property before we put something in the commercial product, but in the MySQL Community Server we are becoming much more open to contributions. We also need to set up websites and systems for broad collaboration in our community.

Planet MySQL has become a great aggregator of MySQL blogs, and the MySQL Meetups are popular local events. The MySQL Forge is a place for contributions, and soon we will launch the Winter of Code program with coding contests. For the most passionate MySQL users and developers there is the MySQL Camp in November (mysqlcamp.org). And of course all of us will get together in Silicon Valley in April for the annual users conference.

But no matter what we organise on behalf of MySQL AB (by the way, AB means "Aktiebolaget" i.e. "Incorporated"), the true value of the community will always be generated by the community itself. We are still on a learning curve in terms of all the possibilities that this offers. So please share your ideas with us. What should we do more of and what less? How could we bring together MySQL users from all over the world to accomplish bigger things? How could we together change the world for the better?

It is amazing what open source has accomplished in the last 15 years, and I am even more excited about what it can do in the next 15. But my excitement is anchored in our community. Alone I can accomplish nothing, but with the help of millions of users worldwide, we can accomplish just about anything.

I hope I answered all questions clearly and conclusively. Let me know if I missed anything.

Marten

------

You asked. Mårten answered. He even added (and answered) a question he wished had been asked, but wasn't. If you have a comment or follow-up question, please post it. Mårten will spend as much time responding to your comments as his schedule permits. 1) Re:Biggest Problem?
by wild_berry

In light of comments here about PostgreSQL being superior (and F/LOSS) technology but MySQL being used because of "MCSE weenie mentality", are you, Marten Mickos, concerned about the complacency of 'good enough' technological solutions or that Free/Libre Open Source Software may never remove the entrenched market leaders?

Mårten: Not the least. There will always be fanatical supporters of various products and projects, but I would look at what real users are saying. As an example, Nortel, Alcatel and Nokia are building new mobile phone network elements that run on MySQL Cluster. That's leading edge open source technology in some of the most advanced high-availability use. Or take YouTube or some of the other new web successes - they are scaling incredibly fast and again it is open source that powers them (MySQL, to be specific).

It is a common misperception that advanced technology cannot be easy to use. But it can! Driving a Tesla is fairly straightforward, but building one is not. Flying the new Eclipse jet is comparatively easy, but designing and constructing the aircraft is extremely difficult. And although it was pretty difficult to fly Spaceship One out of the atmosphere twice in two weeks, it was nothing compared to the difficulty of designing such a modern spaceship.

2) Perception of low quality for 'free'
by OakDragon

How do you fight the perception that MySQL is not suitable for 'the real world' because it is free?

Mårten: That's a challenge we share with all open source products, and I believe that such misperceptions will be removed over time by the overwhelming momentum of open source at large. So it is not so much a product-specific issue. But we need everyone's help to set records straight.

3) R&D Directions?
by eldavojohn

In a market where people are just looking for stability, simplicity & scalability, where do you turn for innovation in your products? Is there a lot of research and development towards new features and completely new products in MySQL's community or do you aim primarily to do one thing well? How do you influence the direction of this research in such a large open source project? Do you attempt to add direction at all?

Mårten: I am very proud of the innovative power inside our company, but even more important is the notion that "innovation happens elsewhere" (that's a great book that I recommend, as is "Democratizing Innovation"). We just announced a new monitoring and advisory service which we have innovated inside the company and with the help of customers.

But more massive is the innovation that happens in the MySQL ecosystem - in the user space. Look at the XML wrappers, RSS converters, full-text search modules or any of the toolkits that have been developed for MySQL. Or take PBXT - a new transactional storage engine, or the custom-built storage engines that Google, Friendster and others have developed. It is in the interaction with these advanced users that we learn what's useful and what isn't.

It takes a degree of humility and openness (and eradicating any Not Invented Here feelings) to be able to make good use of innovations from the ecosystem. But in return it is a much more productive way. I see it as a Darwinian system where the best ideas survive. Nobody can know for sure exactly where the development is heading, but if millions of MySQL users try out various new ideas, all the right things get developed.

We still need to improve our ability to receive innovations from the community, but I am proud of how far we have come already. Look at the MySQL Forge if you are interested: fforge.mysql.com

4) Re:R&D Directions?
by bzipitidoo

A few questions, only one about the biz. For those of us who like designing and researching and wish to spend our time on that, and don't like the thought of spending time on sales and promotion, or begging for money from VC vultures, or figuring out tax forms, stock options, and such, or sifting through thousands of resumes trying to find a few good people to hire, or knowing when a deal is a bad one to be avoided at all costs, or all the other aspects of business, what are wannabe independent software developers to do? If I try to start a business, it'd be so I can make some money doing what I like, not sink time into the business of business. How'd MySQL handle that when it started?

The parent's R&D question reminded me of another question. A goal of future versions of ReiserFS is to make database programs unnecessary and obsolete. Reiser asserts the only reason you even need a database program is because current file systems cannot do things like efficiently store, access, and query thousands of small records. File systems should be able to do everything a database can do. Apart from the bit about being in jail, is Reiser crazy, or is he on to something?

Anyway, do you recommend or favor some file systems over others for best performance from MySQL?

Mårten: I think you had a total of three questions.

First, when MySQL got going, it initially was run day-to-day by the three founders Monty, David and Allan. In the year 2000 they realised it was growing beyond what they could handle, so they reached out to get a CEO, a board of directors, and VC capital. This is a great model for those who want the company to grow and who are ready to share decision-making with others.

Your other question is about what a software developer of today could do. I believe it is easier to start a business today than 10 years ago, and I believe there are some great avenues to follow. For instance, you can develop open source software that wins the hearts and minds of users, and then you can license or sell the software to a bigger vendor if you want to avoid the hassles you mention.

Mark Matthews is a great example. He had developed the best JDBC driver for MySQL, and then we acquired his software and hired him full-time. JBoss has done similar "acquisitions". Another alternative is to build a website that provides the intended functionality as a service. The benefit there is that you don't have to build a distribution channel, because the web is the channel. And then there can be some great outcomes if you don't want to run your own company. Just look at Flickr or YouTube, for instance, who got acquired by web giants for good sums of money.

Your third question is about filesystems versus databases. If I were a filesystem developer, I might claim that databases will become obsolete. But as I am a database developer, I will claim that filesystems will become obsolete! Humour aside, I don't have a specific recommendation, but I do believe that we will always have heterogeneity in the software world. There will always be several alternative technologies for every single problem.

5) Conflict of Interest
by 99BottlesOfBeerInMyF

One of the most common complaints I've heard about the business model of profiting on support for a product, is that it provides motivation to keep the product from becoming very user friendly. After all, if the product is too easy to use, who will pay for support? In my own experience, I've seen a lot of companies that consider support to be insurance, and don't use it for help with installation, configuration, or to overcome usability issues so much as a way to cover their asses in case something goes very wrong. Do a lot of your customers use support to overcome usability problems and if so, does this de-motivate you to solve other usability issues?

Mårten: This is a great question. First, I think closed source vendors are proving the hypothesis incorrect, because they are the ones who have un-friendly products although they have a licence revenue stream. And open source products, which typically lack a licence fee, are the ones with the best user friendliness. Why is that? I think the reason is that popularity is worth more than the marginally improved fees you could get for a user-unfriendly product. Sure, we lose some purchase orders because customers simply don't need our support. But at the same time we gain millions of new users, and they in turn help us develop the product and drive our marketing.

I think one of the reasons MySQL is so popular is that we continue to take user-friendliness seriously. We may joke about leaving some bugs in the product in order to have more support fees, but in reality we take the bugs very seriously. And when we at times have more bugs than we would like to have, the whole organisation is in pain until we get the upper hand and kill more bugs than we produce. Version 5.0 was such an example - when it first came out as GA it had more bugs than earlier versions. It didn't feel good to see the list of incoming bugs, so we rearranged tasks and expanded our QA test coverage in order to fix the situation. I hope you who have used the latest versions of MySQL 5.0.26 have seen the significant improvements.

6) Defects per KLOC
by eldavojohn

Your website touts you as having the lowest defects per KLOC by up to 12 times the industry standard, what do you attribute as the leading factor to your success in this respect? Since cold cash is the traditional method, how do you incentivise code quality in an open source product?

Mårten: I think open source has a wonderful benefit over closed source that often is forgotten: When you know that your work may be scrutinised in public by others, you simply do a better job. It is some magical combination of pride and (fear of) shame.

So I am claiming that the same developers will do a better job of producing open source than closed source software. The analogy is your backyard: why is the backyard never as tidy as the frontyard of a house? You spend more time in the backyard, don't you? And it is you taking care of both the front and the back. The only difference is that your neighbours see your frontyard but not your backyard. It's as simple as that.

Additionally, the reality of the scrutiny, i.e. the peer reviews, ensure that a lot of bugs and defects are detected and removed early. So I am not the least surprised that open source is 12 times cleaner than closed source. The really scary thing is all the stuff that is buried deep inside closed source software, out of the sight of everyone and without any proper stewardship.

7) MySQL business model for niche products?
by MarkWatson

Hello Marten,

First, congratulations on MySQL's market capitalization! My question is:

I have been working part time for about 6 years on software for text/data mining and general semantic information extraction. Almost all of my development is in Common Lisp, but I have ported little bits to Java and released that under the GPL in the past. I view this as a small, niche market, not like MySQL. What do you think that chances are for making money on GPLing a niche product?

MySQL is very widely used so if you capture commercial use icensing costs for a small percent of users, you do very well. For my software, with luck perhaps a few hundred companies a year might start adopting my product. Does it seem like wishful thinking for me to use a GPL based business model like MySQL's?

I want my customers to have my source code for a lot of reasons, but I would also like to capture revenue. I might just end up going to market as a proprietary product that incidently includes source code, with licensing that prohibits redistribution to non-customers.

Thanks for your help,

Mark

Mårten: Good question. I used to state in presentations that open source probably is best suited for products with huge user bases. But I was always proved wrong. Somebody would come up to me and tell about a successful open source niche product. So I have learned that you can be successful with open source in any environment.

I think the key is to provide real value to your users or customers - initially free of charge, but for continuous support against a fee. In that way you can build credibility in your market and gain important insights in what customers need, and you can also build a business. But don't think it is easy to find a business model that fits with free or open source software. It is difficult. But it is possible. You have to take some risks before hitting on the right answer.

8) Commercial vs free - where to draw the line
by Internet Ninja
I work for a good sized business and looked at using the Cluster Jumpstart but when I told my boss the cost and that we'd have to pay for flights he laughed at me, even though we're starting to really use mySQL pretty seriously now for some stuff.

With costs for things like this and gold/platinum support also relatively high on a per server basis it seems there's a wide gap between community based support which costs nothing and enterprise support which appears somewhat pricey.

How do you draw the line for paid vs free support particularly since a lot of SME's are using mySQL and may be unable to afford it? Was it a conscious decision to pitch it high to display value in the product?

Mårten: This is essentially a question of price elasticity. If we lower the unit prices, we may sell more, but will we sell that much more? If we increase the prices, we may have fewer customers but they may be more profitable.

We have an ambition to be affordable and available for all, and therefore our prices are typically a tenth of what you would pay for closed source databases. But MySQL Cluster is a state-of-the art solution that in its own market segment beats all other databases hands-down. The price may sound high to someone from a smaller company, but it is carefully weighed against market conditions. Many customers find that it is not only the best solution technically, it is also the least expensive one. If the price is prohibitively high for someone, then they can always use the community edition and do the work themselves.

9) Achievements & Fallout
by eldavojohn

In your five years as MySQL CEO, what has been your proudest moment? Do you find it difficult to lead a company based on a product that belongs to a community? Do you ever experience any fallout/backfire from running your company on such a business model?

Mårten: Thanks for the question! I felt very proud when we visited Google for the first time and people would come out of their rooms just to see the "MySQL folks". I had scheduled a CEO-to-CEO meeting with Eric Schmidt, but in addition to him we had a dozen developers who all wanted to know about MySQL internals. Fortunately, I had David Axmark (co-founder) with me and he did the technical presentation! Another proud moment was our users conference this year with a thousand or so in the audience, all gathered just to listen to the State of MySQL. Those situations make us realise that we are changing the world - we are making a difference.

The great thing with our business is that we get to pursue goals that we are passionate about. We get to work with the smartest people in the world on the newest and coolest database technologies. We get to stand up to defend the freedom of software and to fight software patents. We contribute to the wealth and happiness of what is called the bottom of the pyramid - people who may not have the financial means to buy software but who have dreams and goals they want to pursue. Of course it all needs to make sense from a financial perspective for everyone involved, so it all needs to boil down to a good business model. But, as I noted, that is exactly the great thing with our model.

But we also need to know that it is very difficult to walk the fine line between freedom of software and freedom to pursue profits. The two are not at odds with each other, but there are overlapping areas where you need to have all details right. We have made our mistakes over the years, and I hope we have fixed them. If we haven't, let me know and we'll get onto it. I am not expecting everyone to agree with our business model, and every now and then we get flamed by someone, but I do hope that people generally respect our business decisions. And so far that has clearly been the case.

A friend of mine just wrote to me: "If you can find a new way for folks to make money from FOSS without compromising fundamentals, good for you. Others can follow your lead. Just don't compromise the fundamentals." That captures the essence of the question very well.

10) Appliance possibility?
by Roblimo (added editorially
because of this discussion)

Now and then we hear rumors that Oracle is going to either come out with its own Linux distribution, sell a single "stack" with Oracle products running on Linux or else sell a complete hardware appliance that runs Oracle on Linux.

Back in 2003 there was a significant PR splash touting database server appliances made in partnership by Pogo Linux and MySQL, but we haven't heard much about that idea since. We know you now have many channel partners, including hardware vendors, but this is not quite the same.

Do you have any plans to come out with either a Linux/MySQL "single stack" software product or a preloaded Linux/MySQL+hardware appliance? And if not, would you change your mind if Oracle started selling either of these products?

Mårten: First, if Oracle launches their own Linux distro, then we take that as a great victory for open source. (And we ask - is the database next?) The most successful distros include JBoss and MySQL and many other products, so we would naturally make sure that Oracle gets all the help they need in putting our product on their operating system.

As for appliances and stacks, I am not sure that we would choose to compile our own stack. There are so many tempting business models and products, but to really grow fast there are just three main rules: Focus, Focus, and Focus. In our ecosystem we have many partners who could provide a stack, and we are very supportive of them. HP has the Linux reference architecture, Pogo their appliance, and Sourcelabs and Spikesource came out with software stacks. The power of open source and the power of LAMP is in the collaboration between the companies, not so much in what one single company can accomplish.

11) What about community?
(inserted by martenmickos himself)

MySQL is enormously popular and has a vibrant ecosystem, but it seems that the core product is primarily developed in-house. Are you at all interested in contributions, and what are you doing to stimulate community engagement?

Mårten (answering his own question): Initially, although open source, MySQL was developed by just one person - our founder Monty. Over the years we have expanded the group of developers both in-house and in the community. But we have more work to do. We need to make the product more modular so that it is easier to contribute distinct modules (such as storage engines or full-text search modules). We need to document the source code better (we are doing it with Doxygen). We need to make the contribution process as easy as possible.

We have very strict requirements on quality and ownership of intellectual property before we put something in the commercial product, but in the MySQL Community Server we are becoming much more open to contributions. We also need to set up websites and systems for broad collaboration in our community.

Planet MySQL has become a great aggregator of MySQL blogs, and the MySQL Meetups are popular local events. The MySQL Forge is a place for contributions, and soon we will launch the Winter of Code program with coding contests. For the most passionate MySQL users and developers there is the MySQL Camp in November (mysqlcamp.org). And of course all of us will get together in Silicon Valley in April for the annual users conference.

But no matter what we organise on behalf of MySQL AB (by the way, AB means "Aktiebolaget" i.e. "Incorporated"), the true value of the community will always be generated by the community itself. We are still on a learning curve in terms of all the possibilities that this offers. So please share your ideas with us. What should we do more of and what less? How could we bring together MySQL users from all over the world to accomplish bigger things? How could we together change the world for the better?

It is amazing what open source has accomplished in the last 15 years, and I am even more excited about what it can do in the next 15. But my excitement is anchored in our community. Alone I can accomplish nothing, but with the help of millions of users worldwide, we can accomplish just about anything.

I hope I answered all questions clearly and conclusively. Let me know if I missed anything.

Marten

------

There have been so many articles written about the perils, pitfalls, and possible rewards of running a business based on free or open source software that we can't possibly link to them all. Instead, let's ask MySQL CEO Mårten Mickos how to make money with a company based on free software, because he runs a company that is almost always touted as one of the world's greatest free software (business) successes. You may want to read some of these interviews with Mårten before you come up with your own questions in order to avoid duplication, but other than that suggestion and the usual Slashdot interview rules, ask whatever you like, however you like.

There have been so many articles written about the perils, pitfalls, and possible rewards of running a business based on free or open source software that we can't possibly link to them all. Instead, let's ask MySQL CEO Mårten Mickos how to make money with a company based on free software, because he runs a company that is almost always touted as one of the world's greatest free software (business) successes. You may want to read some of these interviews with Mårten before you come up with your own questions in order to avoid duplication, but other than that suggestion and the usual Slashdot interview rules, ask whatever you like, however you like.

mikeballer asks: "We currently run our website from a shared hosting environment, with ASP and MS SQL Server. We will be moving to a dedicated host, and to save money, we are considering transitioning to MySQL while remaining in a Windows environment. I had read the Windows-vs-Unix section of the MYSQL documentation, but what is Slashdot's perspective on the performance of MySQL in a Windows environment?"

Sembiance writes "I've been working on a C/C++ source code search database for the past year. It has recently surpassed 275 million lines of searchable open source C/C++ code. The search engine is C/C++ syntax aware so you can search for specific elements such as functions, macros, classes, comments, etc. The site is built upon many open source products including: MySQL and Lucene for the database, CodeWorker to parse the code, PHP and Apache for the website and GeSHi for syntax highlighting. I'm currently looking for suggestions on what sort of 'interesting statistics' I could create from 275+ million lines of open source C/C++ code."

An anonymous reader asks: "What would be the best high availability solution for databases? I don't have enough money to afford Oracle RAC or any architecture that require an expensive SAN. What about open source solutions? MySQL cluster seems to be more master/slave and you can lose data when the master dies. What about this Sequoia project that seems good for PostgreSQL and other databases? Has anyone tried it? What HA solution do you use for your database?"

Aargh writes "Every year a public Internet poll is taken to vote for, amongst others, the "European of the Year". This year, the founder of NoSoftwarePatents.com has been selected as a candidate. Taken from the NoSoftwarePatents.com site: "We now have a first-rate opportunity to make political leaders, media and citizens all over the world realize the significance of our cause. Please give us your vote, and help us gain more votes, so that the founder of the NoSoftwarePatents campaign be elected as the new 'European of the Year'." Non-europeans can also vote, so why dont we unleash the slashdot hordes?" Mr. Mueller had been exchanging e-mails recently on this subject; thanks to an introduction from Kaj Arnö. I truly do think that given his, and the organization's work that they deserve to win. Check out the celebrity endorsements as well. *grin* Also, worth reading their voting guide if you are going to vote.

chicagoan writes "MySQL AB today announced the general availability of MySQL 5.0, the most significant product upgrade in the company's ten-year history. The major new version delivers advanced SQL standard-compliant features such as stored procedures, triggers, views & new pluggable storage engines. Over 30 enterprise platform and tool vendors have also expressed enthusiastic support for the new release of the world's most popular open source database."

jg21 writes "In the continuing saga of the decision by MySQL previously discussed here on Slashdot to make a deal with SCO Group, the company's CEO Marten Mickos has now granted an interview in which he addresses the inevitable criticism that the deal has provoked in the F/OSS community. His main defense seems to be that other companies have ported to SCO too. He admits money too played a part." From the article: "We believe that porting a GPL version of MySQL for the SCO OpenServer platform gives thousands of users more options when it comes to choosing a database -- which is a good thing. The deal produces revenue for us and this allows us to hire more open source developers. We didn't make the decision lightly; we knew SCO was a sensitive subject with the free software and open source communities."

A short time ago, Oracle announced its acquisition of Innobase, the Finnish company that makes the GPL'd InnoDB table storage engine. Among MySQL users, the separately-written InnoDB is almost as popular as the native MyISAM engine, and is considered to be more advanced for most purposes. Slashdot has, except for search, run entirely on InnoDB for the past year or two so we're as concerned about this as anybody. Brian Aker, former Slashdot coder and current Director of Architecture for MySQL AB, comments: "InnoDB is GPL, so once again the beauty of the open source market is at play: there is no lock in, and we can continue to develop Innodb as we see fit. The code is out there and we plan on continuing to support it. The largest database vendor in the world just confirmed that the market for open source databases exists."

Brian "Krow" Aker (Former Slashdot Coder now MySQL Employee) writes "I am pleased to announce the release candidate for MySQL 5.0. This version has been in development now for three years. We have worked to add update-able views, ansi stored procedures, and triggers. In addition we have added a number of fun features that we are experimenting with and resolved issues with bad data inserts (which personally annoyed the hell out of me when we rewrote Slashdot a couple of years back so I am happy to see this issue go away). We look forward to feedback on the candidate and will show some love for bug reports."

Brian "Krow" Aker (Former Slashdot Coder now MySQL Employee) writes "I am pleased to announce the release candidate for MySQL 5.0. This version has been in development now for three years. We have worked to add update-able views, ansi stored procedures, and triggers. In addition we have added a number of fun features that we are experimenting with and resolved issues with bad data inserts (which personally annoyed the hell out of me when we rewrote Slashdot a couple of years back so I am happy to see this issue go away). We look forward to feedback on the candidate and will show some love for bug reports."

OSCON 2005 was held in a convention center this year, instead of a hotel, because it just got too big (2000+ people). Too big, in fact, for pudge and myself to cover more than a fraction of the talks and the ideas flitting around the hallways. But here's some of what I found cool last week. And if you attended or presented at OSCON and want to tell us about all the neat stuff we missed, please, share your thoughts in the comments, or submit a fact-rich writeup and we'll maybe do a followup story later.

Mike Shaver's talk on writing Firefox extensions was packed to the walls. If you've been wanting to try it, Firefox 1.5 makes development easier, and should be out soon, so now's a good time. This talk and the tutorial on Ajax persuaded me to start using the DOM Inspector and debugging some JavaScript to get a better understanding of webpage manipulation.

Aaron Boodman's talk on his extension Greasemonkey was a walkthrough of writing a simple GM user script, a discussion of what's coming up, and some Q&A. Greasemonkey 0.5 ("Now With Security!") is in beta: there are multiple security changes that suggest someone really has sat down and thought the whole model through. GM works with Firefox, Seamonkey, Opera, and Windows MSIE (but not, oh please somebody correct this oversight, Safari).

Ruby on Rails is hot; if you want to develop a web app quickly you can't ignore it. It stresses "convention over configuration" with reasonable defaults. The tutorial went from installation to the "hello world" of the web, a blog (!), in a few hours. Anyone have a real-world example of Rails scaling to a large project and lots of traffic?

DarwinBuild is an open-source project from Apple that aids in building the open-source components of Darwin/Mac OS X. Given a build number of Mac OS X, it will fetch and build the software for that version, allowing you to modify the source as needed, making it easy for any developer to modify everything from the kernel to various utilities (just remember to reapply the modifications after running Software Update, if necessary). You can read more about it from, in addition to the web site, the presentation slides.

Google and O'Reilly gave out the 2005 open source awards, with $5000 attached to each. Congratulations to the winners.

Tony Baxter's Shtoom is a cross-platform VoIP client and software framework, written in Python, for writing your own phone applications.

Novell is still moving its employees from Windows to Linux, which we first heard at last year's OSCON. The migration from Microsoft Office to OpenOffice is complete, and the big step, from Windows to Linux, is 50% complete, projected to be 80% by November. Miguel de Icaza gave flashy demos of some Linux desktop applications that didn't impress this cynical observer very much.

PlaceSite is an open-source project looking to bring physical proximity awareness to Internet access at coffeeshops and other meetingplaces: think "local-only Friendster" and you're not far off. They got feedback from a monthlong trial earlier this year and are working on a new version that will be easy to deploy. Could be neat.

In a great 2-hour session on Wednesday, we got to hear from representatives of four leading open source databases about what they've been working on lately. Here are the summaries...

Ingres r3 has an impressive list of big features. Ingres was just open-sourced by Computer Associates this summer, and it's gotten a lot of attention for being a full-featured enterprise database. Ingres supports table partitioning that can be either range-based or hash-based, which can greatly improve performance in many cases. Its optimizer can now come up with parallel execution plans, which can be useful even on single-CPU machines and non-partitioned tables. There's also federated data storage (one can access data stored in another RDBMS through Ingres) and replication. And they're working on a concurrent access cluster, to allow data to be manipulated not just by multiple threads on one machine, but multiple machines.

A side note: Computer Associates was invited by O'Reilly to talk about its recently open-sourcing Ingres. Its representative, while confessing that introducing a new license was "probably the wrong thing to do," said that other licenses wouldn't have worked for them (the GPL "was seen as viral"). The one question that the audience had time to ask was "is Ingres a dump" -- is CA making it open-source to transfer the responsibility of support from the company to the community? The three-part "no" answer was that there are more CA developers working on Ingres now, that Ingres is at the core of their new releases, and that they've sponsored a "million-dollar challenge" to foster community interest. Time will tell I guess.

Firebird 2.0 has been in alpha since January and a beta is expected soon. Since 2000 much of their development has been aimed at making the product easy to install, and making the code easy for a distributed group of developers to work on. This year they're building features on that groundwork. Their design includes 2-phase commits (since the beginning), cooperative garbage collection (as a transaction encounters unneeded data, it removes it) and self-balancing indexes. Backup has been improved. When 2.0 gets to beta, I'm going to check this out, it sounds like very interesting technology (and apparently it will install with four clicks!).

MySQL 5.0 is in beta, and has been feature-frozen since April. Back in 4.1, its abstracted table-type has been put to advantage with odd engines like Archive (only insert, no update); Blackhole for fast replication; and an improvement to MyISAM for logging (allowing concurrent selects with inserts-at-table-end). Their Connector/MXJ lets you run a native MySQL server embedded inside a Java application. In 5.0 we're seeing stored procedures per the SQL:2003 standard, triggers, updatable views, XA (distribution transaction), SAP R/3 compatible server side cursors, fast precision math, a federated storage engine, a greedy optimizer for better handling of many-table joins, and an optional "strict mode" to turn some of MySQL's friendly nonstandard warnings into compliant errors. And they're working on partitioning, ODBC, and letting MySQL Cluster's non-indexed columns to be stored on disk.

PostgreSQL 8.1 is expected to be released in November or December, after a feature-freeze in July -- and it's an impressive list of new features. Their optimizer will make use of multiple indexes when appropriate, which is pretty darn exciting. The recommendation will be that in most cases it will be most efficient to have only single-column indexes and let the optimizer figure out which combination to use. They're implementing a 2-phase commit, they're bringing the automatic vacuum into the core code, and they removed a global shared buffer lock so they're now getting "almost linear" SMP performance scaling. I've never felt the need for Postgres, but I'm definitely going to look at 8.1.

StudMuffin asks: "I teach graduate computer science courses at a Big 10 university to grad students, who have never programmed before and are studying Human-Computer Interaction or other Information Science specialties. These courses are usually their first dip into the programming pool, so we have tons to cover in three months. This fall, I have been asked to take over and redesign our 'Complex Website' course, which is getting a bit long in the tooth. This course has traditionally been about database backed websites with server-side scripting. My only requirements are that there be coverage of PHP and that we have basic instruction about persistence using a database (which must be MySQL). However, I believe that the nature of 'complex' websites has changed, with XHTML, CSS, Javascript, web services, and so on. Sites like Google Maps make the browser feel like a fat client and are making the web browser a true window onto enormous data sets, and take into consideration the MoRAS of small views on large worlds. What do Slashdot readers consider a reasonable curriculum would be for a redesigned course like this?"

Zontar The Mindless writes "MySQL Community Edition 5.0.3-beta has been released. This version has support for Stored Procedures, Triggers, Views and many other features, including a number of security enhancements and stability fixes. See the changelog for a complete listing of new features and bugfixes. MySQL 5.0.3-beta is now available in source and binary form for Linux, Solaris, MacOS X, FreeBSD, Windows, and other platforms from dev.mysql.com/downloads/ and mirror sites. RPMs for Red Hat/Fedora and SUSE Linux are also now available for the first time in the 5.0 series. This is the first Beta release in the 5.0 series, in preparation for a MySQL 5.0 production release later this year."

Zontar The Mindless writes "MySQL Community Edition 5.0.3-beta has been released. This version has support for Stored Procedures, Triggers, Views and many other features, including a number of security enhancements and stability fixes. See the changelog for a complete listing of new features and bugfixes. MySQL 5.0.3-beta is now available in source and binary form for Linux, Solaris, MacOS X, FreeBSD, Windows, and other platforms from dev.mysql.com/downloads/ and mirror sites. RPMs for Red Hat/Fedora and SUSE Linux are also now available for the first time in the 5.0 series. This is the first Beta release in the 5.0 series, in preparation for a MySQL 5.0 production release later this year."

WinDOOR asks: "As part of a small-mid sized family business, finding a way to rid ourselves from the dependence of using Microsoft products is a very daunting challenge. I've been searching for a good Linux based ERP/CRM software that's adequate for use with about 20 or so users and that can handle light manufacturing and POS type order inputting. I've looked at Compiere, but consider the Oracle tax to be one and the same as the Microsoft tax. We don't have the money nor expertise to design our own solutions like the big corporations that have switched sides. What packages or vendors have you had the most success with? Is Postgre or MySQL an acceptable database backend? Is there a viable replacement for MS Access yet? What language would be best to learn if I had to create my own solution? Do CS Students do this kind of work to pay the bills?"

Jan Wildeboer writes "The three most famous European authors of open-source software have issued an appeal against software patents on NoSoftwarePatents.com. Linus Torvalds (Linux), Michael "Monty" Widenius (MySQL) and Rasmus Lerdorf (PHP) urge the EU Council, which will convene later in the week, not to adopt a draft directive on software patents that they consider "deceptive, dangerous, and democratically illegitimate". They also call on the Internet community to express solidarity by placing NoSoftwarePatents.com links and banners on many Web sites."