Free Software, Get What You Pay For?

An anonymous reader writes "The Xooglers blog is running an interesting article on how big businesses may start out running free software but there is always the continued question of 'Should we go with something "real"?' at some point in their evolution. How often are technologies like PHP, Perl, and MySQL being pushed out once startups get managers who know nothing about the technology and only worry about name brands?"

Comments

sometimes that's the only criterion

[yagu]

I think it's unfortunate but in the IT world it is generally true there is no relationship between quality of software and cost . Some of the best software I've ever used has been free, some of the worst software I've ever used has been expensive.

IT and technology and particularly software can be (is) difficult to understand on many levels: functionality; efficiency; ergonomics; stability; etc. In a book (and God, I wish I could remember the title of this book -- one of my faves) talking about manipulating perceptions one of the discussions centered on the fact that when all other criteria are indeterminate or unavailable, it is human nature to assign credibility and worth based on price or cost. This is rife in the world of software.

Unfortunately, I see this as something taken advantage of rather that properly addressed.... sigh.

Re:sometimes that's the only criterion

[yagu]

shazbot, I hate replying to myself with followup, but for those who care, the book I mentioned above is: Influence: Science and Practice. It's a great book, it's worth reading, it illustrates exactly why people choose things expensive when they have no other criteria by which they can determine value, and, it is on topic! :-)

Go with something real?

[Dadoo]

I don't know about the rest of the people here but, when my company talks about "going with something real", they're asking whether we should stay with Fedora or go to Red Hat Enterprise Linux.

Re:Go with something real?

[GuyWithLag]

That simply illustrates the articles point: RHEL is expensive therefore it's considered real and not a toy.

Re:Go with something real?

[fatrat]

RHEL has a life of longer than six months, therefore it's real.

Re:Go with something real?

[tchuladdiass]

I have to chime in with a "me to" post. There is a lot of flak given to Red Hat over RHEL, but to be honest I'm glad this option is out there. This, and possibly Suse enterprise, is the one of the bigger reasons that I'm seeing wide-spread Linux adoption with the customers I deal with. It basically comes down to that management wants the "warm fuzzys" with the software that runs their business. And I'm quiet satisfied with how Red Hat handles their update policy (minimally invasive patches against baseline package versions, etc.). Also, if you want the same distribution for a lab box but without the Red Hat support baggage, you have the option of installing one of the rebuild projects (Centos, whitebox linux, scientific linux, etc.)

Re:Go with something real?

[turbidostato]

"RHEL has a life of longer than six months, therefore it's real."

So has Debian, but I bet those people wouldn't consider it really "real". It is not corporate-backed, hence expensive, so it can't be "real".

And that's exactly this story's point.

with a DBMS, quality is more objective

[Anonymous Coward]

Since data management has a complete, underlying theory, as well as several decades of best-practices, it's pretty easy to assess quality. Just look at the list of what's possible, and see if product X or Y correctly implements more of the items. If both products are free/open and are equally reliable and secure, what's the point in using the product with the smaller number of features?

Some of the comments here are pretty scary. I'd expect to hear this from a summer hire working on a content mangement system, not somebody working at a big company on a system that involves MONEY:

If you don't have transactions you just roll your own. It's actually not hard at all.

Whoa.. he must be pretty fucking good. Let's suppose you have a DBMS with, say, two major apps using it. You need to adjust a value in 500,000 rows in a single transaction. If one doesn't get changed, and the others do, you are totally fucked. If one application ever sees a state where all 500,000 haven't been changed, you are totally fucked. Power failures, application death, DBMS death, all should maintain this invariant (of not being totally fucked). Your assignment: do this at the application level with a MyISAM table.

I can think of ways to do it, but they aren't transparent to the applications. They involve basically implementing a DBMS layer in between the app and MySQL and putting implementation-dependent columns in the tables (bad design: only business-related data should be explicitly placed in columns).

We have been using MySQL on our site (www.degreeme.com) since its inception and have never had a single problem. ... I am .. a huge supporter of MySQL although I also have heard of instances where MySQL has had corruption problems. Hopefully the new version 5 has fixed that.

Yeah, hopefully... WTF?

People who push the "no transactions" FUD also forget that transaction support often reduces the reliability of applications

Unbelievable... maybe he means "instead of silently accepting crap, the application returns an error when an error actually occured".

The truth is that it's not the end of the world if you mess up a row or two in most databases

Whew, now I know what to tell my clients when they cut checks to their suppliers based on bad data printed on invoices.

Well, that depends on what you mean. In some sense we had no need for [transactions] at all, obviously, because we built the system without them.

Translation: we didn't have transactions, so we just left them out and crossed our fingers.

Question: if you aren't using transactions (or constraints, etc), how do you know if you had a problem? Do you hand-check the data? Do you have a script running ever 10 seconds to check?? "Consistency guaranteed by wishful thinking".

Oh well. Glad to see MySQL backpedaled on all the BS they used to spout and actually implemented some basic SQL features in the latest version 5.0.

Re:with a DBMS, quality is more objective

[moro_666]

rofl, you got my 5 cents.

are these people for real ?


People who push the "no transactions" FUD also forget that transaction support often reduces the reliability of applications


this really threw me off the chair. come on people, if you mess up 10 rows in different tables along in 1 transaction, you can fix it all with 1 blink of the eye (rollback) if 1 insert/update/delete/whatever fails ... where did that reliability just go ? it's the other way around, transactions create reliability, when errors happen, data should not be committed because it disrupts the whole system. imagine that a bank wouldn't have transactions and you transfer 100$ to your mom's account. after it has been discounted from your account, the disk is full and it can't be added to your mom's account. you will lose 100$ and the bank will show you the finger.


The truth is that it's not the end of the world if you mess up a row or two in most databases


Google AdWords definitely is not "most databases", if you create zillion dollar bills for microsoft for advertising excel, you can't mix up this row. actually you can, but microsoft will sue your brains out.

Mysql people that think that messing up some database tables are not a big thing oftect excuse themselves with "to err is human". I'd like to see if you think the same way if your car building factory thought that "to err is human", and forgot to add brakes to your car.

You really don't need to expierence loss caused by ignoring transactions more than once in your life to get your fingers permanently burned.

---
  I hate it when users of superior databases fud over mysql. But even more do i hate that mysql zombies don't realize how mindless and incompetent their own comments on the issue are.

Sweet tiny mysql has kindof added the features but i have seen no sign "stable" anywhere just yet, so using the latest mysql versions and it's fresh transaction is a bit better than no transactions but it still isnt "it", and you shouldn't trust millions of dollars into it's tables.

If i need performance and can avoid transactions without causing dataloss, i choose mysql for speed, otherwise i choose postgresql which still is superior in it's features but a bit slower in speed.

Re:with a DBMS, quality is more objective

[Zontar The Mindless]

MySQL 5.0 was released for production use in October. Some of the less well-known additions in version 5.0 include increased standards compliance in joins and an INFORMATION_SCHEMA database for accessing DB and schema metadata.

MySQL has supported transactions via InnoDB and BDB for several years. Further additions scheduled for 5.1 and 5.2 will make it even easier to use storage engine supporting transactions or whatever other features you care about.

BTW, MySQL now employs several Software Architects (and others) who are *extremely* well versed in SQL 99 and SQL:2003, as well as implementations in other widely used RDBMS such as Oracle, DB2, Sybase, Postgres, and SQL Server. They are tasked with giving developers (and the rest of us who work there) an extremely hard time when something doesn't support the SQL specs, and so they do. :)

MySQL 5.0 is not your father's MySQL, and 5.1+ will be even less so.

Stay tuned for further developments, and check out the MySQL 5.1 Manual-in-progress for a preview of what's to come.

Re:with a DBMS, quality is more objective

[neillewis]

Transactions make applications *fail* reliably, they don't make them *work* reliably. They simplify things at one level by bundling them up and abstracting them, so when they do fail it's hard to know exactly why they failed. The dreaded 'we can't do what you want at this time, try later' horribly vague error message is the result. Transactions are no replacement for serious fault analysis.

Re:with a DBMS, quality is more objective

[moro_666]

4.1.13 was also released officially and was pretty obviously a production release.

But surprisingly it had completely broken the charset matching in the database. so the queries that mixed utf8/local character sets (that worked before and worked again in 4.1.15) were completely broken, and along with them a nice bunch of applications. I was quite pissed to get a bunch of errors from restoring my database from dumps of the previous version. If they had planned this, they should have added tools to overcome the issue.

People that release broken charset issues in production release should now be out of the blue trusted with things as heavy as transactions ?

I call their transaction system stable when it has been "released by them" for a year and a bunch of users have used it more than a year without major problems. There is a thick thick space of testing & evaluation between production releases and stable releases.

Before that happens you can pretty easilly call it beta, you may trust your php webgallery descriptions into it, but you can't trust your bank records to it or any other kind of financially important data, now can you ?

All enterprise people want databases to be mature products that dont break out of the blue, they want databases to be rock solid. And they can't say that mysql with it's transactions is it right now , period.

I like mysql, i use it every day in some projects, but i wouldn't let it anywhere near my transactions for now.

Re:with a DBMS, quality is more objective

[Col. 2.7.0-default]

In the first place, MySQL has supported transactions in SEVERAL YEARS worth of production releases - these are not new to 5.0 or even to 4.1.

In the second place, you had the option of downgrading back to 4.1.12, and you only had to wait about 2 months or so for a production release with a fix for your bug.

I imagine if you'd been using Oracle or MS SQL, you'd've been waiting rather longer for your fix.

Open Source = Release Early, Release Often. Remember?

In any case, WTF have transactions to do with charset support?

Re:with a DBMS, quality is more objective

[duffbeer703]

If you read the fscking article, you'd know that Google wasn't able to implement Adwords without support for real transactions -- they wrote their own transaction layer instead.

The problem with doing things like that is that it is expensive to maintain (do you really want to dedicate programmer time to a transaction layer) and is bound to get crufty and harder to support as time passes. And when you're writing your custom transaction layer, you're tempted to add extra features that add some value, but creates huge headaches down the road when you need to move to a different DBMS (for whatever reason)

The difficulty Google had in porting their system to Oracle belies that.

Its a bad decision for other reasons, too. What happens when MySQL AB goes bust, or gets bought out by some shitty company like CA or IBM, and the quality and frequency of "official" releases goes down?

Re:with a DBMS, quality is more objective

[moro_666]

MySQL has supported transactions in SEVERAL YEARS worth of production releases


sry dude, that was the transaction engine of innodb itself, myisam tables had no transactions until this 5.x. if the mysql people didn't directly copy off the innodb people's invention in the transaction layer, then this is a fresh system, meaning that innodb's experience just doesn't count.

the character set example was just an example that mysql people can make really serious mistakes. if they make it in simple stuff like character handling, what confidence can i have in their transaction engine ?

ps. innodb was a lot bigger on the disk and slower than myisam tables right next to it. and without any real support for stored procedures and proper pointers the transaction support is often quite useless.

Re:with a DBMS, quality is more objective

[hey!]

Block quoth the poster:
People who push the "no transactions" FUD also forget that transaction support often reduces the reliability of applications

I think I can explain this howler. One common attribute I have found inexperienced programmers -- even the bright ones -- is failure to handle error conditions properly. Two things it seems to take experience to grasp are: (1) The problem isn't error messages, it's error conditions, therefore error messages are your friend. (2) Prominent and obvious problems are preferable to subtle ones, especially if there is no data corruption involved. In other words although you don't want your program to crash, it's preferable to crash with no data corruption than to avoid crashing at the cost of losing data. Granted, you want to handle failure gracefully, but sweeping it under the rug doesn't count as handling it at all.

Now, having a transaction fail and roll back everything is a BIG and OBVIOUS failure, and therefore preferable than having one part of a sequence of actions fail.

However, that said, I've run into database vendor confusion over certain aspects of transaction implementation. For example, I've run into this little gem in one version of one vendor's product: if you set a connection's mode to autocommit, but the database properties default the mode to chained, all statements on that connection are rolled back if it terminates abnormally. So, if a user spends all day on an application and forgets to log out at the end of the day, the connection times out overnight and he returns in the morning to find all his work gone.

This happened after the vendor had decided to add two features: nested transactions, implcit transactions on default chained databases to control atomicity of trigger execution. Autocommit connections were (inadvertently I believe) wrapped in an implicit transaction, so that all transactions, even explicit ones were by default wrapped in an implicit transaction. If the network is interrupted or the connection times out, everything gets rolled back. This was a huge, but probably uintended change. The first I heard about it was customers howling in rage.

Naturally, transactions are good, but like anything else you need to beware of the nuances of your platform's implementation.

Re:with a DBMS, quality is more objective

[alienw]

Are you still in high school? That's about how I thought when I was in 10th grade, learning some PHP. Real applications are a lot more complicated than that. I'm sure you would love it if, say, Amazon charged your credit card, but failed to actually add your order to the database (which is exactly what would happen if you didn't have transactions and some error occurred). This is actually a fairly mild scenario.

Seriously, read a database book someday. MySQL is a toy database which is great for things you would normally do with a flat file (content management, web counters, and other single-user applications). The only similarity it shares with a real DB is the syntax. It starts to crap out with very small amounts of data (hundreds of megabytes or so), and it really does not scale all that well. Nobody would ever think of using MySQL for anything serious, with a large number of users and/or data.

Re:with a DBMS, quality is more objective

[Anonymous Coward]

You misunderstand the basic concepts. Transactions are used to guarantee the data matches the business rules. They aren't there for "abstractions" or to make programmers' lives difficult. If an error can occur, it's the *programmers* job to report the error.

Think about it:

Application errors are transient. They may "look bad" but at the end of the day they are *cosmetic* issues. If an application fails but the data is consistent, that's a *cosmetic* bug.

Inconsistent data exists *until it is removed*. If you have a piece of bad data, and then 10,000 pieces of data depend on that bad piece of data, you are screwed. If *physical actions* (payments, shipments, declaration of war) happened due to that bad data, you are really screwed.

There are two types of data problems: bad, but consistent, data (human error for instance), and inconsistent data (order items without orders, for instance). The DBMS is perfectly capable of weeding out the latter if you program it correctly, so there's no excuse for not letting it.

The DBMS (the *foundation* the applications sit on) will survive many applications, in various "flavor of the month" languages and frameworks. The data must be kept consistent, and you can't allow inconsistencies just to avoid error messages. I guess that's not what you're advocating, but it is certainly what you're implying.

Fault analysis, or generally lack of understanding of the business rules, is a whole other problem, a rant for another day...

Re:with a DBMS, quality is more objective

[Anonymous Coward]

I always shake my head when I see programmers blindly accepting MySQL as a relational DBMS (it is NOT a RDBMS, and in fact doesn't even come close to satisfying Codd's 12 Rules). These are the same people who cannot appreciate how important it is to support transaction properties in the face of concurrent access and system failures: Atomic, Consistency, Isolation, and Durability. But I just have to laugh because I make pretty decent money cleaning up their messes. And on the subject of RDBMS and MySQL, how the hell do you develop an application that doesn't support database views?!!?

Free Software has already been paid for.

[Lendrick]

Time is money.

Free and/or open source software such as Linux, the GNU tools, Mozilla, Open Office, GNOME, KDE, MySQL, Apache, Postgres, and many other wildly successful tools have been worked on for countless hours by skilled programmers and designers. Whether out of the kindness of their own hearts, desire for recognition, or a business investment, people have spent millions of hours designing, developing, testing, and documenting Free Software. Consider for a moment how much it woud have cost to pay each and every one of those people for their time. That's the amount of money that hass been put into Free Software.

If someone gives you a mansion, you don't assume it's worthless because you didn't pay for it. The worth is still there; someone else already paid for it.

Free or not, irrelevant.

[Vo0k]

The nice example given in the article clearly shows clueless managers and not convincing enough developers.
In small startups you may pick it because it's free. In giants like Google you pretty much disregard costs of software purchase and just compare features. "Does it do all we need, well?" is the first and ultimately relevant question. All the others are secondary once the only competitors in the field have been estabilished. In case of databases there is no competition here, and all discussions should have ended at that first question. Does it do all we need, well? Yes, NOW it does, all we needed was added, it works fine. Does anything else do all we need, well? HELL NO! MySQL is an absolute master in the field of speed, when properly optimized beats everything and everyone (at costs of all the quirks we had to fight in the meantime). Everything else is much slower, and most choices will be simply way too slow for the expected workload.
Free (Gratis) or not, doesn't matter here at all. Open Source matters, if it doesn't do what we need, we can get it to do it, but that's not essential.

Managers who don't get it, won't work long. Simply because they will keep failing delivering working projects on time.

Re:Free or not, irrelevant.

[duffbeer703]

Nonsense. Where I work we have a DB2 cluster that runs over 6 billion transactions / day I doubt you can even do that with MySQL. We've setup dev environments with Postgre and MySQL, and Postgre generally blows away MySQL, unless you are using MyISAM.

And if need more performance and ISAM is acceptable, I'll cut out the overhead of the MySQL engine and just use Berkely.

Re:Free or not, irrelevant.

[orkysoft]

That's funny, I just saw an advertisement by Microsoft, boasting that their SQL Server program could handle 6 million transactions per day for Xerox...

Re:Free or not, irrelevant.

[Anonymous Coward]

"In small startups you may pick it because it's free"

I don't think that's the point.

I'd say that small startups would pick open source because managers (at least on IT companies) tend to be techies too, and they are not misleaded by some brigth brochures. Then the company grows and here comes a "professional" manager (PHB sp.) and then comes the bright brochures, the buzzwords and the "we only buy the biggest horse".

""Does it do all we need, well?" is the first and ultimately relevant question."

Of course it is. Problem being the one that takes the buy decition is absolute clueless about which product answers properly the question and which one is only selling brilliant colored papers under great benefit margins.

"Managers who don't get it, won't work long."

I think you need some reality checks.

What the fuck

[Anonymous Coward]

How the fuck can you all discuss this sort of thing when an INNOCENT Stan "Tookie" Williams has just been executed.

You people make me sick. I am leaving Slashdot forever.

Propz to GNAA

Re:What the fuck

[Anonymous Coward]

How the fuck can you all discuss this sort of thing when an INNOCENT Stan "Tookie" Williams has just been executed.

Ok, I have a real serious question here. What else would you expect people to do? Let's just assume for a minute that you're right and that Tookie was innocent. Under what circumstances would your average Slashdot reader know (or member of the general public for that matter) that Tookie was innocent? Isn't our judicial system, for all its flaws, designed to get to the truth in cases like this?

So let's take it one step further. Let's assume that the system was totally stacked against Tookie. If he were indeed innocent, how come most people have no clue who this guy was before a couple of days ago. I'm in my early 40s and keep abreast of current issues, yet I had no clue who Tookie was (although I'd heard the Crips name before). Don't you personally bear some of the responsibity for his death if you knew he was in fact innocent and didn't exhaust every resource at your disposal to help save him?

You and I would probably agree that we need to re-evaluate the death penalty in this country and do everything possible to overturn wrongful convictions and death sentences. There's a huge body of DNA evidence that has been used to free innocent people. If Tookie went to his death and there was solid evidence that may have freed him then I'd say that what happened to him bordered on "criminal". But from everything I read, what little last-minute evidence presented was weak. So what evidence do you have that would suggest that he was innocent?

Re:What the fuck

[ObsessiveMathsFreak]

Who the hell is Stan Williams!?

Free vs. non-free misses the point.

[rollingcalf]

MySQL isn't a real database. Certainly not before version 5.0, at least.

However, upgrading to a real database didn't have to involve an expensive commercial brand. They could have moved to another free database like PostgreSQL.

Re:Free vs. non-free misses the point.

[toddbu]

MySQL isn't a real database. Certainly not before version 5.0, at least.

I'm amazed that anyone would say this. Certainly MySQL didn't have certain features like views and stored procs pre-5.0, but these don't make for a "real" database. We've been using MySQL since 3.23 and haven't had a single failure and gotten every scrap of work done that we needed to do. We've leveraged the binary log files to make near-real-time replicas of servers on multiple machines and have yet to lose a single piece of data. It amazes me just how robust MySQL is. Of course I'm not a SQL purist. I think that DBs exist as part of a larger system and play a specific role. I prefer my app-level logic and data where it belongs - in the application driving the db. Call me old school, but I can live in a world without views and stored procs just fine. (I can even live without subselect, but I will admit that was a nice add-on in 4.x).

And how often...

[ArwynH]

are PHP, Perl and MySQL push out because they lack scalability and are hard to maintain?

I mean they are fine for small buisnesses, but when the buisness grows, so do their IT assests (databases, website, in-house software, etc). Large amounts of PHP & Perl code can be hard to maintain when compared to other languages like say Java. And as for MySQL... I'll let the other comments deal with that as I'm sure they will.

I should also be quick to point out that this doesn't mean OSS is bad, just that some of it's more visible members are overhyped and rely on thier brand-name to 'sell'. A good example of scalable OSS tech would be PostgreSQL as a DB, Plone/python for the web and java for large server/desktop apps.

And before someone mentions that java isn't opensource, I'll point out it is a language with an open(ish) specification and has an open source compiler/runtime which is good enough.

Re:And how often...

[Vo0k]

Perl is definitely a prototyping language. Rolling out anything serious in Perl is, let me put it that way, unwise. Performance about 10 times worse than C/C++, bug-prone syntax, source=executable approach, all great for quick and dirty fixes, not for serious projects.
PHP is good for making DHTML and that's all. If you want something serious, get a backend in a mature lower-level language, launch it through PHP to get things done, post results to pages generated through PHP. Anything more in PHP is definitely dangerous, and always a hack.
MySQL - Wrong, wrong, wrong. It's not a toy/prototyping/testing database. Sure it's easy to use and gratis, so integrating it with quick&dirty hacks in PHP or Perl feels natural. But it's like recreational riding a thoroughbred horse, a smooth, easy fun ride. And if you try to put the thoroughbred to a cart, the effects are definitely poor. Jumps? Okay, not impressive though. Cross-terrain, endurance, dressage? Sucks. It's not a versatile horse, and MySQL is not a versatile database. Just put the thoroughbred to gallop and you'll be far first in the means of speed, same about MySQL. Give it a highly specific, simple task where speed is essential. Not synchronizing sales over the whole corporation, not optimizing routes for train schedule, not managing an air traffic tower, where the complexity requires really sophisticated solutions. You put it to pull a single random ad that matches a keyword from a database of ten millions and increase display counter on associated field by one. And do it ten times a second. That's the kind of work which MySQL is made for, and that's where all the alternatives suck.

Re:And how often...

[sl8r]

But [MySQL is] like recreational riding a thoroughbred horse, a smooth, easy fun ride.

Heaven forbid we actually get to use technology that is fun, Fun, FUN! Now get back to maintaining that Fortran app, you insensitive clod!

Re:And how often...

[mangu]

Large amounts of PHP & Perl code can be hard to maintain when compared to other languages like say Java.

I often seen this "Java is easy to maintain" meme being spread, but with little to corroborate it.

I guess the real reason is that Java is so difficult to use that companies must have big teams organized before they start doing anything with it. With Perl you can easily write a ten-lines script that eventually grows into a huge system that no one seems to be able to maintain. Java is so clumsy that, if you are doing something with it, you always start by doing a lot of design before you code. Do the right design, have well established coding rules, document everything, and Perl can be just as easy to maintain as Java.

However, for really big systems that must work and be maintained, nothing beats C/C++. Java becomes painfully slow when the software grows into a million lines of code. Also Java lacks true pointers, it will not allow the programmer to use the right data structures in all cases. In a situation where a binary tree or a linked list is the best solution, nothing else will do. Faking it with array indices won't work, too slow for big (read "professional") systems.

In the end, there is no "right" language that will be the "best" in every circumstance. Perl is good for quick solutions that one needs every day. PHP is good for web-related stuff. Python is good for non programmers to use, a substitute for Basic. Java is good when you have a big team of programmers to coordinate. And C/C++ is the language to use when you need ultimate performance, eiher in program size or in speed. Each one of these languages can become easy or hard to maintain, the team coordinator must be aware of this and establish the right rules for each case.

Re:And how often...

[moro_666]

Java is so clumsy that, if you are doing something with it, you always start by doing a lot of design before you code.


Come again? You code without software design ? Tell me exactly how do you write C or C++ code without designing, or python code ? (and who let you near a computer in the first place ...)

Yeah you can definitely write a 5-line perl hack without designing, but if you write just about anything that is worth a dime, you better plan first and dont fix/patch/alter the stuff afterwards for years. Maybe it's just that i have not had 1-day or 1-week projects for a while, but i usually don't write one line of code on the first day at all. I prefer brainstorming, research and analyzing that will save me a headache and hopefully give me a few extra days at the end of the project to polish stuff.

Java is not slow when code goes into millions of lines, it's the coders who mess it up. I have seen working huge java code, i have written huge working java code which is fast. You are still being fooled around the fact that the swing gui used by java is slow and that java is often used along with oracle that is slow. Don't let this fool you. Java is as fast as C++ in pure arithmetics. If you really need some freaky speed boost, you can write the speedwhore code in C and JNI to it. You can also have your pointer tribble there if you need it, but Java is designed so that it would avoid the need pointers in the first place (all the regular objects in java are actually pointers). Besides, Java is the only thing that will work on almost any platform. And this is what counts in year 2005.

C and C++ are not platform independant and that is what you rarely want to use nowadays. You can never know if your stupid management wants to run this on windows or linux or even sun tomorrow. And once you get your code compatible with all major os's , the source is so #define'd that it's nearly impossible to manage. I won't even start about the library dependancies.

Just use every language where it belongs and don't go around bashing java or anyone else with accusations that can't take 1 mm of water.


Oh well, ok, you can bash php, i have nothing against that.

Re:And how often...

[Welshalian]

Perl is definitely a prototyping language. Rolling out anything serious in Perl is, let me put it that way, unwise.

I assume you are writing this statement on a computer that contains a CPU. You'd be amazed if you saw the tooling used to build said CPU and all the other extremely complex, expensive and risky chips that make up your computer. A few very expensive CAD tools bought from one of the three/four established vendors for top $$$/year, and masses and masses and masses of Perl (or *hack spew* TCL). I call that anything serious.

Re:And how often...

[Vo0k]

Do you call that -wise- though?

Re:And how often...

[Welshalian]

Let me put it this way: most of the work of a chip designer, today, is analysis of GB and GB of information that is generated by complex tools such as 3D field solvers, simulators, etc - all of that is done in Perl. I can't think of a comparably ductile programming language for people with a non-CS degree. You could surely teach an EE to write those programs in C++ using Boost's regexp package to achieve the same goal, but show them a 70-line-long g++ error message and their head would explode.

Perl allows non-CS users to write moderately complex programs such as those needed for chip design whilst shielding them from some of the intricacies of other languages. I think that using it for that purpose is a wise thing, as it enables non-specialist to make progress without becoming experts in decrypting the behaviour of templates, reflection or multiple inheritance.

Re:And how often...

[mangu]

Come again? You code without software design ? ... Maybe it's just that i have not had 1-day or 1-week projects for a while, but i usually don't write one line of code on the first day at all. I prefer brainstorming, research and analyzing ...

Then you aren't a very efficient programmer in an enterprise environment. A large proportion of programming projects today are related to some data mining / data recovery situations where you need to get some result quickly. You need to recover something from a set of old files to include in a report that will be used in the management meeting this afternoon. How do you "brainstorm, research, and analyze" that? Or maybe get some data that has been published in a government agency web site. Try to collate data from a hundred different pages using cut-and-paste into an excel spreadsheet. A quick and dirty Perl hack will save your day in those circumstances, which happen every day in any corporation.

Just use every language where it belongs and don't go around bashing java or anyone else with accusations that can't take 1 mm of water.

That's exactly what I said in the first place: there's a language that's right for each use. I didn't bash Java, I only mentioned that it's not very good for very small projects and it it's not very good for projects where you need specialized data structures. Pointers should be used sparingly and with caution, but they weren't invented solely to confuse software maintenance people. Take the two examples I mentioned, linked lists and binary trees. How do you create and maintain those without pointers?

As you mentioned, for projects that are big or are small at the beginning but could grow in the future, a careful design is needed from the beginning. And a careful design should consider the data structures before you choose a language for coding. After you have your data structures well designed, you start with the algorithms. Only then you will choose the language best suited for the implementation of the code.

I certainly do not "bash" Java, but it's no magic bullet either. I wouldn't use Java for a quick script, I wouldn't use it when I need a binary tree, I wouldn't use it where a careful consideration of the problem shows that multiple inheritance is the best solution, etc.

Re:And how often...

[Anonymous Coward]

That's exactly what I said in the first place: there's a language that's right for each use. I didn't bash Java, I only mentioned that it's not very good for very small projects and it it's not very good for projects where you need specialized data structures. Pointers should be used sparingly and with caution, but they weren't invented solely to confuse software maintenance people. Take the two examples I mentioned, linked lists and binary trees. How do you create and maintain those without pointers?

public class TreeNode {
public Object data;
public TreeNode left;
public TreeNode right;
}
 
public class ListNode {
public Object data;
public ListNode next;
}

Am I missing something?

Re:And how often...

[Billosaur]

I mean they are fine for small buisnesses, but when the buisness grows, so do their IT assests (databases, website, in-house software, etc). Large amounts of PHP & Perl code can be hard to maintain when compared to other languages like say Java. And as for MySQL... I'll let the other comments deal with that as I'm sure they will.

I worked for one of the largest financial companies on the planet and one of the biggest Perl shops around and I can say without fear of equivocation that properly written, maintained, and managed Perl will knock the socks off anything. Java's gotten better in recent years, don't get me wrong, and has a speed advantage over Perl, but the fact is when you need something quickly, Java is not your language. Perl will forever be labelled a "scripting language" because it's more fluid and free-form than most languages, which is its greatest strength.

Ultimately it doesn't matter if you buy your softwware or get it for free. Programs in any language are only as good as the programmers who write the code. And because you need top-notch programmers, free software ends up costing you money in the long run; you save on the up-front cost but the back-end cost may be much higher depending on what you're doing. Nothing is cut and dried.

Re:And how often...

[mangu]

public class TreeNode {
public Object data;
public TreeNode left;
public TreeNode right;
}

public class ListNode {
public Object data;
public ListNode next;
}

That's a textbook example of a binary tree. In real life one needs much more. For instance, how do you create a binary tree from existing data? Maybe you already have a very big structure, which you want to re-organize as a binary tree, without having to duplicate the data records. In real life you need to do both "shallow" and "deep" copies of data. Sometimes you need to create references to existing data very fast (shallow copies), sometimes you need to create a duplicate of existing data (deep copies). Sometimes you need to access a record in different ways.

For instance, I once wrote an orbital analysis software where a state is represented as two 3d vectors, one for state and another for velocity. I used several different libraries for manipulating these states. At some times, to use a plotting library to show the evolution of position in a graph, I needed to have all position vectors continuously in memory; at other times, to solve the differential equations, I needed to have the position vector together with the velocity vector. A commercial satellite has a lifetime in the order of decades, with each orbit calculated in samplig periods of minutes. When doing orbital analyses of satellite lifetime, one may need to keep a few million vectors, each of which may have to be used by several different third-party libraries.

There are many efficiency related problems in deciding whether to copy or alias data. In C/C++ it's up to the programmer to choose well for each situation. The compiler will not limit the programmer, but the choices must be well done. No compiler will optimize a badly designed data structure.

Re:And how often...

[ArwynH]

I'm in complete agreement with you. I've too had experience with well-maintained perl code. I've also had experience with poorly maintained perl code and java code. Out of the 2, the perl code was far harder to understand. The reason for this was due to perl's free-formness. That's not to say free-formness is bad, just if misused it can cause nightmares for later maintainers of the code.

What I meant in my original statement was that while it's possible to have easily maintainable code in any language, it's easier to mess up in some that in others. And since you should always asume the worst, especially with small companys...

That is when they die

[Nice2Cats]

How often are technologies like PHP, Perl, and MySQL being pushed out once startups get managers who know nothing about the technology and only worry about name brands?

That is when they die. Seriously. Look at these guys, what was their name again, ah, yes, Google -- kinda silly but you know these computer types -- anyway, where would they be if they had dumped their Linux and stuff for Windows XP or even OS X (the Unix of the great GUI but crappy thread performance)? Not trading at about $400, that is for sure.

Don't worry. This is evolution in action: The clever ones, the more efficient ones survive. Those who pay $400 for Microsoft Office instead of using OpenOffice for free are not efficient. If Open Source can keep the legal playing field level, the rest will take care of itself.

brand name?

[EngMedic]

perl /is/ a brand name, pretty much.

Is it really that much better?

[TheLink]

Sure MySQL has supported transactions for years, but tell me what is the best way to load 20GB of data into innodb tables?

I tried a fair number of recommended ways, but still got stuck with the disk writing at 2MB/sec even at the _start_ (which is not even the actual rate data is flowing into the DB). Not even got to the "slow down due to big index" part. Yes I did set the sync thing to 0 (sync max of once a second, instead of on every commit).

I was seriously considering putting the innodb log files on a ramdisk ;).

I didn't wrap everything in one huge transaction, because according to the docs if "stuff" happens it takes 30 times longer to roll back everything. So if you are 3 hours into inserting something, it'll take 90 hours to rollback... There are workarounds, but I've looked at the workaround, and "Thanks but no thanks".

In the end I loaded the data into MyISAM tables instead (disable keys then enable keys).

With MyISAM if you want to add an index, MySQL makes an entire copy of the table. The manual doesn't say that Innodb is different, but I haven't tested it.

Fortunately it's what I'd call a small site.

Imagine if you had terabytes of data and to break the news to your boss, "yes sir, we need to double our storage capacity", "Yes sir, our data is only growing at 200GB a year, but we still need to add 2 more TBs _now_ in order to add an index".

By the way by default MyISAM tables are limited to 4GB... Talk about thinking ahead.

Whereas with innodb, say you have a huge ibdata1 file, how do you shrink it once you're done with some tables? While you can have tables in ibd files, it's still something new - the previous reported bugs sure don't give me a lot of confidence in it.

I use postgresql at home. One day i should go load up 20GB of data into it and see how long it takes.

Works for California Wineries, as well

[mosel-saar-ruwer]

In a book (and God, I wish I could remember the title of this book -- one of my faves) talking about manipulating perceptions one of the discussions centered on the fact that when all other criteria are indeterminate or unavailable, it is human nature to assign credibility and worth based on price or cost.

Many California wineries discover that they can't sell their juice at its natural pricepoint of $19.95, but if they add a few choice words to the label, like "Private Reserve", or "Luxury Cuvée", and jack the price [or at least the MSRP] up to $99.95, then the juice flies off the shelves.

[BTW, this strategy doesn't work as well for European wineries, as their clientele is, by and large, a little more cynical when it comes to things vinous.]

I'm Happy

[4of12]

that the question of when and why to migrate to and from free software to commercial alternatives is being asked, that it can be asked.

It means there are alternatives, that everyone can make a freer decision with more options, that free software provides a baseline commodity level that benefits everyone, and that commercial providers can compete on providing genuine value added on top of this baseline offering.

right tool for the job

[willCode4Beer.com]

I agree with your fundamental point, that the language chosen should depend upon the objective. However, I strongly disagree with some of your points.
C/C++ for really large programs - this gave us wonderful bugfree things like Windows, and Office. The power of the languages break down after a few hundred thousand lines of code as they turn into maintenance nightmares. Data dictionaries anyone?
Java hard to use? - This is the first time I've ever seen anybody say that. Java has its issues but, if you think Java is hard to use then C or C++ will be way beyond reach.

Lets really break it down. Loosely typed languages tend to be best for small quick and dirty projects. Strongly typed languages tend to be better for large projects. Speed should just be treated as a requirement. Some things are faster in Java, some are faster in C++. The old law works here: make it work, then make it fast.

Ease of code maintenance will always depend upon the dicipline of the development team, regardless of language. However, strongly typed languages will tend to promote practices of more maintainable code. Loosely typed languages give quicker development in the short term.

When a strongly typed language is used the compiler becomes a powerful tool in the "prevention" of bugs. If you define a method in Java to take an integer, the compiler won't let you give it anything else. If you define a method in C to take a pointer, you can give it a pointer to anything. Perl is so loose about its types that its painful. The larger a program the more this becomes an issue.

Your points about million line Java programs being slow, or about java data structures not working in "big" "professional" systems just shows you might have a bit a learning to do.
Some "big" "professional"'s using Java in large applications:
RealNetworks - Tomcat/BEA
Citibank - Java Web Server (very old version)
Bank of America - IBM WebSphere and Sun One
United States Navy - IBM Websphere
United States Airforce - Tomcat/IBM Websphere
Neilson Reasearch - IBM Websphere
Fidelity - IBM Websphere
Blue Cross Blue Sheild - IBM Websphere

Now these may not all be the best solution in each case but, these are some of the ones I have experience with.

A database is NOT a stupid data container

[wieck]

Of course I'm not a SQL purist. I think that DBs exist as part of a larger system and play a specific role. I prefer my app-level logic and data where it belongs - in the application driving the db. Call me old school, but I can live in a world without views and stored procs just fine.

Where to implement your business logic (app vs. stored proc) is a preference question, granted. But having transactions and referential integrity constraints is not.

Transactions and constraints are your last line of defense against data inconsistency caused by application bugs. Think of them as sort of Assert-checking. Your application code has the responsibility to check, that the customer placing this new order actually exists. But if you forgot to SELECT FOR UPDATE in only one place, you created a bug that will probably not be exposed by any software testing. It is waiting for a race condition with the cleanup procedure that removes inactive accounts once a month colliding with the customer returning after 2 years.

If you define the constraints that represent the relationships in your data, your database will ensure that the above bug does not create an inconsistent state. One of the two transactions will fail and it might cause the application to do some ugly, ungracefull stuff. But trust me, you really want it to do that in this case.

Call me old school, but for some data I prefer belt & suspenders.

Jan Wieck

Re:A database is NOT a stupid data container

[toddbu]

Call me old school, but for some data I prefer belt & suspenders.

I'd love to see that. :-)

I totally agree on the transaction side. As for constraints, we use them where they make sense. Usually they fall within some kind of logical boundary that is meaningful at an application level. For example, our billing system uses referential integrity to ensure consistency, but when the billing system references a customer record then we break the link. We do this so that we can treat our billing system as an independent entity that can stand alone. We can upgrade it whenever we want without impacting the other systems.

As I mentioned before, we've been using MySQL since 3.23, and innodb has supported both transactions and foreign keys for the entire time.