Stored Procedures - Good or Bad?

superid asks: "I'd like to get opinions and real world experiences that people have had with database centric applications that rely extensively on stored procedures. I believe that most enterprise class databases such as Oracle, MS-SQL, PostgreSQL, DB2 and others implement stored procedures. MySQL has been criticized for not supporting stored procedures and will be adding them in MySQL 5. The ANSI-92 SQL Standard also requires implementing some form of stored procedure (section 4.17). So, I'm asking Slashdot readers: if you were architecting a highly data-centric web based application today from a clean slate, how much (if at all) would/should stored procedures factor into your design? Where are they indispensable and where do they get in the way?" "The arguments for stored procedures are pretty straightforward: 1) Centralized code; 2) Compiled SQL is faster; 3) Enhanced security (as our application is over 15 years old, and consists of much legacy code, reimplementation and feature creep that now includes over 3000 stored procedures). At one time we had a client/server architecture so those three advantages were relevant. However, in the past 4 years we have moved everything to web front ends and I have argued that this is no longer true. Does it really matter if my business rules are centralized in stored procedures or in a set of php/asp scripts (ie, in the web tier)? Is it really important to shave compilation time when connection and execution times dominate? (and overall response is ok anyway?) Since the focal point is the webserver, shouldn't security be done there, rather than the DB?

In addition, you either have to have a dedicated T-SQL or PL/SQL coder who then is the weak link in your coding chain, or your pool of developers must become fluent in both your scripting language of choice as well as the SP language. I have experienced both of these approaches and found this to cause bottlenecks when 'the database guy' is unavailable and learning curve problems (bugs) with new coders getting familiar with the db language.

Finally, after staying with our DB engine choice for all these years we are acknowledging that they may not be around forever. Management has asked us to look into migrating our data and business logic to another DB choice. We'd sure love to just be able to point the web tier at a new data source but that is unattainable due to a convoluted tangle of db specific code."

Comments

I don't use em unless I have to

[ph4rmb0y]

I like to keep business logic in one place as much as possible. You are almost assured to have some in your app, so I try to keep all logic there.

Stored Procs and triggers make can make the code simpler and more efficient, but spread out the workings of the application and unless properly documented, more difficult to understand.

Just my $0.02 CDN

Re:I don't use em unless I have to

[nz_mincemeat]

Stored Procs and triggers make can make the code simpler and more efficient, but spread out the workings of the application and unless properly documented, more difficult to understand.

As a developer I've found otherwise. The reason being that when you're examining a bit of code with embedded SQL you often lose context of what table structures it is trying to refer to.

Of course my DBA is very good in helping out and training the developers in SP usage, so YMMV.

Re:I don't use em unless I have to

[flngroovy]

There are many reasons to use them. Performance is always one of the first reasons that people give. When we have applications that run in our enterprise, it is very easy to update a stored procedure if you find a problem. Your clients do not need to update their software when this happens. Personally, I rarely put SQL commands in my code. I leave it all in the procedures on the server. SQL 2005 will allow the use of C# in stored procedures which should increase their usefulness. In a team environment it is easy to have someone work on the procedures while you work on the code. (I know this can be done with interfaces and classes, but this is also very convenient)

Re:I don't use em unless I have to

[severoon]

Stored procedures aren't good or bad...they just are. Passing a value judgment on whether they're good or bad is completely dependent on the situation.

I would argue that the three main points you made in favor of stored procedures are not points that apply to every case (I don't think they were intended to, either, but hear me out):

1. Centralized code. There are lots of ways to divide code up. I would argue that good n-tiered designs for web apps already use methods of organizing code into deployable/organizational units (layers, design patterns, component object models, etc) that render irrelevant the contribution that stored procedures are able to make in this regard.
2. Compiled SQL is faster. This is purely a performance enhancement. Performance enhancements like this belong at the back of the development cycle once you can measure the performance of the app; maybe it's perfectly acceptable. If it's not, then you look for bottlenecks and focus the energy where you get the biggest bang for the buck. If the biggest bottleneck is compilation of SQL, then stored procedures is your answer. Until the performance analysis is in, though, I think implementing performance enhancements maximizes investment of time and resources and minimizes return. Besides, much of the time a good, scalable design makes performance considerations irrelevant...you can have the cleanest code and just scale it up over hardware (within known constraints, of course) until performance meets requirements.
3. Enhanced security. Depending on stored procedures as a key element of security is obviously not desirable. Having said that, in your particular case, though suboptimal, you have to look at the big picture to see if the business justification is there to leave it in for this reason. Having said that, I would try to design the app, security-wise, as though stored procedures don't exist to whatever extend possible. Just as with the business rules, you should be free to change DB vendors from a security standpoint as well.

I would argue that business rules and business logic should be implemented in a vendor-independent way. Also, I would implement the business functionality of the app so that it can support a web front end, but also someday a desktop UI, a programmatic web services front end, etc. That's the soul of n-tiered architectures, they're supposed to bring that kind of flexibility along with the use of tiers...this kind of flexibility is the point.

If an app is not flexible in this way but claims to be an "n-tiered architecture", I'd argue that it is only nominally so. Looking like an n-tiered app without providing any of the benefits is a Pyrrhic victory for the architects and designers. That would be inconsequential except for the (usually large) investment of company resources.

Re:I don't use em unless I have to

[gRa]

Putting business logic on on eplace as much as possible ist good because it is easier to overlook and to maintain. This however is only true for the source code.

If you can organize your project in a manner, where an automated system deploys the compiled or generated code to different places, you can have the advantage of logically centralized business logic, and the advantage of its distribution to many places. The horizontal distribution is relatively easy (clustering), vertical is still quite difficult - but not undesirable. It would be nice, if one line in my sourcecode would be automatically propagated to JavaScript in the Browser, compiled code in the middle tier and stored procedure in the Database.

Re:I don't use em unless I have to

[Atrax]

> Compiled SQL is faster.

Actually, this depends on the database in question these days. SQL Server 2k does a pretty good job of keeping embedded queries hot, so the performance gain is waay less impressive than it was in, say, SQL 7.0

[cue MSSQL Bashing in 3...2....1.....]

Haven't really kept up with competing RDBMSes recently, but it wouldn't surprise me if competitors were also narrowing the gap

> Enhanced security

One incredibly common security hole being SQL Injection, I have to agree with this, but with the following caveat :

I've seen developers create stored procs which do a bunch of string concatenation within the SP, then EXEC the resulting string. This is just as injection prone as doing it in a script in the first place, but the developers in question often cite SQL injection as their one of their reasons for using SPs in the first place.

Again, back to the problem with lack of knowledge on the developer's part causing security holes, rather than the platform.

Re:I don't use em unless I have to

[innosent]

The other major issue is security. If your app has the ability to view tables in the database, then breaking your app, or finding the login and password your app uses compromises the entire system. It is much better to implement security policies in the database/stored procedures, and only allow users/apps to call stored procedures, possibly even using the procedures to validate login information on each procedure call. If someone managed to get the (non administrative) password to your database, would you rather have them be able to do anything they want (or even just view anything they want, such as credit card or health information), or have them only be able to do exactly what they were able to do without the password?

Especially if login information is used as a parameter (I always use username and password as parameters to look up access levels in all user-executable scripts for this reason), if the user cannot access a single table in your database, then they can only do what the stored procedures allow them to do. This allows you to have a central place where security logic is executed, and business logic may either be in the procedures (ideal), or in the app. Just remember, leaving business logic up to the app may pose a risk if the app has a flaw, if the procedures allow the user more access than the app does. If your database security model has more flaws than your app could, then maybe you should consider switching databases.

Re:I don't use em unless I have to

[1001011010110101]

I'm an Oracle DB guy (trying to dive into J2EE, better late than ever), and while I'd agree with your general point of view, I think you are missing something:

1. Centralized code. There are lots of ways to divide code up.

Sometimes is nice to assure that no matter what tools access your data model, only valid/complete information gets into it (usually this is the case with schemas that have existed in a company for some time). Sometimes there are different front ends to the same data, incoming interfaces, etc. They tend to grow on databases as time passes by :). It also helps you reusing business logic in different technologies/applications.

2. Compiled SQL is faster. This is purely a performance enhancement. Performance enhancements [...]

Usually coding thru the stored procedures/triggers is a good way to have access to all the features in the database and programming languages, performance and productivity wise...some things are not available thru a interface to external engines and you need to be pretty close to the engine to get them. These things can make a huge difference.

3. Enhanced security. Depending on stored procedures as a key element of security is obviously not desirable.

Stored procedures are a very good idea if they are the only way to access the related data schema. Just be assured to deny direct write and/or read access to the tables.

[...]Should be free to change DB vendors from a security standpoint as well.

One thing that I've seen around and found quite appropriate: When deciding whether to choose exploiting or not DB specific features, the DB cost is a factor. Why buy a brand name DB and not exploit the features it gives you? The cash you spend on it should be used for something, either performance/resource-wise, or easier development. Otherwise just use an el-cheapo engine like postgress or mysql.

Re:I don't use em unless I have to

[Gilk180]

This is not a problem with the SQL, but with the coding. It makes things very easy to read and more easily maintainable if you isolate each SQL query/command or small sets of queries/commands in their own functions/class.

Not only does this make the main logic more readable, it allows you to modify the SQL implementation without touching a lot of code.

For instance if you split one table with a column with a few large pieces of data that are repeated many times into two tables, one with a reference to a key for the large data pieces. Or if you add an index that makes a different SELECT statement faster than the current.

Re:I don't use em unless I have to

[1001011010110101]

I've seen developers create stored procs which do a bunch of string concatenation within the SP, then EXEC the resulting string. This is just as injection prone as doing it in a script in the first place, but the developers in question often cite SQL injection as their one of their reasons for using SPs in the first place

Only if any part in the string is tainted (like deriving from user input). Otherwise dynamic SQL is pretty safe. Otherwise I agree, I think pearl is pretty neat on this aspect.

Re:I don't use em unless I have to

[Anonymous Coward]

I would argue that business rules and business logic should be implemented in a vendor-independent way.

Let's be real. Not too many people very often suddenly decide to switch enterprise database systems. This is a red herring.

Really, it comes down to data security. Do you want someone with isql/osql/sql*plus, etc., to figure out what the app's login/password is to the database, and then go in unfettered through a tool like this into the database? That is the risk of depending solely on the application for your security model.

As far as vendor-neutral, what if you think of the stored procedure layer as the implementation of an interface? The stored proc definitions are the interface specification, and your app merely hooks into the interface.

Re:I don't use em unless I have to

[AnyLoveIsGoodLove]

Ahhh business logic at the database... why should you do it? Let's start with some open source examples: Compiere. This is one of the most successfuly opensource products based on Oracle in the market place.... but did you ever read the mailing list for database independance? Yup it's a year's worth of mail, but no significant movement. I honestly don't thing there will be. Think about the steps it would take to change:

1) Develop Business Logic Tier (Jboss like tier). Any ever do this? Yup. Lot's of effort.
2) Convert all SQL to standard SQL (hmmm much easier to start this way. This ain't easy either)
3) Test againts all major DBs

Hmmm sounds like a lot of work. So class.. the lesson is:

Do you want to have an OPEN application that is platform independant? Do you want a product that can participate in the enterprise? Do you want to avoid the DB / OS religion wars?

Great then seperate your logic from your data from presentation tiers.. it's simple design and I've made a lot of money off of other people's lazy code

(On the other hand if you have a small app that no one out side your department will use, break out the MS Access. Funny story, I know a federal government agency that has 1200 (yes you read that correctly) Access dbs in production... BTW.. they fear centralization... surprise...

Re:I don't use em unless I have to

[1001011010110101]

Pearl being my coding sister :) (Of course, I meant Perl)

Re:I don't use em unless I have to

[hagardtroll]

I agree with the parent, but I would like to add my $.02. Putting more code in the database takes away from the datbase from doing other things. At my company the DB is used by ALL applications. Putting logic on a middle tier servew allows you to seperate the load. Also it is easier to add more machines to the "middle tier" in a farm/cluster, while in our case adding more database servers cost more $ since we use Oracle and there is a cost per cpu. Also many applications rely on dynamic SQL. I've seen applications that create sql via PL/SQL (Oracle again.) When a middle tier language such as Java or C# provide much better string handling. Just my $.02, now we're up to $.04.

Re:I don't use em unless I have to

[pHDNgell]

There are many reasons to use them. Performance is always one of the first reasons that people give. When we have applications that run in our enterprise, it is very easy to update a stored procedure if you find a problem.

Wow, that almost sounds like a bug. I hope you have a good process around that.

That's where I've always had a problem. Rolling out a new version of my code requires some DB support. The DB support has to be synchronzied and sometimes reversable. When dealing with stored procedures, there are two simultaneous code pushes that must occur during the same schedule. We have the code that updates our servers, and the code that code relies on in the database.

Nowadays, we have a package of code that goes out which may require some schema changes or a bit of migration, but that's about it.

Personally, I rarely put SQL commands in my code. I leave it all in the procedures on the server. ...and how do you access those commands? What I did was create a language that allows you to express a query in pretty much plain SQL, define formal inputs and outputs (with types) and generates a class to interface that particular query.

Instantiating the class in code looks like dealing with any other class. You get an instance, call a few setBlah methods with the appropriate types, and tell it to go. It validates you provided enough parameters, and submits the query for you.

It acts a lot like stored procedures, except they act more like application code, *and* you can chain them together transactionally.

I built this abstraction layer with the intention of using it for accessing stored procedures originally...however, I found that the DB I was using wouldn't allow me to call more than one SP in a transaction. That was an immediate show-stopper.

SQL 2005 will allow the use of C# in stored procedures which should increase their usefulness.

Wow, not to me. All that does is make it so you can never ever use a better database. Our current application runs on SQL Server, but we have customer requirements to run on Oracle. Good luck with that when your DB has to run C# application code.

In a team environment it is easy to have someone work on the procedures while you work on the code. (I know this can be done with interfaces and classes, but this is also very convenient)

Sure, and on my team, the DB guy can write and maintain the classes in this language I made without knowing the language the rest of the application is written in (another original design decision). We end up doing most of them ourselves, though.

Re:I don't use em unless I have to

[Atrax]

> Only if any part in the string is tainted (like deriving from user input).

That's what SQL Injection IS. The whole point is tainted input. Perhaps I wasn't clear about that being part of what I was referring to. let's see...

here's an explanation of SQL injection for those not aware of it. Google also shows up a ton of useful links, a number of which are PDFs so I'm not linking them

Re:I don't use em unless I have to

[Dejohn]

I agree 100%. Stored procedures are nice in certain circumstances, but I try to aviod if I can.

Situations where I use stored procedures:

1. Identical function will be used in many different areas of the software or by seperate software altogether.
2. In places where it would be too dangerous for an inexperienced developer to play around. I.e. when working with a central database that is used by a lot of people and the new app is not the sole tool in use to query it
3. When security is a major concern. In this case, however, one would wonder why you don't trust your developer...
4. Speed, but this seems to be a rare problem that a stored procedure can solve.

Aside from the above, I steer clear from stored procedures because it isn't worth it otherwise...

Re:I don't use em unless I have to

[Hangtime]

Enhanced security. Depending on stored procedures as a key element of security is obviously not desirable. Having said that, in your particular case, though suboptimal, you have to look at the big picture to see if the business justification is there to leave it in for this reason. Having said that, I would try to design the app, security-wise, as though stored procedures don't exist to whatever extend possible. Just as with the business rules, you should be free to change DB vendors from a security standpoint as well.

SQL injection is a very large problem in the enterprise. Stored procedures facilitate better safety. Yes, you can test for all the characters in your code but as soon as your app and password are compromised their is open reign on your database if you gave it data reader, data writer, dbo or heaven forbid SA. All new applications are build on stored procedures not only for data modfication but also SELECT queries. The application itself has no rights to the underlying tables. This ensures if indeed the application was ever compromised the most anyone could do is what the application does today. Also, they would have to figure out the XML strings to manipulate the data with a stored procedure.

Re:I don't use em unless I have to

[cervo]

I agree the business logic should be in one place, but is it not true that the structure of the database itself holds business logic in it?
When you think about it each table extracts the data structure of your business logic. Each Primary Key and Foreign key sets the relationships for your data which is business logic as it is. So you might as well fully store the business logic in the one level. This way if the DBA decides a faster structure to optimize data access he/she can just adjust the stored procedures and the multitude of applications will work unchanged.

Then again, if this database is a one shot deal just for your one application and you serve as the DBA/Programmer it doesn't really matter. If you think more than one application may access the database, or you aren't also the DBA then do yourself a favor and abstract the business logic away with stored procedures, this way the rules can be implemented against all the applications, and as the database structure changes rules can be adjusted.

Dismissing stored procedures altogether is pure foolishness, but using them all the time is also foolishness....There is a balance that must be struck depending upon the needs of the application.

Re:I don't use em unless I have to

[Feyr]

i have another, which is honestly the only reason i ever used an SP for:

5. enchancing an already existing program for which you don't have the source code

Re:I don't use em unless I have to

[1001011010110101]

Oh, it wasn't you , it was me, I wasn't familiar witht the concept. Why would anyone use Dynamic SQL instead of variable binds to replace parameters in a query is beyond me. Well, I dont know if such thing exists outside Oracle.

Re:I don't use em unless I have to

[lewp]

Let's be real. Not too many people very often suddenly decide to switch enterprise database systems. This is a red herring

Not exactly. The point isn't to be able to just hop between databases at will. The point is that if you wake up one day and your vendor no longer exists, has decided to start charging more than you can afford, or has discontinued their database product and won't give you quality support that you can switch to another database without having to rewrite a large portion/the bulk of your application.

I do think stored procedures have their place, so I don't necessarily disagree with your post. I do think writing off avoiding vendor lock-in as a valid argument is a bad idea, though. It's one of the best weapons you have to prevent your vendors from stomping all over you (not that this has ever happened *cough*).

Re:I don't use em unless I have to

[acebone]

Most of the time the DB isn't even accessible from the outside. Why worry about being rooted then ?

Re:I don't use em unless I have to

[fprog]

From my experience...

1. Create the application with SQL within
the code inside some custom DBI object layer.

2. When the query are stabilized inside your DBI functions, morph them to be "prepared" or morph them into stored procedure as needed.

3. This means that your DBA is not the weakest link and tight deadlines can be met.

4. It is better before going in full production to have stored procedure, but it's not required.
You can perform some query analysis to see
if you can improve performance, by adding indexes, sorting, limits, paging, etc.

5. It is better for long term maintenance to have everything in stored procedure.

6. The reason is you know that if you make changes, you won't break any SQL code, since you have access to everything. In big companies,
sometimes this golden rule is broken and you
have to patch tables with some white gloves
and retest everything crossing fingers.

7. Security should be implemented many ways.

8. You should think of a good compromise between getting the app out of the door, having good performance and long term maintenance and security.

9. I've seen too many companies which have a DBA weakest link that if that guy is sick for few weeks the entire IT department collapse.

10. Like I said it's good if you can use stored procedures but don't go crazy about it.

Re:I don't use em unless I have to

[acebone]

> Let's be real. Not too many people very often suddenly decide to switch enterprise database systems.

No, but maybe you are doing a product with a diverse customer base, and you'd like their choice of DB to not be an obstacle ?

The very second you go stored procedure, you're choosing your DB platform, is that a good idea ?

The logic has to lie somewhere, so why put it in the one place that will constrain your freedom to choose ?

Re:I don't use em unless I have to

[Anonymous Coward]

Sometime people create queries on the fly not just use input as parameters.

There are some inexperienced developers who just not familiar with concept of passing parameters to the query.

Re:I don't use em unless I have to

[Anonymous Coward]

Database programming frameworks that support parameterized queries/statements will provide the same benefits that stored procedures do in preventing SQL injection attacks. Applications that are most prone to SQL injection attacks are one's that build the statement completely dynamically.

Re:I don't use em unless I have to

[msobkow]

Well, you certainly know how to pitch the Oracle monolithic viewpoint on database architectures. It works, I did about 7 years on Oracle 5-7.2 before I ended up working with other RDBMS for a while. I'm not knocking your opinion, just saying that it's specific to Oracle.

DB/2 UDB used to have relatively poor stored proc performance, because they ran out-of-process. I think that's changed with the latest releases, but I'm not sure. Yet you could still get blazing performance for certain apps by using precompiler binding stubs instead of stored procs -- the stub is effectively a special case, one-statement stored proc. It doesn't save on network data transfer, but it makes a huge difference for individual statements. Plus it's an interesting approach to database security.

Sybase takes a bit of a hybrid approach. They have the high performance stored procs, but they also support statement caching and precompiling. All in all I'd say it's the most flexible database if you need to build a few different styles of data access into the same repository.

As someone mentioned earlier, none of the approaches is inherently good or bad. There are different ways of slicing your performance, and the decision of what to do and when to do it depends on knowing your specific tools, not just general theory or "rules".

It's surprising how many DBA's and architects keep trying to shove every application into the same design framework they're familiar with, instead of taking the time to learn and try different approaches. There really is only one real "rule" I know of:

I'd do it differently next time.

Re:I don't use em unless I have to

[Anonymous Coward]

> Why would anyone use Dynamic SQL instead of variable binds

Well, the poopy data layers in PHP/Perl/etc don't really support parameters, but they do have all sorts of "quoting" hacks, so an entire generation of new developers have come to the wrong solution to the problem.

Re:I don't use em unless I have to

[bigman2003]

You've got some good points- or at least I agree with you, for whatever that is worth.

My experience has usually been "get the damn thing working" by doing all of the sql in the application.

Then, when things are looking good, customer is happy, you're happy...you can start moving to stored procedures.

Some people might feel that this is wrong, because it should 'be done right the first time'. But usually the reality is that it just has to get done SOON. And if I can sit there and dink around with some queries while I've got someone on the phone who is hitting 'refresh' to see the changes, then I am going to do it in the application where I have better access.

In my experience, which is not the be-all and end-all, but is 6 years or real-world day to day web app programming...(judge that as you will) 90% of the queries are total 'junk' anyway.

I don't really see the need to put things like 'select * from story where storynum=956' in a stored procedure. Looking through most of my code, I would guess that at least 90% of my queries are similar to that.

I've got other routines that do things like aggregate hundreds of thousands of pieces of data- in that case I use stored procedures because they make the difference between 6 minutes and 60 minutes. (but then again, if I were a real genius, maybe it would only take 6 seconds...but then my server might catch on fire or something) but most of my work on the web is pretty straight-forward.

On the other hand, I used to work with a guy who was always 'trying to save cycles'. Like he didn't want the server to have to work too hard. He would spend days trying to cut something down from 500 milliseconds, to 20 milliseconds. Yes, I agree that is all well and good (depending on the frequency of the query...in the 500/20 example, we're talking about a DAILY routine). But we've upgraded servers twice since that time, and that 500 millisecond query IS a 20 millisecond query now.

Not to mention the fact that the server sits at 1% of processor capacity all the time. I hate to say it, but we're living in a fairly hardware-rich environment. Most (I said most) people don't need to optimize the crap out of something, because as the original story said, users are waiting on their connection- not my database access.

Okay- it all sounds like me making excuses for being a lazy coder. But man, you should see all of my pretty comments!

Last thing...where I work, we pretty much upgrade hardware based on TIME, not on NEED- it is one of those 'spend it or lose it' situations. So, saying that if I had optimized the code, I wouldn't have needed to upgrade is a moot point- we would have upgraded anyway.

Re:I don't use em unless I have to

[Anonymous Coward]

They're just another tool in the tool box. I would have to say that I don't use them often, but do when I need to. As far as the point about keeping the code together, SQL Enterprise Manager will generate SQL script files to place into your source archives along with the code, which helps. For other platforms, I don't know how this works. The only problem is that you have to remember to do this whenver you alter the procedures.

Re:I don't use em unless I have to

[1001011010110101]

Well, you certainly know how to pitch the Oracle monolithic viewpoint on database architectures Well, I worked as a tech presale some time ago and that shit ends up sticking to your brain eventually :) I dont think performance is the main issue here..at least not the performance of the queries running. Precompiled queries and executions plans are not a big factor (in fact, at least in Oracle I think they are both re-parsed periodically to assure they are accounting for the updated table statistics, index cardinality and so forth). The idea of huges volumes of data going back and forth through the network just disgusts me :). It just seems more natural to do most of the slicing and dicing of the data in the DB server, that's usually optimized for this kind of work.

Re:I don't use em unless I have to

[CaptRespect]

I agree.

We are current migrating away from ms sql server. Since a lot of our app is written in Stored Procedures, a lot of the app has to be re written.

It's definately a mistake to put everything to stored procedures. Especially if you want to keep your app somewhat database independant.

Re:I don't use em unless I have to

[ron_ivi]

"2) Convert all SQL to standard SQL "

This deserves the insightful mod.

And it's a bit absurd that The Fine Article was talking about the "ANSI-92 SQL Standard".
Wouldn't the "SQL-2003 standard" or at least the "SQL-99 standard" be better goals to shoot for?

For the SQL 2003 standard, you can look here. Yeah it's the last draft, but practically nothing changed from the (costly) finalized standard.

Re:I don't use em unless I have to

[IndigoSkies]

Agreed on that point - the business logic belongs in the application / business object whenever you can.

Personally I DO in fact prefer to use stored procedures, but not the way most do, as a place to write programs with flow control statements, etc. I use SPs as a means to provide "canned queries" (and of course updates etc).

This approach gives you the advantage of being able to persistence your data from any language, and the benefit of pre-compiled speed, and also control over transactions and authorization -- all good things.

The only time I readily put business logic in a SP is when a join won't do -- when doing it in the app would mean too many round trips (to look up and cross ref and look up and cross ref). Too inefficient.

Many people say the other time to put logic in SPs is when you want the exact same complex logic to be runnable from all platforms, such as with order placement. I disagree -- things like that should be made into service interfaces through SOAP, cgi, CORBA, or whatever floats your boat (just set a standard).

Yes, DB SPs can be trated like a service, but that approach is generally only suitable for internal apps, whereas true service interfaces can be invoked from outside more easily (and platform independently).

Re:I don't use em unless I have to

[(negative video)]

Compiled SQL is faster. This is purely a performance enhancement.

One that can be critical for hierarchical data. A stored procedure will save one round trip from the app to the DBMS for each level of hierarchy that has to be traversed.

That's the soul of n-tiered architectures, they're supposed to bring that kind of flexibility along with the use of tiers...this kind of flexibility is the point.

Reminds me of something I read: "We have an n-1 tier approach. No client, just multiple layers of server." ;-)

Re:I don't use em unless I have to

[Tassach]

Depending on stored procedures as a key element of security is obviously not desirable

I totally disagree. Stored procedures should be your primary access control measure, particuarly when you're designing a publicly accessable system.

Stored procedures act similarly to an suid program in Unix -- they run with their owner's permissions, not the caller's permission. The passwd command is a perfect example -- a normal user has no permissions to /etc/shadow, but by using the passwd command a user can manipulate the one part of /etc/shadow which pertains to him, and nothing else.

Typically for a web-based system I'll make a webuser account, which has no permissions whatsoever in the database other than permission to execute some stored procedures. The net effect is that even if an attacker is able to mount a SQL injection attack, he won't have permission to run arbitrary SQL statements against the database. Even if an attacker gains direct access to the database, he'll still be limited to exactly the same set of operations he could perform via the official interface.

Re:I don't use em unless I have to

[innosent]

Most of the time the DB isn't even accessible from the outside. Why worry about being rooted then ?

There's your answer... Most != All. The DB is connected to something, which is connected to something that is accessible from the outside. But attacks don't just happen from the outside, either. Probably 90% of the attacks you really need to be concerned about with most DB applications are an inside job. Can your company trust every user that has some access to the database?

Re:I don't use em unless I have to

[abradsn]

SQL means that it is standard. You missed the point.

Re:I don't use em unless I have to

[Lips]

Just a comment....if you must use dynamic SQL in SQL Server, use SP_EXECUTESQL instead of EXECUTE.

From MSDN:
"Using sp_executesql is recommended over using the EXECUTE statement to execute a string. Not only does the support for parameter substitution make sp_executesql more versatile than EXECUTE, it also makes sp_executesql more efficient because it generates execution plans that are more likely to be reused by SQL Server."

Re:I don't use em unless I have to

[LuxFX]

I like to keep business logic in one place as much as possible. You are almost assured to have some in your app, so I try to keep all logic there.

But if you have a large-scale solution, including multiple applications on multiple platforms that rely on the same algorithm, it makes sense to have a central location for your core logic, so that when you update the logic, you won't have to recompile everything that accesses it.

This doesn't limit you to stored procedures, it could be done just as effectively with PHP and XML.

Re:I don't use em unless I have to

[Anonymous Coward]

I think '92 was toted out since MySQL is a commonly-used *nix database that doesn't even meet a reasonable SQL standard from 1992.

In other words, MySQL is effectively 12 years behind the curve.*

Come on... it's Journalism 101. Create a controversy where none really exists.

* In limited respects.

Re:I don't use em unless I have to

[Lordrashmi]

My former company migrated from MySQL to oracle, because oracle promised the moon, and didn't charge too much. Now, after everything has moved to oracle, using oracle specific features, Oracle decided to more then double the price of each license and that money is not in the budget.... It would be nice if they could change database vendors, but sadly they can't.

Re:I don't use em unless I have to

[Tassach]

Dynamic SQL should be avoided like the plague. Transact-SQL (in both Sybase and Microsoft flavors) encourages the use of variables for query parameters:

create table Foo (
Foo_Key char(5) not null,
constraint PK_Foo primary key (Foo_Key)
)
create table Bar (
Bar_Key char(5) not null,
Foo_Key char(5) not null,
A int not null,
B int null,
C int null,
constraint PK_Bar primary key (Bar_Key),
constraint FK_Foo_Bar foreign key (Foo_Key)
references Foo (Foo_Key)
)
create procedure Example (
@SearchValue char(5) = null
) as
begin
set nocount on
declare @ErrMsg varchar(255)

if exists (
select 1
from Foo
where Foo_Key = @SearchValue
)
begin
select Bar_Key, A, B, C
from Bar
where Foo_Key = @SearchValue
end
else
begin
-- error handler
select @ErrMsg = '"' + @SearchValue + '" is not a valid key.'
print @ErrMsg
end
return
end

AArgh. Does anyone know how to keep slashdot from eating the formatting in a <ecode> tag?

Re:I don't use em unless I have to

[iammaxus]

But the same sort of abstraction can and (I think) should be done inside the application. As one of the replies mentions, store procedures can essentially be replicated inside the software running the DB.

Re:I don't use em unless I have to

[innosent]

What do you do if you have to have 200 employees searching through 10 years worth of live data for 8 hours straight? Those 480ms would be worth every minute spent finding them. Even if it drops to 20 with upgrades, the original 20 would probably dropped to 4 or 5 ms. Make it fast, and if you are upgrading hardware faster than you do business, maybe your company should try doing more business. Technology is a tool to grow business, if it's not doing that, then you shouldn't feel too secure about your job. Maybe your industry doesn't have the strict requirements on long-term data availability that mine does, but I'm sure you can find a use for spare cycles, try using them to grow your company.

Re:I don't use em unless I have to

[einhverfr]

Personally, I rarely put SQL commands in my code. I leave it all in the procedures on the server.

Ah, the joys of vendor lock-in.

Personally, I rarely call stored procedures but wrap them behind nice views or triggers. This allows me to change db backends with very little updating of the client code.

Of course PostgreSQL makes this a little too easy, but you can still do the same in SQL Server using execute instead triggers. IMO this is the proper way to use stored procedures-- keep them hidden from the app so it only sees the relational model.

Re:I don't use em unless I have to

[nettdata]

Stored procedures facilitate better safety.

I agree totally. I've seen too many "enterprise" apps that reside on DMZ boxes that have JDBC connections straight into the data tables in the DB. Not really that great from a security standpoint, IMO.

I prefer to make all remote calls via Stored Procedure API's, via a "proxy" database schema that only has execute permissions on the procedures, and no direct data table access.

This also means that the middle-tier devs can write code for a published API, and the DBA's can screw with the data model to their hearts content without (theoretically) breaking the app.

It also lets the DBA deal with SQL tuning, etc., while the middle-tier (typically Java) devs don't have to worry about understanding SQL tuning details.

I'm sure there are a number of views on this, but this has worked out very well for me over the years.

Re:I don't use em unless I have to

[msobkow]

That depends on your fundamental architecture. For smaller applications environments your app servers and RDBMS are often on the same box. Not great for security, but it often brings the price down to a point where the project can go ahead.

With all of your "traffic" in-server many of the vendors stacks will let you use IPC messaging instead of IP messaging, which can boost performance rather significantly.

The other thing to consider is that for environments like the full IBM stack, you're expected to have a cluster of DB/2 UDB servers with a hefty, hefty backbone in the data center.

Like I said, the vendors have different approaches. Oracles is monolithic -- get it all into one big server. DB/2 is pure database, and expects to have other architectural components in use when building an application. Sybase tries to walk a line that can live in either variant, with interesting ideas of it's own. Postgres has some really slick custom data type support.

The point is that within the overall application cluster, you can get the same kind of performance out of any vendors RDBMS is you're using the appropriate application stack in a properly configured cluster. You just don't architect the solutions for the different products in quite the same way is all.

Re:I don't use em unless I have to

[the quick brown fox]

SQL injection is a very large problem in the enterprise

It doesn't have to be. In Java, for example, just always use PreparedStatement, which disambiguates queries and parameter values. There's an equivalent in .NET; not sure about other programming platforms.

Re:I don't use em unless I have to

[CaptDeuce]

Stored procedures aren't good or bad...they just are. Passing a value judgment on whether they're good or bad is completely dependent on the situation.

And in theory, there's no difference between theory and practice. :-) It's true that every situation is different but from my perspective, sprocs (stored procedures) should be the only way that application programmers can alter a database. I'm an SQL programmer. I carry a coffee mug.

In my opinion, applications shouldn't have any more detailed knowledge of a data base structure than any other OOP language's object.

Back to the original poster:

Is it really important to shave compilation time when connection and execution times dominate?

An easily overlooked advantage of sprocs (and views) is that it allows the data base structure to be changed without changing a single line of application code. I have routinely made changes to a live database with little fuss whereas my front end associates sometimes need to do messier stuff such as (essentially, if not literally) restarting the web server. That is, front end changes are much more likely to interfere with operations than back end changes.

In addition, you either have to have a dedicated T-SQL or PL/SQL coder who then is the weak link in your coding chain, ...

Weak link? After seeing some SQL code of front end people, I totally resent being referred to as a "weak link". :-) I was a dedicated T-SQL programmer and I've been out of work for nearly a year. :-( Most job requirements I come across list SQL almost as an afterthought -- as if good SQL code isn't important.

... or your pool of developers must become fluent in both your scripting language of choice as well as the SP language.

Simple. Stick with SQL as the sproc language. Well ... it would be simple if all SQL sproc implementations were alike. I started working with PostgreSQL after working many years with T-SQL and I was totally amazed at PostgreSQL's shortcomings (sproc parameter definitions are downright brain damaged). After further research the vague nature of the SQL "standard" totally staggered me. Which lead to ...

I have ... found [using a dedicated SQL programmer] to cause bottlenecks when 'the database guy' is unavailable and learning curve problems (bugs) with new coders getting familiar with the db language.

But aren't bottlenecks unavoidable when any absent person's code is causing problems? Conversely, there are some things that can't be done without well written SQL so the problem remains that somebody on the team should be proficient in SQL.

Utlimately there is no clear-cut answer to the question since, as I alluded to, SQL implementations vary wildly. So you either make the final decision based on which SQL server you use OR choose your SQL server based on which approach you decide to use. If my inability to find a job is any indication, you (the original poster) will choose to avoid using sprocs. :-/

Re:I don't use em unless I have to

[Anonymous Coward]

I don't agree with the OP on storedprocs, but I think the point is that many developers are focused on micro-optimizations.

Like they'll say "Ad Hoc SQL is SLOW!", but then the schema is shit and their application code is completely inefficient. In the big picture, it rarely matters.

Re:I don't use em unless I have to

[Hangtime]

This rant is focused specifically on SQL Server. I do not work with Oracle, but if should apply similarly.

If I give you rights to update a table, that means the user you log into the database has all rights to update in that table. Your writing dynamic code against the database so therefore you must be granted writes to the underlying tables themselves in order to work. Now I either get to write some backend code i.e. triggers to ensure that your user has only rights to update the records specifically for that user (think Windows Authentication) or your application logs as one user into the database.

So, I can either add every user in the domain to the database and give them permissions and manage all the underlying rights associated with those users (this means anyone with Query Analyzer can login without any restriction to the database) or I can give your application one login that it then uses for everyone and we build some security tables for users to log in to the application...but of course if you wrote down the login and password or someone saw your code where you log into the database now were compromised and we just gave an attacker free reign on all the tables in the databases because as we said before you wrote against base tables. Or we might go with the best option and have you write all of your code against stored procedures meaning an attacker must use the same procedures that your application does to manipulate the database. Just because your parsing out things like "--" from your input parameters does not mean that your application is secure from injection or compromise.

Excuse grammer, spelling, babbling, etc...I am tired and have a headache from watching the jerky camera work in the Bourne Supremacy.

Re:I don't use em unless I have to

[Westley]

Hang on a sec - you're conflating two issues:

1) General security - yup, SPs win hands down here.

2) SQL injection attack - using prepared statements and parameters, this *shouldn't* be a problem with a Java or .NET app. (I would hope this functionality is available elsewhere too.) You don't write any of the parsing or formatting code - you let the DB vendor do that. (If they've screwed that up, of course, you're vulnerable - but then they might equally have screwed up SP security.)

Given an app which treats all user input as parameters rather than straight SQL, how is the application not secure from injection?

Re:I don't use em unless I have to

[Anonymous Coward]

So put your application on a server. App servers can be secured as well as DB servers, and if they crack that they can likely crack the DB (and probably will, it's quicker to get your data if you're a DBA). I put all logic in the app, and use triggers and stored procs strictly for automating database tasks like preventing connection with tools other than the app or producing audit or snapshot logs, or allowing operations that exceed the normal app authority (like killing app user sessions). There are limits on how far you can scale the database unless you have a mainframe. We spend serious money for performant servers, so I see no reason to weigh them down with the logic of the program when they're working hard to find the data. Even with Oracle 10g you are limited to a handful of nodes if you're looking for performance. But you can scale the app if it's sufficiently modular by using separate app servers. Can't do that if it's all in the DB.

Re:I don't use em unless I have to

[aztracker1]

Beyond the aspect of trying to shove it all in the same hole.. the fact is that *most* stored procedures can be translated from one language to another, with usually only minor tweaks... I've had to re-write a number of Oracle SP's to TSQL for ms-sql use... it was a weird situation as to "why" so I wno't get into it...

But the fact is, that about 99% of it was relatively trivial to translate.. going back and testing was the real pain, because a lot of the views were sloppy and didn't translate well (no wonder the original app was sooo slow), had sp's which relied on views, which were linked to the same tables in the background, with circular queries in the end, what a mess... *sigh*

I like what firebird has to offer, and where mysql is going.. postgre is nice, but I like to dev under windows, so postgre is a beast there.

Re:I don't use em unless I have to

[dubl-u]

Can your company trust every user that has some access to the database?

Yes, if the only thing that has access to the database is the application logic.

I feel that you're better off putting all of your business logic in one place. That can be the database, of course. I know a very clever DBA/developer who built a financial exchange system entirely in the database, accessing everything via stored procedures. It was very cool, as the database took care of many of the hard parts of building a system like that.

But personally, I think putting all your business logic in a database's proprietary language is a pain. The development tools aren't as good, you have a big vendor dependency, and it's hard to find developers who can work in that style.

Instead, I think of a database as the place where the application puts objects when it doesn't need them. In that case, there's no reason to give anybody but the app access to the database. Sure, people may need to run Crystal Reports, but you shouldn't let them do that against the production database servers anyhow.

So these days when I build things, I put no code in the database. And once I made that choice, it freed me up to not even have a database. That makes refactoring much easier, speeding development. Sometimes I still end up using a traditional SQL database, but that's now a choice for me based on performance characteristics, rather than a default.

Re:I don't use em unless I have to

[dubl-u]

Sometimes there are different front ends to the same data, incoming interfaces, etc. They tend to grow on databases as time passes by :)

This pattern, where database-as-repository turns into database-as-integration-layer, strikes me as the road to hell, albeit a tempting and convenient one.

Why? Because as soon as you have two or more code bases trying to work with the same schema, you are effectively screwed. Mortals have a hard enough time evolving a schema when there's just one code base to update; I've never seen anybody successfully and regularly migrate a schema and several different code bases at the same time. Instead, they end up with massive hack jobs, eventually devolving into the classic, "We've got to rewrite everything from scratch," as if the bits had somehow rotted beyond repair.

Forget that, I say. If you need an integration layer, just build it.

Re:I don't use em unless I have to

[Anonymous Coward]

Plus (this is the main point imo) not everyone will access the databases via your application.

What about people who want to link straight up via their PDA, via some embedded tech on the factory floor, via an email gateway, via any method in the future that you don't want to have to include in planning now.

The best way to ensure that all connections to your database are correctly secured and queries appropriately filtered is to use stored procedures.

The alternative is to re-write every data rule for every interface onto your system which is a royal PITA compared to just giving them a basic SQL client.

Re:I don't use em unless I have to

[dubl-u]

But aren't bottlenecks unavoidable when any absent person's code is causing problems?

In a word, no. You may have chosen not to avoid it, but it's not necessary. If you use a team-oriented approach like Extreme Programming, all the developers own the code. If somebody is absent, work goes on. You wouldn't negligently build an app with a single point of failure. Why would you build a team that way?

Re:I don't use em unless I have to

[vk2]

Just for this same reason - we have implemented ip based protection for Oracle databases as provided by oracle (protocol.ora) and also we make very efficient use of invoker rights stored procedures which transperently behave different based on the invoker.

Re:I don't use em unless I have to

[1001011010110101]

That's a use for stored procedures...if you abstract the data access on procedures, the schema can evolve easier.

Re:I don't use em unless I have to

[severoon]

Two points to Westley for proper use of the word conflate. Seriously, nice work my man.

I appreciate all of the discussion that my post has generated, but I'd like to take a second to add a bit of perspective. I've done consulting for the last 4 or 5 years now and worked at several different companies. If the security and performance concerns/solutions I've heard addressed in this thread address the worst problems of your applications, you guys are working on projects that seriously have their stuff together.

As for my part, I'll worry about enhancing security through the prevention of SQL injection once I work on a project that doesn't send customers' credit card information over the wire in cleartext. :-)

Re:I don't use em unless I have to

[bigman2003]

If I DID have that query that I needed to optimized, I would. But I said that 90% of all queries were junk, and didn't need to be optimized.

And If I needed to optimize a query, it would definetly become a stored procedure.

But how many queries on a typical web app fall into this category of 'important to optimize'? From my experience, most do not.

Re:I don't use em unless I have to

[Fulcrum of Evil]

And it's a bit absurd that The Fine Article was talking about the "ANSI-92 SQL Standard". Wouldn't the "SQL-2003 standard" or at least the "SQL-99 standard" be better goals to shoot for?

It depends on what the majority of DB products support. The last I had heard was that SQL-99 was well supported, but SQL-2003 was not. Granted, this was a year ago, but the point remains: there's no reason to be SQL-2003 if your chosen DB doesn't support it.

Re:I don't use em unless I have to

[pinko-rat-bastard]

Plus (this is the main point imo) not everyone will access the databases via your application.

What about people who want to link straight up via their PDA, via some embedded tech on the factory floor, via an email gateway, via any method in the future that you don't want to have to include in planning now.

Which is why web services and "service oriented architecture" (SOA) are such hot topics right now. Rather than writing stored procs (which are, IMHO, hard to read and very difficult to debug), I've been building all my business logic in Java and exposing functionality as web services using JBoss and Apache Axis. The UI can then be built in anything that can consume web services. Calling a web service from PHP is a breeze (not to mention the fact that developing something with PHP is *much* quicker that with JSP's since you don't have that agonizing compile/package/deploy cycle every time you make a change to a page). Also, it makes the UI almost disposable -- you can scrape it off and replace it without worrying about re-implementing complex business rules. It's a great way to integrate multiple business systems which may not only use different databases, but also different database vendors and/or OS and/or hardware platforms (like your "embedded tech on the factory floor" example)

Re:I don't use em unless I have to

[CaptDeuce]

But aren't bottlenecks unavoidable when any absent person's code is causing problems?

In a word, no. You may have chosen not to avoid it, but it's not necessary. If you use a team-oriented approach like Extreme Programming, all the developers own the code. If somebody is absent, work goes on. You wouldn't negligently build an app with a single point of failure. Why would you build a team that way?

Because you work in a shop with only, say, three programmers? I should have said that "bottlenecks are often unavoidable". Extreme Programing isn't always an economically practical option. It's obvious that the original poster expected to have only one SQL programmer on staff suggesting that it's not an unusual situation.

Re:I don't use em unless I have to

[Crash6-24]

I use stored procedures for security and linited access. However, that can be carried to extremes. I was called in to diagnose why a project in production suddenly quit working. A developer had come up with a scheme where every stored procedure would verify the security level of the caller every time the stored procedure was called. The scheme worked fine when installed in a few peripheral procedures but crashed the application when installed everywhere. The problem was that the checking was single-threaded and locked a table every time the security was checked. Parallel operations became serial. Users who were feeding in information from hand-held scanners timed out. The coded-in-to-the-procedure security was removed.
I guess the lesson learned was to review the design by DBAs before implementing, and test under load before going in to production.

Re:I don't use em unless I have to

[Westley]

I know what you mean - admittedly the problems with the databases I've seen haven't been security (usually) so much as just horrible design. Databases don't evolve cleanly :(

The great thing about using parameters though is that not only do they help to secure you against SQL injection attacks, they actually reduce the amount of work you need to do, too. For instance, suppose you have to put a date/time in the database. Using straight SQL this is a bit of a pain - you need to make sure you use the right format for the database etc. With parameters, it just drops out. Same goes for null values etc.

Re:I don't use em unless I have to

[Anonymous Coward]

Hah... I bet you are switching to MySQL... Keep us posted, should be good for a laugh down the road when something really ugly jumps up and bites your asses hardcore.

Re:I don't use em unless I have to

[fitten]

I agree.

We are current migrating away from ms sql server. Since a lot of our app is written in Stored Procedures, a lot of the app has to be re written.

It's definately a mistake to put everything to stored procedures. Especially if you want to keep your app somewhat database independant.

This is completely false. One of the reasons TO use SPs is that you will be using multiple RDBMSs. You write your code (typically compiled code like C, C++, etc.) once and the stored procedures are written to "do the right thing" and each system returns a consistent result set or answer. That way you seperate your code from the implementation that is database specific.

If you are using embedded SQL in your code, every time you have to change a query, you have to recompile your entire app. Every time you want to port to a new RDBMS, you have to write a whole bunch of code to generate the right queries for you. At best, you designed your app from the start to see what RDBMS you are using then create an "object" that encapsulates all of your queries so they are isolated from your app. If you didn't do that, then every function you call that queries the database has to have if/else or switches to see which pathway to go down to get the right query.

Using Stored Procedures is what *keeps* your app database independent. As long as you are using real RDBMSs, you simply rewrite your .sql files to do the right thing and your app runs. Your approach of embedding all the SQL leads to massive redevelopment for every RDBMS you support.

I'm betting that the reason you are down on SPs is that you are going to something that doesn't support SPs, like MySQL (I hope, instead of not just realizing you rewrite your SPs and your app runs as it is already compiled), which you had better understand the implications since you are going from an RDBMS to just a "data dumping place". There can be some things you aren't thinking of with your MySQL migration that can really, really ruin your day down the road if you don't watch it.

Re:I don't use em unless I have to

[Ed Avis]

If you want to implement security in the database then stored procedures are often the only way to do it. With many SQL systems you can either have insert permission on a table or lack it. Clearly, you may want to allow a user to insert rows related to his own department (for example), but not grant blanket insert permission for anything. Some DBMSes like Oracle have fine-grained access control which may be able to handle this, otherwise, you must use stored procedures or perhaps triggers (which are usually written in the stored procedure language).

BTW, if you want to implement security and business logic in a vendor-indepedent way then you cannot rely on features of some particular middleware platform, since this is just as vendor-dependent as the database - usually more so. If you want to allow multiple front ends then security and logic can't be in the front end without duplication. It's certainly possible to write clean code using only published standards all the way through and so become (mostly) independent of some particular vendor, but it's easy to fall into the trap of making the code database-independent in theory, but deeply tied in to some proprietary middleware platform or toolkit. If you have to be tied into something, make sure it's something good. Oracle is a pretty capable system and so are Postgres, Sybase, even MSSQL.

Re:I don't use em unless I have to

[dubl-u]

You wouldn't negligently build an app with a single point of failure. Why would you build a team that way?

Because you work in a shop with only, say, three programmers? [...] It's obvious that the original poster expected to have only one SQL programmer on staff suggesting that it's not an unusual situation.

Doing team-oriented stuff in a small shop is much easier that trying to do it on a large scale. If you only have three developers, there might be only one SQL expert, but that doesn't mean that there has to be only one person writing SQL code. At worst, it means that the SQL expert should pair with (or, less effectively, review the work of) developers doing the SQL work.

Extreme Programing isn't always an economically practical option.

I think that's true, but relatively rare. The productivity, reliability, and business agility gains are substantial. It's my experience that you get more features delivered for the money using agile methods than traditional ones.

Re:I don't use em unless I have to

[Anonymous Coward]

I know a very clever DBA/developer who built a financial exchange system entirely in the database

Er, no you don't.

Re:I don't use em unless I have to

[H09N0X10U5]

"if you abstract the data access on procedures, the schema can evolve easier."

Only if you abstract all the data access.

Re:I don't use em unless I have to

[Anonymous Coward]

Do you want to have an OPEN application that is platform independant? Do you want a product that can participate in the enterprise? Do you want to avoid the DB / OS religion wars?

Apart from the open, that would be SAP. Which does not, afaik, use stored procedures. And does, I know for sure, keep its business logic in the app layaer.

Re:I don't use em unless I have to

[Anonymous Coward]

Good point, and largely still true today.

That was why I mentioned both. That, and the fact that the SQL-99 spec is possible to read, while the SQL-2003 spec is so bloated with useless and sometimes seemingly ambiguous extra verbage that I have to say it's the worst written spec I've ever read.

But still, using SQL-92 is as silly as using the ARM-C++ as a reference for that language.

Re:I don't use em unless I have to

[1001011010110101]

And what's wrong with that?
All my data access is in a single place. The rest of the app access it thru that layer.
Ok, the rest of the app is in procedures too, but it needs to because of the tech. Its an apache mod (mod-plsql) which works very nicely for DB frontends.
Even then, you can code properly. I tend to do a mvc kind of thing with them, even using no oo features from the DB.

Re:I don't use em unless I have to

[Jack Greenbaum]

In my opinion, applications shouldn't have any more detailed knowledge of a data base structure than any other OOP language's object.

But why hide that structure behind a non-portable interface? Isn't this what an API written in raw Java (or J2EE) or behind a SOAP/XML or some other layer is for?

Re:I don't use em unless I have to

[H09N0X10U5]

What's wrong with that is that it doesn't get done. Then you get divided control, which is the same as no control.

Re:I don't use em unless I have to

[1001011010110101]

Believe me, some people do :)

Re:I don't use em unless I have to

[xelah]

What about people who want to link straight up via their PDA, via some embedded tech on the factory floor, via an email gateway, via any method in the future that you don't want to have to include in planning now.

Well, if I had my way there'd be no way in hell someone's PDA would even be able to send packets to a production database - never mind issue queries.

The alternative is to re-write every data rule for every interface onto your system which is a royal PITA compared to just giving them a basic SQL client.

I've seen this argument appear quite a few times, seemingly coming from DBAs. The alternative to having your new application go through the layer that's been put on top of your data is indeed to rewrite that layer inside it. But why would you do that? It really is much easier to go through the code that already exists (or even just reuse it - as a library, for example). It's not like the CORBA or HTTP clients required to do that aren't available or widely ported enough. In fact, it's surely easier to find such a client than it is to find a client library for your favourite database compiled for obscure language x on unusual platform y.

Re:I don't use em unless I have to

[Jouster]

That's a terrible idea, IMHO. It doesn't adapt well to any changes in security architecture, causes difficult-to-debug interactions (coder: "Why is this atoi() failing in the app? I'm logged in to the DB and ran through each step, and at that step, the value is from column 8, which is a valid integer!"), and makes it well-nigh-impossible to switch DB backends (and if you don't think the last one applies to you, think about what DB vendor you would have chosen 15 years ago).

It sounds suspiciously like there were consultants involved in your decision to do this; they probably sold you on EBSO, too, didn't they? All the consultants say is just different words for the attempt to prevent you from ever switching out from Oracle.

From someone who has been there [and is there still],
Jouster

Re:I don't use em unless I have to

[Thundersnatch]

I don't really see the need to put things like 'select * from story where storynum=956' in a stored procedure. Looking through most of my code, I would guess that at least 90% of my queries are similar to that.

If you're actually using "SELECT *" in an application's code, you should be fired.

Re:I don't use em unless I have to

[nzhavok]

I'm not the original poster but...

Personally, I rarely put SQL commands in my code. I leave it all in the procedures on the server.

...and how do you access those commands?

Well, using a business delegate would be the standard way.

There's also a world of difference between calling a procedure and running a query (at least on Oracle) mabye you should look into that.

What I did was create a language that allows you to express a query in pretty much plain SQL, define formal inputs and outputs (with types) and generates a class to interface that particular query.

Sounds like you just reinvented the wheeel again :) I'd be very surprised if your "language" truly extended beyond your own problem domain, or translated well to other databases. You would probably have been beter off using an existing API and using an abstraction layer between it and your business code. well actually you might have been better of using SPs...

Re:I don't use em unless I have to

[SemperFiDownUnda]

Portable code is nice in theory but look what you have to give up for it.

What makes different databases, and programming languages, useful is the tools and functionality that they provide to make your application easier to write.

By the logic of the original question we should scrap everything and build the OS, programming language, dbms ourselves...oh $#!T better design the hardware too. When you design an app you look at the current requirements. If one of those requirements is being able to swap out back end database then you build a robust data access layer. Putting data centric code in your business logic is like putting presentation layer logic in your data access layer. If you choise not to build you application without using any of the unique functionality of a given part of the development environment you are, in my opinion, an idiot.

Also remember if you put all of the SQL in your PHP, or what ever your business logic is mostly written in, and you want to change databases you'll most likely have to change all that code too. But if you have your business logic access a data access layer and that is well designed then you can swap out your dbms, only modify the data access layers internals and the interfaces to the business logic remains the same. Its a lot easier then trying to untangle the code equivilant of a 50 year old tangle of string that is your business, data access and data storage code....and most likely in your case your presentation layer too.

flame me about my spelling...I do not care.

Re:I don't use em unless I have to

[Hognoxious]

from my perspective, sprocs (stored procedures) should be the only way that application programmers can alter a database.

Hmm. Interesting how the most successful ERP system by far doesn't do it that way.

I'm an SQL programmer. I carry a coffee mug.

I'm an SAP consultant. I have someone else to carry my coffee mug.

Good or bad?

[Anonymous Coward]

Bad, of course.

I only put triggers or constraints or whatever in the database for one reason: to make sure only valid data enters the database.

For instance if ColumnA must be between 5 and ColumnB+34, that should go in the database. The database itself should guarantee the data is clean. "Data logic" you could call it.

Business logic and everything else should go in the higher layers. There is some ambiguity about what is "data logic" and what is "Business logic" but it's usually pretty clear.

Why? Maintanence. The stored procedures tend to rust in place over time. If you're an "agile" developer you'll go nuts not being able to refactor business logic or have unit tests check your database procedures. If you're a "BDUF" (big design up front) shop, you might like it, but thankfully many are moving away from that.

Re:Good or bad?

[jfroebe]

for small number of applications you would have a point but if you are dealing with thousands of machines in an enterprise environment, your argument falls apart. Keeping the changes in a central location with the ability to 'roll back' any of the changes quickly is the goal.

Re:Good or bad?

[saberworks]

I totally disagree. Stored Procedures have a lot of benefits.

Speed

Code Reuse - if everyone uses stored procedures, it's less likely that your developers will just write another (duplicate?) query when they don't want to go mucking around in all your php/asp classes/functions.

Bandwidth between the database server and your web server will be reduced - instead of passing huge queries across the link, you send a simple stored procedure call.

Plus, PL/SQL is really easy to use and learn and is relatively portable across at least PostgreSQL and Oracle.

They also provide a good facility for logging or tracking that can be (should be!) completely transparent to the web application.

Re:Good or bad?

[smallfeet]

That's were an application server would come in. If it were J2EE you would put this logic in the EJBs. Not putting business logic in the DB makes it easier to maintain and also makes it possible to change the DB if needed (though THAT never happens).

A previous post mentioned 'data logic' in the DB. I thinks that is a good way to think about it, limit it to data integrity.

Re:Good or bad?

[Miniluv]

Bandwidth between the database server and your web server will be reduced - instead of passing huge queries across the link, you send a simple stored procedure call.

Don't be assinine. A modern production environment is running at least 100MBit Ethernet, if not 1GBit. A HUGE query, say 2500 characters, is 2.5KBytes. Gee, thats pretty rough.

Plus, PL/SQL is really easy to use and learn and is relatively portable across at least PostgreSQL and Oracle.

Easy to learn isn't an argument for or against a technology. That aside, relatively portable is logically equivalent to completely fucking unportable when translated from Marketing into English.

Re:Good or bad?

[johnstoj]

Don't be assinine. A modern production environment is running at least 100MBit Ethernet, if not 1GBit. A HUGE query, say 2500 characters, is 2.5KBytes. Gee, thats pretty rough.

It's not the query he is talking about, it's the transference of the data set from the database to the application server. This is where the data will be processed if the logic is in the app server.

Re:Good or bad?

[l810c]

Don't be assinine. A modern production environment is running at least 100MBit Ethernet, if not 1GBit. A HUGE query, say 2500 characters, is 2.5KBytes. Gee, thats pretty rough.

What kind of shop are you working in? 2.5k is HUGE? Try 500+ users all hitting a database with queries and reports that average 100-200k or more. And that's With highly optimized queries and stored procedures.

If your using a web server or some other application server, you could have them connected via a separate backplane and generally not have any ill effects on the network at large. If, however, you are using some sort of client side front end(Either application or something linke Crystal Reports) then it is definately better to offload the processing to the db server.

Easy to learn isn't an argument for or against a technology. That aside, relatively portable is logically equivalent to completely fucking unportable when translated from Marketing into English.

That may be true for a lot of technolgies, but I've been working with SQL(DB2, Oracle, MS, MySQL, Postgres) for 15 years and with a few exceptions it is a very portable language. I worked on it on OS/390's, HP/UX, Solaries, Linux and Windows.

Re:Good or bad?

[nzkoz]

You're talking about what Rod Johnson calls 'Persistence Logic' vs 'Business Logic'.

If the code is about rules of your application, stick it in your application code. i.e: "A User can only have 3 Widgets checked out at a time unless they're a Super-Widget member"

Alternatively if it's about data, it should be in the DB. i.e 'Users have a Type which can't be null unless the user is Inactive'.

I've seen both sides abusing this, one company I worked for refused to use stored procedures or Referential integrity and built all their persistence logic in code. The app sucked.

Another said 'all business rules must be in SQL stored procedures'. They never delivered in time.

It's all about balance, and identifying what's business logic and what's persistence logic.

Re:Good or bad?

[Hangtime]

If all you have is a couple of web / middle-tier developers, yea your procedures will rust. If you have someone on your team who is dedicated to the procedures then you do not have that problem. You can turn the argument right back around on itself. If I don't have any web developers and only database developers of course my web pages rust.

BTW, I hope you do not plan on using any of that business logic in any sort of reporting...unless of course you decided to go out and build an entire reporting engine instead of buying off the shelf. Then you get to keep that logic in sync between your higher layers and your reports when you could be running all of it from the same stored procedure.

Re:Good or bad?

[Christopher Cashell]

Don't be assinine. A modern production environment is running at least 100MBit Ethernet, if not 1GBit. A HUGE query, say 2500 characters, is 2.5KBytes. Gee, thats pretty rough.

Erm. . . you must be dealing with some mighty tiny databases, tables, fields, and queries, if you think 2500 characters is a huge query. In this day and age, where more and more data is being stored in databases, 2500 characters is nothing.

Or, are you refering to the SQL query itself, as opposed to the result data it returns? Sure, a query might only be 2500 characters, but remember, often the data can be processed locally and reduced prior to transfer, and *that* can have a huge impact.

Depending on your application and requirements, I've seen situations where the query has to return multiple megabytes of data, which are then processed by the application into a usable format.

I'm personally familiar with a situation where a DBA was able to move that processing from the application to the database in the form of a stored procedure, reducing the bandwidth from an average of 6MB to about 4KB.

Oh, yeah, and the time it took for the database to process it directly was about 8% of the time required to transfer all of the data to the application, and then process it there.

Re:Good or bad?

[Anonymous Coward]

I like to set up a set of standard queries, but allow the UI developers to query what they want.

On the other hand, I have seen enough bad programmers that I insist that everything that can modify the database is well controlled. It's annoying to have to straighten out a few thousand (or million) records of corrupted data, but downright embarrassing (and costly) if the error is due to some shoddy programming by one of your junior coders.

Re:Good or bad?

[Anonymous Coward]

Your thinking about it the wrong way.

To have your data 'pre cooked' is great. It will speed things up WAY more than anything you will ever try.

Think about this. I can drag 10 MEG of data across the line and probably not blink an eye. Yet now try that same thing with 200 people all trying to use it at the same time. It is NOT going to happen.

Now what if all you cared about in that 10 meg of data was to sum up a column? SQL can do that great. In fact it was born to do this sort of thing. Total returned data? 4 bytes.

Now if your going to be showing every line and manipulating it stored procs may not be the way to go. But I have not seen very often where the fetch/retrival/rinse/repeat beats out for speed a decently written stored proc. Or in other words what I tell people 'leave SQL to do what it does good. Leave the rest to the code'.

You say its 'bad' but I say PROVE it. I have seen many projects succeed because of it. I have also seen many projects swirl the bowl because they had NO clue what to do with a database. They just wanted a buzz word.

Computer science is not about he who has the fastest computer wins. Its about making due with limited resources and making the thing scream while doing it.

Throwing more hardware at a problem does not solve the problem. It mearly makes the mess worse in a year or so.

I force everyone I work with to use a SQL server on a 400 mhz machine. Why? I want them to feel the pain of slow. Its good for them. Can they use faster hardware? Sure, but will I set it up for them no. They will have to do that themselves. Will they? NO they are lazy. However if they can make it scream on a 400mhz machine they get the bonus of it running AWSOME on a 3.6ghz 16 way...

Now why do I put this damage on someone? My customers will do it. They do not even think twice about it. You do not always have the 'say' of what hardware they are going to buy/give you.

Re:Good or bad?

[Anonymous Coward]

I dunno, did the parent say query or return jackass?

Re:Good or bad?

[Thomas Charron]

In a way, you sort of give good reasons to USE stored procedures instead of against..

Untimatly, it depends on the size of a database, etc..

In a case where, lets say, your doing credit card processing. The database will be large, intensive, and cetralized for a company whos primary buisness is to process credit cards.

And in that case, you have full time DBAs who know what the hell they are doing, and can tke on the full time responsibility to monitor and maintain the data..

It depends on the development approach. If you have alot of data, and limited things to DO with the data, then they make sense..

If not, then your developers are maintaining the data, and hence, stored prox are a bad idea..

It all depends on your environment, really, and whos managing the data, the engineers, or the DBAs..

Re:Good or bad?

[Thomas Charron]

EXACTLY.. ;-)

That's the largest arguement I've had.. If you have the people to maintain the database, then engineers can do the fun stuff, like, write code. If not, they have to write code AND maintain data..

It depends on what you have for a team..

Re:Good or bad?

[rhiorg]

Most large US corporations are by necessity moving to a "BDUF" configuration because of the Sarbanes-Oxley legislation,the regulatory fallout from the Enron and MCI/Worldcom debacles that is intended to ensure that no corporate hanky-panky/book-cooking is going on.

Among other things, the legislation requires that developers not have access to production apps/data, and that rigorous controls are in place to ensure accountability for any changes made to the system. See Section 404 (appropriately enough).

The company I work for is going through the process of "SOx" compliance right now...we used to be a very "from the hip" organization, rapidy responding to the whims of the business. Change the app in production, and move the changes to Dev, that sort of thing. No more.

So BDUF if essential in this new age of accountability...get the requirements from the business, then design design design until the business signs off...then test test test test test until you know it won't break. Cover your bases, think critically, and build a solid app.

I feel that stored procs are very BDUF-friendly.

So, as Golgotha says... "neener, neener."

Re:Good or bad?

[Atrax]

So all the more reason for filtering the data on the database server.

to be honest I'm not sure whether we've all drifted a little away from the point here

Re:Good or bad?

[bwt]

You give the best reason to use stored procedures: to insure "data logic" is preserved. This comes in serveral forms:
- more complex constraints than foreign keys and check constraints
- enforcement of denormalizations
- enforcement of data movement (eg to a decision support system, archive, system interface, etc...)

Your statement that business logic should go in higher layers doesn't rule out SPs, because these are a reasonable choice to BE one of the higher layers.

In some cases, "business logic" can be configured as data in tables. In almost all cases it generally depends on business process state strored in tables. Depending on the amount of data required to make the business logic decision, it is sometimes superior to implement this as a stored procedure. Consider something that loops over all rows in a large data set, and makes a decision about each row. In a stored procedure, you might be rate limited by the speed of memory access and CPU. For the middle layer to implement it, you may have to pipe the large data set over the network, which might be orders of magnitude slower.

My experience is that overall system performance is usually a very strong driver of implementation strategy here. There is no reason SPs can't be just as maintainable as any other middle tier code. The only technical difference is that they run on the most centralized shared resource, but they have much faster data access. So it's a scalability for speed tradeoff.

In fact, often stored procedures are MORE maintainable, because reality in a heterogeneous enterprise environment is that no matter what you pick for your middle layer, somebody you have to integrate with won't be able to deal with it. Your spiffy java API for checking whether a series of transactions is properly ordered isn't callable from their COM object or vice versa. What ends up happening is you end up reimplementing solutions in multiple places, which is fine until the business rules change. Contrast this with the fact that just about every language supports a database connectivity that allows calling stored procudures. So their perl code logs into your database and calls your API -- easy.

Re:Good or bad?

[TRACK-YOUR-POSITION]

This is extremely interesting. But, as we all know, requirements always change. So if you're forced to deliver your product before the customer gets a chance to evaluate it (CAN they truly evaluate it before they test it on their data?) then failure seems much more likely, no?

It's reasons like this that make me suspect that companies like Microsoft, that have shied away from government contracting, tend to have an advantage than guys like Sun and IBM who gravitate towards it. Microsoft releases a product, and (until they become a monopoly in that field), knows that it will succeed for fail based on user opinions and expert reviews after the fact. Sun, on the other hand, has all kinds of awesome technologies like Java virtual machines that guarantee safe execution of code, yet never seems to be able to put all the pieces together and turn awesome technolgies into revolution. Thus, even though Sun has been working for a decade now towards the idea of allowing consumers to execute code from remote web pages and services without endangering their local computers, it will most likely be Microsoft who brings the very same technology into regular usage with .Net.

Re:Good or bad?

[TRACK-YOUR-POSITION]

Er, wait, I didn't make my point clear--MS has the advantage over Sun, because MS is forced to deliver a product and allow the product to stand or fail on its own merits. While Sun is in the habit of selling the product first then building it. So Sun programs have great looking feature sets and use more advanced technologies--because those are the things you can say about a product before you've actually made it.

Re:Good or bad?

[Anonymous Coward]

Rod Johnson, what a great name. Is he in the porn industry by any chance?

Re:Good or bad?

[Christopher Cashell]

Wow, now that's a useful comment. I'm glad you could contribute something helpful to the discussion.

My point was that the poster I replied to either:

• Misunderstood the situation he was talking about, and is therefore incorrect.
• Appears to have very limited experience with the situation that is being discussed

The fact is, the comments he made don't make sense in the context of this discussion, so I responded to both possibilities.

Please continue to add such insightful and helpful comments, though. I'm sure everyone is thoroughly enjoying them.

Re:Good or bad?

[Anonymous Coward]

100-200k or more

Damn! The SQL expression is 200K large! Is it mostly a bloated 'in clause'? If so, a join might suit you better.

I thought I had big queries (joins of 15 tables or so produce a few K or SQL). Most of the results my queries return aren't even that big (though of course when I'm inserting large geom-objects (high-res coastline of california), my insert statement is by necessity tens of K.

Re:Good or bad?

[evil_tandem]

and i have to strongly disagree. using stored procedures as a front end to your database means i can totally resdesign databases and tables, and none of the hundreds of apps i have out there need be the wiser.

also this way i only have 1 point of failure. if something screwy is happening with the databases and info i don't have to wonder what of my many, many programs out there has some weird bug that's just started expressing itself. AND i don't have to debug any new code that i write, i have tested and retested the stored procedures. they just work.

want to change table/column names, locations, or references? knock yourself out. just change the stored procedure to return the required result sets from the new database design.

and to top this all off Microsoft SQL Server has the most awesome tools for looking for ways to clean up your sql code. i've been doing this a long time and even i find new and cool ways to tighten my sql code every day. the fact is that it will almost always run faster as a tuned, compiled stored procedure than an ad-hoc query, and in my world, query result times are what matter. i just don't see what the competition is.

i mean even simple things like analyzing the result set. i'm supposed to send all that data to the client for it to analyze what it wants? for some of my stuff that is a LOT of data. i laugh just thinking about the phone call where i explain that to get that result set i have to send you gigs of data so your computer can figure out what it really wants (granted a situation like this is rare, but they do happen).

and using some 3rd party programming language to handle this as a library or something would suffer from additional suckage. now i can only program in this language. and what happens when i want to update this library? as far as i can tell it'd be the same as just compiling the logic into the app.

i work with this stuff all the time, you can't just be recompiling and shipping out new code everytime a table changes (or is even added or removed). i'd never get any work done.

Re:Good or bad?

[dubl-u]

Alternatively if it's about data, it should be in the DB. i.e 'Users have a Type which can't be null unless the user is Inactive'.

Maybe this is true for procedural languages. But for OO languages, your objects should never be able to get in invalid states. If they can, it's usually a sign of poor design OO or poor unit testing.

If your developers are writing code that's putting bad data into the database, I'd say you should fix the code rather than duplicating rules about the data in two places. And if you can't fix the code, then fix the process that produces the code.

If, in the interim, I wanted to make sure nothing bad got in the database, I might use stored procedures. But instead I mightwrite a consistency checker that ran at night, rather than putting a lot of hopefully redundant code in the critical path for my apps.

Re:Good or bad?

[dubl-u]

Contrast this with the fact that just about every language supports a database connectivity that allows calling stored procudures. So their perl code logs into your database and calls your API -- easy.

Until you need to do a schema change. Then you're stuck, because you've got stuff in just about every language calling your database directly.

Your spiffy java API for checking whether a series of transactions is properly ordered isn't callable from their COM object or vice versa.

Then put a simple socket interface on your Java API. Pretty much every language that talks SQL can talk over a socket. Then you have a single clean interface with just what you need, rather than a big, complicated interface that people will access in all sorts of ways that the original design won't expect.

Re:Good or bad?

[Anonymous Coward]

And ones that don't "talk SQL"...

Re:Good or bad?

[siberian]

Schema change is irrelevant, its hidden in the procedure. Your code no longer cares about mundane 'schema'.

Re:Good or bad?

[dubl-u]

Schema change is irrelevant, its hidden in the procedure. Your code no longer cares about mundane 'schema'.

Yeah, if you completely hide the fact that it's a SQL database, then you can get a clean API. We're proposing the same solution, which is to completely hide the database. We're just talking about doing it at different levels.

Personally, I think the better developer tools, the bigger pool of developers, and the lack of vendor lock-in all point to putting that clean API outside of the database, not in it. But as I've mentioned elsewhere in this thread, I think there are circumstances where it can make sense to put the core of the app in the database.

Of course, either way you lose the much-touted ability to use the database as a general-purpose tool. But I think that's fine. No matter where you put your business logic, I don't think you should let people run arbitrary SQL against your production servers. If they need to do weird stuff, let them do it against a copy of the data on server where their quadruple-nested SELECTs and sloppy UPDATEs won't screw with the production apps.

Re:Good or bad?

[Miniluv]

Even the dataset just isn't that big. Besides which, stored procedures don't even enter into this. The dataset should be the same one way or the other, since query should return the same as the SP would.

Re:Good or bad?

[Miniluv]

My point was that the posters argument was ridiculous. The dataset should absolutely be trimmed on the database server, thats the whole friggin' point of the database. Databases are good at refining the scope of the data set through where clauses, why reinvent the wheel on the app server side? The bandwidth argument is vaguely relevent then, but its still not the compelling reason.

Re:Good or bad?

[bwt]

First off, schema changes are pretty rare once you have production data. Sometimes you add fields and tables, but almost never do you refactor the data model because either you can map your existing data, which proves you haven't achieved any benefit, or you can't which proves you've achieve negative benefit.

Despite this, your argument actually proves a benefit for SPs that I had forgotten about -- they provide encapsilation from the data model. So all your client code written in different languages works fine as long as you preserve the API.

As for programming java-to-COM using a socket, I've actually done that, but trying to get another organization to write their side of such an interface is usually a blank-stare generator. They would probably think for a while and suggest using ODBC instead.

I work in a military environment where the firealarm would probably go off if a port opened up that networking couldn't identify. "Oh, don't worry about that port -- it's the custom socket code I wrote" would probably get me sent to gitmo.

Re:Good or bad?

[dubl-u]

First off, schema changes are pretty rare once you have production data. Sometimes you add fields and tables, but almost never do you refactor the data model because either you can map your existing data, which proves you haven't achieved any benefit, or you can't which proves you've achieve negative benefit.

For you, perhaps. In my experience, schema changes happen fairly frequently as we add new features or refactor the code base so that it more clearly expresses our understanding of the system.

I think the reason most people don't change their schemas is because they're hard to change. Once I did the work to made them easy to change, I was surprised how often we changed them.

As for programming java-to-COM using a socket, I've actually done that, but trying to get another organization to write their side of such an interface is usually a blank-stare generator. They would probably think for a while and suggest using ODBC instead.

There's nothing wrong with doing that, then. If the protocol they want ends up looking a lot like SQL queries or SQL stored procedure calls, then that's a fine way to write a protocol. Or you can point them at one of the pre-built toolkits for web service APIs; that may be enough of a starter that they think it's just as easy as doing SQL.

What's the point of this question?

[joshuao3]

There are no real negatives to stored procedures from a development standpoint. Databases that support them, assuming cost is not an issue, will always be prefered over those that do not. Why is this question even being asked on an open forum such as this?

Re:What's the point of this question?

[bwalling]

There are no real negatives to stored procedures from a development standpoint.

Other than the fact that they suck, there are no negatives! Seriously though, the database is for data, other languages are for business logic. Why split your business logic between your class library and your database server? Keep the logic in one place. Isn't that why you created the class library in the first place?

Re:What's the point of this question?

[heller]

...the database is for data...

Ah yes. so transactions certainly don't belong in a database. Nor do constraints. And definately no sorting! Or reporting. We must remove all aggregation functions because that's logic and not data!

Please. There is always logic that applies to data specifically. If there wasn't, there would be no real reason to have a seperate database. ** Martin

Re:What's the point of this question?

[arkanes]

An RDMBS (as opposed to just a database) is actually for manipulating data, not just storing it. Otherwise you'd just use flatfile for everything and implement all the relational logic in your app code. The database can execute stored procs far, far faster than your app code can perform the same functions. Using database side stored procs gives you the exact same advantages as a class library with additional performance and security options. There's no loss. Why not use them?

Re:What's the point of this question?

[innosent]

Thank you, just to add a bit:

Would you want to send 5 million rows to the application just to check a few fields in each of them, and how they relate to records before and after them? Hell no, sending 5 million rows uses a ton of bandwidth, even for small row sizes. Also, SQL is a language for set operations, while most (99%, Lisp/etc are other 1%) application languages are designed around a single value, not a set of values. For example:
To check if array a[50000] is a subset of array b[2000000], in most languages, you must somehow iterate through both arrays (sorting would help, if possible) and see if each value in a (all 50000 of them) also exists somewhere in the 2 million values for b.
In SQL, something like SELECT COUNT(*) FROM a WHERE a.key NOT IN (SELECT key FROM b) would do the same thing, only much faster than downloading 2,050,000 rows and comparing them. Each vendor handles internal set operations differently, but all of them optimize their internal data structures and abilities to do exactly these types of operations.

Re:What's the point of this question?

[Oligonicella]

"Other than the fact that they suck"

How trite.

"Why split your business logic between your class library and your database server?"

Because there is logic which is involved with the way the data is stored. This is not synonymous with the way it is used at the app end.

"Isn't that why you created the class library in the first place?"

No. The class library is there to provide a view into the data. There may not even be a class library at the data end.

It is my job to provide the data to the app. What the app programmer does with it is of no concern to me. What I do to get it or structure it is of no concern to them. Hence, the two job titles.

Most app programmers only think they know how to manage data.

Stored Procedures are a must

[Anonymous Coward]

Particularly for an application where you are returning large amounts of data, stored procedures hold a distinct advantage over dynamic SQL queries in that, if the SP is designed correctly, the database has pre-optimized the query plan at compile-time and runtime execution is therefore much faster. It also allows for underlying table structures to change without impacting your application logic.

Also, when it comes to long-term database maintainability, putting your database logic in stored procedures allows the db admins to get an accurate overview of what objects/tables are in use and which are no longer needed. At my company, where we have over 20 databases, this is an absolute must.

Generally speaking, I use dynamic SQL during initial development and move to stored procs for QA and production.

Re:Stored Procedures are a must

[Anonymous Coward]

(((It also allows for underlying table structures to change without impacting your application logic.)))

What about views?

Re:Stored Procedures are a must

[Bedouin X]

SPs can also make switching web development platforms pretty easy if all of your special functions and dynamic queries are already coded.

I can't speak for PL/SQL but T-SQL is pretty simple to pick up and anyone who knows a 4th generation language should have no problems designing their own logic. It is probably best to have a database czar to manage the creation of objects on the database though. If the database guy is the bottleneck it's time to get a new database guy, that simple.

Re:Stored Procedures are a must

[Bedouin X]

I may be incorrect but views are still not as fast as SPs as SPs are compiled code. Also, you can't really pass arguments to views unless you're using dynamic SQL, which brings you back to square 1.

Re:Stored Procedures are a must

[complexmath]

Very well said. The DB applications I've written use stored procedures exclusively for these very reasons. If a DB does not support stored procs then it isn't even a candidate.

That said, views do have the same performance advantages, but don't have quite the same degree of encapsulation. If I had to I'd use views but they still aren't as effective as stored procs.

Re:Stored Procedures are a must

[RangerFish]

Views don't let you change the format of the data, all they allow you to do is to present data from multiple tables as if they were in one table.

Re:Stored Procedures are a must

[Anonymous Coward]

Sure they do..

select blegga, case when foobar = 'fu' then 'go home' else 'ok' end as dood
from dual

Hmm... as long as you don't need to do procedural stuff, views can be as simple or as complex as sp's.

Re:Stored Procedures are a must

[Bat_Masterson]

Views are meant to isolate applications from the physical representation of the data (ie. the tables). Their flexibility is in their ability to give a (typically read-only) application a logical view of the database that doesn't fit third normal form.

Re:Stored Procedures are a must

[Anonymous Coward]

Particularly for an application where you are returning large amounts of data, stored procedures hold a distinct advantage over dynamic SQL queries in that, if the SP is designed correctly, the database has pre-optimized the query plan at compile-time and runtime execution is therefore much faster.

Except in most real world applications where querying/loading data is performed with variable paramaters which defeats the purpose of the optimization and renders them effectively useless.

Re:Stored Procedures are a must

[RangerFish]

And if you do...? Point taken though.

Re:Stored Procedures are a must

[Osty]

I can't speak for PL/SQL but T-SQL is pretty simple to pick up and anyone who knows a 4th generation language should have no problems designing their own logic

While it's true that the language and syntax is not difficult to pick up, that doesn't mean you'll instantly be proficient, or understand concepts in the language. For example, and from experience, I've found that it takes most people some time (couple weeks to a couple months) to fully wrap their head around T-SQL's set-based approach. Most people come into T-SQL and immediately start writing cursors and iterative logic. While it certainly works to do things that way, it's going to be horribly slow in most cases (hey, sometimes you have to use cursors). So, the biggest hurdle is not learning the language, but learning how to think in ways the language wants.

Re:Stored Procedures are a must

[Anonymous Coward]

For what it is worth, views are not just used for multi table joins but can also be used to give different users/roles different levels of access to a particular table. The classic example of this is employee information, i.e. employee number, employee name, employee department, salary, home phone number, etc.

Not everybody in the company should be able to look up employee salaries so a view of the employee table can be created to only show the non-salary information. So instead of granting access to the table, you would grant access to the user/role.

Re:Stored Procedures are a must

[kupci]

Yeah, but then you are stuck with a certain database. So I would say your better bet would be something like J2EE if one of your design goals is to be cross-platform or even cross-app servers.

But even there, there are sure to be differences. Main thing is to make sure you keep it simple. I'm currently using a product called omniORB for Corba development, and the designers went out of their way to stick to plain and simple C++, no fancy stuff, so they would have no trouble porting it.

Re:Stored Procedures are a must

[HashDefine]

The distinct advantage of pre compilation is a boon and a bane and actually the pre compilation is helpful ONLY for cases where the stored procedure paramters take typical well distributed values - since most RDBM's create the query plan the first time the SP is executed.

If your SP's take values that vary a lot E.g. a date range that might be as wide as few minutes to a few years you might be better of from a performance standpoint by recompiling the sp each time it is executed.

One BIG benifit of sp is security - you can typically give execute rights to stored procedures without giving rights to the underlying objects.

I often resort to SQL String creation for complex search type screens which have a bazillion options - since the dynamic SQL gives better performance than a complex SP with a varying number of parameters but for simple create, update and delete type of stuff i usually stick to sps

Conclusion - Like everything in life you cannot generalize one approach to be better than the other

Re:Stored Procedures are a must

[ldspartan]

That seems... crazy. It seems like the RDBMS should be keeping statistics on queries and data, particularly those that drastically effect the run time / complexity of stored procedures.

Isn't the whole point of a decent database a decent query optimizer? Doesn't that include adapting to the data? The first time an SP is run, it may very well be against an empty or sample data set. Seems dumb (and unlikely) for the query plan to be completely static.

--
Phil

Re:Stored Procedures are a must

[Spy Hunter]

the database has pre-optimized the query plan at compile-time and runtime execution is therefore much faster

This is a common misconception, but it is not true any more, at least for MS SQL Server. I quote from MSDN:

"SQL Server 2000 and SQL Server 7.0 do not save a partially compiled plan for stored procedures when they are created. A stored procedure is compiled at execution time, like any other Transact-SQL statement."

Re:Stored Procedures are a must

[Anonymous Coward]

Good point, but if the developer doesn't understand "set-based" logic, they probably shouldn't be working with the data store at all. Iterative logic in the DB can be bad, but it's usually much worse when done on the client side.

Someone else should come in and write a layer that gives the junior guys them what they need.

Re:Stored Procedures are a must

[Osty]

Someone else should come in and write a layer that gives the junior guys them what they need.

Think longer-term. Someone else certainly should come in and write a layer that gives the junior guys what they need, but that person should also take to educating the junior guys at the same time. How else will the junior devs become non-junior devs if they're not given the opportunity to learn and grow, and make mistakes? Besides, in the long run, it'll be horribly inefficient to have to push all database access through a single person. Get a good system in place, write queries and such that are needed right now, and then start teaching the junior devs, perhaps by pair programming with them, or code reviewing with them, or giving talks or short classes on best practices.

Re:Stored Procedures are a must

[Jorrit]

It still holds for Sybase though. Sybase makes a query plan for a SP the first time it is used. At least in 12.5 it is still like that. We're using that here.

Greetings,

Re:Stored Procedures are a must

[Anonymous Coward]

That is absolutely correct. It makes more sense to find/teach people in set-logic than to adopt magic tech (whether that be LAMP hacks or J2EE) that pretends that corporate developers don't need to understand databases.

Re:Stored Procedures are a must

[Anonymous Coward]

The part that's left out is that MS-SQL will cache the compiled query plan. Therefore, if you use the SP frequently, there's no compile overhead.

But its a good point that MSSQL doesn't treat Procs any differently than well-defined adhoc queries. (aka those with parameters)

Re:Stored Procedures are a must

[Spy Hunter]

I left that part out because it's irrelevant to my point, which is it doesn't matter if you use stored procedures or not; there's no inherent speed advantage. That's all I was trying to convey.

Re:Stored Procedures are a must

[bug]

Correct me if I'm wrong here, but databases that are worth a damn (e.g., Oracle) will cache optimized query plans for a helluva lot more than just stored procedures. Having dabbled more than a little in database design, administration, and programming, I know that Oracle will happily optimize a query and keep that cached in the shared pool when you do something like DBI's prepare statement in Perl. Later, you can use substitution to change values in your WHERE clauses (and other clauses) and make similar but subtly different queries while still taking advantage of the shared pool. I believe that MySQL 4.x has similar prepared query capabilities. Repeat after me: Bind variables are a good thing.

SPs and triggers have their uses, but in general the language for them is far too weak to be very useful. I think this is true even though Oracle has thrown Java into the system. Oracle made a marketing decision to choose the Java language, which simply isn't a very good compliment to SQL, instead of other languages better suited to data parsing and processing like Perl. I suppose stored procedures look really attractive when you're using another language even more poorly suited for data processing for your application, like C or Java, but given Perl or PHP, I only see limited use for SPs.

Re:Stored Procedures are a must

[Dr_Barnowl]

if the SP is designed correctly, the database has pre-optimized the query plan at compile-time and runtime execution is therefore much faster.

Actually, this isn't true of MSSQL2000. It was true of SQL7, which cached the execution plan as you describe, but SQL2k caches the plan in memory on first execution and flushes it periodically ; this isn't just true of stored procedures, which are now only stored as text, but applies to any statement executed by the db. This basically removes execution plan caching as an incentive to use stored procedures in SQL2k.

SPs are good

[TommyTyker]

I actually miss them in my current job with mySQL. I used to like the way you could run a stored proc every X period to copy "live" data to "public" areas. Or, archive "public" data after "x" period of time. But then again, I am a micro$oft developer at heart, and all this Perl, CGI, Java, (even COBOL) on old RS6000 systems gives me a headache sometimes.

Re:SPs are good

[Osty]

I actually miss them in my current job with mySQL. I used to like the way you could run a stored proc every X period to copy "live" data to "public" areas. Or, archive "public" data after "x" period of time. But then again, I am a micro$oft developer at heart, and all this Perl, CGI, Java, (even COBOL) on old RS6000 systems gives me a headache sometimes

You miss stored procedures because you can use them to fake replication? Ugh. I can think of any number of other reasons to miss stored procedures, but your example isn't one of them. Any good database will have replication support built-in, so you don't have to fake it with scheduled jobs.

Re:SPs are good

[Anonymous Coward]

Well, I wouldn't use replicaiton to "fake" data transformation or archiving.

Re:an important question..

[firebus]

why are you posting if you don't have an answer?
you are only giving me a headache.

SPs are GOOOD

[Anonymous Coward]

Data controlled by the DBA good. Data controlled by developers or users... bad, very, very bad.

good

[menix]

personally I like them. The centralization of code is a big advantage, but I think the biggest advantage is typing the incoming data. This plugs some of the security holes.

My favorite SQL

[Anonymous Coward]

SELECT * FROM users WHERE CLUE = TRUE
0 Rows Returned

or if there are rows, fix it quick!

DELETE
FROM users
WHERE CLUE=TRUE

Re:My favorite SQL

[Anonymous Coward]

Yikes, I hate when people use TRUE like that. Its redundant!

Re:My favorite SQL

[Atrax]

Yikes, I hate when people use TRUE like that. Its redundant!

<retentive type="anal">
Actually, that depends on the language implementation
</retentive>

Re:My favorite SQL

[Anonymous Coward]

Touche. That looked like SQL to me. I dont think ANSI92 even defines a boolean datatype

Re:My favorite SQL

[builderbob_nz]

{begin rant}
The variable 'clue' should really be named 'HasClue'. And people wonder why others have a hard time understanding their coding...
{end rant}

Postgres supports

[Sir_Real]

One of my favorite thing about postgres is it's support of plpythonu (python stored procedures) and the recently added java support.

Just define the function (in java or python) and SELECT it with whatever arguments you've designed it for. I don't know what the overhead involved is, but we used it more for convenience than anything else.

I was told basically, "the fewer the better" and "keep them confined to the innermost loops."

Two answers.

[MisterFancypants]

You're going to get two answers. You'll get the "always used stored procedures" answer from people who actually do real database work and the "never use stored procedures" from the people who hack small websites in PHP.

If I were a bit more of a tinfoil hat wearing man, I'd be Slashdot makes some of these "Ask Slashdot" topics up because the ensuing flamewar will cause more page hits than usual.

Re:Two answers.

[Anonymous Coward]

That's funny, I was going to make the comment but instead of "people who actually do real database work" I was going to say "people who think it's still 1980 and who think changing software should take an act of congress" and instead of "people who hack small websites in PHP" I was going to say "people who use dynamic languages to deliver high-volume applications in weeks instead of months".

I guess it's a matter of perspective.

Re:Two answers.

[TheSpoom]

From the perspective of someone who, at the risk of killing my ego, is in group two, why are stored procedures a good thing? I have a feeling that my later level database design courses in college will tell me, but I'd like to know.

Re:Two answers.

[puppetman]

Actually, I'm a DBA who is responsible for about 20,000 lines of stored procedure code that I did not originally write, and I still think they are a bad idea.

Data constraints in the database, and appliation logic in the application.

Re:Two answers.

[Daniel Dvorkin]

If I were a bit more of a tinfoil hat wearing man, I'd be Slashdot makes some of these "Ask Slashdot" topics up because the ensuing flamewar will cause more page hits than usual.

Based on the sentence preceding the one I'm quoting, I'd say the main flamebait here is your post. How do people like me, who develop corporate LAMP sites with a great deal of "real database work" going on behind the scenes, fit into your neat little conception of who is and is not a real DB developer? Asshole.

Re:Two answers.

[Bedouin X]

They let you abstract data access procedures. It's kind of like asking "why are functions a good thing?"

Because a person who knows what the hell they are doing can write a good function (or stored procedure) containing what the hell they know so that a person who doesn't know a hell of a lot about what the first person knows but a lot about something else on a higher level can interface with that logic and get a result without having to be a guru.

Or something like that...

Re:Two answers.

[Anonymous Coward]

If I were a bit more of a tinfoil hat wearing man, I'd be Slashdot makes some of these "Ask Slashdot" topics up because the ensuing flamewar will cause more page hits than usual.

Call me weird, but I like these types of flamewars. I've done a little PHP hacking with MySQL and I'm curious about the issue of stored procedures, since I've never had the opportunity to use them.

Re:Two answers.

[silas_moeckel]

OK I'll aswer the question not as somebody thats in either camp just the guy that gets to scale the thing. First off I dont care out nice OO design or business logic etc etc thats all just PHB fluff. Stored procedures allow you to scale a DB if you do it well.

Pretty much it boils down to this if your web app can call one stored procedure and generate the responce your golden. The more time your spending going back and forth grabbing data and colating it just takes to long to scale as you have a bigger memory foot print blocking call sockets theads etc for a longer time. Now thats a perfect world were writing stored procedures is just as easy etc etc etc as writing your web app. Your best results are often from good coding in the rough out stage and replacing whole functions with stored procedure wrappers after profiling where your app is spending most of it's time under load.

Re:Two answers.

[Anonymous Coward]

This is GOD. You are a great disappointment. Please do not procreate or as I will be forced to to flood the earth once again...

Re:Two answers.

[Anonymous Coward]

"You'll get the "always used stored procedures" answer from people who actually do real database work and the "never use stored procedures" from the people who hack small websites in PHP."

Can we add "proof by claiming to work in a bigger company" to the list of stupid assertions?

Re:Two answers.

[Unoti]

...people like me, who develop corporate LAMP sites...

No offense, but doesn't this mean you put together web sites in PHP, as the grandparent suggested?

Re:Two answers.

[Daniel Dvorkin]

Yes, exactly. Specifically, MySQL-driven Web sites with a great deal of "real database work" going on in the background. What offended me was his strong implication that the two -- PHP/MySQL programming and real DB development -- are mutually exclusive. It's an attitude that I see a lot on Slashdot, and it pisses me off.

Re:Two answers.

[Anonymous Coward]

Sorry you're maintaining a nightmare. You really think it'd be easier if all that code was all over the place?

Maybe for you, but that's only because you're a DBA :).

Re:Two answers.

[puppetman]

It would be better if that logic was stored in stateless Session EJBs; one place for all that logic that can be accessed from anywhere.

Throw in Hibernate to get rid of the SQL-specifics of the database and reduce the OO-RDBMS nightmare.

I actually do more Java coding than database management (I've set it up so it's pretty much self managing); I wouldn't let it get all over the place, because I would be the person writing the session beans.

Re:Two answers.

[John Courtland]

Yeah... Shit like that gets real old, real fast. Like the 'real' DB guy is doing something hard. Please. I think people like that need to step back and see that there are a lot harder things involving computers than doing 'real' DB dev. Then maybe they'll get off their high horse and shut the fuck up for once.

Re:Two answers.

[Atrax]

...people like me, who develop corporate LAMP sites...

No offense, but doesn't this mean you put together web sites in PHP, as the grandparent suggested?

Linux
Apache
MySQL
PHP

for the acronymically challenged

Re:Two answers.

[Anonymous Coward]

Lets just say that PHP Developers tend to speak for themselves.

(This goes beyond slashdot -- you're fighting an industry-wide perception of where PHP 'stacks up'. There's been some amazing stuff done in Visual Basic over the year, but it's still broadly seen as programming for [cheap] dummies.)

Re:Two answers.

[Master of Transhuman]

If you're using MySQL, people who work on Oracle will consider you as NOT doing "real database work".

This may be incorrect depending on what you're doing, but that's the perception. MySQL simply isn't (or wasn't until recently) in the same class as Oracle, DB2 or even PostgreSQL as a database. And the lack of stored procedures (as well as triggers and other capabilities) was the reason.

MySQL is great for doing what it does - keeping up with Web hits while providing a database backend for a Web site. But it's weak compared to the bigger commercial databases (and PostgreSQL) in terms of database capabilities. That's a simple fact.

Re:Two answers.

[Anonymous Coward]

Originally it was Linux Apache MySQL Perl, so it was a fair question.

Re:Two answers.

[johnstoj]

I'll give you two good reasons:

1) It makes the what the user sees as a frontend much easier to replace. It also makes it possible to access an application from various sources.

2) If there are complex calculations that are part of your business, wouldn't you want those located in one spot? You would also want the data that the query has to process in one location too.

Re:Two answers.

[Hangtime]

That's probably about the number of lines in one small application backend that I would write. My questions for you are:
Were the procedures well documented?
Are there spec documents for the procedures?
Were the procedures written by someone who knew very little or a great deal about databases?

All of these could create problems regardless of where this code was in a web page, a middle-tier object, or a stored procedure.

Re:Two answers.

[johnstoj]

As a DBA it should not be your job to write or maintain the code. There should be a separation of DBA and developer just like church and state should be separate.

Re:Two answers.

[cas2000]

> Specifically, MySQL-driven Web sites with a
> great deal of "real database work" going on
> in the background.

you're not doing "real database work" for the simple reason that you're not using a real database.

you're using a toy - and no matter how good you are at using a toy, it's still a toy.

> PHP/MySQL programming and real DB development --
> are mutually exclusive.

i think the original poster was commenting on mysql, not on php.

you *can* do real programming in php, but i can't see why anyone would want to. it's basically a stripped down weird version of perl for people who are too scared of perl to bother finding out that if they can program php they know about 95% of what they need to program perl.

Re:Two answers.

[Anonymous Coward]

Well...

How do people like me, who develop corporate LAMP sites with a great deal of "real database work" going on behind the scenes, fit into your neat little conception of who is and is not a real DB developer? Asshole.

Since it isn't obvious to you, you're not a real DB developer. You're the equivalent of one of the sixteen-year-olds who open up a Sourceforge project for their Kewl App They Wrote Themselves. But don't worry about it, since most businesses don't need real DB developers, there's plenty of niche space for armchair DBAs like yourself ...

Re:Two answers.

[1001011010110101]

It would be better if that logic was stored in stateless Session EJBs; one place for all that logic that can be accessed from anywhere.

From anywhere that can instance Session EJBs , that is :)

Re:Two answers.

[crazy.tyae]

I'm a DBA who is responsible for about 20,000 lines of stored procedure code that I did not originally write, and I still think they are a bad idea.

And this is a surprise? I'm migrating 591 stored procedures, none of which I wrote, to a different flavor of database server at the moment. As you can imagine, this gets old pretty quick. However, IMO the performance benefit of having pre-compiled query plans dramatically outweighs the downsides of using stored procedures.

Re:Two answers.

[NutscrapeSucks]

That's why you add a SOAP layer. :)

obslash: I have a book around here which describes in painstaking detail how to convert 25 lines of icky business logic PL-SQL to 5,000 lines of EJB code. No wonder people outsource.

Re:Two answers.

[Ded Bob]

You're going to get two answers. You'll get the "always used stored procedures" answer from people who actually do real database work and the "never use stored procedures" from the people who hack small websites in PHP.

Some PHP developers like stored procedures. I added PostgreSQL support to FreeTrade, an e-commerce system. When I added affiliate support, which necessitated RMA support, I found using stored procedures for triggers much easier and faster than creating queries to pull everything into PHP and processing there.

If I were a bit more of a tinfoil hat wearing man, I'd be Slashdot makes some of these "Ask Slashdot" topics up because the ensuing flamewar will cause more page hits than usual.

I have thought that for quite some time. :)

Re:Two answers.

[itwerx]

Replying to both the parent and the grandparent...
I think the answers depend on one's past experience. I have actually worked with a number of different database architectures ranging from extremely light-weight web front-ends with virtually all processing done on the back end to monolithic Access databases (don't ask!) with various odd (and dumb) datastores on the back end. And of course various DB's which were more of a mix (e.g. JDBC clients with on-the-fly query strings against against Oracle or MS-SQL back ends with their own respectable selection of stored procedures and views).
All I can say is it really depends on the project simply because there's usually some sort of existing dataset whose current configuration will usually play a heavy role in determining how best to go forward.

Shrug, just my $2-CDN

Re:Two answers.

[PopCulture]

... and thats different from a programmer responsible for 20,000 lines of java/C++ code that they did not originally write because of how?

Re:Two answers.

[Daniel Dvorkin]

If you're using MySQL, people who work on Oracle will consider you as NOT doing "real database work".

Oh, I'm well aware of that. They are, of course, wrong.

I've worked on Oracle databases. My decision is an informed one: I prefer MySQL, I don't use it because it's the only thing I know. There are plenty of Oracle (and MS SQL, and PostgreSQL, and for that matter Access <shudder>) bigots out there who refuse to accept that this is possible. They're assholes, and not shy about proving it.

Re:Two answers.

[Daniel Dvorkin]

[wearily] My site runs the entire business flow of the company I work for, everything from lead management to financials to supply chain to shipping. We ship some pretty damn complex stuff, which requires managing a great deal of data to get the product out the door. If you think what I do is playing with toys because of my choice of platform ... y'know, I have to wonder how many mainframe programmers there were, back in the day, who sneered at people who wrote software for those little "toy" PC's.

Re:Two answers.

[Daniel Dvorkin]

Yes, that was my point. I do real database work with the platform, something the original poster seems not to regard as possible.

Re:Two answers.

[Daniel Dvorkin]

Since it isn't obvious to you, you're not a real DB developer.

Good thing I have people who can't sign their names to let me know this!

Re:Two answers.

[vegetasaiyajin]

Throw in Hibernate to get rid of the SQL-specifics of the database and reduce the OO-RDBMS nightmare.

Yes. Get rid of SQL specific and go to HQL specifics. I do not know why products like hibernate call themselves Object Relational mappers.
They are Object Tabular mappers. They can only map tables or views to objects, but not other types of relations such as queries.
For something closer to a true Object Relational mapper, try iBatis SQL Maps. Ironically, they say their product is not an object relational mapper, but is the only product I have seen that actually can map any relation to objects.
SQL rules. HQL, JDOQL, EJBQL, and *QL (except SQL suck.

Re:Two answers.

[Anonymous Coward]

Me too, like others have said, go to your manager and say: "Dammit Jim, I'm an administrator not a programmer." DBAs, stored procedures are for programmers, keep to your schemas.

Re:Two answers.

[innosent]

Ok, let's step outside your universe, where MySQL can handle anything you can throw at it. Oracle/DB2 are as good as they are because they scale much better than MySQL. If one machine isn't fast enough, or one array isn't fast enough, there's not much you can do about it with MySQL, while DB2 and Oracle have clustering capabilities designed for LARGE databases. Web sites don't typically use large databases, but in some cases do require them, or may tie in to a much bigger database for all of the other things a company does. Very few companies just sit there and watch their page counters go up, you have to do something with the data when it comes in, and if you're a big company, that's a LOT of data, and it may be necessary, or even legally required, to have all of the data for the last x years available to anyone within 5 seconds.

When you hit the limits of data storage speeds, you have to partition your database somehow, and Oracle and DB2 allow you to do this much more easily and efficiently than MySQL does. I'm not saying that MySQL isn't a real database, it certainly is, and a pretty good one, it just doesn't have the features that larger businesses often require. It may get there, but it isn't there yet. Of course, Microsoft's SQL Server and PostgreSQL don't have those features, either. Then again, MSSQL, PSQL and MySQL don't cost as much (or any) as DB2 and Oracle, either.

Re:Two answers.

[bwt]

It usually comes down to the fact that a database driven system has enormous performance considerations at play. It is very easy to write SQL or an algorithm that uses SQL that requires an incredible amount of resources. Stored procedures allow this data access to be done without having to pipe all the data over the network, and it's often the easiest way to assure that the development work of finding the best performing way is done once and reused.

Having said all that, I do not think there is any way to say "use SPs for everything" or "use SPs for nothing". A well designed middle tier that runs on web servers can be a lot more scalable if it can hold the right data in the middle tier (the "hot" data), and thereby substantially reduced the database load. To have the "hot" data dense on the web server requires leaving data access to the volumes of not-hot data in stored procedures. Also, for things that change data, you typically have greater trust that the stored procedure will abide by all the special data rules. When you have a stored procedure that works, you can use database security to ensure that by god, you can only add one to this column if you subtract one from some row in this other table. Typically DBA's don't trust middle layer programmers as much with data integrity and take the attitude that if the database doesn't guratantee it, it will eventaully be violated.

The ultimate answer is that it depends on your data access patterns and data logic rules, and how well your web layer team can assure that it respects the performance and data integrity needs of the database.

Re:Two answers.

[Trifthen]

That's great! Now, just port your "application logic" to your PHP web front end, your C++ back-end utilities, your Java graphing system, and Forms reporting. See, enterprise level applications often have many vectors of accessing the data, which means porting your application code to four different languages and platforms, or just put the damn data access and integrity logic in the database.

Re:Two answers.

[Nkwe]

The problem with a lot of this logic is that it assumes that there is only one "application" that uses the database.

In larger systems you have a database that holds your data and multiple applications that operate on it. The whole idea around relational databases is that all your data and associated relations are put together in one logical model.

Some applications may be web based, some may be thick client, and some may be some other architecture. If you put your business rules at a stored procedure level, you can ensure that these rules are enforced without regard to which application is accessing it.

If you put your application logic on your web server or on your middle tier transaction server, any application that connects directly to the database is going to bypass those business rules. If you actually care about business rules and security rules being enforced, you will put them into your database.

As has been mentioned, the downside to this is that it will be expensive if you want to change database platforms. Choose wisely. Note that if you are not going to make use of the advanced query functions and the ability to encode business logic and security within the confines of your database, you might as well purchase a less expensive one.

Re:Two answers.

[xelah]

If you put your application logic on your web server or on your middle tier transaction server, any application that connects directly to the database is going to bypass those business rules. If you actually care about business rules and security rules being enforced, you will put them into your database.

There are plenty of good reasons why you'd want your multiple applications/front-ends/whatever manipulating your data through some sort of business rule layer. Yes, you can build this layer in to your database as stored procedures - but this isn't the only way of doing it. You can also have a middle layer which talks to the database and through which everything else goes.

If you need to have disparate applications that, for some reason, can only talk to the database directly then you are certainly going to want your stored procedures. They do have certain disadvantages, though. Are your stored procedure languages as powerful and flexible as the languages you might choose for your middle layer? Can they access third party libraries for, say, matrix algebra or option pricing? Can they make CORBA calls to external systems? Can they charge people's credit cards? Even if they can do these things would you really want to do it from your stored procedures?

Putting your business logic and business rules in to a separate middle layer is a perfectly legitimate design choice. There really aren't that many languages or platforms which can't access such a layer (Perl, Java, C++, Python, Java and many other languages all have CORBA ORBs). This layer can choose to use stored procedures to implement some of its functionality if that's the right decision (to avoid transferring large amounts of data, for example). It can choose to fetch its data from a cache or from an external system, too. You still have a single place to look when you want to modify or optimize your data structure or queries or business rules.

Look at the requirement, look at your system, consider the options and put this layer in the most sensible place for your software.

Re:Two answers.

[Daniel Dvorkin]

I do understand that. Of course Oracle has features MySQL (or MS SQL, or PostgreSQL) lacks. And if my company ever reaches the point where we need those capabilities, I may well recommend a switch. But "real database work" IMNSGDHO isn't just about the number of rows your server can handle. Any monkey can write "select * from hugetable" and then brag about the size of his result set. (Compensation for something else, perhaps?) "Real database work" to me means writing tight, well-crafted architectures and queries that store and deliver the information the user needs, work reliably and handle weird cases, scale well, and execute fast. That's real DB dev, and if you're doing it on Oracle or MySQL or anything else, your work is no less real than anyone else's.

Re:Two answers.

[SiMac]

Well, I guess Slashdot, while serving millions of pages a day, doesn't run on a real database then.

Re:Two answers.

[harborpirate]

I don't think it is merely a matter of perspective. I don't think there is any doubt that at the large enterprise level, stored procedures are an absolute must.

I deal with applications (both web and client) which must get data from a database that contains billions of rows of confidential information. We go through terabytes of data each day that we recieve from various sources.

At this level, stored procedures are a requirement, plain and simple. We need a secure application. We cannot allow a lightly secured account such as a web sql account to have access to database tables directly. Furthermore, because many of our tables contain multiple millions of rows, we have to get the utmost in performance. The only way to do this is careful indexing and highly optimized precompiled queries (Stored procedures).

At the large enterprise level, changing software sometimes DOES take an act of congress, literally. But even under normal circumstances, changes in business logic where I work requires approval from a lot of sources - changes are not taken lightly when millions of dollars are at stake.

I'm not trying to puff myself up here or brag. All I am trying to point out is that at the large enterprise level, all the little things become more important. Mistakes are magnified when your data is critical, and your datasets are incredibly large. Setting the index wrong on a table can mean a statement will take hours instead of seconds. I've literally seen that scenario take place.

The security and performance of stored procedures are a virtually a requirement for many applications at the large enterprise level, no matter what year it is.

I think, rather than a matter of perspective, the relative value of stored procedures depends much more on the actual situation.

I realize this has been somewhat of a rant. There are many cases where the relative cost of implementing stored procedures is not worth the effort. In fact, I would guess that the majority of projects out there do not require stored procedures. But when the data gets huge and confidential, stored procedures are the only way to go.

Re:When did Slashdot switch to IIS?

[earthforce_1]

I know this is OT, but there is nowhere else to post this - I thought I was the only one seeing them. Is it the viruses running rampant, or did something break after their last time they tweaked slashcode?

Re:an important question..

[DaveJay]

A bit trollish, but I'll respond: perhaps he's actually smart enough to seek outside opinions, even though he thinks he knows the answer.

That, or he's preparing some kind of presentation/paper to justify the use of stored procs to a boss who doesn't believe in them (or vice versa), and is seeking real-world examples to bolster his point.

Just a couple of possibilities.

Version management

[Anonymous Coward]

What about version management of stored procedures? Yes I know it's possible, but it creates pain. Everyone must have their own copy of the DB otherwise an error by one developer modifying a SP breaks it for everyone, even if they have their own copy of the app. Scripts must be written to ensure that the latest SPs can be applied at the same time as code updates.

I don't like using SPs, but I think version management for me is the nail in the coffin.

Mark

Re:Version management

[Anonymous Coward]

It's possible through Visual Source Safe and Visual Studio, if you're a Microsoft kind of person

Re:Version management

[Miniluv]

What exactly do you think makes versioning stored procedures hard? Do you think developers go about changing SPs in a production environment with testing, committing the SP to CVS/Subversion/SourceSafe/BobsFilingCabinetOfSourc eContral? If so, please forward me the checksum of your resume so I can ensure I never hire you, work near you, or support anything you ever touched.

Re:Version management

[Anonymous Coward]

We use MSSQL and Delphi at work. Database schema is managed through in-house developed maintenance utility. This utility stores all database objects that are common for all our clients (tables, constraints, indexes, defaults, views, stored procedures etc) in a text file. When database schema is changed, new version of this file is generated through the utility. When we release new batch of modules to testing we also release the latest database update. Utility was designed to upgrade form any version of database to the latest one.
To maintain client specific database objects we use the COM DLLs that contain the SQL scripts for all the objects that need to be created and a function that can be called to execute all these scripts.

Re:Version management

[1001011010110101]

Been there done that. I'm using Oracle for DB and CVS for source control, and its a pain in the ass. CVS/Subversion is for stuff that is naturally in a file, not for something that its on the database. I'm still looking for a better match.... I use Oracle Designer on some stuff, and even if it does a great work as a case tool, it tends to create loads of different objects which are a pain to manage with cvs.

Migration

[Anonymous Coward]

Finally, after staying with our DB engine choice for all these years we are acknowledging that they may not be around forever.

I've never taken this argument seriously. If you put all your business logic into a web front-end, then you're locked into that web front-end. If you put all your business logic into the database, you are locked into that database. Sure, both are pains, but you can't consider it a downside for the database without also considering as an upside for the database.

Re:Migration

[Anonymous Coward]

You didn't use stored procs. You migrate the data, change the DSN's on your app tier, and you're done.

And then the System Administrator learns that even trivial SELECT/INSERT SQL can be vendor-specific. Which he contemplates on the way to the unemployment office.

Frankly, it's much easier to find and port stored procedures than to go through lines and lines of application code looking for embedded SQL. But even then, the app's data access layer is often tied to a vendor. Frankly if you didn't design for DB portablity, you're screwed either way.

(The only agreeable part of your post is that Triggers are big kludge and really suck.)

Good.

[Pig Hogger]

They're double plus good, of course.

The idea of a database is to put the whole data-relation logic in the database, if only to insure atomicity of operations.

Because as soon as you rely on an external process to maintain data integrity, you're bound to fall prey to some sloppy programmer who does not understand the data relationships and will not properly maintain the data integrity.

At least, when you use stored procedures, you can concentrate the data integrity logic in only one place, which is easier to control and manage.

Re:Good.

[arkanes]

Well, if you're doing anything of any size (as opposed to something crapped up in PHP), you're going to want a centralized repository of functions and business logic anyway, to ensure the correctness of your DB functions. You could write that as a library in your application code. But since you're going to need it anyway, you may as well do it on the database with stored procs and get the extra layer of security and performance that comes basically for free. The only point I can see to doing it on the application side is that you can use generic (read: lowest common denominator) SQL for everything to maximise your portability. But again, if you're working with anything more than minimal datasets you're going to need the performance boost that comes with db-specific optimizations - in my experience, query overhead dwarfs connection and processing time. If it doesn't, you should probably take a real close look at your app code.

Re:Good.

[philipsblows]

I agree that they're good.

Aside from the virtues already espoused, implementing a database application within the database app means that as long as you can send a "SELECT ..." query to the database, you needn't re-implement your logic across multiple languages, platforms, etc.

What you get is a nice API for your data that is testable and perhaps even stable outside of the choices and requirements of the client app(s).

Re:Good.

[pHDNgell]

Because as soon as you rely on an external process to maintain data integrity, you're bound to fall prey to some sloppy programmer who does not understand the data relationships and will not properly maintain the data integrity.

This doesn't make sense. At my last job, I built what's essentially an ERP system. We did order capture and processing. An order was basically the following:

* billing address
* shipping address
* list of one or more line items
* list of one or more payment methods
* detail on how much each method pays

[more stuff]

OK, make me a stored procedure that can take this kind of structure all at once and do all the appropriate inserts/updates.

I had a design that made it quite easy to add new concepts to the order object and store, retrieve, and update it atomically without any one person having to know about all of the elements that might make up an order.

Re:Good.

[Tony]

Don't mix up data integrity with business logic. You don't need a stored procedure to do all that; stored procedures make sure the billing address makes sense (like ensuring the zip code and city match).

Stored procedures are great when you have complex relationships between data than can't be defined by simple constraints. Any time you need to apply procedural logic to ensure data integrity, you need a stored procedure.

Taking the aggregate information and dealing with it is the job of the business logic, which is the middle tier. But the middle tier should be unable to insert data that is incorrect, no matter how poorly written the middle tier is.

Re:Good.

[pHDNgell]

Don't mix up data integrity with business logic.

I'm talking about atomicity. You can't make a stored procedure that can atomically store my data.

Another example from my current application:

I've got something like six rows of state for every entity I maintain (something around 2 million currenetly). Those six or so rows represent the state (three or four columns) of six different feature on the device I'm dealing with.

The SQL to do this is pretty straightforward:

start transaction
insert/update device_table [...] where dev_id = 83284842
insert/update state [...] where dev_id = 83284842
[one for each different type of state]
commit

That's a pretty simple one, but I'm not sure how you'd accomplish it with an SP.

Even less-so when you consider that the actual transaction *may* perform an insert or update in up to five or so tables, conditionally, depending on whether stuff changed or not.

Re:Good.

[Anonymous Coward]

Atomicity of operations is insured by the transactions and not by putting code in stored procedures.

Re:Good.

[jonnosan]

OK, make me a stored procedure that can take this kind of structure all at once and do all the appropriate inserts/updates

here's your sproc:
CREATE PROC usp_Order_Insert
(
@OrderXML as test
)

AS INSERT INTO Order (OrderXML) VALUES (@OrderXML)

GO

Re:Good.

[jonnosan]

whoops... that @OrderXML should by of type text, not test.

Re:Good.

[Hangtime]

Learn your favorite RDBMS XML syntax. You would be surprised what you mind find. Various snippets of a procedure currently running in our production SQL Server environment today. Sorry about the formatting, Slashdot sucks in code mode. BTW, go check out a book called SQL Server XML distilled for information on how to exactly do what you did but with XML in all in one trip.

<Root>
<PayoutBatch>
<EmployeeNumber>84 5227925</EmployeeNumber>
<CreateFeeScenario>
<Fe eScenario>
<FeeScenarioID>15</FeeScenarioID>
<Fe eScenarioName>Test Me</FeeScenarioName>
<ActiveFlag>1</ActiveFlag>
<FeeRule>
<TranCode>1156</TranCode>
<StartDate>1 /1/2003</StartDate>
<EndDate>1/1/2050</EndDate>
<FeePercentage>8</FeePercentage>
</FeeRule>
<Fee Rule>
<TranCode>4117</TranCode>
<StartDate>1/1/1 979</StartDate>
<EndDate>4/5/2050</EndDate>
<Fee Percentage>6</FeePercentage>
</FeeRule>
</FeeSce nario>
</CreateFeeScenario>
</Root>

EXE C sp_xml_preparedocument @IDOC OUTPUT, @StrXML

DECLARE @FeeScenario TABLE
(
ScenarioID [INT] NOT NULL,
ScenarioName [VARCHAR] (50) NULL,
ScenarioActiveFlag [INT] NULL,
EmployeeID [VARCHAR] (12) NULL,
ActionType [VARCHAR] (50) NOT NULL
)

INSERT INTO @FeeScenario (ScenarioID, ScenarioName, ScenarioActiveFlag,
EmployeeID, ActionType)
SELECT FeeScenarioID, FeeScenarioName, ActiveFlag, EmployeeNumber, 'Insert'
FROM OPENXML (@IDOC, 'Root/PayoutBatch/CreateFeeScenario/FeeScenario',2 )
WITH (
FeeScenarioID [INT],
FeeScenarioName [VARCHAR] (50),
EmployeeNumber [VARCHAR] (12) '../../EmployeeNumber',
ActiveFlag [INT]
)

INSERT INTO @FeeRule (RuleScenarioID, TranCode, FeePercentage, StartDate, EndDate)
SELECT FeeScenarioID, TranCode, FeePercentage, StartDate, EndDate
FROM OPENXML (@IDOC, 'Root/PayoutBatch/CreateFeeScenario/FeeScenario/Fe eRule',2)
WITH (
FeeScenarioID [INT] '../FeeScenarioID',
TranCode [CHAR] (6),
FeePercentage [DECIMAL] (7,4),
StartDate [SMALLDATETIME],
EndDate [SMALLDATETIME]
)

Re:Good.

[geekoid]

Yes, why can't people see this? I have had this duscussion many, many times. SOme people just don't uynderstand that Stored procedures, front end, and Business should be seperate. It is so much easier to design for this, and so much nicer to deal with the application after it has gone live.

Re:Good.

[dubl-u]

The idea of a database is to put the whole data-relation logic in the database, if only to insure atomicity of operations.

There are other ways to do that. Prevayler is one that works just fine.

Because as soon as you rely on an external process to maintain data integrity, you're bound to fall prey to some sloppy programmer who does not understand the data relationships and will not properly maintain the data integrity.

If you have a bad programmer working on bad code, that's probably true. In good code in an OO language, you have three lines of defense.

• The first is the object interface itself. If the object interface doesn't have a way to change something, then there's no way to break it externally.
• The second is the internal object code. It should be written in such a way that the integrity requirements are clear. A developer mucking around in the object's code should be able to tell what the important bits are.
• The third is the unit tests. Not only will they document the purpose and nature of the object, but they will also prevent a careless developer from breaking something important.

For me, this provides a lot more utility than a set of stored procedures does. YMMV, of course, but using the database to warn you about bad code strikes me as a pretty roundabout approach.

Re:Good.

[Pig Hogger]

If you have a bad programmer working on bad code, that's probably true. In good code in an OO language, you have three lines of defense.

• he first is the object interface itself. If the object interface doesn't have a way to change something, then there's no way to break it externally.
• The second is the internal object code. It should be written in such a way that the integrity requirements are clear. A developer mucking around in the object's code should be able to tell what the important bits are.
• The third is the unit tests. Not only will they document the purpose and nature of the object, but they will also prevent a careless developer from breaking something important.

Typical poppycock from a spoon-fed Java kid.

There are other programming languages than Java in life. Or C-- for that matter. What you are describing is a fine way to lock 1u3ers into using a single platform. That's double-plus ungood. Or are you proposing to use OBJECT COBOL ????

And stored procedures can be made inaccessible to DB 1u3ers such as programmers, thus GUARANTEREEING data integrity..

Re:Good.

[dubl-u]

Typical poppycock from a spoon-fed Java kid.

It works for me. My bug rates in production are below one per developer-month, and have been for a few years now. How about you?

There are other programming languages than Java in life.

Yes! I've used many of them in the 20 years I've been writing code. I'm not saying that good OO code is the only way to program; I'm just saying that it is one way that has worked for me, and sharing some tips on the right way to do it. What are you so defensive about?

What you are describing is a fine way to lock 1u3ers into using a single platform.

And which platform would that be? There are at least three independent Java JVMs, all implemented from a published spec, running on at least five different processors under at least four different operating systems, some of which are available from dozens of vendors.

Using Java could mean you're locked into the Java language. But really, it doesn't even mean that. If you want a public API to your app you're going to have to write one. Whether that's SQL stored procedures or Java-backed SOAP or any one of a dozen other reasonable choices is a local decision; there is no universal answer.

Stored Procedures are Better!

[Anonymous Coward]

Just think about this. You move from client server to web front end was made easier because your business rules were stored procedures. They were not dependant on the end application.

Life without them...

[qurve]

...would be hell for me.

Most of the development I do at my job is Coldfusion+Fusebox with SQL Server on the backend (I don't care if you hate MS, don't bother knocking SQL Server) and stored procedures just make life easier. They're also handy in the instance that you may have multiple front-ends written in multiple languages accessing the same database in many cases. Making a change to the way data is returned is far easier to do in one stored procedure than in X number of front-ends. One of the main reasons we don't use mysql is because the stable versions don't have them.

Re:Life without them...

[Anonymous Coward]

The other main reason I don't use MySQL is because you have to pay a license to distribute your database. Hence why I use PostgreSQL. I too can't imagine doing anything more than trivial database work without using stored procedures, it can certainly save a lot of pain.

Re:Life without them...

[Anonymous Coward]

double agree on ms sql server. without a doubt, a wonderful program. i would spend the licensing fee for it over free mysql or postgresql any day for any non-trivial project.

what many in the movement fail to recognize is that the price of the software is such a miniscule portion of a project in the grown-up world. i have project that have cost over 10 million. do you think anyone cared about 50k in licensing costs?

back on topic now

stored procedures are the way. you really want to give a front-end update capabilities in a production environment? i often use the approach mentioned earlier. i allow dynamic queries in developer test, but it needs to be moved over stored procedures before going to quality assurance. (and yes, i eat my own dog food on that)

Re:Life without them...

[joelgrimes]

Well, that's exactly my situation, and I've always avoided Stored Procs because if the query is written in the cf, then I can see the actual sql being passed to the server in the cf debug info while the page is being executed - and if there's a problem with it it's pretty easy to find.

When we use stored procs, I don't have that luxury - i just get the sql server error number and description, but not the query itself. Worse, if there's a problem that doesn't result in a db error (wrong data inserted, e.g.), I don't have any feedback at all.

If I knew a way around that one little problem, i'd move to stored procedures immediately.

Re:Life without them...

[XorNand]

If you're using MS SQL Server, use the SQL Profiler, chief. It's an app that's installed as part of the SQL client tools, along with Enterprise Admin and Query Analyzer. I use it as a sproc debugger all the time.

Stored Procedures vs adhoc queries in apps

[jfroebe]

If the applications are written in one of the various scripting languages, then this argument doesn't apply:

One major problem with enterprise applications is that when a problem is found in an adhoc query (poorly written, a bug with the DBMS, performance related, etc) then the application would normally have to be recompiled and pushed out to the entire enterprise (could be tens of thousands of computers to push to). This isn't desirable.

Moving the queries into stored procedures (where possible) allows you to correct the stored procedure at a central location and roll it back to the 'old' stored procedure if necessary with minimal effort.

A good rule of thumb: use stored procedures for compiled applications

Jason L. Froebe

Re:Stored Procedures vs adhoc queries in apps

[xmedar]

Alteratively you could use a middleware layer, if your apps are calling XML-RPC / SOAP /CORBA interfaces then you can change the code at that level without impacting users with upgrades.

Re:Stored Procedures vs adhoc queries in apps

[KrisWithAK]

Using stored procs and views properly sets up the interface and contract for data model access. Besides being able to push out data model changes without affecting the client-portion of the app, it will also:

• Assist in setting up a test suite for the data portion of the application by itself
• New or replacement clients can be developed without rewriting or copying SQL code

Good and bad

[Anonymous Coward]

The good side about them is you can put them in one place, centrally, and basically forget about them.

This means several teams can develop apps against the same database (often using completely different languages, and with completely different goals). The business logic isn't hidden in some obscure class that might be in a language you aren't fluent in.

The bad is that they tend to make the individual apps difficult to debug. sp's often throw errors that are non-obvious, and if they are buggy, will do very strange things to your data. Often, hours of debugging latter, you realize it had nothing to do with your code, but was a bug or bizarre error condition in the sp or trigger.

Lately, the databases are allowing sps triggers to be created in more modern languages. Postgres IMHO kicks normalizes everyone elses ass with it's ability to use python, perl, php and ruby (among others) to create your sp's. Then at least, it is code that is a big more understandable.

Good

[Anonymous Coward]

I am in the "always" stored procedure camp. They allow me to centralize a great deal of business logic close to the data source.

They provide a great deal of security if used properly. Not only do I not have to allow my users to have ANY direct access to tables, but I ahve the ability to change underlying structure more easily.

It is wonderful to make the boss' suggested change to his favorite application without having him even close out of it.

The downside is that you have an additional layer of code to manage. However, I have done a lot of code automation, and 90% of my sp's can be written with a few clicks of the mouse...

Only use stored procedures when you have to...

[RobbieJohn]

There are times when there are compelling reasons to use stored procedures. Unless you have one of those situations, there are lots of good reasons to avoid them. In addition to the portability and skills issues, I'd like to add that PLSQL and TSQL are both pretty crude languages. Stored procedures also make it hard to partition your application across servers. I'm in the 'avoid them if you can' camp.

NO WAY!

[matth]

Stored procedures should ONLY be used for sensative data such as in a bank environment, where information should not be passed to the database (such as usernames/passwords/account info etc) but rather the database server should run it. Otherwise the proccessing should be done on the app side!

There is NO REASON for the database server to be doing proccesing.. it's a database server!!

Re:NO WAY!

[jfroebe]

umm.... what do you think a query is? It is a set of instructions to the DBMS of how to retrieve a desired set of data. a stored procedure has less of an impact on the DBMS than an adhoc query because it is already compiled to be used by the engine.

You have much to learn about databases and are likely trolling but I'll give you the benefit of the doubt.

Re:NO WAY!

[Anonymous Coward]

After many years of reading slashdot and never posting, I decided to make an exception in your case, because this is the most oblivious remark I think I've ever heard. Kudos to you.

Re:NO WAY!

[Anonymous Coward]

oh no!... there are several good reasons to get the db serevr to do processing..
1. Reduce network traffic
Processing large numbers of rows is much better to do in place in procss on the db.
2. What is processing?
A qeuery is a process. A database server should have queries run against it rather than just being some kind of shared storage. I mean you could just return the whole database to the processing server.
3. Mysql will get 'em soon so developers can stop having envy for Oracle MS etc or they can just install postgres.( but that isnt as easy as falling off the php/mysl log.
( this rant is not well formed)

Where I've used Stored Procedures:

[temojen]

If you have views set up to restrict which rows a user can select, but you don't want to restrict user creation (with a certain group, and with pre-defined rights) so that your webserver doesn't need administrative rights, a stored procedure fits the bill.

(PL/PGSQL example)

BEGIN WORK;

CREATE GROUP "salespeople";
CREATE GROUP "customers";

CREATE TABLE customerAddresses (
customername name references pg_users(usename) primary key,
address varchar(50)
)

GRANT SELECT ON customerAddresses TO GROUP "salespeople";
GRANT INSERT ON customerAddresses TO GROUP "salespeople";
GRANT UPDATE ON customerAddresses TO GROUP "salespeople";
GRANT DELETE ON customerAddresses TO GROUP "salespeople";

CREATE VIEW myAddress as select address from customerAddresses where customername = current_user;

CREATE RULE updateMyAddress as on update to myAddress do instead update customerAddresses set address = NEW.address where customerName=current_user;

CREATE RULE insertMyAddress as on insert to myAddress do instead insert into customerAddresses values (current_user, NEW.address);

GRANT INSERT ON myAddress to group "customers";
GRANT UPDATE ON myAddress to group "customers";
GRANT SELECT ON myAddress to group "customers";

COMMIT;

-- Now the problem becomes: How do un-privaledged users (ie salespeople or webserver) create all those customer accounts?
-- It's easy to solve with a little PL/pgSQL.

BEGIN WORK;

CREATE GROUP authorizedtocreatecustomers;

create or replace function makeuser (name,text) returns int4 as '
BEGIN
execute ''create user '' || quote_ident($1) || '' with password '' || quote_literal($2) || '' in group customers;'';
return 0;
END;
' language 'plpgsql' security definer;

REVOKE EXECUTE ON function makeuser (name, text) FROM PUBLIC;
grant execute on function makeuser (name,text) to group authorizedtocreatecustomers;

COMMIT;

-- Now, just make sure webserver and all your salespeople are in the group "authorizedtocreatecustomers"

The "Centralized code" argument is ridiculous

[kevingolding2001]

I have heard this nonsense spouted more times than I care to remember. "Keep the code IN the database" the enthusiasts would cry. What you end up with is code spread all over the place. Some of your business logic now resides in source files written in your app. language, and some of it resides in the database in SP language. You can no longer look in one place to see what the program is doing. You have to read two sets of source code side by side. Throw triggers into the mix so that your SP's are silently called as a side effect of other statements and you have a maintenance nightmare. And what happens you need to roll out an upgraded version of one of your programs? You can't just transfer the binary there any more. Now suddenly every new release becomes a database upgrade, requiring scripts to be run to modify the database etc.

Re:The "Centralized code" argument is ridiculous

[petepac]

It is the ONLY way to go!!!!

You get more code spread all over the place when it's imbedded in apps that no one shares the code for. Yes, there's code reusability, yet I've seen very little of it. With stored procedures I, as a DBA, get to control and manager distribution. Most of the database developers I've seen can't write good procedures, let alone a normalized data schema. The think that it's easier to put data integrity in the app. The one that can write them well become DBAs.

If you don't want a multi-tier app, then stored procedures are the only way to go. The front end app should be the data entry, validation and presentation layer. The procedures do a second data validation and process business logic. At one shop we created a standard database with procedures and tables that all applications could use. When there were changes made to the procedures, we still presented the result set in the same way so NONE of the apps had to change.

The was another instance where we were developing an app that was constantly changed to support new user requests. By keeping the data access in the procedures, we were able to keep multiple versions of the application running using the same procedures. Try doing that in just code.

The biggest benefit is security access. Once the user gets access to the procedure, you don't need to give them access to the tables those procedures use. Keeps the power users from mucking up the data using Excel or Access.

Single view of business processes, better security and simplified modification of processes without recompiles and distribution of code. That keeps me from loosing sleep at night.

Oh, and one more thing. NO SQL INJECTION HACKS!!!

Re:The "Centralized code" argument is ridiculous

[1001011010110101]

Yay a nice flame wars , may I join ? :)

Most of the database developers I've seen can't write good procedures, let alone a normalized data schema. The think that it's easier to put data integrity in the app. The one that can write them well become DBAs.

Weird, I've seen the exact oposite, most DBAs (and I've met quite a lot) cannot write decent code/normalize if their life depends on it. They are good at messing with storage and such, but getting into SQL takes a huge effort from them :) I'm a programmer, good with SQL and modeling, and I'd never become DBA.Who would want to get a midnight page from a server because it went down :)

Re:The "Centralized code" argument is ridiculous

[petepac]

It's true what you say about a DBA that can't design a piece of paper that needs to be folded in half. Those aren't DBAs. They're Data Monkeys. In the case of SQL Server, take away Enterprise Mangler ;) and they're lost.

A DBA that can't bring a schema to third normal form and write good stored procedures should be woken up in the middle of the night. They probably screwed up something on the system to cause it. I've seen these people too and we never hired them at the companies I was at. This is like hiring a UNIX admin that can't write shell scripts.

A DBA in my view is a "Database Analyst". I like DBE, "Database Engineer". If the A is for Administrator, that's all you get. If you can't do it all on a database system, you won't be able to do what little you know well.

Developers like you make great DBEs. At least you'll have the knowledge on how to keep the database running and stable. You don't let just anybody design or code in the database. After that comes data warehousing where the real fun begins.

Re:When did Slashdot switch to IIS?

[isbhod]

not sure about IIS, but slashdot has been running a lot slower than normal for me too. Again sorry for the off topic post.

Middle Server

[Cranx]

I almost always use a middle layered server that buffers calls between applications and the database. It's job is really to coordinate transactions, users and to perform tasks that go faster natively than through a database. A middle tier invariably makes it FAR simpler switching from one database engine to another than trying to port stored procedures.

Re:When did Slashdot switch to IIS?

[rbbs]

I've just spent the last 10 minutes trying to get through and this is the first time i've succeeded. It's been going on for about a fortnight now at least. I'm getting maybe a 50% success rate on average....

All business logic in the app?

[Anonymous Coward]

Then you are a moron.

Re:All business logic in the app?

[Anonymous Coward]

Then so are SAP. It would seem you earn considersbly less than a moron.

SP not good all the time

[TechGladiator]

At my work we write apps that are only used inhouse and app that are used by 1000+ users. Our applications are mostly data entry apps with a decent amount of reporting. We only use stored procedures in the software that we sell to our users because it makes it easier to implement any updates/fixes in code than it's in a stored procedure and just send them a program(executable) update. For inhouse we prefer stored procedures because maintenance/updatability is not a problem.

I think if you are looking at large databases where stored procedures may make a huge difference weigh your options between speed and ease of sending updates to the users.

Just my 0.02 cents.

Don't Like Them...

[puppetman]

Stored Procedures are hidden logic, and they're difficult to write, debug and maintain. In some databases, they are nearly impossible to debug without the equivilent of printf statements.

You also place more burden on the database, and lose the ability to scale horizontally. It's easier to add more web/app servers to you site than it is to add more databases.

Not sure I agree with the point about them being tied to a specific database; I don't think it's practical to write vendor-neutral SQL, so you will always have functions like TO_DATE vs DATE_FORMAT to worry about.

Use foriegn keys, check constraints and strongly typed columns to preserve your data. Keep the application logic outside your database.

Re:Don't Like Them...

[Thomas Charron]

Spoken from the mouth of someone who doesnt have a clue..

WHY isnt it as easy to add more database servers then it is to add more web servers?

It's becouse you're not looking at the applications from the side of someone who manages data and databases.

It can work both ways. And in reality, it should work both ways..

Ever seen a database having rows locked by 100 different web servers becouse they DIDN'T use any form of stored procedures?

Re:Don't Like Them...

[nathan_w_cheng]

WHY isnt it as easy to add more database servers then it is to add more web servers?

Because then you have to deal with more of those people who manage data and databases (kidding, of course--50% of the db people I have dealt with have been very good to work with; it's that other 50% that make life miserable and force simple developers like me to get all wrapped up in politics).

Stored procedures come in handy once in a while, but mainly they've caused me enourmous headaches--especially when you run into applications that have a 100K lines of stored procedures mixed with another 100K lines of application-level database layer code, and it's impossible to figure out by what perverted formula it was originally determined what to put into stored procedures and what not to put into stored procedures. In the systems that I've seen, the stored procedure code has over many years become unmanageable spaghetti following no coding conventions, with half of the procedures being completely obsolete. Perhaps this isn't so much an argument against SPs as it is an insult to the kind of people who tend to write them.

Personally, I like leaving the database to being the master of my data, and it being a slave to my application: application wants data, database gives application data; application tells database to store data, database stores data. It just seems proper that way.

I would feel a lot more comfortable and happy about stored procedures in a system where the stored procedures acted as a complete abstraction of the data--so that the application only communicates with the stored procedures and never directly reads or writes tables--in fact, the application has no knowledge whatsoever about the tables. That seems like it would be proper too.

The situation that is not proper is the situation where stored procedures play no clearly defined role, and developers are continually asking the question, "Should I do this in a SP or not?" Then what you end up with is a nightmare that can only be solved by hiring someone like me to come in and unscramble everything and put everything into nice little buckets that make sense.

Re:Don't Like Them...

[Thomas Charron]

In the systems that I've seen, the stored procedure code has over many years become unmanageable spaghetti following no coding conventions, with half of the procedures being completely obsolete. Perhaps this isn't so much an argument against SPs as it is an insult to the kind of people who tend to write them.

That's becouse they where used as developer toys instead of parts of the data store. Take any application thats maintained by sloppy developers over several years all working together, and the same thing tends to happen..

Personally, I like leaving the database to being the master of my data, and it being a slave to my application: application wants data, database gives application data; application tells database to store data, database stores data. It just seems proper that way.

Ok. Whats your 'data'? Letting the database be the master of a char(13), instead of letting it take the commands needed to store a phone number. You have it exactly right. I like the database to BE the master of my data. That's not to say that buisness logic needs to be contained within the store procedures, but the DATA logic does. I don't want to have to worry if Joe Slickback next to me knows how to properly setup his tables. Also, hiding access to tables and allowing access to the procedures allows your data's 'Master' to protect the data it serves.

The situation that is not proper is the situation where stored procedures play no clearly defined role, and developers are continually asking the question, "Should I do this in a SP or not?" Then what you end up with is a nightmare that can only be solved by hiring someone like me to come in and unscramble everything and put everything into nice little buckets that make sense.

BINGO! But the problem is, saying that stored procedures can cause this pasta mess is silly. Anyone who could end up in that situation are going to, chances are, have all their crap mixed together, jumpled, undocumented, and generally held together with digital bubblegum.

Re:Don't Like Them...

[nathan_w_cheng]

Take any application thats maintained by sloppy developers over several years all working together, and the same thing tends to happen.

That's true, but the negative effects are much more pronounced when the language is procedural rather than object oriented with a well-thought-out class structure. It's even worse when the use of SPs is arbitrary.

Ok. Whats your 'data'? Letting the database be the master of a char(13), instead of letting it take the commands needed to store a phone number. You have it exactly right. I like the database to BE the master of my data. That's not to say that buisness logic needs to be contained within the store procedures, but the DATA logic does.

Often times "data logic" is business logic. It's funny you should mention phone numbers, because I'm actually facing this exact problem right now: some of the 3rd parties with which I have to integrate need their phone numbers in one format, and some in other formats. They (probably) have systems where phone number formatting has been hardcoded into SPs, so they can't be flexible with me. So I have to be flexible, and thus hardcoding phone formatting into an SP is not an option for me, unless I want to edit my SP every time I've got a new 3rd party to deal with and get pasta all over my hands.

One big problem with stored procedures is that they're exactly that--procedural. And procedural code _does_ have a greater tendancy to tend toward spaghetti than well-designed OO stuff (both have the tendancy, but it's worse with SPs).

Re:Don't Like Them...

[Anonymous Coward]

Spoken from the mouth of someone who doesnt have a clue..

A fine overview of your post.

Ever seen a database having rows locked by 100 different web servers becouse they DIDN'T use any form of stored procedures?

No.

Re:Don't Like Them...

[puppetman]

" Spoken from the mouth of someone who doesnt have a clue.."

Bet your a popular guy at work.

It's harder to add them because,

1) Commercial databases are still far more feature rich than open source. Oracle still owns the database space. Thus, adding more databases means big licencing fees. Apache is still free, as is Tomcat.

2) It's more complex to set up replication (which is what you need when adding a new database). than it is to add another web or application server. Single or multi-master? If single-master, then what if the master dies? Multi-master, then how do you deal with conflicts.

"Ever seen a database having rows locked by 100 different web servers becouse they DIDN'T use any form of stored procedures?"

No, because I think it's stupid to give web servers access to the database. And it's a huge security hole. Rewrite the URL and an harmless select becomse a "drop table". Web servers access an API on the middle tier, where business logic resides.

In addition, stored procedures can lock rows as well. Locked rows are locked rows, regardless of who does it.

Ever seen a stored procedure fail silently after a table change? Ever tried to debug a stored procedure? Show me a good code editor for a stored procedure language, and I'll show you 10 for any "application" language.

Depends

[Billobob]

If it can magically make 503 errors go away - yes if it does anything else - no

Is it that big a deal?

[Miniluv]

Stored procedures that execute the sme query as one sent over the normal client process usually execute faster. The DB server knows more about the query and the table structure, and therefor can optimize it better.
Having the queries in one place is also generally an advantage, and if the vast majority (or entirety) of your queries are in those stored procedures then migrating from one DB to another means NO messing with DB specific code (and every query ends up being DB specific if it does much of anything at all) except for the query developers.
The shop I work in has two main applications which access the same database. One is a web environment written entirely in Perl where all of the DB logic has been pushed into stored procedures, and then further abstracted into modules. Now migrating from one DB engine to another simply means rewriting the stored procedures from PL/SQL to TransactSQL (for example), and some minor modifications to the underlying modules. Then if we want to change the business rules for that data we can change the modules with only minor changes to the app logic.
Contrast this with a mega app written in C which has tons of queries directly in it, a minimum of stored procedures, and a constant stream of bugs because of the morass this has created. The app moves slowly, ponderously, and half the time wrongly.

Anytime I'd be called upon to architect something, I'd be pushing stored procs as much as possible. They're a logical extension of good, modular design.

Re:Is it that big a deal?

[H09N0X10U5]

The shop I work in has two main applications which access the same database. One is a web environment written entirely in Perl where all of the DB logic has been pushed into stored procedures[...] Contrast this with a mega app written in C which has tons of queries directly in it, a minimum of stored procedures, and a constant stream of bugs because of the morass this has created. The app moves slowly, ponderously, and half the time wrongly.

The system writtemn in C is a 'mega' app? So it's much bigger and more complicated than the other one? Seems we don't have a level playing field here.

Even if the systems were equivalent, your anectodotal argument based on a huge sample of two proves nothing, other than some apps are easier to maintain than others. You could equally well conclude that servers with even (or odd) IP addresses are more reliable.

This might be for nerds, and it might matter, but it sure ain't news.

Re:Is it that big a deal?

[Miniluv]

In terms of complexity the two are roughly equivalent. They do all of the same functions, essentially, through two different interfaces.

Yes, its a small sample, then again I also didn't claim this was some large, statistically relevant sample but instead presented it as anecdotal evidence to support my opinion.

As for proving anything, I didn't think I tried.

Re:Is it that big a deal?

[Anonymous Coward]

"As for proving anything, I didn't think I tried."

You certainly didn't succeed (unless you were trying to prove you're a 'tard).

depends on your user interface

[stonebeat.org]

I started with Oracle Programming, then moved to Objectivity for Pure OO DBs, and now I am using mySQL. Stored procedures are very useful, if your application runs in a console mode, from the server that is hosting the DB. But in real-life this is not usually the case. Most applications are web based, in which case the web server is the middle tier. The latency added by the middle tier and the whole HTML/XML is so much that using stored procedure vs C++ doesn't help too much. Nowadays I use Java/PHP code to process the data, instead of PL/SQL, since it is the middle tier that is the bottleneck, and not the fact that I m processing data on the middle tier instead of on the DB server.
With all advances in the client (browser) technology I am even doing some processing (mostly for presentation) on the client using applets, and XML/CSS.

Re:depends on your user interface

[1001011010110101]

Nowadays I use Java/PHP code to process the data, instead of PL/SQL, since it is the middle tier that is the bottleneck, and not the fact that I m processing data on the middle tier instead of on the DB server.

Right....you add more work to the bottleneck. That will teach that middle tier to behave!

Re:an important question..

[treke]

Fairly simple for someone like me. Over time peoples jobs change, and in a small shop you might have to take up tasks that you previously would not have done. The logical thing to do is try and learn from other peoples experience since you aren't likely to have much in the specific topic. Slashdot probably isn't the best place to go, but you have to give him credit for seeing a hole in his knowledge and trying to compensate rather than just blindly proceeding forward.

I would use them for their intended purpose.

[paenguin]

Stored procedures can be used to isolate field names from the application that calls them. This encapsulation allows you to change data column names without regard for the application that is calling them. The calling application uses the stored procedure to retrieve and store data, and the stored procedure knows how to translate the data column names.

They also allow you to put business rules and other data centric items at the database where they operate.

For example, let's say you have a trigger on a data row that causes a formula to run. Wouldn't it be best to keep that formula with the data it operates on so that it can run rapidly without the need for data to go over the wire several times?

Stored procedures are also a great place to store a complex set of queries so that the presentation to the outside world is more directly understandable. You might have data that is difficult to represent certain views, but a stored procedure can bridge that gap without adding complexity to the calling program.

All of this is more important when you are talking about a web application where there is not a lot of processing power, state is difficult to maintain and lag times can stretch processing times if the data travels around too much.

Stored procedures can solve all of these problems (and many more) neatly and at the server.

My experience

[yellowstone]

2) Compiled SQL is faster;

Usually this fact will trump any other consideration, especially in the minds of those who only have to deal with how fast (or slow) the application actually runs under load.

The main reason I've found not to use stored procedures is in cases (like complex searches) where a dynamically generated SELECT statement will beat any sort of attempt at a generic stored procedure search hands down.

Some time ago, I did some work on a middle tier for some PHBs who got infected with the "stored procedures == faster" meme in a big way. The big sticking point with this was the complex searches they wanted to support, which could be done very quickly and relatively simply with a dynamic SELECT, but became very slow, complex (and thus bug prone) when we attempted a stored procedure implementation. I'm not sure we (the development team) ever really convinced them doing it dynmaically was the right way to go; we just wore them down to the point where they finally dropped the issue.

(PS, to any PHBs who think I might be talking about you: if you keep quite, no one will ever know)

Re:My experience

[bwt]

You seem to assume that stored procedures cannot do dynamically generated sql. Oracle has very nice features for this, so at best this is a RDBMS specific concern.

modeling/modules

[Anonymous Coward]

it's necessary the db is in charge of retrieving desired data. that leaves processing and presentation. in most cases, app will do the presentation, and likely the presentation requires degree of processing at the least. given that, should you split the processing between db and app? where do you draw the line? while sp can be faster (assuming it deals with data stored in db), sp language is typically not as flexible/robust as general purpose programming language, and the designer loses more control due to the use of query optimizer (i think a lot more so than compiler optimizer).

i think sp as way to better define "what" (data) to get is good, but not for actual processing - it seems to amount to optimization hack (and that is when using query optimizier well).

oh, the trigger is the absolutely worst. you thought you were updating a single column, and then whole table get dropped. no one needs that sorta aggravation.

Both

[3770]

1) I'd use both and I wouldn't use security as an argument to use stored procedures. Mere "mortals" should not have access to the database server at all. Just beware of SQL injection attacks (Google it if you don't know what that is).

2) Stored procedures aren't always the fastest because you can't do array inserts with stored procedures, for instance.

3) Queries are cached. So the second time a query is executed it won't be compiled. Just make sure that your queries are parameterized. Don't put your values in the query with string concatenation. Use parameters. Otherwise queries can't be cached. You will also be vulnerable to SQL injection attacks.

4) Use stored procedures when you will gain a clear performance advantage. You may want to try to implement it in your data tier first, and if that isn't fast enough, move it to a stored procedure.

5) Buy or make a code generator that will generate data tier code for you (and possibly other code).

6) As for database compatibility, if you implement it as stored procedures, you are screwed for sure. If you use normal SQL you are probably screwed anyway. Check out this chart this chart for compatibility. And that only points out the parts of these databases that follow the standard. They do have plenty of non standard features as well. If you want to try your queries for standards compliance you can go here.

I have plenty more where that came from, but the wife needs the computer. Good luck though.

Re:Both

[1001011010110101]

2) Stored procedures aren't always the fastest because you can't do array inserts with stored procedures, for instance.

I can in oracle :) Select blah bulk collect into arrayblah from table and for all i in arrayblah insert into table x (blah) values arrayblah(i) Both of them end up as array binds to the engine in Oracle. With the rest I agree, specially with the one about using parameters for reused queries, otherwise they tend to kill the DB.

Re:Both

[russotto]

Not using string concatentation in queries is a great idea, but there's a nasty problem: Some queries cannot be written as parameterized queries, specifically "IN" queries. There's no way to do a parameterized "SELECT B.FOO FROM BAR B WHERE B.BAZ IN ('GEORGE','ALLEN','HAMILTON)"

Re:Both

[NutscrapeSucks]

That bothers me too. Why can't there be an array datatype? (MS-SQL has a Table type, but there's no way to use it as an input param that I know of.)

Re:Both

[Hangtime]

1) I'd use both and I wouldn't use security as an argument to use stored procedures. Mere mortals should not have access to the database server at all. Just beware of SQL injection attacks (Google it if you don't know what that is).

I am not worried about mere mortals, I'm worried about your application ID and password getting into the wild and picking up the pieces as someone with that information goes trouncing through the tables your application had underlying rights to in the database.

2) Stored procedures aren't always the fastest because you can't do array inserts with stored procedures, for instance.

See OPENXML in SQL Server or this site http://vyaskn.tripod.com/passing_arrays_to_stored_ procedures.htm

3) Queries are cached. So the second time a query is executed it won't be compiled. Just make sure that your queries are parameterized. Don't put your values in the query with string concatenation. Use parameters. Otherwise queries can't be cached. You will also be vulnerable to SQL injection attacks.

If you have memory pressure on your database server, your query plan may not be around the next time your run it. From my understanding in all platforms, stored procedure query plans will persist longer then ad-hoq queries.

4) Use stored procedures when you will gain a clear performance advantage. You may want to try to implement it in your data tier first, and if that isn't fast enough, move it to a stored procedure.

What's the difference in time and resources in writing the same code in a procedure one time instead of writing it twice at the end of your long death march of a project?

5) Buy or make a code generator that will generate data tier code for you (and possibly other code).

Yea, because we all know that code generators out there do such a wonderful job in generating high performance, efficient code.

6) As for database compatibility, if you implement it as stored procedures, you are screwed for sure. If you use normal SQL you are probably screwed anyway. Check out this chart [mimer.com] this chart for compatibility. And that only points out the parts of these databases that follow the standard. They do have plenty of non standard features as well. If you want to try your queries for standards compliance you can go here [mimer.com].

Unless you are writing an application to sell to the public trying to achieve database independence this is at best a pipedream unless you write code that is really simple. At that point, I hope the rest of your code is absolutely fast because your application performance is going to suffer due to poor dynamic SQL.

It should be easier than it is...

[a_karbon_devel_005]

There are many promising things about stored procedures, but they are negated by the very thing that should make SQL work strong in many situations.

Supposedly, storing all business logic in the database gives you speed and the ability to change the language used for rendering and UI fairly quickly if need be. In that sense they become similar to the CLR... I can write a PyGTK+ app quickly for a little desktop gui functionality, and use the same validation/business subroutines that the main PHP interface uses. This is certainly sounding like a plus, yes?

However, once you start coding in stored procedures, you're just as married to the database as you would be to any other language. The problem is that while there is generally excellent acceptance of SQL 92 standards, and some decent acceptance of SQL 99 standards, afaik stored procedural languages are not nearly as standarized. Hell, PostgreSQL has like 5! So instead of being married to, say, asp, you're now married to a database, which in most cases is commercial.

Also, I think the point about new coders being unfamiliar with the programming language is slightly invalid... PostgreSQL and Oracle at least have Java (or Java-esque) PL's built in. Pretty familiar for most coders worth their salt.

Encapsulation and balancing of tasks

[Kell_pt]

Stored procedures, imho, are essential for data integrity. It's all fine and dandy when you have just a web app sitting on top of the database, but when multiple applications access the database, sometimes from different sources and programmers, you have to make sure they all follow at least the same base set of rules.
Stored procedures centralize a degree of data validation and integrity. They are also for on-modify creation of data. For instance, one can use stored procedures to make sure some read only table is always up-to-date, by making it so that inserts/updates are mirrored to that table. Some could argue that this can destroy normalization, but in some situations where reply time is critical, normalization can do more harm then good, and sometimes you're better off with more data than with having to join two tables. Stored procedures can be of great value here.
Do web applications need stored procedures? Probably most won't. But then again, "web app" doesn't define an application, it just defines an interface, a media.

Stored procedures are also vital in one of the most important things regarding project development: abstraction. You can customize the database layer to execute code that is implicit to database operations, thus saving time by not having to make them explicit in the application layer AND also saving a roundtrip from the app to the db for each of those explicit calls turned implicit - hope this made sense.

Cheers.

Re:Encapsulation and balancing of tasks

[Anonymous Coward]

Stored procedures, imho, are essential for data integrity. It's all fine and dandy when you have just a web app sitting on top of the database, but when multiple applications access the database, sometimes from different sources and programmers

Don't do that then.

security

[Anonymous Coward]

Stored Procedures -or- parameterized queries can prevent SQL injection attacks.

Dynamic SQL is more more prone to logical errors. Dynamic SQL often involves string manipulation, which can be a performance hit (depends upon your implementation--and helper classes can improve or prevent this--but in general appending strings and variables together inside of code is a little nasty).

Also, consider using strongly typed datasets if you can. Strongly typed datasets reveal datatype problems sooner (compile time vs run time). Sooner = easier / cheaper to fix.

Stored Procs = centralization. Centralization is of course both good and bad. The stored procs can, in theory, be used by multiple applications. Of course, with proper OOP this is less of a big bonus, but it does give some flexibility, esp in an environment where you have different application platforms accessing the same DB.

Stored procs make learning easier in a multi-developer environment, esp one with high turnover. Newbie coders can get up to speed in regards to the DB Schema much more quickly if the library of stored procs is well-named and well thought out. You could also translate this to say it makes maintenance easier over the long haul (think of going back and looking at this stuff again 3 years hence with other projects in-between). It depends on how much of your own dogfood you think you'll have to eat.

Gut reaction based on lots of experience doing both stored procs and dynamic SQL: Stored procs are NOT a silver bullet and can be a pain in the neck early on, but they sometimes pay off in the long run. Once you get into a groove with them, I think you will like them.

Middle Tiers And Different Databases

[Anonymous Coward]

The bad thing about stored procedures is that they are largely database-specific in syntax. Putting the logic in a middle-tier would allow you to target different databases easier, if your database access framework is more flexible/portable (all of the good ones are).

Separation of Duties

[Cornelius42]

If your project is large enough, you separate the development, and allow for your DB admins to create stored procedures.

If your project is small, one programmer, then there is no need to separate duties.

Re:Separation of Duties

[Atrax]

> If your project is large enough, you separate the development, and allow for your DB admins to create stored procedures.

And then have to jump through hoops to get them changed if required, which usually involves some sort of bribery to the DBAs in question, Coffee, Doughnuts, iPods....

Usually this isn't a desirable way of doing it, IMO. Better to give the developers absolute control over the development environment, including the DB, then have the DBAs audit and clean up the Data-Centric components before deployment, just as you'd have a QA process on the code itself

Of course this is just one guy's opinion.

Enforce business rules in the database engine

[Anonymous Coward]

Stored procedures, triggers and check constraints that enforce business rules ensure that bad data won't get into your database from any source. If your business rules are only enforced by your fat-client application, then anybody with an ODBC client can put bad data in your database.

Re:Enforce business rules in the database engine

[Anonymous Coward]

"Stored procedures, triggers and check constraints that enforce business rules ensure that bad data won't get into your database from any source."

Not the only way, though.

"If your business rules are only enforced by your fat-client application, then anybody with an ODBC client can put bad data in your database."

Anybody with a logon ID and password.

Stored Procedures often more harmful than helpful

[1010011010]

Implementing your application logic as stored procedures has some detrimental side effects.

1. SPs turn your database into an application server, centralizing things that needn't be, and raising load on that central machine.
2. SPs invite use of vendor-specific features, and therefore lock-in and loss of portability.
3. SPs are not typically amenable version control and are maintained outside the rest of your code base.
4. SPs represent "premature optimization." There may be a time and a place for SPs, but they are used a lot more than needed in many applications. For example, one application at my company has over 1,000 SPs, and quite a number are just wrappers for simple select statements.

Prepared statements and vendor-neutral SQL are the way to go for portability and controllability of the development process. Use SPs judiciously, if at all, and only when there's a highly compelling need to do so(e.g., order of magnitude speedup, etc).

Schema maintenance

[Clod9]

The major advantage you didn't mention is that stored procedures let you abstract your database schema from the programmatic logic that accesses it. You may have many disparate applications (both web-based and non-web-based) in various languages which all access the database, and if you always do it via stored procedures, then you can change your schema without upsetting the application code.

This is extremely powerful, especially if you have a DBA, but even if you don't your developers can much more easily audit who is doing what.

For instance, say someone needs to add a column to table T; how to know whether there's logic somewhere that will get broken? You have to search for the table name T in all your application code. If you use stored procedures exclusively, then you can know unambiguously what the effect of the change will be. You can ask questions like "does anyone actually use column C for anything?", or "if we change the datatype of column C, what code will be affected?"

You don't necessarily need a skilled stored proc coder -- after all, if your developers are writing their own queries anyway, wrapping them in a stored proc is not hard. If you need complex logic in the DB layer, someone's going to have to learn to do so regardless of whether stored procedures are used.

SP Kool-Aid == yummy

[greengearbox]

I've worked on two mid sized web/db projects now with a couple of old skool Oracle hackers. They are quite sharp, but not really up on all the hot new O/R mapping technology. Hibernate. Castor. etc.
So I figure I'd teach them a bit about the "right" way of doing this. Use the database to store data. Period. Model business objects as Java classes. Implement business rules in Java, and code the views in terms of those objects.

The thing is, the more we talked about this, the more I came to question the way I had done things in the past. For example, it's not always possible (or at least convenient) to define a single set of business objects. A Widget might look one way from a manufacturing perspective, and another from a sales perspective, to take an absurd example. Often I end up fetching more data that I really need, because the BOs have been defined to be as general as possible. And so on.

My particular case is special, because there will more likely always be people with heavy Oracle expertise to maintain these systems than people with Java experience. So the risk of sticking all the rules into the database, only to end up with a maintenance nightmare after the Oracle guys have left doesn't really exist. But even so, I've come around to thinking quite a bit differently about the role of the database in a multi-tier application.

Standardized Data Access

[Mike9000]

I like to keep as much of the data access (read and write) in the SP's. This way you ensure that all of your applications are using the same routines to get your information. This is important when you have a large number of reports. The easiest way to make sure your reports report the same information consistently is to have them access the same SP's.

Just my humble opinion.

As always, it depends

[Percy_Blakeney]

Do you have multiple applications accessing the database, or is it just the single web-based app? If you just have a single app, then it doesn't really matter much -- put it wherever you want. On the other hand, if you currently have or plan on eventually having multiple apps (web-based in PHP/ASP, desktop-based in C++/Java, etc...) then I'd create some stored procedures in the database. It will allow you to keep the database-oriented logic close to the data, thereby reducing how much duplication of near-identical code you'll need to write into the different apps.

In addition, depending on your situation, it shouldn't be too hard for the developers to learn how to write stored procedures for the database. Once you know one language, learning another isn't that hard. The developers might write some inefficient code at first, but they'll get better very quickly. Plus, it will give them a better idea of how the database really works and performs, improving their overall designs.

Of course, IANADBA (I am not a DBA), so take it with a grain of salt.

Forget about performence, it makes your code clean

[lphuberdeau]

One of the huge advantages of Stored functions/Procedures is that they make your code a lot cleaner by removing about every logic from the interface. Of course, it makes it easy to have multiple interfaces but most of the time, this probably won't happen.

Of course, for them to be really useful, you need some way to sort and document them because as everything else, you will end up loosing them and forget they exist. I've seen problems like that with undocumented procedures and views. It ends up being a total mess.

I've used Oracle for a while and procedures/functions are quite useful and easy to create once you figured out all the little syntax problems that come with Oracle... (don't they know about make syntax uniform?). I ran a few tests on MySQL 5 alpha, it's quite unstable now but the implentation seem much better to me. I'll have to get the development version from BitKeeper before going any further in the tests because there seem to be a problem with DROP PROCEDURE in the version I have (not blaming, I know the risks of alpha versions).

When it comes to security and data integrity, there is nothing like triggers, but it will have to wait until MySQL 5.1, which is probably over 2 years ahead since MySQL 5 itself probably won't be released before an other 1.5 year.

Databases should store data

[Anonymous Coward]

My own opinion is that a database was originally made to be efficient in STORING/RETRIEVING DATA, not processing it. It feels wrong to depend too much on a database for storing your data AND taking care of all of the application logic. What if you need to move your data to a completely different database? You are either screwed, or looking at spending a nightmare amount of time porting it over. Develop your logic in your frontend application and just use the database for what it was originally meant for: data storage and retrieval. BTW, I manage an online service that processes millions of database records a day and this is how we are setup. We wouldn't want to do it any differently.

2 tiers

[vurg]

You'll worry about these because you're writing both tiers (db and client). Often times it's not the case. The senior programmer will just tell you this is the database server and these are the stored procedures you can call to read or write stuff. You don't have to think about the table structure because the SPs handle those things.

If you're writing both tiers then I could understand that. I myself sometimes get tired of looking back and forth between the SP and the program code.

USE THEM

[Anonymous Coward]

As a dba for 10 years now I can honstly say that you shoudl be using stored procedures but only for CRUD needs. Imbedding any business logic at all in the proc will only lead to trouble, not using them leads to a bunch of trouble if your trying to tune your db or if you need to change a data structure. PLEASE PLEASE PLEASE do not let your DBA be lazy and not write stored procs for you!!!

An example of good

[Martin Foster]

A year ago, I converted an open source project I write in my spare time from MySQL to PostgreSQL. The primary driving factor in the conversion was to make use of the more robust features of PostgreSQL in order to maintain data integrity.

This involved of course the creation of a schema that made use of referential integrity and stored procedures for certain key operations that would enforce data integrity that the code required but fell outside of relational databases proper.

As I was completing the source code conversion I noticed that a lot of the data checks that had to be done under MySQL 3.x disappeared as PostgreSQL enforced it for me.

The creation of users, and other entities became much simpler as did their removal. Cleaning up the code and making it easier for me to make modifications to the scripts, without having to second guess another script having adverse effects.

The scripts themselves still handle logic, albeit at a higher level. The process of using stored procedures to handle data integrity and enforcing certain rules simply allowed me to concentrate on the bigger picture when dealing with scripts.

Of course, the trade off was in speed. MySQL to this day, still seemed to be capable of handling more loads since the site is dominant on SELECT statements. However that is more of an issue between PostgreSQL and MySQL proper.

Re:An example of good

[imroy]

I've always wondered about the real speed difference between MySQL and PostgreSQL. You said that you were able to take out a lot of code because PostgreSQL was to do more fo the checks for you, and the stored procedures handled the remaining high-level details. Back in the day, the MySQL developers publically trashed the idea of transactions, instead recommending people emulate similar functionality in their client-side code. And that they did.

Now, consider two similar DB-based applications. One is connecting to MySQL with all sorts of client-side code to emulate transactions and referential integrity (foreign keys, etc). The other is connecting to PostgreSQL where most of the work is done in the DB engine. Foreign keys are used, as are transactions and stored procedures. Thus the number of app->db requests will be far fewer than the client-heavy MySQL app. Now, even though the MySQL server may beat the PostgreSQL server in raw speed from simple selects, how would that change when you factor in the extra work that the MySQL app has to do? And what about more complex queries? I've found that PostgreSQL can handle complex queries (sub-selects, unions, aggregrates, etc) very well. It's much more efficient to do a single big multi-row query than lots of single-row queries. It probably takes longer for the programmer to write the query, but once that's done the optimizer gobbles it up and produces a plan that gives you all your data at once. Very nice.

I'd bet that all or most of the MySQL/PostgreSQL benchmarks don't take into account the very different ways these two DB's are used in real apps. MySQL is traditionally used as a simple data store with an SQL interface. Whereas PostgreSQL can become a more intergral part of the application, with stored procedures, triggers, foreign keys, etc.

Been there, done that

[DanielMarkham]

First of all, your question sounds a lot like "What's your favorite color?"

Secondly, where do you want to put your cycles at? For larger applications, that's an important question.

I've done this and seen this a dozen ways from Sunday. For every bad implementation of SPs I can show you a good one. Good coders write good code. Bad coders write crap. Sorry, but there is no one answer. For a compentent code mechanic, you can put it here or there or anywhere. All the same. (lots of "devil in the details" goes here)

Having said that, tiering is a great thing. Use it when you can for better code maintainability. But at the end of the day, you usually end up with business rules all over the place anyway. If you stick the business rules locked into the database, and lock out access otherwise, at least you know they'll always be executed. And that, of course, is the entire reason for having business rules in the first place.

I can argue this from every angle, which probably makes it a great slashdot topic! Endless speculating without any conclusion! (grin)

Maintenance

[bboyers]

In my experience stored procedures are the way to go but please take care with maintenance activities. Since database objects can't list external dependencies like internal dependencies (e.g. views/procedures/trigger/etc). Without knowing the dependencies a certain about amount of fear creeps into maintenance work. The fear of updating a procedure and breaking other web pages (or the lack of time for analysis) sometimes cause copies of the same procedure to be create with slight changes but under different names. Over time these "new" procedure become a problem since it just leads to more "fear" of change. Fear is greatest problem when dealing with an application code base.

Of course with good execution and experienced programmers this (and all issues) are not a problem, but over time poor execution and less experienced programmers will compromise the integrity of an application design.

I can only go from what our DBA does, but...

[RangerFish]

...she uses stored procedures like crazy.

I can definitely see the sense in this. Although it does make the db design more complicated, it puts a layer between your app and the actual tables and fields. That means that you can change the fields willy-nilly, and as long as your app's only using stored procs to get to the data, that's all you need to change.

There's whole books on using similar concepts in code.

Shhhh! Don't tell ASP/PHP/Perl

[Percy_Blakeney]

There is NO REASON for the database server to be doing proccesing.. it's a database server!!

Likewise, there is no reason for a web server to be doing processing... right?

Re:Shhhh! Don't tell ASP/PHP/Perl

[Anonymous Coward]

ASP/PHP/Perl are not Apache/IIS.

The web-server takes a request, forwards it to ASP/PHP/Perl, which process it, then send HTML to the web-server. So the web-server does not process anything.

(Aside from technicalities.)

Re:Shhhh! Don't tell ASP/PHP/Perl

[Atrax]

> The web-server takes a request, forwards it to
> ASP/PHP/Perl, which process it, then send HTML to
> the web-server.

Sort of. in a CGI context, the web server spawns off a new process to do the work. in an ISAPI environment (ASP is an ISAPI extension for instance, and Perl and PHP are optionally in-proc) the script runs in-process, as a thread within the Web server's own process space. (This is true of IIS, I expect the analogy to transfer to Apache - someone confirm/deny?). SO in essence, the processing is part of the web server [process]

There are performance benefits in moving in-proc, but it does mean a disaster in the extension can take down the whole web server process. Which is bad.

The situation is actually a bit more complex these days (IIS 6.0 Worker processes and application pools for instance, or FastCGI type stuff), but I think the general gist stands

and yes, I do work for them. I admit it. happy now?

Re:Shhhh! Don't tell ASP/PHP/Perl

[matth]

Some situations where stored procedures can be particularly useful:

* When multiple client applications are written in different languages or work on different platforms, but need to perform the same database operations.
* When security is paramount. Banks, for example, use stored procedures for all common operations. This provides a consistent and secure environment, and procedures can ensure that each operation is properly logged. In such a setup, applications and users would not get any access to the database tables directly, but can only execute specific stored procedures.

Stored procedures can provide improved performance because less information needs to be sent between the server and the client. The tradeoff is that this does increase the load on the database server system because more of the work is done on the server side and less is done on the client (application) side. Consider this if many client machines (such as Web servers) are serviced by only one or a few database servers.

Re:Shhhh! Don't tell ASP/PHP/Perl

[Anonymous Coward]

* When security is paramount. Banks, for example, use stored procedures for all common operations

I call bullshit. For financial information the bank I work for has COBOL code from 30 years ago manipulating VSAM files on the host. Not a Database in sight. The Databases they do have are accessed through SQL & COBOL with triggers for audit logging. No stored procedures there either.

GOOD!

[Anonymous Coward]

Stored Procs allow the clientside portion of an application to pass queries to a Server-Side processing done by a backend "industrial strength" database housed MOST LIKELY, on a FAR MORE POWERFUL (in RAM + CPU hardware-wise) system than the client machines usually & far more efficient system for data processing (since the queries get compiled too) & return of said data in a return recordset to the client program & client system...

At that point? All the client program has to do is populate say, a grid (AND in the doing of this? I strongly recommend the use of the Sleep API to stall that grid population every 3rd record or so (this you have to tune for, & also this allows SOME users to have more "oomph" than others too, like mgt. which always demand this)).

This use of that API call for external-to-application multitasking & ceding of processor time between apps running on the client system vs. the amount of CPU time given to the middleware (program that passes data back to serverside & acts as interface to server dataengine) helps multi-user setups NOT so much on ordinary LANS locally located @ say, main company site? BUT MORE those that use say Citrix WinFrame/MetaFrame or Windows Terminal Server setups to NOT "lockup" each client node coming thru usually a single remote site pipe over the public internet (this type of solution is often used to save money & create what I call a 'cost-effective 2.5 tier application... it works like a dream, & keeps the client rigs from sailing up to 100% CPU use one by one because on Citrix/TS? They share a SINGLE desktop & middleware technically... you flood that single instance? In effect, you DDOS your citrix/ts desktop shared access point... I've seen it happen, this cured it, & drove clientside remote TS user rigs CPU use down to 4% each from 100% each. Keep this in mind when designing!).

ANYHOW:

The clientside application, all it does using stored procs? Is ask a question & passes it to the server via middleware (how you pass data to/from dataengine on backside), the server-side dataengine processes it, & send the client back its answer!

Most efficient, most effective, & keeps business logic off the clientside app. as well especially in multi-user environs which most apps of this type? ARE!

APK

Re:Two answers - not always

[filesiteguy]

Actually I fit both molds.

I do PHP and use "static" mySQL in code blocks for my personal sites.

For my day job, I tend to use ASP/VB and perform most of my calls within massive stored procedures - often times creating multiple temp tables. Generally my databases have >10M rows in the primary online tables (not to mention some archive tables), so stored procs work well there. I use MSSQL, DB/2 and IMS interchangebly.

I tried out the stored procedrues in PHP5 and wasn't impressed. Since I'm coming up with a new online app in PHP/mySQL, we decided to go with "quasi" stored procs where we have a php page that can accept "parameters" and generate the SQL code accordingly.

I like stored procedures

[Kentamanos]

Besides the optimization the DB might do on SP's as opposed to dynamically created SQL statements, SP's are nice from a security point of view.

You have to be extra careful with dynamic SQL due to SQL injection bugs that we all know about. This isn't really an issue when you're dealing with stored procedures that take defined data types as opposed to creating SQL on the fly based upon your data (which could have injected SQL).

Controlling which DB accounts can use what stored procedures is also handy mechanism for determining permissions. Stored procedures represent what all your application might do, so picking which DB connections (which have credentials) can access these is a nice place to control those permissions.

Granted, you can still do lots of stupid things to mess up security :).

Also, there are places where SP's are not really possible. Severely dynamic reports are a good example (assuming you allow that functionality in your application). There's definitely times when you HAVE to generate SQL on the fly. In any event, try to create a "data access layer" in your code and if you have to dynamically generate SQL, run all sorts of checks on it with regexp's etc to check for injection.

stored procedures bad

[fred fleenblat]

1. Each vendor has a special language for stored procedures that is incompatible with other DB's...so it's a vendor lock-in mechanism.

2. Stored procedures which are triggers operate in a weird out-of-phase universe where they can do stuff before, during, or after a transaction. Yes, this makes them powerful, but the transactional behavior of the procedure depends on the context in which it is invoked. So you end up making a mental map (it never seems to be documented) of which SP's operate in which phase. One odd but effective way to deal with this to put the word "before", "during", or "after" (optionally row also) into the name of the stored procedure.

3. It kind of breaks the (sometimes adhered to) idea of keeping a clean separation of the algorithms from the data, in the same way that writing into the instruction cache is best not done.

It depends (as always)

[chochos]

One case where I find that SP's can be useful for storing business logic is when your system will have different front ends using different technologies. For example, if you have a web frontend with PHP or Java but also have a rich client written in .NET or VB.
Of course you can also solve this using an additional tier (like an app server and use web services) and it could be easier to maintain, but if performance is too much of an issue, then you could go for SP's for some of the logic.
I don't think it has to be an all-or-nothing decision, though. You usually end up with some logic in the app code and sometimes some logic in SP's.

Neither

[Anonymous Coward]

Use Hibernate (or an equivilant) and never worry about either again.

Re:Neither

[fishbowl]

"Use Hibernate (or an equivilant) and never worry about either again."

Hibernate is a wonderful middle tier solution, but there are any number of age-old problems that it won't solve. At the end of the day, it's still JDBC, Prepared Statemets, Collections of objects on a round trip, and authentication issues, as ever.

There are always areas where such things as stored procs and views are useful, and there are always areas where they are a liability.

Have data access objects helps the most

[Invisible Now]

I work for a big bank. For small web based apps we put the sql inline with the ASP or JSP page code. Makes the web page essentially self documenting if you have an ER diagram.

But for big apps - say like www.bigbankxyz.com - what really matters is robust stability and the abilty to tune app performance over time. To do this you need to build a layer of data access objects that abstract and wrap around the SQL or SP.

With a data object layer your frontend folks can then use these objects to do things like:

accountbalance.retrieve acctno

in the page code.

The big advantages in addition to simple productivity are:

1) If the underlying data source or even the source system changes you can update the data access objects and the frontend folks are fine and oblivious

2) If your site has performance issues you can selectively build data caching into slow objects (queries) after deployment and the frontend code runs faster without any code changes

3) You can apply a number of load balancing, regionalization, clustering, fail over etc strategies across frontend servers by manipulating the objects while the page code, again, stays the same

On the other hand, it's not worth the effort if you're always going to have only a few developers running a two server (webserver and DB) setup.

Re:Have data access objects helps the most

[NineNine]

Every single app I've seen designed like this turned into an abject failure. That's about a half dozen. You're adding in a tremndous performance hit and complexity that is usually not needed if you have some good database people. All of the performance things that you talk about are available on RDBMS', and are more powerful than any of these new fangled middle-tier thingies.

Re:Have data access objects helps the most

[pyrrhonist]

Every single app I've seen designed like this turned into an abject failure. That's about a half dozen.

That's not unusual. Most software projects do fail.

You're adding in a tremndous performance hit and complexity that is usually not needed if you have some good database people.

I think you're misunderstanding him. He's merely abstracting the query from the display code, not adding a middle tier.

Furthermore, every "good" database person I've had the extreme displeasure not to be able to avoid working with has caused the other developers more pain due to the complexity of their designs than was necessary for the project to work.

The part that infuriated me the most about these people is their inability to provide basic access to their system. If they had bothered to provide a basic access library or stored procedures for the simple queries we were required to do initially, they could have tweaked the tables at their leisure. Pleas for basic access went unheeded, because performance tweaks were obviously so much more important, even though we knew from system profiling that the performance issues were not with the database. Subsequently, we had to munge SQL everytime they decided it was time for a change. I managed to shield the other developers on my team from some of this by making a basic abstraction layer, but it meant that I bore the brunt of tweaking the database code for my team, when I should have been spending all of my time implementing my own team's customer requirements. This meant I had to go to the DB "expert" on his timetable to beg for the performance tweak du jour so I could add it to our set of queries (the same set being used by other teams) so that we'd actually get rows back instead of errors.

All of the performance things that you talk about are available on RDBMS', and are more powerful than any of these new fangled middle-tier thingies.

Right, and a lot of these performance enhancements require proprietary SQL. Thus, if you abstract the queries, you can tweak the SQL without breaking the page display.

In general, it's a good idea to abstract your most commonly used queries, because you don't know initially how your backend database is going to change over the course of development. For instance:

• You may have to change queries for performance reasons, and it's better to shield the team from this kind of tweaking. In this case, the entire SQL can change drastically. You may have to combine some statements, or change tables, or change datatypes. Sometimes, the old query won't even work anymore, depending on what the DBA changed, and it's definitely better to be able to fix this issue in one place quickly.
• You might also have to change database vendors. I've been on two teams in the past where we were forced for political reasons to change our database vendor, and both times, query changes were required. In fact, in one instance, we were forced to change the database vendor twice. Fortunately, most of our queries had been localized to one library in this case, so only that library needed to be changed.
• Your datasource may change. On one project another team decided to stop storing CORBA IORs in a nameserver, and store them in a database instead (don't ask). Fortunately, this code was abstracted from the client code, and I only had to change one method. Since the other team told us they might switch back at any time, I kept the old code as well. This way, the same method would just work from the client's point of view.

And Programming Logic, Too...

[BrianMarshall]

Making a change to the way data is returned is far easier to do in one stored procedure than in X number of front-ends.

Stored procedures can be good for programming logic that uses the database and that should be the same across multiple applications.

Re:And Programming Logic, Too...

[Anonymous Coward]

Stored procedures can be good for programming logic that uses the database and that should be the same across multiple applications.

An abstraction layer called by these multiple applications would be as good, if not better.

Both have their place

[Anonymous Coward]

Stored procedures are definitly a good idea once the requirements/design are finalized, but it's a hard arguement to make while developing unless the developer also is the DBA. While some may argue that it enforces better design up front, I would remind you that most requirements/designs rarely make it from paper to code unchanged, and the development costs both in time and talent are significant with stored procedures.

If you went with stored proceedures, everytime there is a business logic change, or requirements change, your front end/business logic developers would have to wait until your DBA makes the appropriate changes to the data layer before being able to continue. For me, that's just too much wasted time

With most applications that i've done, one of the major requirements is database interoperatability. I've always been asked to make my systems portable between at least Oracle/Sybase/SQL2K. With that requirement, you would need someone trained in porting stored procedures in both PL-SQL, and T-SQL. Once you start adding in less familiar databases ( MySQL/PostgreSQL/SAP etc ) things get dicey quickly. The talent required becomes much more expensive and difficult to find.

Contrast that with writing your business logic in pure SQL92. Database porting this way is simple. translate the tables. watch for weird database quirks. translate your value objects. done. If you use java, then things are even easier now with O/R Mapping libraries such as Hibernate, Castor and JDO.

If you want the best of both worlds, and you have the skills required, just put the SQL in the code, but write it in TSQL/PLSQL as if it were a stored procedure. That way it's easier to develop, and when it's time to write the stored procedure, it's already half written. Just remember that there are very few instances where the performance requirements really justify writing a stored proceedure. Be smart where your optimize your database.

put the logic in the DB and fire the programmers?

[Anonymous Coward]

A friend of mine worked at a company that put a lot of logic in stored procedures. And, when development was done, kept one cheap programmer and a DB guru. Everyone else was layed off. Could be cost effective i suppose.

Stored procedures == Database API (a good thing)

[FlyerFanNC]

I'm an Oracle DBA, and I like creating packages of stored procedures and functions (and especially table functions) that represent an API of sorts to the database. This means that the application code doesn't care how the data is stored, and the DBA is free to rearrange the data for tuning purposes, without requiring any app changes (assuming the API remains constant).

In the past I've supported keeping more of the business logic in the database, but I no longer believe this is an optimal design. Now I keep business logic out of the database as much as possible and limit the stored code to enforcing data integrity and making the database look like a "black box" to the apps.

portability

[MORTAR_COMBAT!]

I've found that, say, writing an app with a lot of code in Oracle PL/SQL, using cursors, etc, means your app will only and forever support Oracle, without a whole lot of re-write and likely re-design.

So unless you like vendor tie-in... stay away from db-specific stored procedures.

Re:portability

[FlyerFanNC]

If you spend a lot of money for a database, you might as well use all the features of it; otherwise you're wasting your money. Why spend a lot on something like Oracle and then only use, say, 30% of its capabilities? Portability is a ridiculous argument for throwing money away.

Also, a lot of SQL is database-specific anyway; even SELECT statements and datatypes lock you into a specific product.

Re:portability

[kupci]

Yep lots of talk of portability, be interesting to see how many people *really* use it.

Re:portability

[NineNine]

So unless you like vendor tie-in... stay away from db-specific stored procedures

Actually, I'd say that if you're switching databases willy-nilly, you have much more serious problems.

Re:portability

[Chazmyrr]

The database applications I develop run on dedicated servers. There is no reason to ever change the RDBMS once the application is put into production. If it ain't broke, don't fix it.

Not using stored procedures doesn't buy you much in portability anyway. Take an application written for Oracle and try to run it against SQL Server and see how far you get. If you use any conversion functions, you're screwed, and if you use dates, you're going to be using conversion functions.

Re:portability

[rollingcalf]

"There is no reason to ever change the RDBMS once the application is put into production. If it ain't broke, don't fix it."

As long as the database vendor doesn't jack up licensing costs by 100%. It has happened before, and they do it because they know that their customers lock themselves in.

Re:portability

[KontinMonet]

We have to use it. We develop Java 'shrink-wrapped' apps that use the db of the corporation. This is usually Oracle, MSSQL or DB2. At the same time, these corporates also have their own reporting tools (Business Objects etc.) and I don't want to have to duplicate logic in code and in the reporting tool. SPs are important in this respect. I'm absolutely sure we're not the only guys who need 'portability'.

Like any good tool, it can be used for evil

[Anonymous Coward]

A stored procedure can be a wonderful thing:

triggers that keep history tables

triggers that keep referencial integrity that a foreign key simply can't do

stored procedures that loop through large amounts of data that only return a small amount back

A stored procedure can be an evil, evil thing:

implementing complicated algorithms that can't be debugged

creating HTML

I've seen both sides. It's like asking if javascript is a good thing for web browsers.

John, who gets a 503 when trying to log on :-(

My take

[ZeroConcept]

Stored procedures are a performance optimization, consider the following scenario:

Retrieve the 20th page using a page size of 50 records for all the SKUs under a catalog (potentially millions total) for a specific user which could or not have visibility permissions for each SKU. Assume the security provided by the database is too coarse to fulfill the business requirements, therefore some set of rules must be evaluated to determine SKU visibility for a particular user.

That query would normally be very fast if implemented as a stored procedure because:
1) Only one round-trip is needed.
2) You don't have to move all the data to a middle-tier and then filter out information.
3) RDBMSes are usually faster at filtering data out (by using indexes, denormalization, etc.) that what a developer could code in a middle-tier to filter out information.
4) Most RDBMSes are very good at scheduling tasks, caching, managing memory, etc. The more you move logic away from it, the more you would have to implement it yourself.

You could send all the SQL statements to the database and achieve the same effect, but it might make debugging harder and you still have all the SQL logic in some place, only a different one.

On the flip side:
1) It's harder to write stored procedures than it is to write code in a managed language like Java or C# (thirty-line SELECT statements are not very intuitive).
2) Generally speaking, the compiler of a managed language does a better job at catching errors than a compiler for stored procedures, where a lot of the errors will be caught at runtime.
3) Stored procedures are not portable.

My advice is, if you are only using the RDBMS as a persistence device and your data size is not huge, avoid stored procedures and create some sort of middle-tier object model. Only when performance is a impediment, use stored procedures. You might as well use a hybrid approach, try to model as much as you can in the middle-tier and implement stored procedures for those tasks which are performance intensive.

I work with people that worship UML and patterns as well with RDBMS Gods that can plow through pages and pages of stored procedures without blinking. As much as I love ULM and patterns there are some tasks that must be done in the RDBMS for performance reasons, and tasks that are simply more maintainable when done in the middle-tier. Both approaches have advantages and disadvantages, the trick is to use the best approach according to the situation.

Re:My take

[Anonymous Coward]

You could send all the SQL statements to the database and achieve the same effect, but it might make debugging harder and you still have all the SQL logic in some place, only a different one.

I disagree. You can't step through a stored proc. Your app can spit out the SQL that you can test against the DB.

Re:My take

[Anonymous Coward]

I agree - stored procedures are a necessary evil for maximizing performance of database applications. They add complixty to development and maintenance of applications - but can make the different between an application achieveing less than one HTTP transaction per second and one achieving more than 50 transactions per second.

I have been involved in architecting system - as well as performance tuning other people's messes - and stored procedures have proven critical to satisfying transaction rate performance requirements. There is no way around them.

If you don't care about performance and scaling of the application - don't use them. You don't need them. Things could be faster - but who cares?

But if you need to scale up to support large number of users at reasonable transaction rates, you should be designing them into the system from the beginning.

As has been pointed out, the typcial secnario for DB interactions of an application written Perl or Java or what have you running on an web/application server(s) is:
- the web server / app server generates a DB query and gets back some results from the DB
- the web server / app server processes the results and issues another query for additional results

This pattern of DB request, app server processing, followed by additional DB requests *kills* system performance and scaling. Time spent serialiing data across network connections between servers wastes cycles. Time instantiating data from the DB in the app server memory space for subsequent business logic processing in order to simply issue further DB queires is a complete waste. If the bulk of the data exists in the database OR several independent DB queries in an HTTP request may be consolidated into a single database query running in the database via a stored procedure - then the performance win with stored procedures can be huge - especially when large amounts of data transfers betwen app server and DB can be eliminated.

Summary - stored procedures are likely necessary for applications with large data sets and high transaction rates. If you don't have demanding performance / scaling requirements - then don't complicate things.

Finally - most large projects I have been on have used DB SQL specialists to design and tune schema, queries, and stored procedures. These people are typcially different from the web front end and application logic people. So stored procedures can help organize responsibilitues amnong people on a team with different skill sets.

Re:My take

[Anonymous Coward]

First of all, there are SQL Debuggers. Visual Basic supposedly has one.

Second, there's no reason you can't just "spit out" the stored proc SQL and test that.

Re:My take

[Osty]

It's harder to write stored procedures than it is to write code in a managed language like Java or C# (thirty-line SELECT statements are not very intuitive).

This depends solely on your own discipline. I've seen quite a bit of horrible T-SQL code, but in 90% of the cases the worst part about it was not the logic. That could be followed, somewhat. It was the formatting. In the same way that you wouldn't write C code all on a single line (even though you could), you shouldn't do that with SQL code, either. And yet, it still happens all the time. Which is easier to read:

select fb.foo,fb.bar,fz.baz,fx.xyzzy,fr.froboz,u.unf,p.po o from foobar fb inner join foobaz fz on fb.foo = fz.foo inner join fooxyzzy fx on fb.foo = fx.foo left outer join froboz fr on fx.froboz = fr.froboz inner join unf u on fz.baz = u.baz inner join poo p on u.poo = p.poo where fb.foo = 1 and fx.xyzzy in (select xyzzy from oldxyzzy where bump = 13)

or:

select fb.foo
, fb.bar
, fz.baz
, fx.xyzzy
, fr.froboz
, u.unf
, p.poo
from foobar fb
inner join foobaz fz on fb.foo = fz.foo
inner join fooxyzzy fx on fb.foo = fx.foo
left outer join froboz fr on fx.froboz = fr.froboz
inner join unf u on fz.baz = u.baz
inner join poo p on u.poo = p.poo
where fb.foo = 1
and fx.xyzzy in (select xyzzy from oldxyzzy where bump = 13)

I know which one I'd rather read, and which one will more quickly make sense to me. And yet, most programmers still write their SQL code as the former, and not the latter. Nasty, nasty "programmers"!

Generally speaking, the compiler of a managed language does a better job at catching errors than a compiler for stored procedures, where a lot of the errors will be caught at runtime.

That's a deficiency of your build process, then. At work, we have our build process setup to "compile" (ie, load objects and stored procedures into a database) T-SQL code at build-time, helping us catch errors long before we get to runtime. It's not perfect, but it's quite a bit better than waiting until runtime to find our your query is broken.

Stored procedures are not portable.

Big deal. Chance are, you're not going to be writing portable SQL code in the first place if you're doing anything more advanced than simple select, insert, update, or delete.

Re:My take

[bubbha]

Data Warehouse "marketing" applications scan millions of rows and have to cache intermediate result sets of 50 - 100 megs each....

Stored procedures did not work well for this (in 1999) because it was much faster to sort these intermediate result sets outside the database.

We did this stuff in perl on the database box.

Re:My take

[Chazmyrr]

If you can afford a multi-tier architecture, you can afford a good RDBMS person. The Java programmer writes the Java code and the database developer writes the stored procedures. The database developer is used to set operations and understands how the RDBMS handles things under the hood.

Re:My take

[Anonymous Coward]

Yuck, you put your commas at the beginning of the line! Are you insane?

Re:My take

[Osty]

Yuck, you put your commas at the beginning of the line! Are you insane?

Nope. I just follow the path of least effort. If putting my commas at the beginning of a line rather than at the end lets me quickly and easily add a new line later without the possibility of accidentally breaking code because I forgot to put a comma on the previous line, then I do it. In this case, it makes it painfully obvious that you're going to need to add a comma if you add a new line, and it keeps additions to a single-line change. Commas at the end of the line mean that it's easy to forget to add the comma, and it's a two-line change if you add a new line to the end of the list.

Re:My take

[tuomoks]

Two old programmer tricks - too often forgotten.

Some SQL engines can do the same filtering, sparse field selection and sorting but only if told so and I haven't seen (for a long time) any DBAs who can understand the physical access (time) differences between sequential and random access speed. This is a must for any large reporting queries, one time queries are different.

Osty : "I just follow the path of least effort. If putting my commas at the beginning of a line"

Another trick and very useful ( and very readable )

the poisoned apple

[Anonymous Coward]

SP's are the poisoned apple... shiny, polished, better benchmarks, they seem ok, and, besides, everybody is doing it, right?

Any SE on a large project will tell you: stay clear of SPs -- no matter what the DBA's say in the meeting.

...mmmm, poisoned apple.

Advantages of stored procedures

[Orion Blastar]

As a programmer, I find that making a change to a query or table can cause me rewriting code in every application I've developed.

With stored procedures, I just refence the stored procedure name and leave the query tinkering to a DBA.

The only thing that I have to make changes for is when the DBA changes a column name in a table or a parameter for the stored procedure. Also when a stored procedure is in use, and it needs to be changed, I have to make the program use a second procedure name and switch procedure names each change, because if the procedure is changed as my program is running, it will break if a parameter is added or removed.

I had to work on a docket calendar program for a law firm and we used stored procedures with the reports. The managers and lawyers were always adding things to the reports which needed changes to the stored procedures. We eventually maxed out the max number of tables allowed, and each stored procedure was five pages long with if else statements because of all the things that the managers and lawyers wanted.

Using regular queries would not work because of the flexability that T-SQL had to meet the law firm's demand. MySQL would not have cut it. The reports were in Crystal Reports.

Re:Advantages of stored procedures

[AbbyNormal]

"a query or table can cause me rewriting code in every application"

You actually bring up a good point though. Who does the Database maintenance? In my small shop, if I make a change to a query/table, I need to go back and re-write every stored procedure that is affected. MSSQL does not provide any functionality (to my knowledge) that allows me to search through all SP's in particular DB (DBA is part time). Working in a small shop, I'm still undecided about our use of SPs. I've inherited most of the work, but most of my new development I've only used SP for very complicated queries. I've started abstracting the queries to a Data access layer instead.

v. Architect?

[Anonymous Coward]

Architecting? Why not use the much simpler (and... REAL) word "building" or "creating"? That's sort like saying, "Were carring up to the mountains this weekend." "Driving" would've worked much better.

Data/Logic co-location

[linuxhansl]

This question is almost as old as computing itself.
You want to have your logic physically close to the data. Stored procedures is one way to achieve that.

It is a balance to strike which very much depends on the application. So I don't think there is good/bad.
If I had an application that needed to crunch a lot of data that is stored in a database and the logic is such that it cannot be expressed declarative (i.e. a SQL query), than I have no choice but using stored procedured or to ship large amounts of data between the database and the Application.

In other words...

[Anonymous Coward]

Stored Procedures. What are they all about? Are they good, or are they whack?

Re:In other words...

[dhakbar]

Perhaps we should ask Borat...

It's all style

[Anonymous Coward]

The style in which you manage the project is how you decide. If you do lots of planning on your db apps, you'll find it easier to put data functionality in the database as stored procedures, and you'll put your methods for getting and displaying the data in your web code. If you are just throwing something togather, stored procedures slow you down because you find yourself making adjustments to them to support additional fields or features, causing you to re-test all of the previous calls to that store procedure to be sure they still work.

Distributed Computing

[Anonymous Coward]

I have developed in major environments. I have been a DBA and a dev. I have spent some time going through the pros and cons of this question. Over the years, I have come to believe:

1. Nearly all the valid arguments in favor of stored procedures have lost validity as technology has shifted.

1a. Arguments like the one for compiled apps used to make sense. These days, apps tend to be n-tier web apps, so the compiled argument goes out the window.

1b. Argument like "stored procedures provide atomicity" were never valid. Atomicity is attained via transactions whether you use SP or not.

2. SP rely on a single implementation of a sometimes arcane language. Such implementations can almost never advance the way independent languages can. PL/SQL will not get the same advances as say, Python. Oracle's implementation of Java will not advance as well as Java in general. If you have a stored Java procedure, you're stuck with how Oracle does it. If you write an external procedure, you can choose the best JVM available from Sun, IBM, Oracle, or whoever.

3. SP centralize your code. Many people see this as an advantage. I see it as a disadvantage. Every single SP must be executed on your DB server/cluster. If I have a single puny P4 DB server behind 4 load-balanced app servers, I can distribute the load of the external procedure across the 4 app servers. I can load the DB with super fast disks and lots of RAM and modest CPU, then load the App servers with no disk, lots of CPU and decent RAM.

External procedures allow me significant flexibility in my architecture. They allow me to develop in the most effective language. They allow me to use collaborative tools more easily. They allow me to load balance in almost any way I can imagine.

For 2 things only

[Anonymous Coward]

I only rely on stored procedures for 2 things:

1. Transactionality
2. Performance

For everything else, middleware is better. This can mean web services, message queues, etc.

A number of factors

[Anonymous Coward]

This is quite a broad topic that needs to have many, many factors taken into account. In no particular order, here are some random thoughts:

1) Hardware budget.

The company that I used to work for had a large investment in applications servers that at one time ran their own individual database engine. When they made the decision to convert to Oracle they found it quite a bit cheaper to leverage this existing harware setup and spend far less on a few database servers and keep business logic in a middle tier.

On the other hand, if you only have budget for a database server and a webserver, you are far better off keeping business logic OFF of the webserver itself and securely behind a firewall in the database server (you do have a firewall between them right?). There has yet to be an unhackable webserver.

2) Human resources budget.

Pure database developers demand more money than straight programmers. Also, the more workload you place on the database, the tighter you need to have it tuned. DBAs are some of the most expensive IT people out there.

3) Flexibility.

I have yet to see any stored procedure language that can as easily do the same logical tasks and with as few lines of readable and maintainable code as a C++, VB, or Perl program. This is not to say that one can not writed unreadable and unmaintainable in these languages - bad coders come in all flavors.

Far more people know one of these languages and that is part of the reason for the market factors behind them being cheaper. Plus this eliminates the weak link in the chain scenario if you can afford 3 of these type of programmers for every 2 stored procedure programmers.

4) Performance. Yes stored procs run faster than ad hoc SQL. But there are a lot of things that a DBA can do to make ad hoc SQL run faster with query caching.

Your developers should all be familiar with the concept of bind variables too. More often than not, the same queries get executed again and again with minor changes in a WHERE clause. Bind variables can mean orders of magnitude performance difference in these situations where business logic is kept in a tier that isn't the database tier. Not to mention that bind variables are one way to help avoid SQL injection attacks.

Proper (some might say clever) use of local storage and state in a tier outside of the database for commonly used data can eliminate the need to hit the database as often.

Like I said, random thoughts. Mod how you will.

Annother advantage

[temojen]

Stored procedures can be created as having the rights of either the current user (eg. apache) or the person who wrote it (eg postmaster). Thus write a stored procedure that can access things the webserver normally can't, while doing some required security checks and not giving access to everything.

Re:Annother advantage

[Anonymous Coward]

eek. Basically a SUID program.

Re:Annother advantage

[Anonymous Coward]

Yup, and that's a whole lot better than giving someone access to everything all the time

(Flame Bait) SP = bad, RDBMS = bad

[Trinition]

I'm asking for it, aren't I?

Personally, I think way too much stuifdf is stored in RDBMSs. I work as a Java programmer in a non-IT industry, and everyone is happy-go-lucky about making every object map to a table. But its a huge impeadance mismatch. We have layers of DTO, DAO, VO, etc. in the way.

I think the world would be a better place if most of the typical day-to-day was stored in an object-oriented, transparent database, and the relational database was left for storing things where an RBMS really shines (arbitrary relational queries, etc.).

Once you've gone the way of an OOBMS, you have objects, so naturally all of your logic stays in your objects. The fact that your objects happen to be persisted for you is irrelavent. All you car eis that you have your objects.

Re:(Flame Bait) SP = bad, RDBMS = bad

[puppetman]

We looked at them. Either way too expensive, or they had weird "features" that killed performance (thinking specifically of Matisse, that locked all indexes on a table on insert/update/delete).

Hibernate is the way to go to avoid the impedance between OO and RDBMS.

It's also been argued that there is no such thing as a relational database; a SQL database is a SQL database, not a relational database (ie no SQL database implements domains). Fabian Pascal has belaboured the point in several books, and I tend to agree.

Re:(Flame Bait) SP = bad, RDBMS = bad

[fishbowl]

"Hibernate is the way to go to avoid the impedance between OO and RDBMS."

I have production systems deployed using both Castor and Hibernate, and these have been quite successful.

There are issues, of course, it's not magic bullet, and it only really applies if you're doing (Enterprise) Java.

But I love Hibernate. With careful design and planning, it's a great way to abstract out ALL the database stuff from a java application. Once you learn to work within the idiom of Hib, you can make your code so that managing Connections is hidden, and so that all you have to worry about is a persistent object with respect to a Session. Most of the database semantics are cleared off your plate, including relational stuff.

www.hibernate.org

Re:(Flame Bait) SP = bad, RDBMS = bad

[Chazmyrr]

And when management wants to run business information reports on typical day-to-day stuff stored in an OODBMS, what then? Are you suggesting that the information found in your objects be duplicated in a relational database so that useful queries can be run? Or are you going to crush your server by having it walk every object?

If the impedance mismatch is so huge, maybe OO isn't always the answer? Have you considered that creating a well defined procedural interface may be more suitable for many applications? OO can result in easier to read front-end code but can also add a lot of complexity under the hood.

Re:(Flame Bait) SP = bad, RDBMS = bad

[dubl-u]

Hibernate is the way to go to avoid the impedance between OO and RDBMS.

Let me give an amen here. If you are using an SQL-based database with Java, Hibernate keeps about 95% of the database-related shennanigans completely out of your Java code. Sooooo many Java object models are hideously distorted to match some unmigratable schema, and a good O/R mapping framework Hibernate makes it much easier to write good, clean OO code.

Re:(Flame Bait) SP = bad, RDBMS = bad

[Trinition]

Is what you're saying that is because management may want to run arbitrary queries, everything shoudl be in an RDBMS? What I was sayign is that it makes sense for some things to be in an RDBMS when you need to run arbitrary queries. BUt often times, those things do not need to be done.

And besides that, I am suggesting that other arbitrary queries do walk the the portion of the object graph they're concerned with. If you're familiar with XML and XPath then you may want to look at something like JXPath (it is Java-specific, though). Other OOBDBMS do have their own query languages. I've heard there are standards too. The point is, a good OODBMS will be able to efficiently walk the object graph looking for data.

The whole point of using an OOBMS is to have fast, transparent persistence. With object-relational tools, you can get the convenient transparency, but there is the impeadance mismatch -- objects are not rows. Something that can persist your object directly does not have to worry about that.

With an "object" is that the data is only accessed through the objects methods, so integrity can be asserted in one place. As soon as you stick the data in an RDBMS, people can change it all they want. Your data integrity can be screwed, unless you duplicate or soley contain your business logic in the database, say,as stored procedures (the whoole point of the /. post, right?). An OODBMS would instead still force people to go through the object, not bypassing any methods.

There was a nice site,s omething like oodbmsfacts.com, that I can't find anymore. But they had some very interesting points about it all. Personally, I just want simple object persistence for the general case where I don't want my objects in memory lost. If I feel I need general querying, or dyanmic reporting, then I'll stick that data in an RDBMS (insurance policy database, electronic yellow pages, etc.).

Oracle Supports Java (and other comments)

[Stone316]

You can have Java stored procedures in Oracle so you don't have to rely on learning another 'language'. (I know its not python)

As DBA's I like to see stored procedures so I can gain a better understanding of how the application works. Depending on the application it may be easier to fix/improve a stored procedure rather than application code.

In reality, I don't think alot of people use stored procedures for inhouse applications. Alot of application vendors these days are trying to bypass database specific features for database portability. That may sound good but what you have then is an application that doesn't take advantage of the database which usually results in decreased performance.

For instance, Peoplesoft (if i'm wrong, someone please correct me) doesn't use sequences when its plugged into an Oracle database. Instead they use a table and when they require a unique value they lock the table, increment the appropriate row/column and unlock the table. This may be fine for a few users but for a mid-sized company locking issues are a major issue. Unfortunately for us this isn't addressed until the next version of peopletools and they seem to have an allergic reaction to backports.

The big reason for putting your SQL in the database is because most developers unfortunately don't understand how to write efficient and good SQL. If for instance, you have a PLSQL developer on your team (s)he will understand how to do this. Also, by using stored procedures you will be able to take advantage of all the database features.

Good Ask Tom article which sorta talks about this issue.

Unless your using a 'free' rdbms like mysql or postgresql a database system is a rather large investment. Your not going to be able to cutover to a new environment in a flash for reasons other than application dependence. If your company is anything like ours you have alot of $$ invested in training and experience. Its not a decision to take lightly.

One thing superid asks is In addition, you either have to have a dedicated T-SQL or PL/SQL coder who then is the weak link in your coding chain,. Like I mentioned above most developers can't write good/efficient SQL, so as long as your hired properly then your PL/SQL developer shouldn't be the weak link. (But don't get me started on hiring practices, most interviews are a joke and rarely ask difficult technical questions even for senior positions.)

Re:Oracle Supports Java (and other comments)

[Osty]

One thing superid asks is In addition, you either have to have a dedicated T-SQL or PL/SQL coder who then is the weak link in your coding chain,. Like I mentioned above most developers can't write good/efficient SQL, so as long as your hired properly then your PL/SQL developer shouldn't be the weak link. (But don't get me started on hiring practices, most interviews are a joke and rarely ask difficult technical questions even for senior positions.)

Also, it's possible for a T-SQL or PL/SQL coder to also be proficient in another language. It's good to have an expert on the team, but unless your project is almost pure database, the best expert will be someone who's also proficient in your main application language (java, C/C++, C#, php, perl). Your expert doesn't have to be a weak link, but not having one is certainly a detriment.

The only reason SQL Server doesn't suck...

[bani]

...is because microsoft didn't write it. SQL Server is Sybase.

Re:The only reason SQL Server doesn't suck...

[Anonymous Coward]

Explain why Sybase sucks so bad then.

Re:The only reason SQL Server doesn't suck...

[0racle]

Hate to burst your bubble, but that was a long time ago, its like saying Windows XP sucks because you didn't like Win3.1.

Re:The only reason SQL Server doesn't suck...

[PaulMaximne]

Yeah, but if you've ever looked at the assembly code for SQL server, you'll realize that it's still sybase with many hacked in patches. How can you tell? If you've ever looked at compiled C code you NEVER see long unconditional jumps, while SQL server has many many unconditional jumps 1 Meg down in the code and then an unconditional jump back to the same spot. Alot of that stuff hacks in the loading of external DLL's so that they can add functionality written in a HLL, instead of assembly.

So, instead of actually starting from scratch and writing their own, they blew off Sybase, reverse engineered the BINARY, and hacked in new functionality.

No wonder that piece of trash has so many security holes.

Re:The only reason SQL Server doesn't suck...

[Anonymous Coward]

Someone dissasembling Ms Sql server to that level of detail must either have a lot of free time or just be into S/M (mostly into the M part I think).

Re:The only reason SQL Server doesn't suck...

[Anonymous Coward]

Considering that MS-SQL was ported to Alpha/PowerPC/etc, I rather doubt they are hacking around on a Sybase OS/2 binary they got 15 years ago. Its more likely written in PL1/Lisp/COBOL/Other obscure language that doesn't interface directly to C DLLs.

The security record also isn't any worse than Oracle, etc.

Good, no question

[prophecyvi]

Especially for a web shop.

I run web for a medium size telecom, and recently hired two people. I questioned all the applicants extensively about web security and just about none of them had anything remotely resembling a clue. Most of them listed web sites they'd worked on on their resumes, and more than half of these sites were vulnerable to both Form and URL SQL injection attacks, which are largely (in our case, completely) defeated with stored procedures.

(Even more of a good thing when the PHBs insist on being an MS shop...)

Multiple Interfaces

[Galaxie]

Like everyone has already mentioned, if it's a server based application that doesn't have to be pushed out to a ton of clients, using or not using stored proc's doesn't make a whole lot of difference.

Any enterprise level application that is going to be used throughout a large organization now-a-days almost requires a possibly lightweight web interface, more robust or heavy weight application based interface.
Using the same routines to access data makes stored procedures a godsend when trying to share functionality at that level.

With the heavy use of webservice like technologies now-a-days, you could really put those in place of (or in conjunction with) stored procedures and have many different interfaces like personalized list of products/prices for a client, and an application interface for those on the LAN that may do some very data intensive or transaction intensive work, and more heavy duty web interface for those on the road.

Having all the raw data access in one area, no matter how it gets accessed, makes things much easier.

Just my opinion, i've really come to rely on them when it comes to something more than a small website type solution.

Your "Database" Guy Here

[Hangtime]

Let me start out by saying, developers work well in there chosen language but the difference between the code and schemas I have seen between people that understand databases and people who code apps/web pages/middle tier objects is quite shocking. I wouldn't sit down today and pretend to write ASP, PHP, Python code, etc. and be in expert at it. Quite frankly my code would probably suck.

A few bullets

1. Application coders write suck performing SQL and running SQL dynamically makes performance problems worse
2. Forget your pipe dream about writing SQL one time and running it on all database platforms, the SQL is so generic and so inefficient that it inherently leads back to number 1
3. Its better to enforce data integrity in the database, coders who leave all their code in ASP and middle-tier objects don't put things like foreign keys on their tables, have flags flipped incorrectly, and wonder why there are orphan records all over the database. True story, during a conversion of another bank's records I found a customer hadn't been billed for approximately $70,000 worth of interest because of poor data integrity
4. Rolling out changes to stored procedures is much easier then pushing out changes to application code along with being easier to unit test
5. Stored procedures facilitate the use of transactions and set-based operations

I have found VERY few app coders (10+) that actually could write good, clean, efficient SQL code and design schemas to match. The problem lies when you're dealing with a large data-centric application, ALMOST ALWAYS YOUR PERFORMANCE ISSUE LIES IN THE SQL! If you have someone writing dynamic SQL looping through a 500,000 record resultset doing something like this (this comes from an application we purchased), your performance will be horrendous:

BEGIN TRAN
UPDATE IAR SET ARIBDATE = '7/30/2004' WHERE ARRECID = 38196.810543788582
COMMIT TRAN

Your application coders will write this sort of crap instead of using one set-based operation and you will all wonder why your application takes so long to complete an operation.

Your best bet, find two or three good SQL developers and have them do traces and find out how many of those 3000+ stored procedures written actually are getting used. My guess is a whole bunch of them are not being used or just were never cleaned off after their retirement. Next, go in and let them figure out how many of these procedures are one-off of one another, you probably take out a good 20% from there. Now you're getting towards a more manageable number. Very few people are good at database development. It is a unique skill set that can have a major impact on the maintainability and performance of your application. Stay with stored procedures and find some additional database developers if you need them. Let your app and web coders code what there good at, let database developers code what they are good at and you will have a wonderful application.

s-procs are not for data access

[ncaHammer]

Use s-procs for maintaining the data integrity only !!

1) Centralized code

if you want to support all the RDBMS you mention then it's obvious that the code is not centralized using s-procs. You have to maintain 3 different versions of s-procs (Oracle,MSSQL,DB2)
Second you code, in some languages oriented in set logic, problems that are mainly procedural. The result is very poor IMHO in both speed and maintainability. If you use Oracle and PL/SQL the problem is not obvious, but if you plan to use MSSQL, T-SQL is the worst language i am aware (to the point i dont consider it as a language at all)

2) Compiled SQL is faster

Execution plan of Ad-hoc queries is also cached

3) Enhanced security

It must be really lazy the admin that demands to access the database using s-procs only. All RDBMS offer finer methods to secure the data than this.

and last
Since your application is "highly data-centric" using s-procs for data access is actually an SQL sterilization
No matter how clever you may code your s-procs nothing beats in speed and flexibility the adhoc (dynamic) queries, so the power that SQL has, is never delivered to you or to the end-user.

Stored Procedures are good...

[Anonymous Coward]

For the reasons mentioned above (speed, database control on the database where it belongs etc). But also because you can do things you cannot do in dynamic SQL. In ORACLE you can increment a sequence using dynamic SQL, but you cannot do this eligantly in SQL Server, which is a very big problem. The only way in SQL Server to get the ID of a row inserted reliably is to use a stored procedure which returns a record set or output parameter. Anyone who uses SQL Server for complex applications should agree.

Can't do without stored procedures

[slobber]

Sometimes it is just not possible to obtain any decent performance without stored procedures. For example, suppose you have to do some complex calculations on a large data set and then crunch it down to a much smaller result.

Without stored procedures:

Application server - run a large select on DB, transfer a huge amount of data, crunch it. Very inefficient since large data transfer (usually done over network) is very costly.

With stored procedures:

Application server - simply run stored procedure on DB and get small results data set back. Much more efficient since no large data transfer is required.

Depends on your design methodology...

[JohnA]

Personally, I find Stored Procedures to be a very difficult thing to manage in the long term of software development.

If you are designing a web application, then I find it much more maintainable to utilize DAO interfaces & impls since this allows you to make changes that might be necessary should you experience an unexpected change in your environment.

Need to move from MySQL to Oracle? Simply override any db-specific code from your ANSI Impl, and go.

Although if there is no chance of an environment change, stored procedures become much more attractive.

For reports at least

[matdodgson]

Use stored procs when you have complex queries like reports at least. Putting those large queries in your code just looks bad.

Good for what they're for; crap otherwise

[sparks]

Generally a database should be where the data is kept. Nothing else. If there is some functionality which is absolutely 100% to do with how the data is stored, then it *might* make sense to use a stored procedure for it. Better that than filling your actual business logic with the minutae of a particular DBM.

On the other hand, you should never, ever put actual application logic in a stored procedure. The reasons are several. The most important is that stored procedure languages are all, to a greater or lesser extent, crap. This comment will cause me to be flamed to death by those who only know PL/SQL etc, but the fact is it's true. They are not general-purpose programming languages.

Sure, you might not RIGHT NOW want to fork off sendmail from your application, but some day you might. Or, horror of horrors, maybe you'd like to write directly to a system file? Or use a neat SNMP library you found? Although there are twisted, hacker-like ways to do these things in most DBMs they are hardly the model of reliability or professionalism. [1]

Secondly, they tie you in at a fundamental level to a particular database vendor. Database software is generally neither Free nor free. They want you to put your business logic in their stored procedure language because it will only run in their database products. Lock in is bad. OK, you'll be locked in whatever you do, but I'd rather be locked into Java or Python than PL/SQL.

Thirdly, you are losing control of your application's performance. You have very little control over how the code will be optimised or run.

Fourthly, you are breaking abstraction. It is very, very hard to write stored procedures which aren't entirely dominated by the structure of the underlying database.

Finally, assuming you probably will have to have a middle layer between the client and the database anyway, it's a bad idea from a maintainability point of view to bits of the same functionality among your layers.

[1] have you ever written a cron job to run a query to dump a table to a file to be parsed by a Perl script to send an email? You might be an Oracle Portal user.

Re:Good for what they're for; crap otherwise

[1001011010110101]

[1] have you ever written a cron job to run a query to dump a table to a file to be parsed by a Perl script to send an email? You might be an Oracle Portal user. I'm an Oracle Portal user, and for that I'd create a DB job that runs a java sp that uses javamail apis :P Some people might create a PLSQL proc that uses DBMS_SMTP to send the mail and run that from a DB job. Sure, you can also create a cron job that runs a c++ prog that does some of that crap, but why bother?. Dont put a perl coder to do DB work unless they know their stuff. Your solution is hackish at best, try RTFM :)

Re:Good for what they're for; crap otherwise

[NineNine]

Generally a database should be where the data is kept. Nothing else.

You know nothing about database-backed application development. I would recommend that you find some people who know what they are talking about and learn from them. Find an old Oracle or DB2 DBA or developer and talk to him. You know nothing about what a database is.

Thirdly, you are losing control of your application's performance. You have very little control over how the code will be optimised or run.

You've never used a database, obviously. I'll argue that there are zero high level programming languages that allow the granulaity of control that a real RDBMS allows.

Lock in is bad. OK, you'll be locked in whatever you do, but I'd rather be locked into Java or Python than PL/SQL.

You've never worked for a real company, have you? Front end app languages change more often than the weather, and in one company, many are usually used at the same time. One department uses VB, another uses Oracle Forms, another uses Java, another uses Perl. Companies don't change databases willy-nilly.

Seriously, I think that you should probably work with some real database people before you try building any apps on your own. You have a tremendous amount to learn.

Re:Good for what they're for; crap otherwise

[Anonymous Coward]

You forgot to mention CVS (http://www.cvshome.org/). Usually database developers commit their changes to their stored procedures directly into the only database, where they are stored. That is not good, because other Java developers are using the same database at the same time, so they see intermittent bugs.

It is better if all code goes into CVS, even stored procedures. Not to mention that every developer must have a copy of the database in their own machine, usually MySql (http://www.mysql.com/) or HSQL (http://hsqldb.sourceforge.net/) is better for that.

Then from CVS you take out a build (you use CruiseControl to generated builds that are compiled and unit tested using JUnit http://www.junit.org/) so you know all the Java code works with the stored procedures, the database model and the example data. Then you can add your code, with your unit tests, into CVS.

My $0.02 CLP (= $0.0000318238 USD according to http://www.xe.com/ucc/)

Re:Good for what they're for; crap otherwise

[Chazmyrr]

I think you've missed the point of stored procedures. The point of stored procedures is to provide an abstraction layer between the underlying database and applications. Additionally, they are the best place to enforce data integrity and they are extremely useful in implementing row level security.

Control over performance? I have just as much control over the performance and optimization of stored procedures as I do of functions written in any other language. And I don't have to do anything special to instrument the code. All I have to do is review the trace logs and execution plans to find bottlenecks.

Sure, you probably don't want a stored procedure to go out and pull a credit bureau report or something equally asinine but any application logic that doesn't have to access anything outside the database is fair game.

Re:Good for what they're for; crap otherwise

[Anonymous Coward]

You might be interested to know that PostgreSQL supports Ruby as a procedure language, among others. So, it's entirely possible to code your database logic, your gui applications, and your web server logic all in one language which is arguably one of the best.

Failing that, I think it's also easy to add new languages to PostgreSQL.

Re:Good for what they're for; crap otherwise

[dubl-u]

You know nothing about database-backed application development. [...] You've never used a database, obviously. [...] You've never worked for a real company, have you?

Say, you may not get out much, but there are other ways to build an app besides on top of an SQL server. You'll note that Doom 3, Microsoft Word, and the Google search engine don't use them. You'll concede that those are all real apps produced by real companies, right?

I'll argue that there are zero high level programming languages that allow the granulaity of control that a real RDBMS allows.

A little reminder here: databases aren't made from magic pixie dust; they're written in other languages. An RDBMS can't be more granular than the language it's written in, it can only be less. Indeed, it probably should be less, as its purpose is to make development simpler, easier.

The only question here is when it's more cost-effective to use an RDBMS on a given project. Back before DBMSes existed, people wrote a lot of low-level code to get things done. But eventually people pulled some of that code into common libraries for managing records, and the DBMS was born. If you're doing the kinds of things that Oracle expects in a way that Oracle expects, using an Oracle server may save you a lot of money and development time.

But if you're Google, an off-the-shelf DBMS is a big waste of time. Oracle's a great general-purpose engine, but Google engineers don't give a damn about the general case. They are building an utterly specific application; the generality of a database server mainly gets in their way.

So ease off on the "If you don't use a database you must be an ignorant fool," crap, ok? The parent poster was clearly inflamatory, but he describes one perfectly reasonable way to build successful applications. His isn't the only way, and neither is yours.

Re:Good for what they're for; crap otherwise

[NineNine]

Say, you may not get out much, but there are other ways to build an app besides on top of an SQL server. You'll note that Doom 3, Microsoft Word, and the Google search engine don't use them. You'll concede that those are all real apps produced by real companies, right?

Notice that I said app development with a database back end. I know that not every app uses a database. Not by a long shot.


A little reminder here: databases aren't made from magic pixie dust; they're written in other languages. An RDBMS can't be more granular than the language it's written in, it can only be less. Indeed, it probably should be less, as its purpose is to make development simpler, easier.

Oh, I understand that, but Oracle & DB2 & other "real" RDBMS' aren't written in high level laguages. With Oracle, at least (my experience), you can control queries and other processes down to the memory paging level. You can't control where *exactly* you put memory with C++ or Java. They have pointers, granted, but that's nothing compared to the fine tuning you can do in Oracle (which I've only *seen* done... I was a developer, not a DBA).

Re:Good for what they're for; crap otherwise

[sql*kitten]

RDBMS can't be more granular than the language it's written in, it can only be less.

You're right, that's why JVMs don't have garbage collection or strong typing, because JVMs are written in C, and C doesn't.

Oh, wait. It looks like software can have features that its host language doesn't. How 'bout that?

Re:Good for what they're for; crap otherwise

[drooling-dog]

Oh, wait. It looks like software can have features that its host language doesn't. How 'bout that?

But the parent speaks of granularity, not features. E.g., if you're building a library that furnishes/requires bit-level access to data, you don't choose a high-level language that doesn't provide that capability to implement it. The example you give is actually a reduction in granularity.

Re:Good for what they're for; crap otherwise

[dubl-u]

You're right, that's why JVMs don't have garbage collection or strong typing, because JVMs are written in C, and C doesn't. Oh, wait. It looks like software can have features that its host language doesn't. How 'bout that?

That's an interesting point, but a) it's one I agree with, and b) it's nothing to do with my point. I was answering NineNine's claim about about granularity of control. In Java, you can have garbage collection precisely because you give up control over memory allocation and deallocation. I think that's a worthwhile tradeoff most of the time.

An SQL database gives some very cool features, like the ability for non-experts to sit down and hack out queries to do ad-hoc reporting. But you give up a lot of granularity of control to get that. Various proprietary vendor extensions let you claw some of that control back, but I don't know of any that let you get all of it back in the same way that Java's JNI does. Indeed, I wouldn't trust a database that gives you that kind of power. Why? Because some of a database's best features, like the ACID guarantees, require never giving certain kinds of control back to anybody except a handful of really, really smart guys at Oracle.

Re:Good for what they're for; crap otherwise

[sparks]

I think you've missed the point of stored procedures. The point of stored procedures is to provide an abstraction layer between the underlying database and applications. Additionally, they are the best place to enforce data integrity and they are extremely useful in implementing row level security.

I completely agree, which is why I said their use was appropriate for things which are 100% to do with the data itself.

Re:Good for what they're for; crap otherwise

[sparks]

I'm not going to engage in a petty debate about my own experience, which by the way is substantial, but your post has proven several of my points.

• 
  This comment will cause me to be flamed to death by those who only know PL/SQL etc

You know nothing about database-backed application development. You know nothing about what a database is. You've never used a database, obviously. You've never worked for a real company, have you?

• They want you to put your business logic in their stored procedure language because it will only run in their database products

don't change databases willy-nilly (Of course they don't, they're locked in.)

Re:Good for what they're for; crap otherwise

[sparks]

Your solution is hackish at best, try RTFM :)

I agree that the solution posted is hackish. However it's not my solution; rather it was implemented by staff consultants with Oracle UK on an Intranet project for British Energy.

Good...think long term.

[chevybowtie]

The web is your current interface. If your app is 15 years old, you had another interface before. Don't believe this is the last one. Keep you SP and data logic in the database and it will grow even if/when your interface changes. You'll change/add interfaces more often than you will change databases.

from many years of experience..

[BigGerman]

pros:
easy to switch the frontend
maybe faster
Ability to add more security. Important these days when everything runs under "web/web" or "prod/prod".
cons:
no easy way to put under the source control - there is absolutely no way to be sure what exactly this particular installation is running. Too easy to change the source.
Flat, simple languages. No comparason with Java, etc as far as creativity, OOP, libraries, etc.
Different style of programming. Good SP developers are never on good terms with neither "enterprise" or LAMP programmers.

sprocs rock for transaction processing

[jonnosan]

I started working on existing app 3 years ago. At that time the app was a 3 tier ASP/COM/SQL Server app, where all the DB queries where dymanic SQL in the COM layer.

For performance reasons, we started pulling the SQL out of the COM layer and in to stored procs. The performance benefit came not because the sproc is precompiled, but because we can pass all the data to the DB in one hit, and not have to wait for network round trips between each step in a multiple step process.

Say you have the classic 'pay money in to your checque account from your saving account' example. You're business logic layer will probably be something like:

1. start a transaction
2. get the current savings account balance
3. make sure there's enough money in the account to support the request
4. debit the savings account
5. credit the cheque account
6. write some log (so the txn appears on the next bank statement)
7. commit the transaction

In a typical 3 tier app, between each of the steps above there will be a network round trip between each step, which will take say 10ms. neither the DB nor the app server is doing anything useful in that time, they are just waiting for data to go back and forth over the LAN. But you've got uncommitted transactions, which means rows or tables are locked so other processes have to wait.

So anyway, in order to scale up, we had to put all our business logic in to sprocs. The downside of this is writing stuff in TSQL in Query Analyzer means you don't get any nice IDE perks like code completion & intellisense etc.

The thing about having a DB guy is a red herring - you've got to have someone write SQL at some point, whatever layer you put it in.

Re:sprocs rock for transaction processing

[Anonymous Coward]

Good point. "Distributed Locking" is also much more expensive than a transaction lock which can be added and released locally.

Stored Procedures vs. Not

[glh]

In my experience, this is a difficult question to answer. There are many factors that you should consider in making a decision to use all stored procedures or not.

First, every business has different needs. Every software development group is also different in what they can or cannot provide. There are camps on both sides- many people in the database discipline will say "put everything in the database" while hard code developers will sometimes opt for queries in code.

Some considerations:

1) Consider the needs of your application. Is there a good chance your application will need to talk to another database platform or backend at some point in the future? This could be an argument for not using stored procedures. AS far as centralizing business logic goes, that can be done in just about any tier.

2) Where is your current bottleneck? How possible will it be to scale out your database server? If you are in a web farm scenario, your database server may be under significant load. Putting more logic on the database server can be a lot more expensive- it is typcially a lot cheaper to sacrifice performance on the backend for scalability. In other words, if you can keep your database server relatively load-free you can always add more web servers. I currently support a site that has over 2000 concurrent users at any given time, and currently our DB is the biggest bottleneck. It is a lot cheaper to cluster web servers than DB servers, since the DB is centralized and web servers can be duplicated easily.

3) Consider the experience of your staff and the culture of your IT department. If you have a lot of developers/dba's that are used to programming with stored procedures, and management is used to that paradigm, it may be difficult to change architectures without a compelling reason. "If it ain't broke don't fix it".

I'm sure there are other considerations, but those are probably the most three important ones I can think of right now.

Square hole, round peg..

[wfberg]

Is it just me, or are stored procedures used mostly for trying to shoehorn non-relational data into the relational data?

You know what I'm on about.. The umpteenth stored procedure to do some recursion to manage some sort of tree in a rdbms. Can't do recursive SQL (usually), so we'll solve it in a stored procedure (rather than take a small performance hit and sort it out in the application logic), because a tree just doesn't fit all too well in a relational model (unlike, say, in a hierarchical model)..

While I'm sure pros love the optimization capabilities SPs give them, and some might enjoy the abstraction that can be achieved, I think this is what suckers most people into the SP camp..

(Of course, SPs can be downright evil, in that they encourage vendor lock-in to quite an extent, but then, switching to a different RDBMS is hard anyway).

Re:Square hole, round peg..

[Chazmyrr]

It's just you. I use stored procedures to:

1)Provide a consistant interface to applications regardless of changes to the underlying database structure.
2)Enforce row level security.
3)Enforce data integrity.

Trees in relational tables without recursion is a problem that has been solved for years. http://www.sqlteam.com/item.asp?ItemID=8866 It is trivial to implement nested sets with a few lines of SQL and after the tree is constructed, a simple SELECT will give you children, descendants, or ancestors.

scalability

[Anonymous Coward]

If you dont have the luxury of using a clustered setup like Oracle RAC your DB will always be the performance bottleneck and and the toughest spot to scale.

IMO, SP's that are used to implement business logic are not optimal to get the best scalability out of your architecture.

Logic costs performance, thus the DB load overall increases.

Applications can be created so they can work joyfully in loadbalanced architectures, DB's are much more expensive to scale this way.

Version control with SP's is a freggin headache.

Logic IMO should stay in the code, the DB stores data and ensures data integrity.

Avoid corporate-speak like the plague it is

[Anonymous Coward]

"Architect" is a noun, not a verb.

Meeting

[cubicledrone]

DBA: "I think we should use more stored procedures"

Project Manager: "That would be consistent with our n-tiered strategy"

Programmer: "But all of our business rules are in object libraries"

Division Manager: "Could I get the ranch dressing? Thanks."

DBA: "The right way is to put business logic in stored procedures. It's faster"

Project Manager: "We're standardizing on faster programs across the enterprise"

Programmer: "But we'll have to re-write the entire system!"

Division Manager: "Pass the pepper. Thanks."

DBA: "Oh, that's alright. We were going to upgrade the servers too. We can finish both jobs at the same time"

Programmer: "We're upgrading the hardware too?! So basically we're building the entire company from nothing again, right?"

Project Manager: "You're not being much of a team player"

DBA: "The new system will be more robust and have new features"

Division Manager: "Lets get the system built as soon as possible. We're laying off the entire division in a few weeks"

Programmer: "WHAT?! I just signed a new car lease!!"

Division Manager: "Pass the croutons please"

Re:Meeting

[mitchy]

Project Manager: "So we need to start this project off as soon as possible, as this will become the foundation of several different applications."

Programmer: "Wow, kewl, we should do it all in Perl."

DBA: "We will definitely need to have a robust database if it is to power all these different apps."

Programmer: "MySQL rules man! Besides, it is free."

Division Manager: "Hmm, free is good. Can you manage the data in MySQL?"

DBA: "Well, actually MySQL doesn't do..."

Programmer: (interrupts DBA) "We don't need any of that relational integrity crap, that's just marketroid speak for Oracle, dude! All we need are a bunch of tables. I can wrap all of the SQL in a Perl package."

DBA: "Then you will have dirty data, and we can only write apps in Perl if we have to use that library for access. This is really putting all of our eggs in one..."

Project Manager: (interrupts DBA) "So this is free, right? I like free. My stepson talks about Perl, so it must be a totally hip language." (snorts)

Division Manager: "So all of the database logic is now going to be written in Perl, right?"

Programmer: "Absolutely! The database is only there to store data. We can check everything in the application. It will all work perfectly, because I am 3l337."

DBA: "But that's like just using flatfiles, how are you going to..."

Division Manager: (interrupts DBA) "Hey, if this is free, and you can do all the code, then we can cut half of our database specialists to save costs. That is a great idea."

DBA: "You can do that, it is your company and all, but without triggers and some sort of stored procedures..."

Programmer: "What's a stored procedure? Trigger? Isn't that a horse from the 60s?"

Project Manager: "Heh, that's funny."

DBA: "And if the next application needs to be done in Visual Basic, or some other language? Then you will have to write the whole thing over again because..."

Programmer: "Look, this is easy, me and my buddies from the high school can pound this out over the weekend."

Division Manager: "Now THAT's team spirit!"

DBA: "But how can you ensure consistency of data? Your database cannot even enforce minimal compliance of..."

Programmer: "Look, grumpy dude, I don't make mistakes. My code would never create dirty data, and besides, I'm 3l337."

Project Manager: "Wow, my budget is going to look great after this."

DBA: (leaves room, goes home and beats dog)

Re:Meeting

[0x0d0a]

Programmer: "Wow, kewl, we should do it all in Perl."

I don't like reading perl much, but it's not bad to the point that you should just throw it out.

Programmer: "MySQL rules man! Besides, it is free."

Division Manager: "Hmm, free is good. Can you manage the data in MySQL?"

DBA: "Well, actually MySQL doesn't do..."

Programmer: (interrupts DBA) "We don't need any of that relational integrity crap, that's just marketroid speak for Oracle, dude! All we need are a bunch of tables. I can wrap all of the SQL in a Perl package."

I've used MySQL very little, so I can't speak with authority about it's features. However, I do know that it supports transactions. I can't figure out how a DBMS could have transaction support and be missing any features required to guarantee data integrity. I'd be facinated to hear what you're thinking of.

DBA: "Then you will have dirty data, and we can only write apps in Perl if we have to use that library for access. This is really putting all of our eggs in one..."

They can't interface code to Perl? I mean, yeah, there might be some overhead, but the DBA is seriously stretching the truth.

Re:Meeting

[Anonymous Coward]

I can't figure out how a DBMS could have transaction support and be missing any features required to guarantee data integrity

MySQL Gotchas. Basically, it will silently convert datatypes and NULLs rather than throwing an error. Truly the VB of databases.

Re:Meeting

[Anonymous Coward]

Indeed. There exists no single situation where MySQL is a good choice if you care about the integrity of your data.

Re:Meeting

[Anonymous Coward]

integrity sm-integrity. who needs all that krufty data anyway?

Re:Meeting

[Anonymous Coward]

Neither could the programmer.

Nice story.

dyn vs. static, poorly written SPs, other thoughts

[kburkhardt]

In our enterprise, SPs are the norm. I've also worked on projects where embedded static sql with host vars was the standard (high volume order processing app), and where dynamic sql was the norm (quick build web site).

As with most architecture choices, SPs have their place. Here are the guidelines we try to enforce:
1. For online actions, KISS. Do data access only, unless the alternative is incredibly difficult biz logic in the app when it would be simple in the SP.
2. For batch, SPs are okay, but you must use cursors, checkpointing, control table, etc. In short, a proper batch arch.
3. Keep dynamic sql in the SP to an absolute minimum. Exceptions must be approved by God.

Despite the guidelines, main issue we face are shitty, unmaintainable SPs, where business logic is spread between app logic and SP. Second issue is unreadable SPs because they are building dynamic sql and executing it. Bad, bad, bad.

Whether or not the compile time affects you is another decision. If you can throw cash at the problem, more CPU is often a pretty good answer and cheaper than a good programmer. If you're trying to run your website on a 486, well, SPs are a better call.

Re:Stored Procedures often more harmful than helpf

[dheltzel]

Your point #2 is the most valid and important of your list.

That said, you missed a number of advantages of SP's, most of which have been covered in other posts pretty thoroughly. Still, the "flip side" to vendor lockin is that it's possible to perform some significant, vendor-specific optimizations. A good Oracle DBA understands the data structures and the way they are accessed at a low level, giving him/her the opportunity to write code that does the right thing, all the time. This low level code that encapsulated basic logic can prevent a lot of silly mistakes or poorly written code.

One DBA can make the SP's for a whole lot of Java,PHP, or whatever coders and make their life much easier. Since the DBA is ultimately responsible for the DB performance, it's preferable to give them the visibility into the code they need to keep everything running well.

Does this create vendor lockin? Absolutely. If that's a big concern, you should probably not use them. If you need to squeeze the last bit of performance or reliability from your DB, then vendor lockin might not even be a consideration. Just buy the best and fastest DB (and DBA) out there and don't look back.

A good read...

[Skim123]

A good discussion on this topic can be found at this blog entry: Don't use stored procedures yet? Must be suffering from NIHS (Not Invented Here Syndrome). The content of the blog post (by Rob Howard, a former Microsoft employee who most definitely knows his stuff) is definitely a good read, but the real gems are in the comments, which there are plenty. There's an equally interesting thread on this discussion by another blogger with his entry called Stored Procedures are Bad, M'Kay?. Both worth reading, if nothing else for the comments.

Why stored procedures are bad.

[killjoe]

I know there are exceptptions but in most cases....

1) Stored procedures are not written in an object oriented language and are almost always not written in an object oriented way.

2) Stored procedures are not checked into a version control engine.

3) There is no sane way to organize them beyond manimg tricks. No breaking up your stuff using directories for example.

4) No global compilation. No way to check ahead of time whether you just broke another SP by passing a string instead of a number in as a parameter. You won't know that till it runs.

5) No unit testing frameworks.

6) No cohesive way to examine code flow. What you end up with is a mountain of code snippets scattered all across your database. Cross your fingers and hope each step gets excuted properly.

7) No real debugger. No stepping through the code, no breakpoints, no watches.

8) Most commercial databases charge you per CPU. This means your CPU cycles are best used to keep data integrity, process queries and return recordsets. Most middle tiers are not licensed on a per CPU basis so you can afford to throw a lot of CPU cycles into executing code.

9) Last but not least you can couple your middle tier using a high speed interlink so there is no real need to use SPs.

Feel free to add to the list. SPs are not good.

Re:Why stored procedures are bad.

[Vancorps]

Are you for real?

First off, what do you think a table is? It is by definition an object, a view is another object. Temp tables and datasets are also objects. Now you're approaching the limits of TSQL but in the Oracle world you have PL/SQL in which case you have a full object oriented language to play with. If you feel you need to use TSQL then you have the option of creating an extended proc where you again have a full object oriented language you can use.

In both the Microsoft and Oracle world's you can see exactly what a stored procedure will do to your data.

I'm actually unsure what the hell you mean by version control. Changing one SP inherently does not change another SP. In order to create the SP you the server does a check against data constraints on all tables/views involved. Again, this is the same for both Oracle and MS SQL.

Debugging, now I'm starting to think you are on crack. In both Oracle and MS SQL you can rollback any transactions you just performed assuming two things, first the syntax had to be correct before you could even create the proc let alone execute it. Now you can watch exactly what parts of the database are effected in as much or as little detail as you like.

Since most commercial databases charge per CPU you would be wise to use optimized code which by definition would be a stored procedure which is precompiled and just awaiting execution. This is far fewer steps than doing a JIT compile.

I think I'll stop here for now

Re:Why stored procedures are bad.

[freedom_india]

My organization uses CVS to check in SQL procs. It then daily updates Oracle with data from CVS (stored procedures). Tools like TOAD, PL-SQL etc., allow you to test the proc a lot before you go off deploying it. They also offer deugging points, etc.

Re:Why stored procedures are bad.

[spideyct]

You have some good points, but some of them seem uninformed. You mentioned you know there are exceptions - but I think they are pretty big exceptions, to the point that they are closer to the rule.
I would agree with a previous poster that said stored procedures are neither good nor bad.

1) As of SQL Server 2005, you will be able to write stored procedures using C# or VB.NET. Oracle already allows you to write stored procedures using Java. These are all object oriented languages.

2) You CAN check in your DML scripts (I'm not saying a lot of people do...)

3) Agreed

4) I'm not sure what you mean by this. SQL Server, and the tools I used for Oracle, both give me compilation errors if I try to save invalid code.

5) None that I know of.

6) You COULD create test scripts that execute SPs with various parameters.

7) Not true. SQL Server with Visual Studio .NET (and I think Query Analyzer) allows real debugging. Stepping through code, breakpoints, and watches. As does SQL Navigator with Oracle.

8) Interesting point.

9) I wouldn't recommend SPs for performance reasons anyway. At least on SQL Server, there is no performance gain (the execution engine caches dynamic queries the same as stored proc queries).

Re:Why stored procedures are bad.

[aristotle-dude]

"1) Stored procedures are not written in an object oriented language and are almost always not written in an object oriented way."
Tables are objects but OO programming is not the be all and end all. I think you have some catching up to do. The hot thing right now is Object oriented Analysis and Design. The implementation phase can involve OO programming, Component based development and stored procedures, especially if you design you database to represent business objects discovered in the analysis.

"2) Stored procedures are not checked into a version control engine."
Sorry but that simply is not true. You can extract stored procedures easily to text files.

"3) There is no sane way to organize them beyond manimg tricks. No breaking up your stuff using directories for example."
Directories? LOL. Have you ever heard of namespaces? I hope you don't suggest using complex directory trees.

"4) No global compilation. No way to check ahead of time whether you just broke another SP by passing a string instead of a number in as a parameter. You won't know that till it runs.

5) No unit testing frameworks."
You obviously know nothing about test driven development.

"6) No cohesive way to examine code flow. What you end up with is a mountain of code snippets scattered all across your database. Cross your fingers and hope each step gets excuted properly."
See above.

"7) No real debugger. No stepping through the code, no breakpoints, no watches."
Um, how about using the debugging tools/query analyzer?

Sorry pal, but you are totally out of touch.

Re:Why stored procedures are bad.

[1001011010110101]

1. Oracle has a OO extensions (that I woudn't touch with a hundred metter pole :) ). It also has Java SPs.

3. Try packages.

4. Most databases have a dictionary.

Like:

select *
from user_objects
where status='INVALID'

That catches compilation errors. Runtime errors cannot be found, but you neither can you in Java.

7. Oracle has all that natively.

Re:Why stored procedures are bad.

[JamesKPolk]

All of my PostgreSQL code is in GNU Arch with no problem, complete with unit tests, and without restrictive licensing.

I like them for one reason: I put all data sanity checks in one place. The database keeps the data sane, and my applications do the work keeping the data up to date. It's a clean, clear division of responsibility.

Re:Why stored procedures are bad.

[Anonymous Coward]

Table is an object, view is an object too. The question is what kind of objects. These objects allow only very primitive methods of manipulations. Select/Update/Insert/Delete/Create/Drop is all you can do with them. You can't define business methods of manipulating those objects and encapsulate them with the object. You can't have inheritance and polimorphism for those objects either. So even though they are objects they are by no means the object oriented programming "objects".

The lack of version control referred is probably that there is no good way of going back to any previous version of the stored proc. You have to create DDL scripts and store them in Visual Source Safe of CVS. You also can't have several people working on the same stored proc. (I know it is somewhat bizare example but if you too reliant on stored procedures they have tendency to grow to obscene sizes)

The parent is not correct on this one because it is possible to debug stored procedures the same way you can debug any other code but it is not related to any transactions.

There are few things that RDBMS can do very good: Access the data with blazing speed searches and perform pretty fast data modifications.
What they can't do well is algorithm execution. The sooner developers will realize that Oracle and MS SQL Server is not a good execution environment for their algorithms the better.
That's why RDBMS vendors trying to include Java and .NET environment into their products. Even the JIT compiled code is so much faster then your beloved stored procedures.

Use the right tool for the job! The data access work should be done by database and the business logic should be in middleware.

Re:Why stored procedures are bad.

[Vancorps]

Select,Insert,Update,Delete,Create,Drop are most definitely not the limit of what you are given.

First, you can create datatypes, or tables, or views, or users, or groups, or databases, or procs, or extended procs... the list really goes on. Second, you can have inheritence as that is the very definition of a relational database. If I make a change to a table with a relationship to another than that other table will be adjusted accordingly without any further developer interaction.

As for polymorphism I tend to wonder if you fully understand procs and how you can create params which can adjust themselves depending on who called them, what action they perform, or any number of user defined criteria.

As for version control, in both Oracle and MS SQL you can rollback your procs. By default you could rollback past its creation with Oracle. MS SQL needs to be configured to do this but it is quite capable of the same function.

Back to debugging, yes it does relate to transactions, by default everything you do in Oracle is a transaction. MS SQL can be configured to behave the same way. That means even if you throw untested code out there you can roll it back negating any damage. In addition to this you can do two things with query analyzer. First you can look at the execution plan by running a simulation. It will show you what calculations it will perform and how many cpu cycles it will take. Second, you can run the proc against your test database because no one develops directly on a live mission critical database ;)

Just about the only areas I agree with you are with algorithm based execution. In many situations it is both faster and easier to develop it in the a different language.

As for business logic, most applications place a good chunk of their logic in the application itself. Even larger applications will have several middle tiers, a basic chaining of databases to minimize the load on the master.

You're right though, use the right tool for the job. Stored procs aren't for everyone, but it sure makes a much easier way to develop applications. You are essentially building a database api when you build them and I rarely ever hear anyone saying APIs are a bad idea.

Re:Why stored procedures are bad.

[ron_ivi]

" 2) Stored procedures are not checked into a version control engine."

Your DBA should be fired. All source code should be checked into the version control system. In fact, since the version of your stored procedures and the version of the application almost certainly have dependancies on each other, they should be checked into the same repository and branch in your source control system. How else could you reconstruct a checked in application?

Re:Why stored procedures are bad.

[Anonymous Coward]

"First off, what do you think a table is? It is by definition an object, "

C'mon silly, that's like saying an array or a struct is an object.

It's true that some databases (postgres, oracle) support the object-like properties of tables, but how often to you really use features like table inheritance?

Re:Why stored procedures are bad.

[bwt]

Gosh, what a crummy analysis.

1) (not OO) If this is so important, use an OO language to write them. Eg: java SPs and Oracle types are both OO. More importantly, databases themselves are not OO, so this is desired most of the time anyway.

2) (not version controlled) What do you think would happen if you checked your DML into a source control system??? would the world explode? This is a moronic point. Your DBA's shouldn't install anything on your production system other than from source control.

3) (can't organize other than by naming) You refuted your own argument -- use a naming convention. Also, use packages.

4) (No global compilation) I'm not sure what this even means or why it is desireable. If it was desirable, you could write a simple script that would issue the compile statement for an explicit list of packages or all packages in the dictionary. Compile errors like the one you claim is not reported in fact are generally reported.

5) (No unit testing frameworks). If you use a langage that has one, they this is false. java SPs can be tested with JUnit, PL/SQL SPs can be tested with utPLSQL. Bzzzt.

6) (mountain of code snippets). Maybe you have this problem, but I don't. Hmmm, I wonder why.

7) (No real debugger. No stepping through the code, no breakpoints, no watches.) Oracle has this. So does postgres. So does MS SQL Server. So does Firebird.

8) (commercial databases charge you per CPU) If you don't like the way they charge you, don't buy it -- that's why it's COMMERCIAL. Oracle has a per named user licence if you don't like the per CPU one. There's always postgress or firebird if you don't want to pay.

9) (couple your middle tier using a high speed interlink) Sure, and I can have my middle tier call my stored procedures if I want.

Re:Why stored procedures are bad.

[attonitus]

I'm actually unsure what the hell you mean by version control

That much is clear.

Version control means CVS, RCS, Perforce, Clearcase, Sourcesafe, or the like. Strictly speaking, the original poster is wrong, in the sense that it's quite possible to check your stored procs into a version control system. However, they're also on the right lines, in the sense that publishing stored procs from the version control system to the db is usually more of a pain to manage than building source code.

Re:Why stored procedures are bad.

[killjoe]

"As of SQL Server 2005"

I am sure that will be great once ir hits the market.

" You CAN check in your DML scripts (I'm not saying a lot of people do...)"

Most people don't.

"I'm not sure what you mean by this. SQL Server, and the tools I used for Oracle, both give me compilation errors if I try to save invalid code."

That's great for the SP you are working on at the time. What it does not tell you is if you are passing a wrong type of parameter to another SP (that possibly somebody else wrote). For example if I write a method in java that calls another method of another object and passes the wrong type of argument the compiler will tell me and the build will break.

"You COULD create test scripts that execute SPs with various parameters."

Just make sure no real data gets changed.

Re:Why stored procedures are bad.

[killjoe]

"If this is so important, use an OO language to write them"

Great if you have oracle. Sucks if you have SQL server.

" You refuted your own argument -- use a naming convention. Also, use packages."

naming tricks are stupid. Paackages again great if you have oracle sucks if you don't.

"I'm not sure what this even means or why it is desireable."

If I write call a method in my middle tier and pass in the wrong data type the compiler tells me that and the build fails. Do the same thing in T-SQL and you don't know till it executes.

Re:Why stored procedures are bad.

[bwt]

Sounds like you are making an argument for why SQL server is bad, not why stored procedures are bad. I don't know a whole lot about sql server, so I can't agree or disagree with your facts or conclusions.

Postgres has OO stored procedure languages, so it isn't only oracle that is reasonable. Does SQL server really not support the .net languages in the database?

Re:Why stored procedures are bad.

[aristotle-dude]

One point, I wanted to hammer home to you. OO languages or OO programming don't mean sh*t without OO Analysis and design. I don't don't know if you have noticed or not but the industry has moved away from strict OO programming to using Component based architecture built on an OO design created by an analyst. You can take a OO language and start coding without coming up with a product that is neither OO or scalable.

I guess you know more about OO than Sam Rostam, formerly of Sun Microsystems. I have attended some lectures put on by him.

I suppose you are some kind of guru who knows so much about OO programming and RDBMS. Please enlighten us oh wise one.

Re:Why stored procedures are bad.

[bit01]

I'd add:

10) Stored procedure languages are usually kludgy and encourage bad programming practices.

11) Applications written in two or more languages are usually a significant maintenance headache. Not only do you require experts in all the languages but the interfaces themselves can be problematic. This is particularly true for applications that store significant state both in and out of the database. Synchronisation can be a major issue.

I use stored procedures when I have to, usually only because I need to reduce the size of a retrieved dataset before it hits the network and a simple SELECT won't do. Other than that I avoid stored procedures like the plague.

DBA's in the organisations I've worked in have generally had an unrealistic view of their importance and technical abilities. They get away with a lot more than they should mainly because they are gatekeeper to the organisation's "crown jewels", the data, and because database management is a mess that most developers try to avoid.

I've found from a practical standpoint it's best to keep the business logic out of the database and treat the database as a simple data store. Your mileage may vary.

---

It's wrong that an intellectual property creator should not be rewarded for their work.
It's equally wrong that an IP creator should be rewarded too many times for the one piece of work, for exactly the same reasons.
Reform IP law and stop the M$/RIAA abuse.

Re:Why stored procedures are bad.

[Anonymous Coward]

'"2) Stored procedures are not checked into a version control engine."
Sorry but that simply is not true. You can extract stored procedures easily to text files.'

That's quite the ass-backwards way of thinking of it. You don't "extract stored procedures" any more than you "extract C++ code" from a .dll file. Check i the stored procedure first, and then import it into the database.

Re:Why stored procedures are bad.

[Anonymous Coward]

the first and last sentences in the first paragraph mean something. The rest is just a puff piece. Ignorance and arrogance together again, eh.

Re:Why stored procedures are bad.

[Anonymous Coward]

FYI, the last two projects I worked on, I used the MS-SQL->VS.Net->VSS integration to store and deploy SP drop/create scripts.

It worked out very well, I thought. The IDE will pull the current state script out of the RMDB, store it in a sub "Database" project under your main solution VSS project, and offers 1-click deployment against multiple targets as well as T-SQL syntax checking.

Was slick. And against SQL Server 2000, there was even step-through debugging of the SP against the actual server.

Re:Why stored procedures are bad.

[killjoe]

TO be fair most people don't probram using java SPs in oracle. Most people use PL/SQL which has the same kind of defeciencies as T/SQL.

No SQL server does not have .net languages (yet). It will be "RealSoonNow" which in MS terms means when longhorn comes out.

Re:Why stored procedures are bad.

[Vancorps]

I think you misinterpreted what I was saying. I was suggesting that such products do exist and the parent was implying that such things did not exist.

I have seen many version control systems that do a fine job of it.

Re:Why stored procedures are bad.

[attonitus]

I did misinterpret you, but that was because the following statement didn't seem all that relevant:

Changing one SP inherently does not change another SP. In order to create the SP you the server does a check against data constraints on all tables/views involved. Again, this is the same for both Oracle and MS SQL

Re:Why stored procedures are bad.

[Vancorps]

That is quite understandable considering those "sentences." I don't feel right calling them that since its pretty obvious I didn't look it over. I was merely saying that there is inherently a bit of version control because if you realize ten or so revisions down the line that your proc isn't doing something properly you can role back the proc specifically and be where you left off. Its a form of control versus management.

Basically I was just trying to say a whole lot at once because the parent so blatently did not understand anything about modern database development.

I suppose I could have just said he was wrong and been better off numbering everything. Oh well, tis what I get posting on slashdot right after a long day of fixing sql server cluster while trying to make Panda not suck so bad.

Re:Why stored procedures are bad.

[nzhavok]

Not sure if you are trolling, karma-whoring or just plain ignorant. However for the sake of anyone that actually reads you post lets correct some things.

1) Stored procedures are not written in an object oriented language and are almost always not written in an object oriented way.
Tables are objects and they are treated like objects in stored procedures so this is wrong. You can also write SPs in Java (on oracle).

2) Stored procedures are not checked into a version control engine.
WTF does that mean? Of course it is, you do know you can export it and save it right? In most environments there's a "file" menu with a "export" option, sorry if that's too hard for you.

3) There is no sane way to organize them beyond manimg tricks. No breaking up your stuff using directories for example.
erm, "packages"!

4) No global compilation. No way to check ahead of time whether you just broke another SP by passing a string instead of a number in as a parameter. You won't know that till it runs.
Hmm, how about you compile it and take a look in your IDE. You can do this in your own schema as well so it won't affect others. Or were you thinking all SP developers use vi and sqlplus on a single production schema? PLSQL Developer for example has a button for compiling dependent objects.

5) No unit testing frameworks.
Well there's JUnit for Java SPs, but I guess your ignorant of that. There are many unit testing frameworks for PLSQL.

6) No cohesive way to examine code flow. What you end up with is a mountain of code snippets scattered all across your database. Cross your fingers and hope each step gets excuted properly.
Now your just talking utter crap, in most editors you can click on a function call and open the SP that's called.

7) No real debugger. No stepping through the code, no breakpoints, no watches.
Of course there is a debugger, of course you can step through the code, of course you can set breakpoints, utter lies.

8) Most commercial databases charge you per CPU. This means your CPU cycles are best used to keep data integrity, process queries and return recordsets. Most middle tiers are not licensed on a per CPU basis so you can afford to throw a lot of CPU cycles into executing code.
What an ignorant viewpoint. The bottleneck of database engines is disk speed not CPU. What, are you doing FFTs in the stored procedures or something?

9) Last but not least you can couple your middle tier using a high speed interlink so there is no real need to use SPs.
What an excellent example of "Fallacy of Exclusion". This statement would only be true if speed was the only reason to use SPs, however this is not the case so your statement is void.

You have so much wrong here I can hardly believe you are serious! Do yourself a favour and learn a bit about a subject before posting.

Re:Why stored procedures are bad.

[bwt]

PL/SQL has almost none of the supposed defeciencies discussed here.

the myth of portability

[jonnosan]

having your apps portable across DB's is one of those things people always think is important but in my experience it's a total crock for an app that is going to be installed in a single location, i.e. a commerce website or an enterprise specific app.

You are just as likely to want to change your middle tier as your DB. More likely, in my opinion. If your business logic is in sprocs, then it's as easy to call from Java as it is from .Net.

Re:the myth of portability

[barjam]

I used to work for an ecommerce (java/ejb) website that needed to ditch it's oracle licensing (we were running out of money) and had to switch everything over to mysql. Our site featured a custom built ecommerce site, forums and a few external services (feeds like weather etc).

Because we designed everything using EJBs and a decent middle tier we were able to convert everything over in two days (two senior developers).

Saying that I do agree with your point. Changing DBs doesn't come up very often in the real world.

Re:the myth of portability

[geekoid]

It comes up often enough to keep it in mind. When Oracle (or whoever) changes the pricing, it's would be nice to be able to tell your boss that it won't be a complete rewrite.

"It depends"

[enkafan]

The answer to this types of question will always be "it depends." I personally use all kinds of stored procedures. It is far more easy to secure a database based on sprocs and views than it is on the table or field level. So sprocs bring security to the table.

If I'm doing an operation that would be better suited "close" to the data, the answer is again stored procedure. It saves you round tripping that data back and forth potentially multiple times. You also get the added plus of the procedure cache, but honestly products like SQL Server 2000 already do an excellent job of caching ad-hoc statements.

Probably the best case against stored procedures is if you are using an O/R tool that generates code dynamically based on your tables and rules that you set up.

I'm not for putting all your business logic in a sproc. That's a bit silly. If you are concerned about having to update clients if it changes, you might want to look into developing some middleware components. It's pretty easy if you haven't done that in the past. I'm not for accessing everything via ad-hoc queries either. The risk of sql injection is too great (yeah, yeah I know your an expert, but what about that intern we hired to make some UI updates...), and security is just too difficult to maintain at that level.

Re: Prepared statements and vendor-neutral SQL

[Anonymous Coward]

Dream on, pal!

Don't make me get my soapbox!

[t'mbert]

This is a hot-button topic for me, so I'll try to keep it simple.

When I came to my current job, my opinion was to keep as much of the logic in the application as possible, stored procedures should help make the developer's life easier, and make the database more robust (triggers mostly do this part).

When I came here, however, they forced EVERY SINGLE QUERY into a stored procedure, and like you said, it brought development speed to a halt, because every project was assigned "a database guy" to do the work, but there weren't enough database guys for the projects.

It was pathetic, and when our big layoff occurred (we got bought out), we found alot of really really bad stored procedures that the so-called experts made for us.

Worse, it makes code releases a nightmare, because not only does the codebase have to be updated, but the database too, every single time. Unlike code bases where you can checkout your latest release, with databases you have to a) find all the altered database scripts, b) package them up for the release, c) halt the database for the release and d) run the script at go-live and hope you didn't miss one more thing.

Mind you I still use stored procedures, views and triggers where they make sense (when it's obvious that it just belongs in the database), but mostly I try to put as much logic as possible in the app.

Now you want to make me get into NULL values and candidate keys issues too! Bah, no NULLs anywhere, #1, and #2, don't make up integer primary keys when an existing field will work. Don't let your database administrators talk you into it, I beg you!

Re:Don't make me get my soapbox!

[PingPongBoy]

It makes sense. I try not to store any queries in the database except maybe some one-time queries because of the difficulties remembering for so many different databases serving so many different applications what query is doing what. Most of the time I find that a query is called by one part of a program and not by many different parts of many programs. Also a query in a database might be changed by someone and then the program using the query stops working - some people who would never change the program have no compunction changing something in the database feeling they cannot hurt the data. Databases have very friendly interfaces for managers that want to quickly peek at the data. Isolate management from the code.

Nulls are useful - a program that hasn't entered a value into a field yet knows that the field has to be null. It's a good way to keep track of whether anyone has gotten to the point of actually entering a value.

Autonumber primary keys are useful too. If I'm looking at a database that I haven't been looking at for a few months in order to do a couple of small tasks, I won't remember exactly which fields need to be unique. However, by having number foreign keys named after the table they are related to I know by looking at the table what relationships exist. This also allows the tables to be modified later to add/remove uniqueness and add/remove fields that could have been used as keys.

One danger is the possibility of mistaken removal of uniqueness and then creation/alteration of records with nondesired duplicated values in certain fields, especially the omission/alteration of information identifying a record.

Best of both worlds

[zyridium]

Where I used to work we generated stored procedures for all tables based on the referential integrity and indexes defined in the database. This code was regenerated when the data structure changed. Along with this code a set of small classes to allow simple (typed) access to the data were also created. The classes also supported batch updating of mutliple rows for a set of data.

These generated procedures allowed for the majority of data access that was required to be done using all the benefits of SPs without having to write them, and made the number of SPs that needed to be handcrafted manageble..... From the application side where the logic was enforced the interface to retrieve and save data was very clean and neat and typed.

They can help a lot...

[Eric Damron]

I work with several in-house database applications implemented using PowerBuilder. They run against a Microsoft SQL server.

We decided to use stored procedures instead of inline SQL so that we could make modifications on the server instead of having to change the code in the application, recompile it, repackage it and deploy it to over 100 PCs.

The drawback is that it's not always easy to know what's going on because the application is broken into various parts. That is there are business rules being enforced in the PowerBuilder code and in the stored procedures on the server. I guess once you get to know the application really well it doesn't matter but it can make it a real pain in the ass for a new programmer.

It depends on the complexity of the procedures.

[Lord Kano]

I am working on a few fairly complex Filemaker databases and for many functions, stored procedures are the way to go.

If the calculation needed is fairly comples, I prefer to do the work at runtime using scripts. That allows me to tweak the results for the situation the user is in when the function is called. If it's something simple like percentages, go for the stored procedure. It's simpler. Set it up and forget about it.

LK

Shallow thinking

[Anonymous Coward]

Because as soon as you rely on an external process to maintain data integrity, you're bound to fall prey to some sloppy programmer who does not understand the data relationships and will not properly maintain the data integrity.

1. You're relying on the external process for all your data anyway - if the app screws up data, it screws up data.

2. Stored procedures are hardly immune from sloppy programmers who don't understand data relationships.

3. Stored procedures are prone to being hacked by DBAs with no concept of software CM or development processes....

Migration

[IGnatius T Foobar]

The big downside of stored procs, from a system administration point of view, is that they make it difficult to migrate to a different brand of database. Let's say your organization just bought a brand spanking new Sun Fire 9990000 and it's got Oracle 20i running on its 512 processors. You've been granted permission to move your application onto it!

Scenario A:

You didn't use stored procs. You migrate the data, change the DSN's on your app tier, and you're done.

Scenario B:

Not only are you a moron for choosing MS SQL Server, you're a double dumbass moron because you wrote lots of stored procs and triggers. Now you can't just move the data because every RDBMS uses a different language for stored procs. If you're lucky, there might be a conversion tool. If you're really lucky, it might even convert 50% or more of your code without errors. But you're in for a lot of work.

It Depends

[CynicTheHedgehog]

How large is your application? What architecture are you using? Are you using an application server? Does the application server manage persistence? Is the application distributed? Is vendor lock-in an issue with your database? Do you have a staff of knowledgeable DBAs to code and debug stored procedures? Do you have a strategy for passing exceptions to the business tier? Is performance really that much of an issue?

Some things to consider:

There is no standard (or convenient) way to pass or return complex data structures (objects, arrays, etc.) to and from a stored procedure.

Stored procedures can return cursors, but implementation differs from databas to database and driver implementation to driver implementation. In my experience it's more trouble than its worth.

Application servers nowadays have very efficient persistence management that monitors object properties for changes using dirty flags and whatnot. Tools such as Hibernate can even track multiple of changes to object in memory before actually committing the data to the database, reducing the number of transactions, which may increase performance more than a stored procedure.

Stored procedures aren't good at logic. The looping and conditional structures are very primitive in most cases. Oracle has Java stored procedures and SQL Server 2003 has .NET stored procedures, but that's all fairly new and I'm sure there's a catch somewhere.

It's always more difficult to debug a stored proc. You can't exactly compile it -f and run it through an IDE. You'll have to use printlns or SQL traces in most cases.

Transaction management can get complicated using stored procedures. Many application servers/environments have transaction management through message oriented middleware...if you build all of that logic into stored procedures then that defeats the benefit of using configurable transaction management.

Stored procedures can be good, but they are really at their most useful in 2-tier architectures (client/server) on set-in-stone platforms. There is a lot of PowerBuilder/Oracle stuff out there, especially in the Government, because Oracle ain't going nowhere.

Personally, I'd avoid them (makes everything simpler and is the most flexible option), but it's up to you. Use constraints and triggers (where you have to) to ensure integrity, but handle business logic in the business tier.

Insightful?

[Anonymous Coward]

Mods, how is this insightful again?

Stored procedures make CM difficult

[Anonymous Coward]

And on a large program, good CM is critical to maintainability and therefore reliability and availability.

And on the large programs I've worked, I've seen too many DBAs who think nothing of going in and changing things on the fly on an operating system.

At least if your business logic is compiled into a binary, it has to go through some sort of CM process.

Stored Procedures Dont Help

[antarishk]

This is a great discussion. I think there are three camps of technologists here.

1) The DBAs

2) Junior Microsoft developers (typically VB)

3) J2EE developers (N-tier developers)

My experience has been that DBAs are usually proponents of stored procedures. There are two simple reasons for this.. familiarity of database engine and job security.

The Junior developers who typically develop two tier apps prefer stored procedures for performance reasons, because there is usually a big-a*s backend database server which can lift a lot of the data processing load.

Then there are the N-tier developers who are seasoned enough to pick the right architecture. These people prefer non-stored procedural approach. They understand that the database is a persistent store and business logic should be in the middle tier. It gives them flexibility to swap database engines and business logic modifications are localized to the code modules they are working with.

Re:Stored Procedures Dont Help

[Inthewire]

...because swapping DB engines is a sign of maturity.
I don't know your situation.

I work with a big ole AS400 (really, one of the larger ones around, and yes, I know this isn't the epitome of computing) and a clustered MS SQL-2000 DB.

I doubt I'll walk in tomorrow and find we've been converted to Oracle.
Internal software isn't focused on conversion, it's focused on getting the most out of a platform.

Re:Stored Procedures often more harmful than helpf

[Osty]

SPs turn your database into an application server, centralizing things that needn't be, and raising load on that central machine.

When was the last time you saw the CPU sustain a high load on your database server? Database machines are constrained by I/O (disk and network) and memory, not by CPU in most cases. Therefore, your 8-way SQL box is sitting at 10% utilization. Why not get a bit more use out of it?

SPs invite use of vendor-specific features, and therefore lock-in and loss of portability.

Portability for the sake of portability is a waste of time. More importantly, given that every vendor has non-standard extensions, and the SQL definitions (SQL-92, SQL-99) don't go far enough, you'll find that you almost always need to use some vendor-specific features, whether you're writing ad-hoc queries or stored procedures. At least with stored procedures, you only have to change it in one place when you migrate, rather than changing it all over your code. Tell me, how do you add an AUTOINCREMENT/IDENTITY/auto-numbering column to a table in vendor-neutral SQL?

SPs are not typically amenable version control and are maintained outside the rest of your code base.

Says you. In my team, all of our stored procedures are stored in the same code tree as everything else, controlled by the same source control. We take it even a step further, and check in all of our database object generation code (tables, keys, indexes, triggers, etc). So, just because you don't track your stored procedures in a source control system doesn't mean it can't be done.

SPs represent "premature optimization." There may be a time and a place for SPs, but they are used a lot more than needed in many applications. For example, one application at my company has over 1,000 SPs, and quite a number are just wrappers for simple select statements.

You're assuming that the speed benefit from stored procedures typically comes from the fact that they're compiled. While that's true, it's also not much of a benefit (you might win 50-100ms per query, yay!). No, what's more important is that your queries are centralized, so that you can optimize them more easily. What's easier to optimize: A single query to <insert operation here> in a stored procedure, with the procedure called by 100 different methods; or 100 different queries to <insert operation here> across all of your code? Yeah, sure, you can centralize that as well, but what's stopping a developer from writing his own query rather than using the optimized one you've already provided for him? With stored procedures, you can deny access to individual tables and force that person to use your stored procedure.

If you do....Naming conventions

[N8F8]

I only have one compmaint about usign stored procedures. Fro ma programmign prespective if can be rather difficult to debug queries when you add this extra layer between you and the underlying query. I highly recommend you decide on an elaborate naming scheme before you begin. It can be very difficult to 1) Look at a stored procedure and identify the code that is using it. 2) Find a stored procedure in your DB managment tool if you have a buttload of stored procedures. The more well defined the naming scheme the easier it can be to find your stored procedures.

I would recommend some sore of pattern like [Program Module]-[Verb (Select/Delete/Update)]-[Noun]... or somthign along those lines. Sorting out a SP with a name like "USER_ADMIN-SELECT-USER-BY-USER_ID" is much easier to find than "Get-User" or "SelectUser".

Re:Stored Procedures often more harmful than helpf

[Anonymous Coward]

SPs are not typically amenable version control

Buddy of mine at work says that at his old job, they kept the stored procedures scripted in a directory under version control, and it worked out nicely for them. Haven't really pushed for it at current company, since the only source control is MS Sourcesafe...if we succeed in our Subversion advocacy we might go for it.

Database stored procedures

[accvio]

I'm a long term oracle DBA that has worked (and still works) for several large companies, and the general agreement is that business logic should be kept in the database, to ensure data consistency. Business logic can only be stored as a stored procedural object, because, by definition, business logic is procedure applied to data in order to conduct given business. Problem with holding the business logic in the application is that while applications have to access the data in the database, they may may have different understanding of business logic and thuc, very good chance of creating a mess. Putting the business logic in the database means that all the applications will have to use the database version of business lohgic.

Bad, Bad, Bad.

[Anonymous Coward]

The only major advantage of stored procedures that I can think of is that of reduced communication latency. So, if your query is particularly enormous (and sent zillions of times by jillions of connected clients over slow connections), then I *might* consider a stored procedure.

But, other than that, stored procedures tend to pose very big problems from the standpoint of maintaining an application that evolves over time. (I know, because I used to deal with this at a company that I worked at.)

Imagine, for a moment, that your Software version 1 has a particular DB schema, and uses a set of beautifully-crafted stored procs. Then, when you come out with version 2, your schema may have changed; so, as part of your software installation procedure, you now need to run SQL scripts to update the stored procedures. (If the code had all been in your C/PHP/etc. program in the first place, you wouldn't have to worry about that.)

So far, you're probably thinking: "What's the big deal? It's just a change to the install routine." True, but also not true. What happens, then, on the day in which your software installation routine dies or the user installing your software does something unexpected (e.g., power goes out, etc. etc.).

Well, I'll tell you from experience (and I've seen this over and over and over): your customer's multi-gig database now probably has half of its stored procedures working for Schema 1 and half for Schema 2. And your application probably works fine, until one day, when the user tries to do a certain particular thing, which then opens up a big black hole of nightmarish debugging & tech support doom.

I have seen variations of this happen so often that I have just become convinced that stored procs are not worth the trouble -- far better to modify your app, and keep all of the necessary SQL there (and in one consolidated place) than to have to worry about keeping the customers' DBs in sync.

Re:Stored Procedures often more harmful than helpf

[Hangtime]

1. SPs turn your database into an application server, centralizing things that needn't be, and raising load on that central machine.

And having all that load sit on an application server while also taxing your database server with poor performing queries is better?

2. SPs invite use of vendor-specific features, and therefore lock-in and loss of portability.

Unless you work for a software development company, I see the purchase of a core database server being one of those one-time-only purchases.

3. SPs are not typically amenable version control and are maintained outside the rest of your code base.

SQL Server and Visual Sourcesafe can be integrated together and all SQL scripts can be checked into your version control suite of choice.

4. SPs represent "premature optimization." There may be a time and a place for SPs, but they are used a lot more than needed in many applications. For example, one application at my company has over 1,000 SPs, and quite a number are just wrappers for simple select statements.

Would you rather write it once where you KNOW your going to get the best peformance or figure out at the end of your development cycle that your loop through 500,000 rows is slow performing considering that both cost you about the same amount of time. Also simple selects could be there for the purposes of security. If you run your applications exclusively through stored procedures when working with the database you can deny all rights except for the procedures to your application thereby enhaning the security of your application.

Re:Stored Procedures often more harmful than helpf

[Anonymous Coward]

Sorry I must disagree with number 3.

Source control is rather easy to use with stored procedures. I do it every day. I have a 'dev' database with a snapshot of some data. The 'real' database is built out of the source control. So for my work to get in it MUST go through source control. I put that in place so people would stop 'hacking' the real databases. It cleaned up lots of big files laying around. Simplified installs and made changing things a snap.

Its only hard if you LET it beat you. Total time to make it happen? About 3 days of work. Time saved? I would say months of 'do you have the main file?' 'can I change this stored proc or are you?' 'who the hell did this?' 'why did they do this?'. Its all in source control the who what when and why.

Basic economics

[Anonymous Coward]

I'm migrating 591 stored procedures, none of which I wrote, to a different flavor of database server at the moment. As you can imagine, this gets old pretty quick. However, IMO the performance benefit of having pre-compiled query plans dramatically outweighs the downsides of using stored procedures.

Let's do some math here:

If you assume it takes you 4 hours per procedure, that's a bit more than 2,000 hours total time. Call it a man-year.

Now, your program will need to test all those procedures. Call that another man-year or so when you account for all the overhead of coordination between you and the testers (and all the other BS that goes on when more than one person is involved...). And you are going to test, aren't you?

Now we're up to two man-years.

It costs a company at least $100,000 per year to field a decent IT person (counting taxes, insurance, benefits, overhead like lights, heat, air conditioning, and computers...).

So, to migrate those 591 stored procedures just once costs damn near $200,000.

You could have just done the work in ANSI SQL and buy a lot faster hardware with that much money. And any time you want to change databases, you'd just have to modify a few makefiles to use a different SQL precompiler.

Of course, it depends.

[Anonymous Coward]

Does it really matter if my business rules are centralized in stored procedures or in a set of php/asp scripts (ie, in the web tier)?

Only if you care to reuse the logic anyplace beside the PHP/ASP scripts (I won't comment on putting the logic in these scripts in the first...won't even go there).

For example, you already have a bunch of SP's in your system. I don't know what you C/S clients are written in, and it doesn't matter. What does matter is all of those SPs are available right now to you as a PHP/ASP programmer.

In addition, you either have to have a dedicated T-SQL or PL/SQL coder who then is the weak link in your coding chain, or your pool of developers must become fluent in both your scripting language of choice as well as the SP language.

And you have that exact same problem with your HTML/Javascript/VBScript/PHP/ASP coders. How is this any different? You have to write SQL code in your pages anyway, it's a small leap to writing that SQL as a SP.

As for DB lock in, you're just as locked in to PHP or ASP, so how is this any different in that regard? You can't just up and move your PHP app to ColdFusion or whatever.

While DB portability is "nice idea", performant applications need to take advantage of DB specifics no matter what (whether it's SPs, query structure, or "hints", or special SQL syntax).

The way around that problem is to isolate your SQL. SP's give you a nice place to do that (since somehow I doubt you'll be doing that in your PHP/ASP code).

An outsider looking in

[mblase]

I'm a web programmer who's worked on databases externally, and built a few in MS Access, but my new job has a big ol' MS SQL Server database and my project has a LOT of stored procedures. As in, there's nearly no SQL code in the ASP at all -- just calls to the stored procedures.

This is difficult for me, because in order to understand the code I've inherited, I need to read it in two places -- the ASP page and the SQL stored procedure. Both are somewhat weakly organized and incompletely commented. I can do it, but it would be much easier for me if all that logic were in the ASP. That's the down side.

The up side, I understand, is that it's more efficient to use stored procedures. Many of them involve procedural logic in addition to SQL queries, and (I'm told) it's more efficient from a processing perspective to have it there.

I'm confident I wouldn't have any problems if the ASP code were THOROUGHLY commented, especially when it calls a stored procedure, to tell me exactly what was going into it and what was coming back out. In other words, stored procedures should be treated as included functions by a programmer and explicated somewhere outside the database to make the programmer's life easier.

Personally, I disagree with putting too much procedural logic in a stored procedure. It just doesn't seem to belong there. But if I'd started out in databases instead of in Perl, I'd probably have a different outlook.

Re:An outsider looking in

[Anonymous Coward]

Easy solution: Connect another monitor to your box.

Python's rexec disabled

[jesterzog]

One of my favorite thing about postgres is it's support of plpythonu (python stored procedures) ....

Do you have them working at the moment? Last time I checked, the whole python side of postgres's stored procedures was broken (at least with recent python releases) due to Python's rexec module being disabled for security concerns.

Looking at the current online Python documentation, it appears that the latest python release still has rexec disabled.

I guess you could hack around this as long as you trust the code that you're going to be running, but it seems a bit ugly. Fortunately I'm happy enough with plpgsql for the stored procedures that I write.

Re:Python's rexec disabled

[Sir_Real]

plpythonu for me built as of 7.4.2 . It's in the src/pl tree. Just make, install, createlang and you're done (more or less). (They just don't use rexec)

From the src/pl/plpython/TODO

* Develop a trusted variant of PL/Python. Now that RExec has been shown to be full of holes, this may take a while :-(

From the article you linked:
module contains the RExec class, which supports r_eval(), r_execfile(), r_exec(), and r_import() methods, which are restricted versions of the standard Python functions eval(), execfile() and the exec and import statements. Code executed in this restricted


plpythonu is python "untrusted" (or unrestricted... I can't remember) meaning it doesn't garauntee that functions written in it will only have access to modules and functions that are "deemed safe" And, the worst attack that can be launched using an exploit of this module is a DOS. (Not that that's ok)

If someone managed to pull off an attack like that, I don't see how it could possibly be anything but an inside job. I imagine that the exploit would have to be via an sql injection attack. Which, if someone can perform one of those, they would need to know the language the function was written in, and how to exploit this particular vulnerability.

For my little family online photo album, I'm not going to worry about it. I know many people who are running it in production fearlessly. (foolishly?)

PORTABILITY! PORTABILITY! PORTABILITY!

[Anonymous Coward]

Only use stored procedures if you want to be locked in to your database vendor for the life of your system.

And now you can thank God you didn't pick Sybase...

Re:PORTABILITY! PORTABILITY! PORTABILITY!

[brunorc]

This apply to every product you use, not only database. You use it == you tie yourself to it.

Then you can start to convinvce me that PHP will last at least 25 years more than PostgreSQL :P

Business Logic In Stored Procedures

[yintercept]

I agree with the point about including all business rules in a single location.

My tendency with data intensive applications is to put all of the business logic in Oracle stored procedures. I then have a variety of front end applications accessing the stored procedures. When the integrity of the database is the main concern of the application, I might write all of the business logic in a Java, PHP or C++ layer, hoping that no-one dinks with the data.

The big advantage of putting all of the business logic in the PL/SQL layer is that it helps make a very clean separation between the different tiers of the application.

Re:Business Logic In Stored Procedures

[smittyoneeach]

You have to consider security, usability, maintainability, stability, and maybe a few other -ities.
On one painful project with Oracle 8i on the backend, ASP/ADO on the server, and requiring IE on the client, I actually spanned the logic across all three tiers for the heavy reporting.
All of the criteria for a heavy report were gathered in JavaScript in hidden form fields, and the VBScript arranged it all through a clever use of the ROWNUM and MINUS features of Oracle to select just a small piece of a huge dataset.
Even though the code was well factored, and I put great tracing in all of the code, turned on with a simple DEBUG flag, I couldn't ever explain the idea well enough to the colleagues...
The upshot of this was that the otherwise impossible reporting became do-able on that project, because of judicious resource leveling across all tiers.

Re:Business Logic In Stored Procedures

[captain_craptacular]

When the integrity of the database is the main concern of the application, I might write all of the business logic in a Java, PHP or C++ layer, hoping that no-one dinks with the data.

Thats absolutely backwards. If the integrity of the database is the main concern you should use the database integrity contraints IN THE DATABASE. Implementing your logic in some random language and hopeing no-one "dinks" with the data is insanity. If you put your logic in the database (ie stored procedures) and made proper use of constraints, no-one COULD dink with the data.

On a related note...

[MSBob]

On a related note, would you recommend Emacs or Vi for general text and code editing?

Stored Procedures are critical

[Anonymous Coward]

We find that database procedures (the ingres variant) are critical as they allow us to expand the capabilities of the various systems we run.

For example out finance system stores it's data in Ingres, and with the ability to create database procedures and set them to trigger on events, we are able to identify activity that is of interest and automatically export data. This allows us to link our finance system (commercial off the shelf product with no source code) with our inhouse reporting systems (full source code access).

It gives a level of functionality that would otherwise require customsing a commercial product (read $$$).

It also allows us to build smart audit code into the database that allows us to in a single place implement audit rules irrespective of what applications (past, present, or future) access and use data in the database.

Also, where complec transactions require a large amount of network I/O, and a view is not suitable, it is better to invoke the database procedure from the front end and just read in the data as it comes back, rather than sequence SQL transactions back to back, processing data at either end of the wire and throttling activity to match the speed of the network connection.

Database procedures rock !

The right answer is in the middle

[iabervon]

Don't use stored procedures in your initial design. But also don't rule them out in your original design. Once you've implemented your application, remember that stored procedures are available as an optimization technique, if the portion which turns out to be slow is amenable.

There are trade-offs involved in using stored procedures; primarily that they are separate from your application code and therefore that two parts have to agree, and that it is more effort to modify stored procedures. They should therefore be used only if and when they provide a necessary benefit.

If your design rules out stored procedures, you'll probably have performance problems at some point. If you use stored procedures all the time, your DBA will never get a chance to optimize them (and the rest of the database), because you'll be constantly needing them tweaked.

It is actually worthwhile having stored procedures duplicate application logic when you use them, because then you can test their results against each other, and your application developers can follow what is going on; otherwise your developers will keep bugging your DBA and prevent her from getting anything done, you'll have to have someone less skillful write stuff while she visits relatives in China, etc. (not that I have any experience in this...)

To SP or not to SP...

[udbknight]

As usual the answer is probably: That depends... If it's a one off application against a database that was built by a programmer and it will only be used in the "one application one database" scenario, it probably doesn't matter too much.
If it's important data that will be accessed by multiple interfaces/applications it's probably better in the database (performance/maintainability/resuability/etc). I've seen very successful deployments where the only access to the data was through stored/procs. The developers were able to focus on the application and the user interface and didn't get bogged down in religious arguments about database design and data management (which they probably had less background/experience in anyway).

In the end the developers liked developing code that way. All they needed to know about the database was inputs and outputs.

Re:Stored Procedures often more harmful than helpf

[Anonymous Coward]

The main difference between SP and application servers is that SP in th DB grow vertically, that means you need more powerfull servers to run the logic, and the AS grow horizontally, which means that you can add more servers in parallel...

With an AS is much easier to mantain and escalate an application :)

I like dynamic SQL better

[tsotha]

I've implemented complex database logic in stored procedures and also using application code. I've found it to be easier in all cases, both from a development and a maintennence standpoint. My experience applies to Sybase and Oracle, though.

What I've found in most shops is people use stored procedures because they don't understand anything but the most basic aspects of SQL and it's the only way they can have nested loops without sacrificing too much performance. They don't understand how to use subqueries, outer joins, complex select logic, etc. So they break the problem down into multiple small peices or nested cursor loops. I simply don't buy the argument I've seen by other posters which goes "use stored procedures so you only return what you need". SQL will do that if you use it correctly.

Also, stored procedures tend to have limitations dynamic SQL doesn't have. For example, suppose you have a partitioned Oracle table that's using a data-based partition key. You can't create a partition from a compiled stored procedure without jumping through a lot of hoops (with associated ugly code).

As far as the speed goes - I don't know about other vendors, but Oracle caches the execution plan for dynamic statents, so if you send the same statement twice it'll run about as fast as a stored procedure. In any event in most business applications the query plan generation takes much less time than the actual query execution, so it doesn't matter that much unless you have a statement that gets executed thousands of times.

And you can get bit using stored procedures, too. If your statistics have changed drastically since you compiled your procedure you might not be using the correct query plan. Not a big deal for in-house applications, but do you expect your customers to recompile stored procedures?

Re:I like dynamic SQL better

[accvio]

On the other hand, using dynamic SQL will push parsing through the roof and, in the era of smart, cost-based optimizers, parsing a SQL statement is a very expensive operation which will impose significant burden on the system. In other words, an application like that would slow down and unnecessarily tax even big database servers. Developers have a problem with stored procedures because there are no fancy Gooey tools that one can point and click and have a reasonably quick way of developing an app. Also, debugging a trigger or a stored procedure can be a real nightmare. Oracle moved in the right direction with making Java a stored procedure language, because there are many tools for developing things in Java and no additional learning curve is necessary. Now, if Oracle would only make a perl interpreter, I would be a happy camper.

If used use them for simple tasks

[Portal1]

Depends but,
Well i would only use them for simple tasks that need atomisity. Like autocounters. All other application specific datacode should reside in the busines layer. Or in the mapping layer if possible. It is the same as not putting busines logic in the GUI. There are exceptions, but you should think otwice trice or four times before using stored porcedures.

SPs

[Anonymous Coward]

By having SPs, istead of hardcoded sql, a skilled dba might be able to optimize / update the query as teh database structure/organization changes. . .

Mostly good

[vadim_t]

You should use them where they're appropiate. For some things they're great, and in general I think using stored procedures and views for even simple things can bring great flexibility. For example it can be convenient to be able to fix bugs by changing stored procedures instead of distributing a new version of the program, and it helps writing good code too. Things look much nicer when the executable does the interface stuff and doesn't consist of lots of embedded SQL.

Also, they can bring huge performance benefits. For example, by turning the function that calculated an article's price (which depends on client, which may have VAT apply or not, may be discounted or not, etc) to a stored procedure.

The intent of the function was to get the final price. By retrieving such things as rows from the client and discount tables you're introducing unneeded latencies into your program which will now have to wait while this data gets delivered, used, and then almost certainly forgotten unless you implement some kind of cache.

By keeping all this stuff on the database server and only sending the result to the client you save bandwidth and time. My rewritten function executes more than 10 times faster than the original written in VB using ADO. By the way, if you really want to get the maximum performance possible in VB, first move the code to a stored procedure, then write a VB wrapper that caches the Command object. Turns out creating ADO objects is quite noticeably expensive.

I'd say the main considerations are:

Are you getting things from the database you will forget soon? If you will be retrieving data from the database to only use it temporarily in a complicated operation to forget it later, then it's probably better to avoid making it go through the network at all, so use a stored procedure.

Are you writing code with lots of SQL? If so, it'll look much prettier in a stored procedure than as a mess of string concatenation in whatever language you use.

Are you passing complicated data types? If whatever you're sending to the database is going to contain quotes, floating point numbers or dates, it might be safer to put the code in a stored procedure and call it with ADO or whatever you use so that it deals for you with issues like quotes in strings and date formats. Sure you can do that by hand, but it's very easy to try to send a number as a string by converting it with str(), forgetting that in some countries the decimal separator is a comma, and the SQL server won't like that.

Big, fixed queries probably can be safely moved to stored procedures and views too, just because huge joins that fill the whole page are often messy and distract from the main thing.

Now, one thing that should definitely go in the application is interface related code. I'd avoid excessive sanity checking in stored procedures, since that's often much better done in the interface, and some types of user permissions would belong there as well.

Also, big loops almost always should go in the program. Your server probably supports the use of cursors in stored procedures, but that has pretty high chances of freezing your application for a long amount of time with no progress report, and makes error handling more difficult.

Technology moves on

[almax]

To me this sounds like the argument for using assembly level code - faster, more control, etc. Things should be moving away from dependence on platforms. If you use stored procedures, then you can't take advantage of highly leveraged, portable and productive development platforms like Open for Business (www.ofbiz.org). Discussion about using SPs will seem pretty silly in the not-too-distant-future when all coding is done at higher levels of abstraction than it is now.

Stored procedures not complied, but separation def

[Grimace1975]

Procedural code and SQL code are two separate programming language processes. The first directs the computer from a singular point of view. For instance "do this, then that, then go here and check this. etc.". And the other deals with groups of items. such as "everyone wearing blue shirts go to room 103", or "we don't need these anymore".

As far as intermixing these code bases, your procedural business logic and data business logic should be split when it makes sense. The database is optimized for merging and managing sets of data, and procedural code is good for binding this to a functional form. The business logic should be split into these two zones and implemented appropriately. It would be inappropriate to return a set from a database then loop through that set searching for some name or value. And at the same time it would be unwise to return two sets and join them in your code. With experience it seams cleaner to maintain these two zones of code. This doesn't mean that you need to use stored procedures though.

As far as stored procedures, they are a convenient way of separating these two types of languages, another way is to in place the Sql code into your procedural code, but it seams advisable to centralize this type of code in one place for visibility, and manageability. If stored procedures are not available or undesirable, then using classes or function that are located in some central, or locatable place, is recommended.

As far as for speed, implementing the data and logic in the appropriate place will speed your application, but stored procedures will not in there own right speed anything up. At least in MsSql server, stored procedures are not precompiled. They exist as plain text, just like issued queries. They do however get their own query cache, separate for the issued query cache, which could be of a little assistance.

Anyways. I am over talking about this. Take it, as u wants it.
-- Grimace1975

Both Actually

[Anonymous Coward]

Lets take a Web Application for example.

Strategy 1 : Front end logic : Bad.

Writing database logic in the front of a web application is bad, very bad. You are decentralizing bussiness logic and creating a maintenance hell. Every page that access data must be updated or changed for any small change in the system. Such strategy is very common with PHP and classic ASP for example. You are not only rewriting code but also possibly creating a security and consistency nightmare. The same code on a pletorah of pages can lead to small variations that could create anomalies in the data, and therefore ruin you entire system. Most of us start this way, and works great for simple apps. You can copy and paste code you find on the web and develop apps quickly. You must FORCE yourself to leave this style behind. It was VERY hard for me, but I am so gratefull that I did.

Strategy 2 : Back end logic : Good

Stored procedures centralize the application logic in the database. This is better than the first strategy, but still has very large limitations. For example, you begin your application with Postgre SQL, your system works fine, but then your application grows, and you decide to use another suplementary database server, lets say MySQL. Now you must also create new code and systems to make the application work. Later, the company grows, you are getting a zillion hits, and you decide to buy MS-SQL, Oracle, DB2, etc. Another rewrite of the system ensues.

Strategy 3 : Object orientation and encapsulation: Best.

This old strategy is best implemented in systems like J2EE and ASP.NET, perhaps it works with php5 but I haven't looked in depth into it. With MVC implementations like JSF and Web Forms you can create the views. You then create classes, web services, beans, enterprise beans to encapsulate the bussiness logic and create "linker" classes to interact with the presentation layers. In the database, you create restraints and triggers to avoid bugs and sloppy programming to ruin the data. If the system updates databases, you just change the triggers and restraints. Perhaps change one modular piece of code or two.

It takes discipline to create a modular, n-tier system. But in the end, It is worth it.

My Humble Recomendations
1. Learn Object Orientation and practice until you think and breathe and eat in this manner. Pick a good system like J2EE or even ASP.NET if you don't care too much about portability. Yes, ASP.NET can use MySQL and Postgre gracefully.
2. Learn the specifics of different database system. Some may use LIMIT and others may use TOP. Design good relational models and create triggers and restraints to ensure that the data cannot get in without being perfect.
3. Read, read, read, and practice practice practice.
4. This is the good part. Profit ;-)

Just my 2 cents. I am talking from experience, because I have grown from my mistakes. In fact, MANY of them.

I work for Microsoft

[Anonymous Coward]

I work for Microsoft. I have to post anonymously so I don't get into trouble with management but basically as you all know we are developing a new file system based on treating the filesystem as a database. It's called Cairo. Maybe you've heard of it. Anyway, long story short, we are implementing stored procedures in our database filesystem to make it easier for developers. They will be able to do more with less commands by .netifying everything. See, many years ago viruses were very small like maybe a few hundred bytes. Over the years their size grew until finally in disgust we began including a VB-like macro language in all our products. That helped to make viruses smaller and easier to develop again (as well as required the antivirus companies to license our code to decode our proprietary file formats) but over time the effort required to write a worm or virus for our products has ballooned until it now exceeds what a 15 year old computer dork is willing to invest. But by including prewritten lines of evil code as stored procedures in Cairo we believe that our products will once again hit the sweet spot with our target audience. For example in only a few commands you will be able to write a totally new virus. So don't count us out yet. We are innovating our asses off. Just wait and see.

Stored procedures

[Mock]

For the web app I'm working on (1 million hits a day, processes 30 million records every 30 mins, which are fed in from another company via FTP... go figure), we use stored procedures to massage incoming data into a more useable form, usually involving denormalized cache tables and precalculations to lower the amount of data the server app has to request from the DB back end.

It works fine so long as all developers are aware of the nature of the data.

Thats like saying a Humvee is a toy

[MichaelPenne]

because its not a M1 Abrams. The real problem is that there are so many Abrams drivers out their trying to sell their clients on a Main Battle Tank of a DB when for most DB projects, something faster, lighter, and easier to drive is a better fit.

Re:Thats like saying a Humvee is a toy

[Anonymous Coward]

Except MySQL is less of a Humvee and more of a 3-wheeled car that only drives backwards. (But the alpha release adds an optional extra wheel and sorta allows you to drive sidewise.)

There's no problem with faster/lighter. There is a problem with bad practices memes, baseline features that are optional, and grossly incorrect SQL parsing.

They mostly help

[pvera]

I personally like stored procedures in SQL Server because the query is pre-planned, so when the procedure is called SQL Server does not have to waste precious time parsing and planning execution for the stored procedure. I have not touched a stored procedure in Oracle for over 3 years so I could not honestly tell if Oracle does it the same way.

It is also nice to get all these SQL queries out of the web code, and it gives its own little bit of (job) security by obscurity, but when it comes to it the main reason I do the stored procedure is because I want the damn thing to run faster.

What I don't like is when people write stored procedures and then they build a dynamic sql query inside, which is retarded since it will have to parse and pre-plan the damn query every time the stored procedure is running!

Good, like most things if they are used properly.

[IpSo_]

It's a slippery slope...

I used to work for a large web hosting company that wrote there billing software frontend in Delphi, and kept all the business logic in stored procedures on MS-SQL. It got so bad, they even ended up having a stored procedure that generated HTML/Text invoices for customers! Ever tried doing text layout in a stored procedure? It was absolutely nuts, but once they had started putting all the business logic (and much more) in stored procedures, its hard to stop without "splitting the code-base".

They were also scared to upgrade from MS-SQL 6.0 to anything newer for fear of it breaking their stored procedures. They tried at least once and failed miserably. As far as I know, to this day, they are still running MS-SQL 6.0.

This whole issue basically put a strangle hold on the company, it took forever to "innovate" and they eventually got bought out. The new parent company has spent over a year trying to migrate away from this "stored procedure" mess.

I think stored procedures are good, but only in very specific circumstances. If you design and code your application properly, there is usually very little need to start down the slippery slope of stored procedures.

Stored procedures are for data integrity...

[aralin]

We use stored procedures for data integrity. You get only SELECT right on any table at best. If you want to do any DELETE/UPDATE/INSERT you have to call a stored procedure that ensures the data integrity. Basically we use the PL/SQL to define operations that are permited on that particular data model. It has several advantages.

1. If the database schema changes, you don't need to rewrite the application
2. Your data are secure and the model is always in stable state
3. The operations on the data model are well defined and well optimized. Nobody is reinventing a wheel.

Of course, any business logic has to go to your applications.

Re:Stored procedures are for data integrity...

[Anonymous Coward]

You use stored procs for Integrity? Why don't you just use, you know, the _DBMS_ to do that? After all, that IS what it's made for.

Re:Stored Procedures often more harmful than helpf

[Anonymous Coward]

I am in full agreement,

SP's are not all they are cracked up to be. One of the major trade offs is that you lose flexibility. for example in order to offer a an advanced search page for say a BBS you might not know ahead of time all the possible variations of your select statement therefore you have to have the ability to construct one at runtime. A good example of such a search tool that would not be possible using SP's is:
http://www.linuxquestions.org/questions/searc h.php

Another reason that SP's are not cool is that they are often a cause of the rift between DBA's (who don't allow app developers to touch the DB) and application developers.

I suppose programming to SP's is like programming to an interface, but for some reason it just feels like they get in the way more often then not.

Middleware

[kupci]

They're also handy in the instance that you may have multiple front-ends written in multiple languages accessing the same database in many cases. Making a change to the way data is returned is far easier to do in one stored procedure than in X number of front-ends.

Sounds like you need some middleware. There are database servers, and there are integration servers, for example WebMethods. They might be overkill in your situation, or might be a big improvement.

You already answered your own question

[nurb432]

Using the arguements for their use in your post, what else do you need to know? They are the best way to go, unless you are doing something that wont matter, or want to do things really sloppy..

But if you are making something that people will actually use, then using stored procedures is the only way to go.

does/may /. beneffit from stored procedures?

[urbieta]

I know /. uses a db, so could this beneffit it? Ive seen a few error messages latelly, so....

SQL 2000 and table variables....

[PoJoCatDog]

In SQL 2000 you also can use table variables in stored procedures, functions, etc. instead of temp tables to reduce io to the tempdb. This allows for easier manipulation of large data sets with faster output.

Man - why the question? THEY ARE 100% NEEDED !!!!!

[chrisdrop]

I can say from real experience in the enterprise (Implemened the 30th and 60th busiest sites on the net today - MediaMetrix). These COULD NOT HAVE BEEN DONE without stored procs. The level of performance increase is completely non-trivial. YOU WILL GET HUGE PERFORMANCE GAINS - especially when you have a complex AND PROPERLY RELATIONAL db. You CAN NOT use string SQL at the enterprise level like this. You CAN NOT maintain string SQL all over. When you do things like 1 click place an order, register a user, enter some survey questions etc - do you think you can maintain 5000 concurrent users while each are making 5 string based SQL trips????

The level of business logic being embeded in procs can be maintained so you KEEP YOUR REALL OO MODEL. It is KEY from an app development perspective to maintain a good OO model. Everyone acts like if you put ANYTHING beyond 5 lines of SQL in a proc - you are commiting an evil act of embedding 'business logic' in a proc - God forbid. The REAL truth is that you CAN put LOTS AND LOTS of SQL in a proc - have it work REALLY REALLY fast and STILL have a GREAT OO model.

The caveats; you need to have people that write good procs and have a properly structured DB.

Data Model Abstraction

[FritzAtlanta]

I am have been an Oracle DBA/Developer for more than 10 years. I also got into Java and more hardcore OO programming around 1996ish. In my opinion, from a design and implementation standpoint, the use of stored procedures is extremeley useful to provide an abstraction layer for data access. This is valuable because the stored proc can handle the renormalization of tabular data to best fit whatever object model is used in the application. Simplifies the application and protects it from necessary database changes (normalization/denormalization) in terms of how the object is implemented into tables. The great benefit of this is that it lets dbas and other database developers implement the data model independently of the application! This forces the relational data model design to resources that can program it most efficiently. Obviously there will be a small number of cases where this is not a viable approach to follow, but in general, it is a huge time saver, risk reducer, and productivity enhancer.

Sheesh, why not just resurrect vi v. emacs

[theAtomicFireball]

Come on, this is a troll if I've ever seen one. A lot of people have strong opinions about this, and tend to lump those who disagree with them in some category of people (e.g. "small website hackers").

I deal with large applications that access large volumes of data in the real world, and I'm not a big fan of stored procedures. But, I use them in a limited fashion.

My personal opinion (and nothing more) is that you start with your business logic (exluding data validity constraints) in your controller objects of your application. Once you're into system and load testing, you look for places where the benefits of a stored procedure really buy you something that you need to buy. "Centralized code" is not a benefit of stored procedures: just the opposite, unless you manage to do your entire application in stored procedures. I also disagree that security is a benefit of a stored procedure. Unless you're dealing with client/server applications (and poorly designed ones at that), there's relatively little difference in the risk of having logic in the application versus in the database, and I can think of a few known exploits that could give somebody a look at your application logic.

The true benefit of a stored procedure is that it is easier to improve performance. Notice how I phrased it, because it's important. Many people will state flat out that stored procedures give you better performance. Not true - on most databases, you can implement optimizations to give you the same benefits (e.g. declaring cursors, using prepared statements and bind variables, etc.). But, databases know themselves pretty well, and one of the quickest and easiest optimizations you can do is to move poor performing logic into the database - achieving the same thing through SQL or an interface can be time-consuming, and you have to have good developers who really know their stuff.

Even at that, I try to tune the logic first before moving it into the database. A lot of "first try" SQL is inefficient, so spending a little time on your more complex querys often allows avoiding having to separate pieces of business logic. Oh, and if you do use a SP, put a big obnoxious comment where the logic used to be with the name of the SP, how it's used, and when the logic was put there. Or don't... it's your system (and career).

There are a few exceptions to my approach of avoiding stored procedures. Bulk moves between databases is one I can think of. But as a general rule, I avoid them during initial design, and use them as an optimization when I have a reason to optimize.

Your mileage may vary. I know many people who have 180 different opinion on this, who are just as successful with their systems. It is possible to make a well-designed, easily maintained system using SPs, but it just doesn't really fit the way I work.

it's performance, stupid

[SethJohnson]

If you can avoid having the DB recompile the query everytime it's run, then that's good for business, period. Data Dictionary, blah..blah.. yeah, repetitious queries get stored there and aren't recompiled. But in a web application, there aren't a lot of 'repetitious queries.' Think about something like Ebay, or travelocity. Yeah, the stuff on the front page is pretty static. It's probably even cached at the webserver level. But as soon as visitors start entering random crap into the search field, repetition goes out the window.

So for web applications, put as much as you can into your stored procedures.

use store procedures with restraint

[justanote]

Our company standards require we use store procedures. The procedures provide a window to the database and are meant to hide the complexity from the programmer. They don't even get to see the tables. They provide better performance, and you have people who understand the data writing the queries. The database is meant to hold data, not business rules. Keep them in your code. Let the database maintain data integrity. If your concern is to be able to use a different database down the road, then use a data abstraction layer to hold all your code that accesses the database. Use a factory to provide you with the data abstraction object you need. You could have an Oracle and MS SQL implementation of your data abstraction layer. Your web programmer never has to care which is being used. A simple change in a property file can tell the factory which to provide. This approach requires some division of labor, and depends on having the resources available. It also takes more time to develop it right. However, if done right it can save a lot of headaches in the future.

keeping the logic in the code vs. performance

[flacco]

my instinct is to put the logic in one place, and since i need that logic in the code, my preference is to centralize it there instead of duplicating it in the db. this extends even to operations that databases are traditionally used for, like aggregating data.

i'm quite unsure of myself about this, though. at the moment i'm working on a budgeting application, and both performance and productivity are becoming an issue.

example: to aggregate budgets over a time period, i retrieve the budget objects for each budget period individually, and accumulate the aggregate data in code. this takes quite a bit of extra coding, and execution time is quite slow, however; doing aggregation queries in the db would certainly give better performance, and it would be a lot easier to slap together some queries instead of writing all that code.

so, the way i look at it, it comes down to a question of science vs. engineering. the scientific impulse is to adhere to the theories of keeping logic in one place, and respecting the objects' ownership of their data. the engineering impulse is to use the technique which is faster and easier to implement.

i guess at heart i'm more of a scientist than an engineer.

i'd love to hear others' takes on this question, btw.

Re:keeping the logic in the code vs. performance

[Anonymous Coward]

example: to aggregate budgets over a time period, i retrieve the budget objects for each budget period individually, and accumulate the aggregate data in code. this takes quite a bit of extra coding, and execution time is quite slow, however; doing aggregation queries in the db would certainly give better performance, and it would be a lot easier to slap together some queries instead of writing all that code.

If you're talking about aggregating historical data for reporting or other uses, making summary tables is a clean way of doing it. Your other option is to use OLAP, which uses bitmap indexes to build a cube. How to apply summary tables and triggers for aggregating data depends on your requirements. There's no simple answer and you'll have to stress test it to see if it really performs better.

Re:keeping the logic in the code vs. performance

[jargonCCNA]

That is not an engineer's impulse. Not after Tacoma Narrows and the Montreal bridge incident. An engineer's impulse is to do it right the first fucking time and not waste anybody's money.

On behalf of my father, sister and everyone who can append P.Eng. to their name, I'm insulted.

Re:Stored Procedures often more harmful than helpf

[Anonymous Coward]

So, you refuse to source control your SQL because you lost the fight about which source control program to use? People like you make IT suck.

Re:Stored Procedures often more harmful than helpf

[Anonymous Coward]

"Source control is rather easy to use with stored procedures. "

Real world experience tends to be that IT depts with functional CM, break down along a division between business software developers and database admins.

Every line of your application code might be meticulously maintained in CVS, while your DBAs don't give a damn about that, don't have to give a damn, and aren't accountable to your IT management.

Re:Stored Procedures often more harmful than helpf

[fishbowl]

"but what's stopping a developer from writing his own query rather than using the optimized one you've already provided for him?"

You can do things in a SP that the app connection lacks credentials to do. So what's stopping the developer? How about hard, drop-dead, grant-level security?

One more advantage of sprocs

[pherthyl]

One more thing that I haven't seen mentioned in the comments yet.
Stored procedures allow easy updates, especially if your application is compiled.
You found a bug in how data is accessed/returned from the DB? No need to recompile anything, just give the client a script to run that patches the affected sprocs. Easy, non-disruptive, and fast.

This was a huge timesaver for us and our clients at my last job.

architecting

[/dev/trash]

I'm just trying to figure out what: architecting is.

Re:architecting

[popeyethesailor]

Drawing block diagrams on a whiteboard with lots of illegible acronyms and connectors.

Not quite.... learn SQL

[NineNine]

SELECT COUNT(*) FROM a WHERE a.key NOT IN (SELECT key FROM b)

Actually, SELECT COUNT(1) FROM a WHERE a.key NOT IN (SELECT key FROM b) is much faster.

Re:Not quite.... learn SQL

[innosent]

No, actually, according to the SQL-92 standard, SELECT COUNT(1) should return the number of rows for which the value 1 is not null, which is equivalent to COUNT(*), the number of rows. The database does the exact same thing, possibly even more for a SELECT COUNT(1) if always-true assumptions were not made equivalent to COUNT(*) by the DBMS vendor. Most DBs would probably be able to further reduce execution time of the query, but how to go about doing that depends largely on which DB you use, how the tables are organized, etc... A "SELECT TOP 1 a.key FROM a WHERE a.key NOT IN (SELECT key FROM b)" should be faster, but it depends on the type of index and DB implementation, as counting a hash join of indices may be faster than actually retrieving the data for the first match.

I used the simpler SQL statement because I wanted to make a point, not give someone an optimized query for some imaginary database table.

Re:Not quite.... learn SQL

[innosent]

Just as a follow up, google things before you write them: http://asktom.oracle.com/pls/ask/f?p=4950:8:::::F4 950_P8_DISPLAYID:1156159920245
http://www.oracledba.co.uk/tips/count_speed.htm
and a DB2 example illustrating my point about it possibly taking longer: http://www-106.ibm.com/developerworks/forums/dw_th read.jsp?forum=292&thread=31551&cat=19

Re:Not quite.... learn SQL

[Anonymous Coward]

Your database is broken. Please report a bug to your vendor and/or the open source project. This one should be pretty easy to fix.

(though I think you're just wrong... postgresql and mysql have no such problems, and I'm sure oracle and DB2 have at least as good optimizers so they'll be fine too.)

503's for me!

[simetra]

I've been getting 503 errors on and off for maybe a week.

For whatever that's worth

Like all tools...

[kabdib]

Like all tools, stored procedures can be used to good effect, or they can be very, very evil. In my experience, projects with people who "knee-jerk" to stored procedures often wind up as train-wrecks, with the engineers begging for mercy-killing. Sure, you can write crap in any language. But it seems that some languages attract the crap writers more than others.

On one project I converted a stored procedure that was an undebuggable mass of 300 lines that took 18 hours to run, and recoded it in Java, using in-memory operations; it ran in a few minutes. Tried to explain that to the developer who wrote it, got a stoney face and some shrugs. Guess who doesn't work there any more?

Stop MySQL from implement SP

[w3c]

Calling SQL from within scripting language,
we don't need SP,.. unless we need workaround,
because of DBMS has missing
basic "must be" features.

Consider example: lack of identity in oracle
(auto-increment ID) required to write trigger
and SP to emulate it.
Another example: lack of timestamp data type
required to write another trigger.
(BTW, even with triggers, it's impossible
to implement in oracle,
but that's another story...)
Third example: lack of *atomic* multi-table update
may require transaction defence line
and SP code around.

On the other hands, do we ever pay attention
that all above is not required when we
operate on regular filesystem with files and just
rely on atomicy and consistency of well-defined
filesystem's operations.

Same was nearly true until MySQL didn't forget
about web-centric usage base
where it was growing from.
Then, instead focusing on benefits of
copy-on-write snapshotting,
LIMIT and caching improvements,
they migrated to "market demands" with cursors,
sub-selects, SP and other attributes of
"respectable DBMS"-es.

As of my personal opinion,
if someone *really* need stored procedures,
person need to take a look onto 4GL,
(for example) Progress DBMS,
and realize there is no any reasons
to return back to any SQL.

As long as SQL is claimed to be
*non*-procedural language (at least by idea;),
adding such workarounds as [stored] *procedures*,
must be considered honestly, isn't it?
If so, why create any standards for
mis-conception at all? SQL-99, SQL-2003,...

Well, well,.. reason is always there: the market.
People know SP-s, people use them,
create more and more workarounds,..
and you have to deal with them sometimes.

Of course, create or not your own workarounds, --
always still been your personal choice.

Compiled is misleading

[Anonymous Coward]

Saying stored procedures are compiled and straight SQL is not is misleading. Oracle and I believe a number of other databases have the equivalent of a SQL caching layer. In Oracle it is called the shared pool area. Once the database sees a SQL statement they save it off so that next time they don't have to generate an explain plain. They in effect "compile" a SQL statement the first time they see it. With these types of database all one has to do is make sure that the SQL uses(Oracle terms bind variables) variables in the part of the SQL that may change. This makes the SQL reusable by the database. If you fail to take advantage of this very nice feature then yes you can see some big preformance differences. It is amazing how bady some programmers can understand whats happening and right extremely inefficent code in both Stored Procedures and in off database languages.

I have seen an OE system that was written in straight stored procedures(PL\SQL) that did extremely poorly and then large parts rewritten in a Java layer and it did much better. However its not really a fair test because the original stored procedure layer was ultra badly done with lots of needless and badly done abstraction that only slowed everything down.

Good Luck.

Re:Compiled is misleading

[popeyethesailor]

PL/SQL stored procedures are indeed compiled, they are compiled to byte-code. You are mixing up things here, there's a PL/SQL engine and a SQL engine within the DB.

The PL/SQL engine just executes the stored byte-code. The SQL engine parses & executes SQL statements. Two different things. The Shared pool (to be more specific the Library cache)stores both SQL & PLSQL code.

And from 9i, you can even compile a SP to Native code.

Cheers

Don't look for a universal answer

[Anonymous Coward]

It shouldn't have to be a generic good or bad answer to the question. It depends on the circumstances and the context of the problem.

These n-tier architecture argument often overlooked on it's applicability. If you're a vendor selling application, it is wise to make it database independent and n-tiered. If you're a in-house development and has chosen your strategic database and the database is centrally used across multiple applications, SP might make more sense.

Performance importance is also contextual. I work for a bank and the volumns are counted in millions and there is limited window for batch processing. For an smaller online apps it is probably not significant.

If your database is extremely sensitive, there is no substitution on SP. Any security logic implemented outside the database can always be bypassed.

Flat Address Space

[HairyTiger]

Do all DBMSs have a flat address space for stored procs? The guy above who had 3000 procs in the database - how can you manage such a load without packages or libraries or directories to organise them?

To address the issues of vendor tie-in, what if the stored procedures are written in a common language, such as Java? I know that Sybase and Oracle both allow Java stored procedures instead of Transact SQL and PL-SQL. The J-Procs are basically JDBC code, and would be much more likely to be vendor-neutral. Is anyone in the world using Java stored procedures in these two DBMSs?

Good for combining data from different sources

[simetra]

One thing I maintain is a database that combines and stores a lot of data from a few different places. This data needs to be combined and have various functions performed on it daily, and stored in a way that the client apps can access it in fairly organized, fast manner, to run reports, add additional data, etc. Stored procedures, triggers, and (ugh) DTS in Sql Server do this pretty well. It would be a lot less efficient to just store the data in its raw form in several tables and force the clients to compile the data as needed. This is especially true as some of the clients have pretty slow connections to our LAN (I believe 56k frame relay).

The answer to this question really varies, depending on what your resources and needs are. There are many variables. It shouldn't be looked at as a simple yes/no question.

How about this answer

[Tangurena]

How about you put the following in the log in prompt for the web pages that access that DB...

User: bob
Password: x' or 2>1 --

and see what breaks. If you compose SQL on the web page script, the bad guy is in.

Welcome to the cruel world of sql injection. Just imagine if the abuser types in the following for the password:
x' or 2>1; truncate table customers; drop table users; --

All it takes is one page somewhere to forget to trap/escape out bad stuff, and you are hosed.

Re:How about this answer

[mgkimsal2]

Wow! You bother to write distinct data cleansing input filters for every single 'page' of a web app? Many (most?) people doing that sort of repetitive work have determined that setting up the application to do all that basic protection processing before control is passed to the logic is a more secure way of programming. Feel free to keep putting your input filtering code on all of your 'pages', but don't lump everyone else in with your vision of web app development.

Re:How about this answer

[Aeiri]

That is, unless you secure the site PROPERLY.

If you use a security module loaded into each page, you can have it scan each get variable, each post variable, each cookie variable, etc, etc, and clean up everything. It isn't as hard as it sounds.

Sure you could say you can forget about loading the security module, but include it in the header and that doesn't matter either.

Necessary

[vandan]

We make extensive use of SPs, especially in the billing section of the database. And no, not for adding / editing data - just for viewing it.

If we didn't have views / stored procedures, then accessing billing information would be:

a) Very difficult coding - you'd have queries all over the place and you'd have little discrepancies pop up that some manager would have my head roll over. Centralised queries is a very good thing indeed.

b) Very slow. I don't care that SPs are pre-compiled. Being able to use nested views / SPs is where we get most of our speed from.

Triggers I'm not so fond of. They are handy in some small cases, but as I'm sure everyone else will be pointing out, they can make debugging interesting.

Varies

[dtfinch]

Just your typical answer:

For most applications, plain SQL works fine. Stored procedures take more time to write, which is undesirable if you're aiming for rapid development. And they aren't that portable.

I say use them if you'll see a performance benefit. As my pages have always been fast enough, I haven't used them outside of school.

The greatest bottlenecks for many web developers seem to be in their code to read the resulting recordset, following unoptimized samples like seen in most tutorials, or forgetting to index certain fields, using multiple queries where a join would be more appropriate, using the wrong provider in their connection string, excessively concatenating large strings when a string buffer could do it 1000x faster, not knowing to turn on reponse buffering, etc. The kinds of things that lead to multimillion dollar it purchases.

When something isn't fast enough, and you've determined that the database is infact your primary bottleneck, and stored procedures have the potential to give a visible improvement, then and almost only then will it be a great idea to use them.

If you're working on a very high traffic system where RAD techniques no longer apply, and performance is your highest priority, then of course you'll do whatever you can, including planning from the beginning to use stored procedures everywhere they'll help. But at this point I would be considering alternatives to a sql driven relational database.

But what do I know, I'm only 22.

Not at all

[bugninja]

I use MySQL for everything. No need - even for smart apps.

The gift of abstraction...

[$ASANY]

Database structure gets changed all the time in a real-world environment. Unless you want all your app developers to be reacting to every minor database change, you create stored procedures as an abstraction layer in between the database and the middleware or applications that depend on it. This allows your middleware and front-end developers to code to their strength, and your DBA's to work on their strength.

If you allow developers to access database structures directly, every time you change your database structure they have to change every program that depends on that structure. Stored procedures allow you to provide logical interfaces to (generally speaking) physical data structures that can withstand maintenance
efforts.

Try letting your developers deal with transactional and logical units of work, particularly in the OO web applications environment, and you'll find your data severely screwed up. The J2EE methods of dealing with data are not at all the way that relational databases expect interaction. Provide a compatibility layer of stored procedures, and your data won't get hosed and your application developers will appreciate the convenience of not having to deal with transactional logic.

It's a great way for DBA's to enforce logical constraints that affect data management. It also allows them to employ the tweaks and tricks that modern RDBMS's provide. The end result is that those who have the strongest ability to execute well in a particular portion of a project can assume the authority to do what they know best. That can't be a bad thing.

sp's are the way to go. .

[Anonymous Coward]

This is the classic DBA vs. programmer argument. Ideally developers can write app code and database code. Our shop made the move a couple of years ago. All, data access is througt sp's, and when it makes sense, app logic as well.

Our applications are *very* fast, *very* easy to maintain, *very* easy to debug, *very* easy to test, *very* easy to optimize, *very* easy to deploy, *very* secure and if need be, we can switch app servers with minimal effort.

Need to change/extend/optimize/re-purpose the data model? No problem. The sp's abstract the datbase structure completely.

A couple of schema changes can wreak havoc on application code. To be fair, they can wreak havoc on database code as well; but it is much easier to manage if your data access is through a centralized api.

We also are careful to divide access to the database by functional roles early on in the development process. This allows us to scale an application with hardware very quickly.

The argument that you are locked into a db vendor if you implement an application using sp's is silly. You are locked into the db vendor anyway, get over it. The fact is you are just as locked in at the app server level. End of this argument please!

If you are a developer and don't know how to write t-sql crack the books. You can't build an enterprise application without sp's, triggers, and udf's. At least you shouldn't in my opinion. Stop whining about sp's and pick up a book! You will learn a lot, create much more robust applications and up your job security a couple of magnitudes of order.

Scripting languages were never meant to deal with large amounts of data or complex data models. Databases are in fact designed to do this exactly. Database connectivity was an afterthougt in almost all web programming languages.

Just my 2 cents. I've been building applications for 12 years. The more I build, the more I rely on this architecture.

vendor lock in

[Anonymous Coward]

all this talk of vendor lock in is bogous. if you write any kind of serious application, i'd defy you to move databases without a code rewrite.
bottom line is you upgrade/migrate your code base far more then you move db servers. that combined with the benfits of SP's makes them a the best choice.

I'll Never Use Stored Procs Again

[Anonymous Coward]

Finished a HUGE project last year that involved, oh, I don't know, about 2,500 Oracle PL/SQL stored procs that were called by a Java middle-tier. About 3 months ago I was told by the PHB that the app now needs to also support MS SQLServer. I'm in the middle of slogging through all that pl/sql converting it to platform-neutral Java database calls. Shoulda just passed on the stored procs from the get-go.

Oh yeah, and apparently there's some evidence to suggest that calling Oracle stored procs from Java is actually much slower than using Java prepared statements and the like. Something about all the overhead required for Java to set up the call or something. Here's a link:

http://www.oreilly.com/catalog/jorajdbc/chapter/ ch 19.html

*sigh*

Architect

[Anonymous Coward]

Architect not verb. pls use actual words. k? thx.

Unstable Environments

[Anonymous Coward]

Not sure whether to get involved in the flaming but there have been some good points for using Stored Procedures; however the conscious choice not to use stored procedures has saved my butt at least once so let me share. Application developed with Perl/PHP4 and Postgres on a Linux desktop box because the NT Admin didn't want anyone playing with his SQL Server baby. Management insisted that deployment had to use the SQL Server hence I developed without using stored procedures. Well four months after deployment the NT machine was hacked SQL Server compromised and the decision was made to switch to Oracle. (Hey I'm just a code monkey not the decision maker) So because I had not used any stored procedures my application was working perfectly after the move to Oracle while other projects were still rewriting their apps. The moral: unstable environments are not the place for stored procedures.

Sometimes Indispensible

[Anonymous Coward]

IMNSHO...

Take a scenario where you have to accept data from another system. The data may be structured in such a way that a series of interdependent lookups need to be performed to gather surrounding information that you need to store with the data or to understand how the data relates to already existing data (insert vs. update)...

In one case we encountered (which is one of many) this required a potential of 13 separate database accesses with a minimum of 8.

No problem. I wrap all this up in a proc and make one network transfer and database call from the system accepting the data to the database.

13 network round trips with 13 un-compiled SQL accesses vs. 1 round trip and pre-planned and pre-compiled SQL calls.

Now add in the fact that this particular operation happens half a million times a day and you start to see that in some cases stored procedures can be indispensible in certain cases.

Also, I have found that the security benefits as well as data integrity benefits more than outweigh the small "development time" penalty that they incurr.

One last point: if your writing all your sql in stored procedures you most likely have a dba minding the code and he/she can be much more proactive in maintaining the overall health of your data "ecosystem".

Of course none of this applies if your doing some dinky data based web app, but for heavy data lifting I have years of experience that says I'm right.

Java stored procedures: best of both worlds

[mmacdona86]

Lots of the problems people have had with stored procedures are because stored procedure languages have sucked. Now RDBMS' are giving you the option of writing stored procedures in real, OO languages that are familiar to most of your developers, Java and C# soon if you've drunk the .Net Kool-Aid. If you decide that portability is an issue (most people don't) and you code carefully, you don't have to decide where the code executes (app server or DBMS) until run-time.

How to be a "real" DBA

[0x0d0a]

What offended me was his strong implication that the two -- PHP/MySQL programming and real DB development -- are mutually exclusive. It's an attitude that I see a lot on Slashdot, and it pisses me off.

Just:

*) Ensure that your company blows inordinate amounts of money on software with expensive support to cover your ass. It doesn't matter whether it works better, but you should be able to point at someone else to blame if something goes wrong. This also means that you don't actually have to understand the product, and can trust your DBMS vendor's salesmen for all your information, which will chew it up into bite-size pieces and spoon-feed it to you.

*) Act very self-important, talk down your nose to anyone that *doesn't* waste as much money as you, and treat your money-wasting as a mark of pride ("I have a $N million dollar budget this year!")

*) Place "real" before every class of software you work with, and vaguely insinuate that there are technical (especially reliability) failings in any cheaper product than the one you use. It worked marvelously for "real UNIX admins" for years.

*) Mention "Fortune X" frequently. For example "Fortune 500 DBAs with $5M budgets like me use *real* database software -- *we* can't afford downtime".

*) Avoid, if at all possible, knowing anything about software or computers other than database interfaces and possibly a scripting language or two. It's also good to choose a single OS, learn only that, and then slip occasional comments about how other OSes are unreliable.

*) Slowly begin slipping corporate acronyms into your speech. Use "ROI" whenever it's appropriate, and even when it isn't, to make it sound like other people are just slipshod and guessing numbers. "Enterprise class" is also good, since it implies, with only very vague factual requirements, that you work at a larger and more stodgy company that the person that you're trying to put down.

You'll be well on your way to doing "real" database work in no time. :-)

Re:How to be a "real" DBA

[Anonymous Coward]

brilliant. so sad, but so true. Funniest thing is that Oracle trains people to be like that.

Re:Stored Procedures often more harmful than helpf

[Rary]

I have responses to your other points, but it's point #2 that I hear the most and want to reply to.

SPs invite use of vendor-specific features, and therefore lock-in and loss of portability.

First of all, is this really an issue? I mean, I'm sure that there are companies out there that actually switch their DBMS at some point, but how often does this occur? Most of the companies I've worked for made a DBMS decision ages ago and stuck with it. It seems to me that a lot of people talk about the idea of DBMS portability as if it's just one of those things you plan for regardless of whether or not you actually have a reason to plan for it. I think it's an issue that becomes a reality only in a tiny fraction of situations.

Also, as another poster pointed out, why would you spend $x00,000 on DBMS product X, and not actually use the features it gives you? The use of vendor-specific features should be, to some extent, the very reason you went with that vendor in the first place.

Just some thoughts off the top of my head. For the record, I'm somewhat on the fence on this issue. I think SPs are essential in a client/server environment (yes, there are still tons of them out there, even new projects), but I'm still debating their use in an n-tiered environment.

Scalability

[OldButNotWise]

Sorry if this has already been noted, but...

While putting logic in the db can result in large performance improvements (by reducing round-trips between client and server), it can also reduce performance by forcing all clients to contend for a single shared resource (the db).

In a typical client-server situation, you can add compute power on the client end pretty easily. Adding additional horsepower to the db engine is usually a lot more expensive.

Like people used to say back in the old days, when workstations were new: the nice thing about the network is that it doesn't get faster at night.

Re:When did Slashdot switch to IIS?

[Anonymous Coward]

Just some gossip... but I hear a CmdrNaan might be replacing CmdrTaco soon.

Re:Why stored procedures are bad. (10)

[attonitus]

10) They make team development hard. When I'm writing java or C++, or whatever, I can usually build and run whatever application I'm writing locally, so I can develop new functionality without affecting other people in the team. Most projects that I've worked on use 1 or 2 development databases. This means that if you're developping stored procs, you have the following choices:

Risk breaking something that someone else is relying on whilst you debug your code.

Create a replica db to develop your stored procs in. But this can be a pain if your application requires sensible data to be in the db.

Piss around creating duplicately named stored procs in the main dev DB.
None of which are particularly great.

Why not use an Object Database like Versant?

[SupplyChainCoder]

If you write an application that needs persistance for it's data model then an Object Database is the way to go. You get to keep all logic in a central domain model, great performance and all the features you need like object level locking, transactions etc.

Eminantly sensible!

[sg_oneill]

Fantastic thinking.

You are quite correct there, and that also comes down to the competancies of the developers and software.

Oracle is fantastic data storage and retrieval, and is it great 'business logic'?. Probably not.

Likewise , A DBA is not necessary great at such logic, but a magician at getting and storing the data required for such logics, keeping it safe and as bookishly-correct atomic and/or normalised as wisely possible.

And theres some fantastic coders out there who's knowledge of DB's probably just extends to joins on selects and inserts.

So the DBA just presents a unified interface to the coder via stored procedures, and the coder does his job. The coder doesnt even have to know how the data is stored and retrieved.

OO in action!

Re:Why stored procedures are bad. (10)

[jkarlin]

wow, maybe you should take 5 minutes before you start hacking your code together to talk about your database and the things you need it to do. I know developers like you that hear half the problem and race to their computer to start coding. I'd rather find a whiteboard, draw some pretty pictures, and figure out what the hell we're trying to do first. And you know what, except for the simplist projects (which wouldn't have 2 people anyway), I'd finish before you and work a hell of a lot less.

One Place SPs are Useful...

[endofoctober]

...is report building/distribution. Build your report SPs, design reports around the SPs with Crystal or something like it, then let users call up real-time reports from a mirrored db with them. The queries are optimized and can be built with indexed parameters for user interaction. If done well, everything runs fairly quickly and, more importantly, no one in Sales can TOUCH data with ad hoc reporting tools. Bad touch, Marketing Drone!

Re:Good, like most things if they are used properl

[leabre]

I've worked for 3 companies that exclusively uses stored procedures and haven't seen any problems. In fact, the two places I worked that didn't use them and instead used embedded SQL were the ones that were difficult to maintain, especially in the places where the same embedded SQL scripts where in multiple places in the application. One other place I worked for used a special Data Access Layer that had a "Select" method and a "Update", "Insert", and "Delete" method that took parameters and then just dynamically generated the SQL script to execute. It worked well and was low maintenance source code, but a very fragile DAL that was hardly touched for fear of breaking it.

In all, properly used, SP's aren't bad and are very useful. Especially for security and data integrity. I know some prefer to use SQL for multiple database platforms without having to rewrite the SP's on each platform, but in reality, I've never seen a place (not saying they don't exists) that wrote such generic SQL that it would without database-specific changes, anyway.

Thanks,
Leabre

Think Different

[FranTaylor]

Why is the application environment different from the database environment?

The database is really just persistent program state, right? So why do you have a separate application, etc. for the database?

When you start putting constraints, triggers, etc. on tables, you are already sliding down that slippery slope. Whenever you have a slippery slope, it's a clear sign that a new way of thinking is neeeded.

Why not write the whole application in the database? The stock answer is that the embedded language is not insert adjective here enough to run my application. Why does this have to be the case? Why can't a powerful language have its own database? Why do you have to retrieve variables from the database? Why can't the variables just be in the database in the first place?

Imagine perl tied hashes, but you can also do SQL queries on the same data if you like, no performance penalty. Why not?

If the inventors of motorized vehicles had gotten stuck in this same brain-rut, we would be driving horse-drawn carriages pulled by mechanical horses, instead of automobiles designed from scratch to use engines.

Yesterday we saw that the filesystem is the database. Today we see that the database should be the language. See where this is going? Go read As We May Think again. Prof. Bush is still out in front of us. There's no operating system. There's no file system. There's data, there's the user, and there's the glue (a language) that binds them together. Everything else is just artifacts of the haphazard way we implemented it.

Good and bad of stored procs

[einhverfr]

First, I think that stored procs are not only often good but lead to better, more powerful, secure, and flexible applications that would be feasible without them. But on the other hand, they lead to hard to maintain applications which are explicitly tied to one database. So they are necessary but often misused.

Triggers are more important and usually use stored proceedures to ensure that the information in the database is always meaningful or that some other automated activity happens on an insert or update.

Unless I absolutely have to, I try to avoid having my application call stored proceedures directly. A relational database manager *should* be able to hide the stored procedures behind a view allowing a nice standard interface for your data. This means that if you have to move to another RDBMS later, porting is much more simple and mostly confined to the backend.

BTW, I agree with the points about having your business logic in one place. Stored procedures allow you to move business logic which is inherent to the database into the database, thus making it available from all clients regardless of the language they are written in. For a single app/db pair this is not an issue but if you have a large DB with many different clients, it is a larger issue. Maintaining your application in one location is a LOT less work than reimplimenting it all in every one of your apps.

Triggers, BTW, as I mentioned before are very powerful mechanisms. They are not called by the app directly but are run every time a row is inserted, updated, or deleted (some RDBMS's also allow select triggers, though some have alternate ways of implimenting this). They can be used to impliment additional security restrictions, enforce referential integrity, or more advanced stuff such as send email when a new order is added to the database. Again, this is done regardless of whether the order is filed using a web app or a Windows app for the rep in the call center. Since the logic is unknown to the app, it isn't even aware of what happens after the order is placed. Talk about clean interfaces..... This requires stored procedures.

So, these are the great things about stored procedures. But when they are used badly, you end up with the stored procedures reducing the agility of the application because they tie it in too closely to the database. What do you do when your app is tied to Sybase and your customers want MS SQL Server? What if *all* your logic is in that database? Do you rewrite *all* of it for MS SQL Server? Probably Not. You are stuck and out of your investment.

In my opinion, it is important to maintain a clear line between what is the database's job and what is the application's job. If this line is blurred, bad things can result. Stored procedures are a very easy way to blur this line.

I design my apps with the following layers to them:

UI
Application-specific logic
Database Access Layer
Information Presentation (SQL API using views)
Information Management and Data Logic (usually stored procedures)
Information Storage (highly normalized, usually).

This allows the database to be a server app in its own right, and the client logic to run in the apps themselves. HERMES (see my sig) is mostly built this way, but there are a few things I need to change before I am happy with the interfaces. This is one reason it no longer supports MySQL.

Re:Good and bad of stored procs

[Anonymous Coward]

If you were designing an application from scratch, you're saying you'd really use triggers on purpose?
Triggers are not performance optimised, and seem to only exist to tack on functionality to an existing system.
I'll explain: if your app uses triggers to insert to another table once a row is updated, why can't you code all of that into your SPs as well?

Re:Good and bad of stored procs

[Darby]

I'll explain: if your app uses triggers to insert to another table once a row is updated, why can't you code all of that into your SPs as well?

I'm sure that there are at least a few different reasons, but let's address the obvious:

Update a row. (through any of the craploads of methods possible which isn't a stored procedure).

Oh holy crap.
Nothing triggered and it wasn't taken into account that something ( Anything at all.) happened in one row of one table which required something (Anything at all, remember we are talking about anything at all that could ever happen at any company in any industry) to update somewhere else.

Oops, it didn't happen.
6 months down the road during an audit it turns out that you have lost the integrity of your database.
Doh!
Well, if you can polish a turd you can polish a resume

We are, of course, talking about the real world where you don't actually have to know SQL (or anything about DBAing) to interact with a database.

Oh, but we saved an hour over that 6 months ( divided by the, say, 30 employees )
Neat.

Re:Good and bad of stored procs

[einhverfr]

If you were designing an application from scratch, you're saying you'd really use triggers on purpose?

Absolutely.

Triggers are not performance optimised, and seem to only exist to tack on functionality to an existing system.

In which database manager? And can't triggers call stored procedures?

It is true that there may be a performance penalty for triggers in certain cases. For example,PostgerSQL allows tuple-level and command-level triggers on insert, update, and delete operations. Triggers in PostgreSQL MUST call stored procedures, so if SPs are performance optimized (they can but don't have to be), the triggers are too. In PostgreSQL you cannot do a trigger on a select operation, but you can wrap the function in a view which calls a function either for the whole command or for each row returned. This is trivial.

PostgreSQL also has a "rule" system (NOT equivalent to the same name on MS SQL) which allows the database administrator to write macros which dynamically rewrite incoming queries (VIEWs are implimented using RULEs in PostgreSQL). Rules can add to or replace incoming queries altogether and can be attached to SELECT, INSERT, UPDATE, or DELETE statements (though you can only have one rule on SELECT and it must replace the current operation).

Also referential integrity enforcement in PostgreSQL is implimented using triggers. I.e. when you create a foreign key, it creates appropriate triggers. These are tuple (row) level triggers and if you are not careful these can have performance impacts, especially if your data types mismatch (which is usually a design mistake-- foreign keys SHOULD have the same data type as the primary keys they reference).

I'll explain: if your app uses triggers to insert to another table once a row is updated, why can't you code all of that into your SPs as well?

You could. Then you could put your SP's into triggers too so that your apps can use ANSI SQL without worrying about how the database backend wants the procedure to be called.

Re:Good and bad of stored procs

[Anonymous Coward]

Because he's fucking stupid?

Re:Good and bad of stored procs

[Jouster]

Somewhere in here, someone missed how much of a headache it is to backup and restore the data from these trigger-based DBMS's. Obviously, if you did a data dump in SQL, you can't just pipe it all back in as INSERTs, or things go haywire. So you're stuck with copying the DB files themselves (which usually requires a DB-specific tool if you want to do it when they're live), and I hope you didn't want to do incremental backups, and.....

Jouster

Re:Good and bad of stored procs

[FuzzyBad-Mofo]

This is one reason it no longer supports MySQL.

Then you should probably move your demo server off MySQL in that case.. :)

Warning: Access denied for user: 'demo@10.5.1.36' (Using password: YES) in /home/groups/h/he/hermesweb/htdocs/demo/hermes/DBA L_mysql-1.0.0-b.php on line 40

Warning: MySQL Connection Failed: Access denied for user: 'demo@10.5.1.36' (Using password: YES) in /home/groups/h/he/hermesweb/htdocs/demo/hermes/DBA L_mysql-1.0.0-b.php on line 40

Warning: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) in /home/groups/h/he/hermesweb/htdocs/demo/hermes/DBA L_mysql-1.0.0-b.php on line 81

Warning: MySQL Connection Failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) in /home/groups/h/he/hermesweb/htdocs/demo/hermes/DBA L_mysql-1.0.0-b.php on line 81

Warning: MySQL: A link to the server could not be established in /home/groups/h/he/hermesweb/htdocs/demo/hermes/DBA L_mysql-1.0.0-b.php on line 81
Error connecting to database. Was unable to process your request!
Login failed

Hits a nerve with me here

[aldousd666]

I am so incredibly emotional about this topic, because I have seen so many awful database implementations. Ad hoc calls are often times confusing when reading the code, so stored procs provide readablility (if named mneumonically of course) and the I absolutely believe 100% that all business logic possible should be encapsulated into stored procedures. Without stored procs, databases are not much better than a giant expensive spreadsheet application. DBMS's are made to handle, twist, turn and file data in order, forward backward and sideways. When arranging and smashing data Your code is *almost* (see I said 'almost' leaving you code hungry byte crunches in the clear) always the bottleneck. Why possibly bottleneck your app, or possibly risk storing data in less efficient flat in memory/on disk structures that don't collate, or neatly turn themselves into XML? I write some databases with enormous stored procs that accept and parse XML directly for insert or update, which would take a lot more work to do in client code. It's really nice when you can ship XML into the DB, and pull XML back out completely unprocessed, and the DB takes care of all of it with a stored proc. Why not use this kickass functionality? (It's even easier to get XML out of a db -- at least in Oracle and SQL server) When you put code in the DB, you can run whatever front end consumer or producer application you want -- for example a webservice, or a server demon. In my xml example, both could either produce or consume the data in the same format, with only the call of a few file reads and dblib calls. To me it's a no brainer. I love MySQL, but I can't wait for the stored procs. I haven't had time to mess with the beta yet... I'll wait for the release probably.

As for the argument that programmers don't want to do it becuase they are 'only java programmers' or only 'visual c++' programmers, that's bullshit. Programmers write code for a living, they need to learn whatever language it takes to get the job done. I do, and everyone I work with does.

"The" application?

[alessio]

It's funny how a lot of answers use the explanation "put the data in the database and the logic in the explanation", like there is only *one* application talking to your database. So it's not really an enterprise database, rather the data of an application... Real databases have tens or hundreds of different applications talking to them, written in different languages, with different aims and different bugs. Better to centralize the checks.

In my personal experience, I use a lot of PostgreSQL triggers to keep tight consistence of my database.

Sybase

[POds]

I've used Sybase stored procedures before on a project i worked on. I thought the idea was great. The project was split into two different teams. Team A worked on the Database and Business logic which went into the stored procedures and I was on team B which was in charge of writing a UI for the system. For me stored procs where great because it ment i didnt need to know heaps about the tricky business logic and i could just do what i was ment to be doing, I.e pretty UI.

It's was a good seperation of skills for this particular project.

Stored procedures vs. prepared statements

[Anonymous Coward]

There's at least one situation where stored procedures are a big win over prepared statements, or pretty much any other approach: where bandwidth is a factor. A stored procedure can invoke a number of statements, and then iterate over result sets and the like, all without involving any round trips over interprocess communication lines.

Granted, a lot of times folks think they can only do something procedurally, when it's actually possible to do something relationally, but not only can this be really ugly, long, hard to understand, and buggy, but sometimes it's not even possible to do in straight SQL!

If performance doesn't matter too much (like when you're driving a Web page and a second or two to render the page isn't going to kill your app), or where you have abundant bandwidth (same machine, or a very fast LAN), then sure, there's no huge performance benefit to keeping complicated database manipulations in the backend. However, if you're driving your app over a laggy line with minimal bandwidth (say, a satellite link), or you're doing a large volume of transactions where processing power is relatively cheap, while bandwidth isn't, then this approach is indicated.

Notice all of this is conditional on certain circumstances. Like all things relating to database operations, it all depends on what you're using it for. Almost everything a database has is useful in one context or another.

It all depends...

[GolfBoy]

It all depends on how you want to view your database. Fundamentally, you can look at it in two ways:

a) It's a persistence mechanism. At root, this is fundamentally what databases arose from. It's a fine way to look at the issue, and one that personally appeals to me. In that case, don't use stored procedures, but use triggers and constraints to ensure database integrity. Your access to the database should be through class loaders (in your language of choice). Don't try to get tricky with the database. Its job is to persist object instances and load them. Essentially what you are running is an OO database that actually works, since relational databases are - in practice - stunningly robust.

This the the normal 'application' use case.

b) The database is the company store of information. The understanding is that many applications will be written by disparate groups to access the data, and those groups will certainly not be willing to communicate and may not even know the other groups exist. In that case, for God's sake, make plenty of use of stored procedures and encourage their use. Stored procedures are the API to the company jewels, and are the only thing that allows changes to be made to the data structures.

This is the normal 'IT / internal business' use case.

Decide which category you are in follow that model.

(There is also the use case of the application with performance requirements that require the use of stored procedures. If you are in that situation you already know who you are.)

Consumer choice

[TooTechy]

In the same way people swap car brands and jam brands, people will want to swap their proprietory DB brand. You use their stored procedures, you tie in. They have you.

Other languages for SPs?

[RevAaron]

Do other databases systems give some other options as far as the language you can use in a Stored Procedure (SP)? For instance, perl, python, java? Most of the database programming I've done is for one object database or another (GemStone/S, Magma, MinneStore), and while there is no special distinction for a SP as such, I've created similar structures to obtain the same benefits as are described here.

But then again, in these OODBMSes you don't query with SQL or anything like it- you just query in the language you're coding in, which ends up to be completely seamless and very slick. Well, it can be a pain in the ass in less flexible languages like Java, but in Smalltalk or Common Lisp it's a dream, I like it a lot more than SQL.

So, do any DBs allow you to write your SPs in a something other than PLSQL or T-SQL or some other SQL-based thing? It'd be slick if you were coding your brand-new web-app in Perl 6 to just write your SPs in Perl 6 too, perhaps with inline SQL statements. And you could still call it from any other langauge presumably, just like a regular SQL SP.

Just me wondering...

Re:Other languages for SPs?

[Lazy Jones]

So, do any DBs allow you to write your SPs in a something other than PLSQL or T-SQL or some other SQL-based thing?

Sure. PostgreSQL for example supports PL/Perl, PL/Tcl and PL/Python. Look here.

Re:Other languages for SPs?

[RevAaron]

Ah.. Slick! Those PostgreSQL folks know what's up. I would hope and presume so, but any language can call those stored procs right?

It's like having a dedicated RPC, XML-RPC or SOAP server .... BUT REALLY BIG! :)

we use a lot of stored procedures

[ammoQ]

In many of our projects, we put all logic into stored procedures (packages) in an Oracle DB.
Although the vendor-lock-in is definitely not desirable, the advantages are strong: writing business logic in PL/SQL requires less code and is more readable than any mixture of SQL and Java or other languages. We never ever have to worry whether or not a number(14) fits into an integer variable. Using stored functions in SQL statements is much more natural if the rest of the logic is also in stored procedures.
Some of our programmers need only PL/SQL-skills, only a few are required for the surrounding framework.

Speaking as a Dev Manager

[LadyLucky]

I've just had to sink months of time in to a port from one database to another, because of the overuse of stored procedures. A good portion of them didn't do anything anyway, just some selects and so forth.

These days, so long as you are using a real language, you can find really good tools to perform persistence automatically, freeing you to have a wonderfully tidy place to put your business logic.

Having said that, it's not cut and dried. Stored procedures allow you to perform logic on your data. And if you use it for that, i.e., Data related logic, then you're fine. If you intrude into the world of business logic sitting in the database, you're going to run into problems very quickly... no extensibility, harder to test, etc.

So:

1. Make sure you're using a language that allows you to perform logic somewhere that is not a stored procedure and not a script inside an HTML page
2. Understand the difference between business logic and data related logic
3. Do your best to understand the potential for you to want to alter databases

Then find your happy medium!

Re:Stored Procedures often more harmful than helpf

[bwt]

The SP may or may not substantially raise load on the central machine. You seem to assue that SQL issued from the middle tier takes less work than SQL issue locally. The reverse is actually true. In addition, SQL from the middle tier has to push the rowset over the centralized machines network interface, which can be a very slow operation relative to local memory access, and uses a shared resource. You only get a resource savings if the data can be cached in the middle tier are reused enough to eliminate subsequen SQL calls to the server. For data accessed with low frequency and high volume, this is mostly a losing bet.

As for using vendor specific features, you aren't considering the ROI. By coding to least common denominator SQL capabilities, you are wasting your users time, because the vendor specific features make a big difference. In an enterprise system, nobody gives a crap about theoretical niceties like portability of the RDBMS. They want the thing that helps them do the company's core business process faster and better. And since the latter is what pays your salary, you will lose that debate every time.

SPs are text and can be version controlled as easily as anything else. That's two people who have made this completely moronic argument today.

Why do you call SPs "premature" optimization? What's premature about it? It doesn't take any extra time to code in the SP language (in fact for data-centric operations it's often easier).

Business rules engine..

[popeyethesailor]

I like stored procedures especially for one reason- They are targeted towards munging data within a relational database, and they do it damn well.

Take for example PL/SQL. Now there are damn lot of quirks in the language, but its base is rock solid. The in-built exception handling, oracle specific features just rock. The Data is stored relationally, and the best language to leverage it is the DB's own language.

Each DBMS has its strengths and weaknesses. For eg, a T-SQL developer would keep creating tons of Temporary tables to accomplish a task, which Sybase or SQLserver can execute effortlessly. The same cant be said of Oracle, in which DDLs are expensive, and a different technique has to be used. If the whole thing is written a middle-tier language, the performance most often sucks.

Having said that, I would really love a Open standards - Business rules language specification, with just one goal- to manipulate data in a relational database. A translation engine can convert this spec to optimized PL/SQL/T-SQL/$SP language. This should keep both the DBAs and the Portability gurus happy.

Re:an important question..

[DeadChobi]

Headaches are better than being stupid and having found out you've screwed up later on. Obviously he's got some sort of marketable skill in that area, or he wouldnt be designing the database. He's asking the question in order that he may bask in the collective wisdom of many hundreds of geeks, and their songs of databasery resounding through the halls of the frozen server room of life. Obviously.

Shared libraries

[MrCreosote]

You might as well ask if using shared libraries is a good idea.

Security in Stored Procedures

[rfc1394]

One advantage a stored procedure has over a direct SQL statement is that an SP has the priveleges of its' owner. (Or at least it did with MS-SQL; YMMV.) This means you can connect to a database without most priveleges and can only access the database by way of the SPs. In theory this should provide more security because even if you can hack the website or the application, the only functions you can perform are to pass arguments to stored procedures, you don't have the priveleges to perform other (more dangerous, or protected) actions on the database.

This also caused a problem because there was one instance where they wanted a stored procedure to be able to dynamically create an SQL statement to do certain things the user wasn't authorized to do, and while a stored procedure could create dynamic statements, the SP would only run with the priveleges of the user calling it, even though it was saved by a user that had admin priveleges. This was a safety and security factor and I thought was a good idea.

yes yes

[geekoid]

YES!
It is still relevant. You need to keeo your business rukes, database, and interface as seperated as possible*.

Security, reusability, readability, maintainabilty all demand it.
Security - You can encrypt stored procedure, and it's another layer someone has to get threw if the are trying to gain unauthorized access.

Reusability - If you have your data access broken down into Stored Procedures and triggers, the next time you need to do something similiar it will save you time.

Readability. - The code will be easier to read. You have an interface issue? now you don't have to wade through thousands of lines of data code to get to it. If it turns out to have been a data issue all along, you will find out a lot faster if it is a stored procedure.

Maintainability - You need to change some data rules? now you just change a stored procedure. This means your risk of an unexpected bug is much lower, and it's easier to rollback.

*There are some exceptions. If you need a phone number with an area code, thats a business rule. However it is so trivial you could put it into the GUI interface.

Newer SQL standard

[VGPowerlord]

FYI, Stored Procedures (AKA SQL-invoked routines) are also covered in ANSI/ISO/IEC 9075:1999 (aka SQL 99) Part 2 (PDF), Section 4.23. Triggers are in the same document in section 4.35.

In a perfect world, database vendors would be using the SQL 99 standard rather than sticking with SQL 92...

Re:Good, like most things if they are used properl

[geekoid]

I ahve done a lot of database work, and I have never seen anything that bad. The strongest arguement for stored procedures come down to have the data access seperated from the businesss, and GUI code. Onces they started putting business logic in it, they violated the whole point.

"Ever tried doing text layout in a stored procedure?"

yes, lots. Not really that hard.

Re:Difference is out here

[Anonymous Coward]

There's no operating system. There's no file system. There's data, there's the user, and there's the glue (a language) that binds them together. Everything else is just artifacts of the haphazard way we implemented it.

There is no database with data and logic.
There is web form on user's fingertips, and distributed services behind the screen, with distributed transaction's processing, messaging between nodes, and data distributed across the globe.
Any DBMS-centric (read Oracle-centric or MS), is truly artifact of pretendors on the role of big brother.

Good ? Bad ?

[Anonymous Coward]

Err...bad. No wait..good. No take that back...err... I am confused.

Use them where ever possible

[chriseyre2000]

There are certain things that cannot be parametrised such as joins across databases. We keep our archive data in a seperate database on the same server. The only way to access this via a stored procedure is via dynamic sql.

object oriented approaches...

[kurisudes]

by putting the logic for the common manipulations in the database, you can more easily use the data from many sources. Like say a java app, perl app, etc. The point is by not accessing the data directly with sql every time, you can reuse the data and logic for it in many front-ends... call it encapsulation, call it MVC principle, whatever you like, but it just makes sense. (The caveat being able to easily work on that code you've stored and not treating them as macros)

i make simple web apps...

[the_greywolf]

and i can see that the 8,000+ character SQL query i wrote for MySQL last week could have been very much simplified by replacing large portions of it with stored procedures. it's a feature i could have used, but it wasn't necessary.

of course, the query itself is a maintenance nightmare, and that's the way i like it. it was a contract job, so they'll *HAVE* to come back to me if they need to update it or move it to another platform.

that said, i'm wishing i would have had the opportunity to learn how to use SPs. the query on its own was an incredible learning experience for me, and i can truthfully say in job interviews that i can do amazing things in SQL if needbe.

i can see where SPs have their place. i can also see the arguments against them quite clearly. in fact, i would go so far as to say that if portability is at all your goal, reduce your SQL code to the bare minimum, dynamically generate them and never touch SPs. not all databases support SPs, and of those that do, not all support the same standard subset. MySQL and PostgreSQL are prime examples: MySQL has no support in the current releases, and PostgreSQL supports Python and Ruby, which few (if any) other databases support.

(for those curious, the query i'm specifically talking about was designed to generate the Calendar section of a local periodical. i chose to do it entirely in SQL both to strengthen my SQL skills and to stress my knowledge, abilities, and my locally installed MySQL server. so, the query returns four fields: the calendar category [exhibits, film, community, etc.], the event date [of the form "[THU|FRI|SAT|SUN|MON|TUE|WED|THU] xx{(, | - )([:DAY:]|[:MONTH:]) xx}"], the event title, and description. needless to say, the date field quickly became quite monolithic and complex, but i had a hell of a lot of fun doing it, and learned a great deal about MySQL, Apache [which i had to reconfigure twice to be able to use phpMyAdmin to test the query], and the SQL language itself. i'm a better programmer because of it. mainly because i know never to do shit like that again. ^_^)

Organization !

[Anonymous Coward]

I'm a java guy and I've done a couple store procedures in Oracle.

I understand the benefits in term of performance of using store procedures but IMHO they are not woth the trouble (unless you can't do otherwise: ie: you would have to send too much data to the client).

I prefer to have the code in Java classes because I can organize the code better! Code organization for big projects in the problem number 1 (if you want to understand you application of course:-)

With the modern IDE it's simple to localize code and that's an added benefit when you have access all you application code in a centralized place.

security in depth

[moocat2]

Since the focal point is the webserver, shouldn't security be done there, rather than the DB?

Security should be done in every layer of the system. If you only did security on the web server, if crackers are able to compromise it, getting into the database is simple at that point. But if they can only run stored procedures, they then need to find a hole in the database which halves the likelihood of getting in.

Of course there are all the costs of maintaining the extra level of security. So you have to consider the cost/benefit of the security you put in place.

Great for *processing* data

[PizzaFace]

I understand some people use SPs for everything, as a security measure. I'm not that paranoid.

Stored procs are great, though, for processing lots of data on the server, without wasting resources (network bandwidth, client memory and CPU) sending the data to a client for processing and then posting results back to the database. Even if you have to do record-by-record processing instead of set operations, it's much more efficient to use a cursor on the server than a loop through a resultset on the client.

Stored procs are good at encapsulating a series of operations done periodically, such as end-of-month processing. In some databases you can define an event on the server that will automatically fire the SP. With such an event on the server rather than on a client machine, you don't need to worry about whether the correct client is running and connected when the event is supposed to fire.

Complex reports often require more data crunching that just a single query. I use an SP to process the data and leave the results in a temporary table. My client program runs the SP, then reports with a simple SELECT on the newly generated result table.

Missing Named Parameters

[Tablizer]

I am not too fond of over-use of stored procedures. However, they can come in handy in a multi-language environment.

But one thing they usually are missing is (optional) named parameters. Named parameters make it easier to add parameters in many cases because you can add an option without having to change a jillion existing calls. I suppose that optional fixed-position parameters sometimes can do this, but if you have lots of parameters, it can get tedious to match positions.

Not so far fetched.

[Anonymous Coward]

I didn't like 3.1, and XP does suck. The core philosophy of windows has been inherited:

1) taking stuff an admin should be doing, and giving users simplified access to it, while removing any of the complex options that a good admin needs.

2) a by-product of (1), but important enough to mention seperately: ignoring security etc. in favor of simplicity.

3) fundamentally single-user.

4) code that's so fundamentally flawed that it even looks ugly when you're running it.

One word: SEPARATE

[brunorc]

Of course - GOOD. And one more argument - this way you can truly separate code from presentation. This is the same way MC(C)V systems work. Let database handle all "this-id-is-not-allowed-to" issues, this will make the code of your presentation layer smaller and easier to debug.

Of course, you will still have to debug triggers/procedures, but believe me - it's worth. And - if the word SEPARATE doesn't convince you, think about the word REUSE. You can apply policies on the database side, then wrap it with PHP, Perl, Python, C, Java - whatever you want.

Braindead assertion...

[Anonymous Coward]

"[snip]At one time we had a client/server architecture so those three advantages were relevant. However, in the past 4 years we have moved everything to web front ends and I have argued that this is no longer true. Does it really matter if my business rules are centralized in stored procedures or in a set of php/asp scripts (ie, in the web tier)?[snip]"

You've argued this because you've only been around for a we fraction of time and have never watched technology change from one generation to the next. If you stick the logic in the database, what is the cost of developing a new platform or changing web technologies? What's the cost of change if you don't put your logic in the database? In five years, are you going to assume your PHP code is a tomb of business process logic that can be trusted because its logic has been segmented and acts independently from data presentation hacks or frufy one-off hack jobs?

"[snip]Since the focal point is the webserver, shouldn't security be done there, rather than the DB?[/snip]"

Web software is a moving target. For every platform, web program, service, etc., you have to enforce the same set of rules. Stored proceedures are to databases what functions are to programming languages. Use them or your business will suffer from inconsistencies ranging from data management to more serious problems such as data access and theft. I bet companies that process credit cards or people who work on medical records probably have strong and justified opinions on the topic of data security... think the cost of implementing consistent database proceedures is bigger or smaller than the cost from a screw-up that lets the data walk out the front door?

You mention MySQL in your post... use a real database such as PostgreSQL for two months and let me know how your opinion on databases has changed at the end of your trial run. Compare and contrast this experience with your experience stemming from application development with MySQL. If you think MySQL in its current state is still a valid core for an organization at the end of that duration, you obviously suffer from some form of brain damage.

Depends on the project and the team.

[tunesmith]

I regularly freelance for projects where we have one template programmer, one middleware programmer (me), and one DBA. With the same team, we've tried projects with stored procedures and without. In general we've moved away from sprocs.

The first time we tried sprocs, we basically treated them as functions. I would pass in a bunch of arguments (db column contents) to sprocs that would insert a new Activity, for instance. This got old very fast. It was much faster for me to write the business object that would insert itself from the middleware layer, than it was to wait for the DBA to create the sproc, after which I would have to create a middleware layer to the sproc anyway. It also didn't make financial sense for the client, because DBA's usually charge a higher hourly rate than middleware programmers.

The other sprocs were ones where I would supply several search criteria to a sproc (basically portions of a where clause), where he would assemble it into a sql query and then return the result set back to me. That was a bit more useful, but ended up kind of silly too, because it wasn't efficient to involve the DBA in actual application logic - we kept on having to go back to him whenever someone wanted to add a new dropdown to the search form.

If you're been an intermediate programmer, you've painted yourself in the hellacious corner of trying to dynamically generate a sql query that may or may not join across multiple tables. It ended up being a lot easier for the DBA to simply create a view for each family of search queries. Then I'd assemble the sql query on the middleware layer. Easy then, because I'd never have to worry about dynamic joins - the view would already have the joins, and I'd only be querying against the single view. And if there was a query change, I wouldn't have to involve him unless is actually required adding new columns to the view.

Right now the complexity of our projects don't require the remaining cases where a sproc would really make a huge positive difference. One such case would be a multi-step atomic transaction where we were worried about performance. A sproc would be perfect for that. But in general you can do just fine with inserting into tables and selecting from views without having to deal with the cost of having a significant amount of your project in sprocs.

Finally, an important tiebreaker between having logic in sprocs and in middleware is a pragmatic one - system resources. If you're making a lot of changes, you're going to be dealing with source code management - trunks, branches, and multiple staging installations. It's much easier to do this with your code repository than your database. With ost companies I've worked with, it's a lot easier to set up a new vhost for a code installation than it is to set up a third oracle installation. If you have a lot of quick changes to make, it's easier to make them in the codebase.

Beyond that though, it really depends on the team. If we had a full-time DBA rather than our 10-hour/week guy, and a less competent middleware programmer than myself, and a project with more fixed requirements, then we might defer more to sprocs. But our DBA is swamped, our projects tend to have ever-changing scopes, and I'm quite comfortable with MVC and keeping the control layer thin, to be able to respond quickly to the scope changes without having to majorly rework business objects on the model layer. It works well for us - and these are for large scale bank intranets, not simple little webapp one-offs.

Many people that know too many buzzwords think that "the business object layer" by definition MEANS the database and sprocs. It doesn't have to. It can just as easily mean the Model layer of an MVC middleware layer. With my work style, it's faster to leave it there and then use the database for storage and data-level calculations that can be embedded in the queries themselves.

I agree, you effectively get a black box

[SmallFurryCreature]

If someone else has to read the code then you got data being manipulated outside the code. I put in A=4 and when I look it says A=6. WTF? BUG BUG BUG. Oh no. Stored procedure. Yippy.

If you really have to figure out undocumented code you are going to follow the path, so if a function is called and you don't know what it does you look up the function and see. It is a nightmare but at least it can be done. Stored procedures add hidding function calls. You don't see it. Sure it can be figured out but I see the world divided in to two groups. Those who use stored procedures and who never have to figure out someone elses code. Those who do. The first like stored procedures. The latter hates them.

I like the three tier approach. Seperate presentation from business logic and storage. It is kinda unixie. Small applications that each do what they are best at. Databases are not programming languages.

Of course it is easy to use stored procedures as it enforces database rules, it is much easier to use myscl auto_incremenet feature then to write your own code to make unique ID's. However if you were looking at the code and you are any good at programming you should be wondering were the hell that ID number is filled. As said it is bad for a programmer when variables just change value without it being reflected in the code.

But of course this is just from my experience and from being taught by people who didn't like stored procedures. Others will have been raised differently and be capable of giving excellent reasons for their use. I think the only way to reach a compromise is for the code to reflect exactly via comments when a stored procedure will be called. Comments in code that are usefull. HA, that will be the day.

Why are macros a good thing?

[munch117]

'They let you abstract data access procedures.'

Still I don't recommend using them, because I know there are better ways of reaching the same aim.

Same thing with stored procedures.

Re:Why are macros a good thing?

[Bedouin X]

What ways would those be?

Re:Stored Procedures often more harmful than helpf

[NFNNMIDATA]

1. Opposite day on that one, SPs decrease the load on the server by increasing its efficiency. That's kind of the point.
2. Vendor specific sql is unavoidable anymore.
3. You can version control SPs like any other code, and maintain them within your codebase if you so choose.
4. Anything can be over-optimized. But tiny improvements in speed can make a huge difference to the user experience when the query load is heavy. It only takes 10 100ms delays to add a second to the user's wait time. Maybe your 1000 SPs were created with that in mind.

Also, I don't understand all these guys saying "let the database and the business logic each do what they do best" and then coming down against SPs. SPs allow the db server to do what it does best, and then hand off a final dataset for the app logic to use (to do what it does best). With SPs you can take advantage of the folks that designed your db, who know a hell of a lot more than you about how to optimize for performance on the thing.

Question

[sceptre1067]

Complex reports often require more data crunching that just a single query. I use an SP to process the data and leave the results in a temporary table. My client program runs the SP, then reports with a simple SELECT on the newly generated result table.

Just curious... why the choice of a temp table versus directly returning the results to your client?
(to partially answer my question is this a matter of amount of data transfered, or based on your choice of language/architecture... the temp table is easier to manipulate then storing locally?)

thanks.

Re:Question

[PizzaFace]

Just curious... why the choice of a temp table versus directly returning the results to your client?

It can be simpler just to return a result set, as you suggest. I often use a temporary table to allow pre- or post-processing of the results. For example, if I have a payroll application that prints paychecks, I will typically use a temporary table so I can first display a list of checks to be printed, then after the user accepts that list I print the checks, then if the user confirms that the print operation completed successfully I record the checks in a permanent table in the database.

Sometimes in the preprocessing phase I allow editing of the temporary table. Payroll is not a good example of this because it needs to be controlled closely by the application, but something like an address list can be created in a temp table by an SP, then the user can edit the list before printing it.

SQL is Flexible

[MrSteveSD]

The company I for deals with major UK power companies and a lot of them now have a policy that all internally developed systems use Stored Procedures for security purposes, although they are no yet forcing this on externally purchased systems. Obviously SP's are important for security but there are times when dynamic SQL makes things so much easier. For example, imagine you have a form that shows the details of all of your stock. You might want to throw all different kinds of queries to get the information you want. e.g. "Show me all items from electrical goods made by Phillips that cost under £300." or "Show me all kitchen products that come with a 2 year guarentee" These sorts of queries need flexible WHERE clauses. In our applications we use a flexible Where Clause generator I created. You just set the constraints on the WhereClause object and it generates the necesary SQL without lots of messy if-then-elseif type logic. How would you do the same thing in a stored procedure? Lots of if clauses? Writing a flexible clause generator in SQL? I'm sure you could do it some way but it think its a lot neater doing this sort of thing in the application.

Re:Stored Procedures often more harmful than helpf

[xelah]

When was the last time you saw the CPU sustain a high load on your database server? Database machines are constrained by I/O (disk and network) and memory, not by CPU in most cases. Therefore, your 8-way SQL box is sitting at 10% utilization. Why not get a bit more use out of it?

This depends very much on your application and dataset. The one I look after has a small but actively used dataset (less than 1GB is heavily used) and regularly nearly fully uses the CPUs in the database machine. Moving load off the database (and, indeed, caching data elsewhere where appropriate) is the only way to survive in these circumstances (bearing in mind that buying better database hardware or software is not an option thanks to the cost).

No, what's more important is that your queries are centralized, so that you can optimize them more easily. What's easier to optimize: A single query to in a stored procedure, with the procedure called by 100 different methods; or 100 different queries to across all of your code? Yeah, sure, you can centralize that as well, but what's stopping a developer from writing his own query rather than using the optimized one you've already provided for him?

If you're doing the same SQL query from a hundred different places then I think you've got more fundamental problems than a few bits of poor SQL...

Using a database properly is about more than optimizing queries. Everything from system architecture to the detailed behaviour of this or that object affects the load on the database and the quality of its structure. Using a database properly is also about designing your application appropriately and choosing what and how it uses the database - not just how it expresses those things in SQL.

Yes, you probably want a layer which is responsible for accessing the database, enforcing security and implementing business logic (or, at least, business logic relating to persistent data). This layer can be a set of stored procedures or it can be a middle layer which the rest of your code accesses (or both, if you've got a very good reason). Whatever you choose it DEFINITELY needs to be written by someone who knows about databases (and, in case you hadn't noticed, DBAs are not the only people who know about databases).

Stored procedures are tools

[mlwmohawk]

These debates are as silly as they are pointless. Stored procedures are an important tool within a SQL database. They are part of the SQL standard and should be there. Trying to say they are good or bad is like asking if a socket wrench is good or bad. There is a lot of overlap between what can be done in a "business logic" language, like Java or VB, and what can be done in a SQL stored procedure. Like all "gray area" arguments, the simple problems are a matter of choice as to where you want to solve them and which method is easiest. In the "real world" "real problems" often dictate where they must get solved. When you need a stored procedure, it sucks not to have them. That is why I NEVER use MySQL.

Programming using relations

[Baldrson]

The problem isn't with stored procedures, its that the entire environment isn't relational, persistent, triggered, transactional and journaled.

There are very few experiments let alone developments advancing this direction.

The Stored Procedure Nightmare

[daem0n1x]

I coded tons of stuff in PL/SQL, back in the good ole days days I worked with Oracle (sigh). Despite its limitations, it's not a bad language for business rules, and I did pretty complicated stuff with it.
Now I'm forced to use stored procedures in MSSQL. I hate the T-SQL language with all my guts, and I wish I could have a dime for everytime I told my team we should be dropping them and favor business logic in the middle tier. But they love that crap, so I'm mandated to use it, too. :-(
Looking back now, I realise that my extensive use of stored procedures and triggers really wasn't a good choice. They tend to complicate the databases and make them hard to understand. I favor a clean object-oriented or service-oriented model (depending on the domain) on top of a vanilla relacional model, with thorough and well-thought use of integrity constraints.
An example:
Some years ago I wrote a system that imported a massive amount of data to an Oracle data warehouse. It envolved parsing some weird binary files and stuffing some tables with data every single day by 2AM. The data had to go through some complicated calculations before being stored.
I wrote a program in C++ that parsed the files and called some stored procedures to do the business stuff. The parameters for the calculations were stored in a dozen tables with some thousands rows. This took 5 hours/day when the system was designed, which was considered acceptable.
A year later, due to business growth, the system was taking 21 hours to do the import. This had to be fixed. For days, I reviewed all of the PL/SQL code I had written, and found nothing that could be optimised. I ended up reviewing all the database parameters (IANA DBA) and ended up shorting the processing time to 12 hours, which is barely acceptable. If the daily data volume continues to grow, a major refactoring is to be done.
One thing that occurred to me was to change the business logic from PL/SQL to C++ and do all the calculation stuff inside the parsing program, using some tricks. The program would load the parameter data to some hash tables and other data structures and perform all calculations this way. I think I could save a lot of processing time this way, but the business logic is so complicated that I cannot possibly do that without creating another project from scratch. The golden rule is: "If it ain't broken, don't fix it", and the business logic in this system is one of the triumphs in my engineering career. Since the day it was written, it never had a single bug, not even in the testing phase.

Makes maintenance simple

[Anonymous Coward]

Sorry if this has already been mentioned... haven't read all the replies yet...

I suppose this doesn't apply to thin client applications, but I work on a thick client application and at the minute, most of the business logic is in the client. Whenever we ship a bugfix or new version, the client has to go round and update dozens of machines with the new EXE. We made this a bit simpler by writing a program that sucks new EXEs out of the database and replaces the client app, but it's an optional extra and people don't like paying for stuff.

I recently did a major enhancement and put all the business logic in the database via triggers and stored procedures. The only code in the client app is used for viewing the data the database business logic produces. If there's a bug, we ship a SQL patch to update the database with the new trigger/stored procedure. No wandering round updating EXEs. Clients love it.

Word of warning: If you're putting business logic in an Oracle database using triggers and stored procedures, keep the triggers as small as possible by putting common code in stored procedures. When the database boots, it compiles stored procedures and functions, but not triggers, which are compiled when the trigger is fired. If you've got a large trigger getting fired frequently, your app is going to grind to a halt pretty damn quickly. This might have been changed in Oracle 9i/10g (I'm using 8i).

This common discussion has one answer...

[pebs]

It completely depends on the requirements.

.NET response

[mr. mulder]

"We'd sure love to just be able to point the web tier at a new data source but that is unattainable due to a convoluted tangle of db specific code."

I'm not proclaiming that .NET is the answer to our problems; however, Microsoft has made a good step in the right direction with .NET. With a few changes in lines of code, you can switch data source types.

I know the above is a bit vague, but as we move into "more managed" code environments (which is where we're headed), programming language interface with the hardware and other software products will become much easier and robust.

The JVM from Sun was just the beginning; .NET is version 2.0. I'm excited to see what's coming in the next 10 years.

God Bless You - You Speak the Truth

[Anonymous Coward]

In the past 10 companies I've worked for I've never worked with a DBA that was not a complete lying tool. You forgot to mention that most only use TOAD/SQLNavigator because they don't actually know database commands.

Stored Procedures Solve a Non-Existent Problem

[smack.addict]

Stored procedures solve a problem that predates languages like Java and C#. Once upon a time, you built client server apps using a client built in C, C++, or (god help you) PowerBuilder or VB. Database code either had to go in the user interface or the database. There was no other choice. So, to centralize business logic, stored procedures came about and people began placing the logic there.

In a three-tier or web architecture, stored procedures have no place. Centralize your business logic in business objects on the server. This makes your application independent of any underlying table structure or persistence mechanism. You can get the speed of stored procs by using prepared SQL in your database mapping code. These days, you can use mapping tools like JDO to avoid any database mapping code.

Where stored procs still have a place is inadministrative functionality, such as background batch processing. That's it.

DB Coder

[Anonymous Coward]

Disclaimer: I am a DBA

"In addition, you either have to have a dedicated T-SQL or PL/SQL coder who then is the weak link in your coding chain, or your pool of developers must become fluent in both your scripting language of choice as well as the SP language. I have experienced both of these approaches and found this to cause bottlenecks when 'the database guy' is unavailable and learning curve problems (bugs) with new coders getting familiar with the db language."

I've seen the SQL our Java Developers write *shudder*. Seriously, sounds like resource (i.e., people) allocation problems. I write all the sps for our web apps (Oracle's PL/SQL), and would argue that putting it in the db gives you:

1. Better speed because the DB is made for munging the data, and it's one trip to the db server. The DB Engine is processing all the SQL anyway. In the days of CGI and Cold Fusion the db was the fastest place to put data manipulating code (including selects) and it still is.

2. More security through secured passwords and roles (password protected). Commercial RDBMS support things like login-triggers and 'Virtual' Databases within the db to limit data access.

3. Arguably simpler maintenance/enhancements. If the logic is on the db side, and you need to make changes, you have one point to run your changes - depending on the scope, you may not even need to re-compile/re-deploy code on multiple app servers. Using Java to store business logic guarantees that I have to a) 'nicely' drain all servers of active sessions b) take each server offline, redeploy the java c) bring each one back online. A process that will take hours. My maintenance window for db changes typically takes 15 minutes (often less).

4. Freed of SQL tasks developers can focus more on application design and GUI performance.

non-proprietary stored procedure languages

[Anonymous Coward]

FWIW, PostgreSQL implements Python,Perl, and C as stored procedure languages, as well as several others. They aren't automatically installed, you have to add them yourself; however, that would mitigate the proprietary language problem in SP, as well as simplifying the learning curve for most developers.

Re:non-proprietary stored procedure languages

[abirdman]

I agree. However it should be noted that just changing the prodcedure language doesn't avoid proprietary lock-in. Writing a stored procedure in any language requires writing to a specific API with specific structures in a specific context for accessing, manipulating, and returning data. The fact it's written in C or Python or PHP or PL/SQL may make the stored procedure easier for someone to understand (if they're more familiar with those languages-- and I agree with a previous poster that PL/SQL beyond the most basic is difficult to understand, and don't get me started on T-SQL in MSSQL!), but most definitely doesn't make it portable. You can't really pull the procedures out and use them elsewhere--another DB or in some middle layer of software. Creating any significant DB functionality with stored procedures will result in a reduction in the portability of the application. You're locked in, even if it's a lock in to PostgreSQL.

Now don't get me wrong, I don't think this is a bad thing, and I'm in favor of using stored procedures, and have written quite a few of them, for Oracle and for PostgreSQL and MSSQL. They keep key functionality centralized, and allow less experienced coders to do useful work without having to understand all the complex data relationships, integrity rules, security policy, and such that are a part of any non-trivial DB application. I've also noticed that PostgreSQL's "native" procedure language is getting more and more compatible with Oracle's PL/SQL, making Oracle ironically perhaps the least subject to proprietary lock-in.

Re:Stored Procedures often more harmful than helpf

[Anonymous Coward]

That's two people who have made this completely moronic argument today.

You're such a moron.

And then ...

[jdkane]

... there's that cool murky middle ground, where you don't put project-specific business logic in the stored procedures, but you do create generalized stored procedures (e.g. can be easily applied to other data sets/projects) that support your programs. Of course that still means porting stored procedures if you change database platforms, but not as many as in the extreme case. It's a fine line.

You didn't define the assumptions

[Uzik2]

When you ask "what's the best..." you always have
to give your assumptions. If you asked "What's the
best cutting tool" you'd have to tell me if you
were going to do surgery with it, or chop firewood.
An axe is great for one, but terrible for the other,
and vice versa.

Are you trying to get the 'best' application for
the users, or for the programmers? What makes
it best for users might make it harder for
staff to support. What's your budget? If the
best choice is to rewrite in 'X++' will they
pay for it? Are there language and support
availability constraints? Will someone there
dictate to the programming staff that they must
use Microsoft products? Will they refuse to use
'X++' because support staff is hard to find?

It's hard to get someplace when you don't know
where you're going.

Configuration Management

[Crash6-24]

A point I haven't seen mentioned is configuration management. In a disciplined team any member knows where to go to get/inspect the code that the app is executing.
Years ago I insisted that the developers work in SQL stored procedures so I could see exactly what they were running / testing in our app. It was amazing how often the code that the developer thought we were testing wasn't what was in the database. Using sp_helptext on the procedure I was able to defuse many "testing errors" as "configuration management" errors.
In later groups I've been more adamant that developers and testers and the project lead follow a configuration management scheme. Whether the code is in the database or the app, interpreted or compiled, I want to know what we are running / testing / developing.

Control and performance

[awol]

This is a great question. I have read a lot of responses here, and for what it's worth I have been writing complex database applications requiring high performance for ten years.

Stored procedures are critical.

First, they are critical for situations when you, as designer, know a piece of information about your database application that the query optimiser cannot really work out (if you are asking what is the query optimiser then find out! :-). The classical example of this is if you have a particular process that is going to require mulitple scans of most of the rows in a relatively large table but that only returns a limited amount of data, then what you should do is design a stored procedure that does the table scan once and collects all the data along the way. The performance imporovement will be out of sight. Particulalrly if the data is used in a client server environment.

A corollary of this point is procedures that can take variable parameters to reduce vastly the number of rows returned from standard queries. This enables efficient deployment over a network where the client can help the server reduce the number of rows sent over the network.

There are a lot of other good reasons to use stored procedures (and if you include "triggers" in the mix then number increases), all good comp sci reasons like modularity, reuse, etc etc. But the critical one for me is "design time" knowledge about the application of which the RDBMS tools (indexes etc) cannot take advantage.

The hidden benefits of stored procedures

[1iar_parad0x]

s/Stored procedures/embedded-SQL/

On DB2, a stored procedure (written in SQL-PL|stored procedure builder) is nothing more than a package (compiled SQL with a generated query plan) created with Embedded SQL and C. I imagine Oracle is the same. Technically, a well written piece of software using SQL-J (or the C equivalent) or a well designed app (in some cases) using prepared statements would generate the same performance.

C is faster than PHP|Cold Fusion|ASP|Perl

In a world of slow, interpreted scripting languages, C code is fast. Which would you rather run, prepared statements with PHP|Cold Fusion|ASP or thin layer of front end code calling massive middle-tier business logic.

Stored procedures should be the number one tool of web developers. However in a world of large scale, n-tier business apps, I'm not so sure. Of course, I believe a lot of enterprise software ends up being architectual overkill, but that's just my opinion. However, EJB/COM/Corba/XML-RPC pays better so.....

My advice is to get a fast, beefy db server and simplify your app.

Stored procedures can hide database structure (normalization)

Stored procedures help protect the end code from being affected by changes in DB design. Sure, you may be able to write a complex query as a workaround, but stored procedures offer you more flexibility.

SQL

[Anonymous Coward]

I write all my apps in php with a mysql db, I don't use db specific routines wich makes porting to any db quite easy I stick to simple statements and do all data manipulation in the script. I use a single persistent connection into the db for each application. A simple preg_replace of query and fetch, and connect code to whatever they used for oracle, postgres, etc... Would do the trick to migrate.

Re:Why stored procedures are bad. (10)

[attonitus]

On the other hand, maybe I'm also not so arrogant that I to think that I'll get it all right the first time. And even if I do manage to get it right first time, maybe I'm imaginative enough to think that maybe there will be future requirements or changes in business processes that I couldn't possibly anticipate at the moment.

Are you seriously suggesting that once you've designed the DB you'll never have to go back in there and change things?

So you've moved to a webfront end ...

[Anonymous Coward]

And what will you move to next?

Let's be completely honest with outselves -- software changes with time.

I admit, it's easier to just work in one language, be it JSP, Perl, PHP, whatever -- but it's not as modular as you think it is, because then you're bound to that language. Of course, likewise, if you're using PL/SQL stored procedures, you're then bound to Oracle, so it's a bit of an even tradeoff.

Unfortunately, for every advantage of stored procedures, there seems to be a corresponding disadvantage, so in the end, it matters what you know about your project, and what the plans are for the future.

[oh, wait, one advantage -- less network traffic... I don't think there's a disadvantage on that one]

Moo

[Chacham]

One other aspect on centralized code is that standard procedures are followed. Such as, if the is a business rule (as opposed to an enforced data rule) that mentions how when A gets a new line, B must be updated as well, a TRIGGER is not a good idea (that's for data rules), so an SP does it all for you. Now, anyone can use the SP and forget about the rule.

Sprocs are useful - how should you access them?

[Anonymous Coward]

Stored procedures are valuable for efficiently conducting data-intensive operations and enforcing data validity rules. But, if you do not use them for all database interface operations, you have the problem of the client-tier developer trying to figure out whether functionality is contained in a sproc or not. The solution is a good middle tier that encompasses all database interface operations. Decisions about how extensively to use sprocs will be made in the creation of the middle tier.

If you choose to not have a middle tier in code, you gain the flexibility of allowing access to the database from any code language that has the needed db access libraries. The problem with that scenario is, you must use sprocs for every operation in order to ensure correct use of the database by client-tier code.

In regards to creating the middle tier, using generated code to create sproc access functions is advisable.

Concerns about database vendor / programming language lock-in may end up forcing your decision one way or another, but these only apply to the individual situation and are not helpful in a general discussion about the use of sprocs, IMO.

MySQL must not become an Oracle/DB2/... killer

[lbo_ninja]

Sorry but I'm in Oracle db since 10 years now and I use MySQL too but not for the same goal. I think that many developers need heavy database with many options (procedure, triggers, replication, objects and so on) and lightweight database too. Each have his own range of applications and utilities.
So i'm not really against procedure and triggers in MySQL as long as it stay a low requierement hardware database. If it change that way to be, I'll drop MySQL and stay in Oracle for all usages.
To all speaking about being stuck in one db vendor , I'll reply than in company front-end langages change far more than database. Databases are expensive so the choice is generally done for long and migration of data are expensive too so company say generally stuck in Oracle, DB2 or SQLServer for a very very long live...
But the front end is another story. For various reason, often many front end are used. Only here in my firefight departement, we use: access, excel, openoffice, php, oracle forms and oracle reports. It lead the dba and the developer (oops it's the same person: me) to use views and procedure to offer a kind of abstraction level to all that tools. It's the better way to be sure that a same question will give a same result whatever is the front end.
The couple MySQL/PHP is great because light and easy. I want it stay like that.
So if you thing procedure is requiered go for it but keep it light in requierement and esay to use.

Statment Queuing, Network lag, result set processi

[Anonymous Coward]

I am a DBA and a web developer. Heres some quick facts for ya.

SQL Server 2005 has .NET as the stored procedure programming language. So you can write vb.net or C# stored procedures. It also has XML documents as a DATATYPE, like int, float, varchar, XML.

SQL SERVER 2000 has NO performance difference in executing external SQL statements versus stored procedures IF those external sql statments are "parameterized queries". Sql server can parse, precompile, and store and execution plan for these EXTERNAL sql statments that are "paramenterized queries".

Heres the pros and cons of stored procs in SQL server.

* Faster than non parameterized SQL queries ("standard sql")

* A MUST when your application is internet facing in order for SECURITY. You wouldnt want some hacker being allowed to shoot SQL at your database if they got the application creditials.

* Reduce unneccessary network traffic.
* If the database has to *wait* for the application to supply it with sql statments its going to run slower than if all the SQL is queued up in a stored proc. And I can tell you this, it takes way longer for a web server to talk across a network to an application tier that talks across a network to a database and supplies sql than a stored proc executing inside a database.

* if you have any kind of result set processing that must occur its WAY faster to handle the result set ON the database server than to SHIP it back to the application.

* You can use stored procs to alter the behavior of applications at runtime without locking users out of the application.

The cons:
* If your not familar with tables, indexes, temporary tables, views, triggers, transactions, error handling at the database level, and in general relational database performance tuning you will need help writing them.

How much did you pay for Oracle?

[lorcha]

Probably too much to just write DBMS-neutral code and not bother to use any value-added features that Oracle provides.