Gee, funny. As a developer, I could've sworn that it was so that the libraries each program used would be accessable to any program installed on Windows. God knows I'm thankful I can link to IE's HTTP transfer engine, rather than having to either code my own or getting a third party library. Or when I have to show a video, I can link WMP's libraries rather than having to code my own video rendering routines...
Of course you can do this. But, there is no reason why the libraries you use should be bundled with Windows, or should not conform to a common standard so that other companies can provide the same service. Why should there be a problem with a third party driver if it conformed to the same API as the Microsoft HTTP transfer engine?
When you use DirectX on Windows, you are using a video driver written to a common specification. Use of third-party DirectX drivers is common, and largely transparent to the developer. There is no reason why MS should not have devised a web-engine specification (say WebX). You could have then used the either Microsoft IE WebX driver, or the Mozilla WebX driver or the Opera WebX driver. Same with the media player.
But nope. Clearly there's no technical reason why this was done. It's all because Microsoft is evil and monopolistic. Never mind that Konquerer in KDE is quite similar in integration as IE is to Windows, for similar reasons...
You are completely missing the point (Several points, in fact):
1. KDE is not in the position of being a monopoly.
2. KDE has not attempted to influence Linux distributors not to supply alternatives.
3. The KDE services have a published API and are open source - competitors could write an alternative to Konqueror and the rendering components which would work the same for developers.
They became services because it made sense for them to become services. Applications need to be able to render HTML and access web content - applications need to be able to play and work with media files.
And on other operating systems they do this. They use libraries which are shared between applications, and there can be alternative suppliers of the libraries.
Making an API that other media players could implement... while this sounds nice in theory it would take many many years to get everyone to agree on such an API
Windows versions take years to appear - plenty of time to do this.
and in the meantime everyone would need to include whatever multimedia app their app builds on top of with every little shareware program - or they would need to write the code themselves.
But this works fine on other platforms.
Furthermore the bug-finger-pointing that would go on if there were many different browsers implementing the IE APIs or different media players implementing the WMP APIs would be a nightmare for consumers.
Absolutely not. This creates a market for the best quality provider of the API. In the same way as there is finally an increasing market for the browser, there could be an increasing market for the browser engine or the media service.
You could also argue for defining an API for filesystems (which does exist) and allowing different vendors to ship with different filesystems should be a government mandated requirement. Of course when the drive gets corrupted who is going to take the blame?
The vendor who has shipped the driver, in the same way as the vendor who provides the computer hardware or hard disk or network card gets the blame.
When someone calls MS are they going to have to debug a 3rd party filesystem?
No, they call the driver provider. Its like the provider of a COM service or ActiveX control.
Who cares if these APIs are bundled with the operating system?
The end user, who gets poor quality bundled products with the OS, and the developer who wants to provide a better alternative, but has to get over the barrier of the default bundled service being... already there.
It makes it easier for other developers to write applications on the platform
But only by tying in the developer. As I said, having a standard for the API would not raise this problem for the developer.
- and it sets a bar beyond which 3rd party equivalents must pass before they are interesting.
A bar which is unfair because it is imposed by a monopoly company. There should not be this bar. It is illegal.
In a similar way WordPad sets a bar (albeit a very low one) beyond which all word processors must pass before they become interesting to end users.
Well, this example is fine. The problem with IE and WMP is as if Microsoft bundled Word or Excel with Windows.
It is fine for Microsoft to provide products such as WMP which provide Windows services which make life easier for users and developers. What is illegal is for them to bundle these products with Windows raising a huge bar to adoption of alternatives. It is illegal because MS is using its monopoly on desktop operating systems to try and establish a monopoly in other areas (browsers and media players). It is trying to do this by falsely re-defining the term 'operating system' to include these areas.
What issues do people have with WMP anyway? Don't people realise that WMP and IE are not "applications" but more like "services"?
The point is that they WERE applications. They have been transformed into services for business reasons: in order to crowd out alternative application providers. In a monopoly, that is illegal.
There was another way things could have happened. Microsoft could worked with suppliers of applications in order to develop an API for these services that both Microsoft and other companies could have written to. That way applications could target the 'standard HTML rendering API', and use IE or Mozilla as the engine. They could target the 'standard media playing API' and choose either WMP or Real.
Removing these portions will severely affect third-party developers. Now, a zoom player download is increased from a couple of megs to well over 20. Genius.
As for downloads, there is nothing to stop vendors supplying multiple media players on CD/DVD ready to install in the PC. This is the way it used to be with browsers.
The problem is that, as more and more power is put into the GUI or the database, the application gets squeezed down to just bare essentials. For those people that standardize on a particular GUI or database, this is a good thing.
Surely the majority of applications are now moving to very thin-client or web-based GUIs. Almost all the code now sits on a server, in 'middleware'. Personally, I last wrote a client-side GUI years ago, and now everything I do is web-based.
Recreating existing functionality outside of the database is even slower. It doesn't matter how "modern" it is.
This is not recreating functionality outside of the database. This kind of Java functionality can be run within an embedded app server within the database engine (as is available from Oracle). There is no network overhead, because there is no network use.
If you looked at the way current object-relational (OR) systems work you will see that there is often minimal activity outside of the database. The OR system uses highly-tuned SQL that makes good use of DB-specific functions, but this is transparent to the developer. What you do with Java object instances is analysed by the OR system and batched into often just a few SQL statements - it is not replicating functionality - just an alternate, more portable way of expressing it.
Congratulations on making the transition from programmer to pointy haired buzz word compliant salesman:)
I'm not a salesman, I'm a developer. Everything I am talking about is based on years of actual practical experience on past or current development projects. I am the last one to listen to salesmen - I am after proven high-performance strategies. I have benchmarked my uses of OR systems as compared with embedded SQL and have found the differences to be insignificant in most cases for real applications.
The dinosaurs took over a million years to die out.
There is no evidence for such a long die-out. The latest evidence suggests that it was practically instantaneous, which fits very well with an asteroid impact.
Is there really an advantage to running Hibernate on the java side of things when the JVM is only running to run JDBC, on the same box as the RDBMS?
Absolutely - portability, and speed of coding. I find things far, far faster to code in terms of an object model than in terms of direct access to the DB.
One would think that the RDBMS could make better caching decisions about data [locality of reference] than the application layer -- and you'd be looking at two caches now, instead of one. My usual goal is to have enough RAM around for the RDBMS to rarely need to hit the disk under normal application load. Hibernate has one concrete disadvantage from where I sit -- reports run against the RDBMS engine (i.e. SQL*Plus command line) are not always up-to-date, which also makes me nervous about crash recovery.
This is a good point, and the answer is to turn off all Hibernate cacheing, and let the RDBMS do the work. I don't use Hibernate - I prefer standard APIs, so I use a JDO product, but the principle should be exactly the same. I prefer JDO as I find that specific JDO products have very detailed cache controls.
I'm a bit concerned about things not being up-to-date - at the point of committing work (transaction.commit()) the database should definitely be up-to-date! The answer may be to use turn off any write cacheing.
I'd suggest you take a look at JDO - its very similar to Hibernate - almost identical in terms of the coding, but there are multiple vendors who compete to provide products, so you get good quality (not that Hibernate isn't good). They are highly optimised for commercial databases like Oracle. There are, of course, open source free versions such as JPOX.
And that's a fine argument as long as you don't care about performance.
If on the other hand you have real work to be done, a project that has to be delivered, revenue at stake, and a defined environment then things may not be quite so simple neat and clean.
Java Data Objects is a widely used system that is used for critical applications by large commercial companies. JDO is a standard and there are commercial implementations that compete to give the best performance and scalability and ease of use.
One of the benefits of a good JDO product is that is effectively an expert system which knows in detail how to produce optimised SQL for the underlying database. It will use, for example, specific Oracle features or specific MySQL features to gain performance benefits, while maintaining transactional safety and data integrity, all with no effort from the coder.
There is a lot of skepticism about the performance and practicality of object-relational mappings, but usually from people who have never used a modern product. They really do work, they are simple and effective. There may be the occasional need to drop down to DB-specific SQL or to call legacy SQL stored procedures, but almost all of your code can be portable, and that is a huge business advantage, as it saves both on current development time and the inevitable future migration costs.
I was just having this conversation with someone who felt just like you and developed many applications at a large company here in town. In his words "everyone will talk to my app, which will talk to the database". I then said what about those Microsoft Access and Toad users that are now starting to hit your system? He turned white... well more white:-) He didn't even have any constraints on the system.:-)
You make good points. I hope I can answer them!
Firstly, of course you need constraints and other rules for data integrity at the level of the database. I have seen major problems where developers have not done this, and relied only on the O/R mapping. Fortunately, my JDO product sets all these up for me.
Secondly, I think we are talking about different things when we talk about 'logic'. I am talking about data processing and calculations. I have no objection to simple triggers and other processes being in the database in a way that it makes it suitable for access by other methods. What I am really against is having tens of thousands of lines of PL/SQL code embedded in the database.
Thirdly, there are simple ways to talk to the Java (if that is what is on the server): CORBA is a standard for remote access to functionality that even Microsoft clients can use. Of course, these days there is SOAP, which can painlessly link Java and.Net. If someone wanted to access more than just the underlying tables and views, I would give them the WSDL describing my SOAP services. I can even use the Java ActiveX Bridge to provide functions to the MS Access users!
I think you make good arguments, but they are mostly out of date.
The practical difference between "write-once-run-anywhere" and "relatively easy to port" is not an overwhelming issue on a bespoke project.
This is a serious mistake that has been made in major projects. I just don't believe in 'relatively easy to port'. I have never seen this work in practise, and when it has failed, it has been disastrous.
Point 1: M:M application:database relationship
The database is not your personal persistent object store. Other people have to use the data.
This is irrelevant. I am responsible for my use of that store. The issue of whether or not I use an O/R mapping has no bearing on how others use the data.
Point 2: You still need to regression test
Even if the application is write-once-run-anywhere portable, you still have to devise, maintain and find the warm bodies to execute a comprehensive regression test.
I fail to see how this is an argument for not having write-once-run-anywhere portability for the software. Testing can be done very effectively with test harnesses and frameworks at the Java level (with systems such as JUnit/Ant). This works fine no matter what the underlying relational store.
Point 3: Java is mostly used for bespoke development
You seem to be making a spurious connection between portability and bespoke software. I write bespoke software, and the ability to have it potentially ported between different operating systems and databases is vital. Bespoke software is expected to last for years or even decades. Having such software tied in to one operating system or one relational store is extremely unwise and potentially disatrous over such a timescale. These days, portability has to be a primary concern.
I am currently involved in a long-term project to port legacy bespoke DOS software to something more modern.
Point 4: O-R mappers are not a substitute for competent developers
If you're developing a database app without reasonable SQL skills on the team you've got bigger problems than portability.
You are right. But competent SQL developers are not a substitute for the code reduction and efficiency obtained by the use of quality O-R mappers as well.
The notion of abstracting away data access into a black box is utopian.
No it isn't. In real, large scale applications this is exactly what is being done.
Data access performance is actually subject to mechanical constraints of disk performance.
That is a matter of fine-tuning the underlying database, is should have no relevance to the Java developer. Nothing about the use of a decent O/R abstraction prevents a good DBA from fine-tuning the underlying system.
For this reason, database independence is just as big a myth as network transparency. Hands up those of us who design EJB components without regard to network round trips. You cannot ignore what the database is doing behind the scenes.
Me. I don't use EJB. I use JDO. It is far more lightweight and flexible. EJB is well known for its performance problems.
An application using POJO's and a reasonably competent data layer gives you flexibility to use database features (and drop out into stored procedures where this is sensible to do). Most work can be done with generic table/record classes. See Fowler's POEA book for deeper discussions of such architectures.
You have just described much of JDO.
Lightweight data access libraries get you most of the practical benefits (reduced need for boilerplate code etc.) that a heavyweight O-R mapping layer does. They also give us more control over this part of our application.
I think you may have been using rather old-fashioned heavyweight O-R mappings rather than the far more lightweight systems (JDO/Hibernate) that are coming into widespread use today. It sounds like you have co
If you cared about licensing issues, you wouldn't be using Oracle. Oracle know this. Therefore they know you're bluffing and will ignore you.
Their customers are self-selected not to care about money or licensing restrictions.
This isn't true. I am an Oracle customer. I use Oracle because of its feature set, and because I have been using Oracle for nearly 20 years and understand it's advantages. Now I am in the process of migrating some of my databases to PostgreSQL because version 8.0 has the features I need for some applications.
If it's written in ANSI SQL then it *can* move from one DB to another without much pain. The upside to using all SQL is that in most cases (not always, I admit) it is MUCH faster and more efficient overall to let the RDBMS manage the data instead of shuffling it off to an app to do the work.
No, it really isn't that much faster. Code embedded in an app server in the database engine (such as that provided with Oracle) can run at very high speed. The idea that all data has to be transferred back and forth between an app in this way is a myth. Good object/relational systems simply don't work that way.
The problem arises when the 'SQL addicts' in question are using platform dependent bonus commands which would certainly cause a problem. Just like all those annoying business website that require MS Explorer because they use special MS extensions.
The problem is that it is not bonus commands that are the problem. With databases such as PostgreSQL and Oracle there are absolutely core differences - porting PL/SQL (Oracle) to pgSQL (PostgreSQL) is a major task. The differences between these and MySQL is huge.
Wouldn't it be nice if either Sun or Microsoft finally got a clue and added persistence to their platform so the whole SQL crap could be left in the garbage dump?
They are. Sun already has JDO as a standard extension to Java, and Microsoft are soon to release a standard extension to.Net for the same purpose.
It is completely illogical to use a database back end with a different data model to the programming language.
Not necessarily. The object model can map reasonably well to the relational model.
If entity relational is the way to go, then build Java arround it. Since its not build a back end data store that works.
The JDO standard is intended to work with more than just relational - it is general purpose.
I am certainly not advocating anything as primitive as sticking to a basic SQL dialect and a subset of database features - that is a waste of the database.
Decaff, I think you would be shocked to see how much of that goes on out there....
I'm not surprised at all, and I can see how it would give portability a very bad name! But, it really doesn't have to be like that. There are very high quality object/relational mapping products for Java and.Net that really do make best use of the underlying DB.
It seems the main problem was that network latency to transfer small bits of data was taking up a quite noticeable amount of time.
This is exactly why I use Java + JDO. My code can be run either on an app server on the same machine as the database, or even in an app server that is part of the database product. There is no network latency. This is the 'middleware' principle.
but there are indeed cases where everything should go in the database.
I would say that in some cases, some small sections of logic may need to be in the database, but I would disagree that there is any situation where everything should go there - there is simply no need.
Saying your not going to take advantage of a databases features is like someone programming in java saying they aren't going to use netbeans cause it won't be as easy to port the code. Ok, probably a bad example but i'm not a developer.
These days, developing portable code does not mean you aren't using the database optimally. For example, I use a Java Data Objects product that has detailed knowledge of certain databases, including Oracle, MySQL, PostgreSQL, SQL Server. My portable code is translated into very optimised SQL that makes good use of the specific database (if there is a way to do a particular action very fast in Oracle, the product will use it).
I am certainly not advocating anything as primitive as sticking to a basic SQL dialect and a subset of database features - that is a waste of the database.
I think it's really a trade off between performance and portability. Sending a message to the DB to transform some data is faster than pulling it out, operating on it and doing an update. You save a netowrk trip and marshaling and unmarshaling your data into objects.
This is true in theory, but I have found it not to be the case in practise, for me. The portable code need not be on a separate machine from the database - it may even run within an embedded app server in the DB engine. There is then no network trip. If you examine the technical details of a technology such as Java Data Objects, you will see that there is actually very little marshalling and unmarshalling. The manipulation of object instances in Java often simply acts as a proxy for database operations - there may not be a need for any data transfer in some cases, simply the generation of SQL.
Whereas for my part I am absolutely sick of dealing with software that does not perform well on ANY platform and cannot be moved rapidly to a new technology.
Me too. That is why I use Java+JDO, and not DB-specific SQL.
Too bad they don't support the neato language where we put the business logic.
Good point. Show me a platform that does not support Java. I would rather have the logic there than in some neato DB language that has to be ported, at great expense.
whereas if the business logic had been stored in the database, reimplemention would be a few weeks work.
A few weeks work? Have you actually worked on such a re-implementation? This is nonsense. A moderate project can take months, and a large scale project years, especially on a live system. I know this from personal experience.
and the result will be 2 systems each faster, more scalable, and more secure than your portable system.
This is simply a statement with no foundation.
There are no security, scalability or speed issues with the system I use - JDO. It is designed to be secure and scalable, to work at high performance on clustered systems and to generate optimal SQL for each version of major databases. Large corporations use it for this purpose.
Which doesn't even touch on the topic of data integrity...
Why should the matter of data integrity be relevant? Systems like JDO and Hibernate and Toplink fully support all aspects of transactions, clustering and cache management. Data integrity is, of course, not an issue. If it were, these products would not be so widely and successfully used in critical projects.
I wish jdbc/odbc had been around in 1997 (on BSDI and Solaris). I'm thinking that maybe I should replace the client interface guts with an odbc abstraction.. Hmm.. I wonder what the performance hit would be?
I have found that in practice there is almost no performance hit, if you are running the Java process on the same machine as the database engine, then most logic placed within Java runs about the same speed as PL/SQL in Oracle.
If you use a quality JDO product, or a good alternative like Hibernate, then that product produces very efficient SQL for your database.
If you have a "high-volume high-turnover commercial project" and you can't cheap-out with PostgreSQL, then pay Oracle its damn money and get on with your business.
In a few years, using PostgreSQL won't be the cheap option, any more than Linux is now the cheap option. There is no point using Oracle if PostgreSQL has all the features and is reliable, and you can get support for it.
Oracle's stubborness says, time to start looking at DB2.
Absolutely. But how many can easily switch?
For a long time I have had (occasionally heated) arguments with SQL addicts who insist that almost everything about an application should be coded in SQL and stored procedures. Meanwhile I have been moving all my logic away from the database engine, using APIs such as Java Data Objects, which makes my code very rapidly portable between databases. Now I am in a position to switch my code (and data) easily between different database vendors if there is a licensing or price issue.
I strongly believe we should start to think of databases simply as engines for storing and retrieving inter-related objects and not as platforms for writing applications.
XML has a huge share of warts. I'll go so far to say that it sucks
I really don't understand this sort of comment. XML is based on years of research into how to add meaningful markup to textual information in a way that is easy for software to parse and process. It is also designed for internationalisation and extensibility.
Saying things like "hardware is cheap" leads to the performance problems of today. It's extremely bad thinking by a developer and leads to bad code.
The is completely the wrong attitude. Performance problems should be the last worry of a developer. Other concerns like 'does it work right?' and 'Can others make use of this software?' and 'Will this software go out of date because no-one can understand the format of my text files?' are FAR more important than raw speed.
There are plenty of areas where text files are fine but XML is not.
Slashdot Mirror
Gee, funny. As a developer, I could've sworn that it was so that the libraries each program used would be accessable to any program installed on Windows. God knows I'm thankful I can link to IE's HTTP transfer engine, rather than having to either code my own or getting a third party library. Or when I have to show a video, I can link WMP's libraries rather than having to code my own video rendering routines...
Of course you can do this. But, there is no reason why the libraries you use should be bundled with Windows, or should not conform to a common standard so that other companies can provide the same service. Why should there be a problem with a third party driver if it conformed to the same API as the Microsoft HTTP transfer engine?
When you use DirectX on Windows, you are using a video driver written to a common specification. Use of third-party DirectX drivers is common, and largely transparent to the developer. There is no reason why MS should not have devised a web-engine specification (say WebX). You could have then used the either Microsoft IE WebX driver, or the Mozilla WebX driver or the Opera WebX driver. Same with the media player.
But nope. Clearly there's no technical reason why this was done. It's all because Microsoft is evil and monopolistic. Never mind that Konquerer in KDE is quite similar in integration as IE is to Windows, for similar reasons...
You are completely missing the point (Several points, in fact):
1. KDE is not in the position of being a monopoly.
2. KDE has not attempted to influence Linux distributors not to supply alternatives.
3. The KDE services have a published API and are open source - competitors could write an alternative to Konqueror and the rendering components which would work the same for developers.
They became services because it made sense for them to become services. Applications need to be able to render HTML and access web content - applications need to be able to play and work with media files.
... while this sounds nice in theory it would take many many years to get everyone to agree on such an API
And on other operating systems they do this. They use libraries which are shared between applications, and there can be alternative suppliers of the libraries.
Making an API that other media players could implement
Windows versions take years to appear - plenty of time to do this.
and in the meantime everyone would need to include whatever multimedia app their app builds on top of with every little shareware program - or they would need to write the code themselves.
But this works fine on other platforms.
Furthermore the bug-finger-pointing that would go on if there were many different browsers implementing the IE APIs or different media players implementing the WMP APIs would be a nightmare for consumers.
Absolutely not. This creates a market for the best quality provider of the API. In the same way as there is finally an increasing market for the browser, there could be an increasing market for the browser engine or the media service.
You could also argue for defining an API for filesystems (which does exist) and allowing different vendors to ship with different filesystems should be a government mandated requirement. Of course when the drive gets corrupted who is going to take the blame?
The vendor who has shipped the driver, in the same way as the vendor who provides the computer hardware or hard disk or network card gets the blame.
When someone calls MS are they going to have to debug a 3rd party filesystem?
No, they call the driver provider. Its like the provider of a COM service or ActiveX control.
Who cares if these APIs are bundled with the operating system?
The end user, who gets poor quality bundled products with the OS, and the developer who wants to provide a better alternative, but has to get over the barrier of the default bundled service being... already there.
It makes it easier for other developers to write applications on the platform
But only by tying in the developer. As I said, having a standard for the API would not raise this problem for the developer.
- and it sets a bar beyond which 3rd party equivalents must pass before they are interesting.
A bar which is unfair because it is imposed by a monopoly company. There should not be this bar. It is illegal.
In a similar way WordPad sets a bar (albeit a very low one) beyond which all word processors must pass before they become interesting to end users.
Well, this example is fine. The problem with IE and WMP is as if Microsoft bundled Word or Excel with Windows.
It is fine for Microsoft to provide products such as WMP which provide Windows services which make life easier for users and developers. What is illegal is for them to bundle these products with Windows raising a huge bar to adoption of alternatives. It is illegal because MS is using its monopoly on desktop operating systems to try and establish a monopoly in other areas (browsers and media players). It is trying to do this by falsely re-defining the term 'operating system' to include these areas.
What issues do people have with WMP anyway? Don't people realise that WMP and IE are not "applications" but more like "services"?
The point is that they WERE applications. They have been transformed into services for business reasons: in order to crowd out alternative application providers. In a monopoly, that is illegal.
There was another way things could have happened. Microsoft could worked with suppliers of applications in order to develop an API for these services that both Microsoft and other companies could have written to. That way applications could target the 'standard HTML rendering API', and use IE or Mozilla as the engine. They could target the 'standard media playing API' and choose either WMP or Real.
Removing these portions will severely affect third-party developers. Now, a zoom player download is increased from a couple of megs to well over 20. Genius.
As for downloads, there is nothing to stop vendors supplying multiple media players on CD/DVD ready to install in the PC. This is the way it used to be with browsers.
The problem is that, as more and more power is put into the GUI or the database, the application gets squeezed down to just bare essentials. For those people that standardize on a particular GUI or database, this is a good thing.
Surely the majority of applications are now moving to very thin-client or web-based GUIs. Almost all the code now sits on a server, in 'middleware'. Personally, I last wrote a client-side GUI years ago, and now everything I do is web-based.
Java is slow.
:)
Nonsense. Java can match C++ speed.
Recreating existing functionality outside of the database is even slower. It doesn't matter how "modern" it is.
This is not recreating functionality outside of the database. This kind of Java functionality can be run within an embedded app server within the database engine (as is available from Oracle). There is no network overhead, because there is no network use.
If you looked at the way current object-relational (OR) systems work you will see that there is often minimal activity outside of the database. The OR system uses highly-tuned SQL that makes good use of DB-specific functions, but this is transparent to the developer. What you do with Java object instances is analysed by the OR system and batched into often just a few SQL statements - it is not replicating functionality - just an alternate, more portable way of expressing it.
Congratulations on making the transition from programmer to pointy haired buzz word compliant salesman
I'm not a salesman, I'm a developer. Everything I am talking about is based on years of actual practical experience on past or current development projects. I am the last one to listen to salesmen - I am after proven high-performance strategies. I have benchmarked my uses of OR systems as compared with embedded SQL and have found the differences to be insignificant in most cases for real applications.
The dinosaurs took over a million years to die out.
There is no evidence for such a long die-out. The latest evidence suggests that it was practically instantaneous, which fits very well with an asteroid impact.
Is there really an advantage to running Hibernate on the java side of things when the JVM is only running to run JDBC, on the same box as the RDBMS?
Absolutely - portability, and speed of coding. I find things far, far faster to code in terms of an object model than in terms of direct access to the DB.
One would think that the RDBMS could make better caching decisions about data [locality of reference] than the application layer -- and you'd be looking at two caches now, instead of one. My usual goal is to have enough RAM around for the RDBMS to rarely need to hit the disk under normal application load. Hibernate has one concrete disadvantage from where I sit -- reports run against the RDBMS engine (i.e. SQL*Plus command line) are not always up-to-date, which also makes me nervous about crash recovery.
This is a good point, and the answer is to turn off all Hibernate cacheing, and let the RDBMS do the work. I don't use Hibernate - I prefer standard APIs, so I use a JDO product, but the principle should be exactly the same. I prefer JDO as I find that specific JDO products have very detailed cache controls.
I'm a bit concerned about things not being up-to-date - at the point of committing work (transaction.commit()) the database should definitely be up-to-date! The answer may be to use turn off any write cacheing.
I'd suggest you take a look at JDO - its very similar to Hibernate - almost identical in terms of the coding, but there are multiple vendors who compete to provide products, so you get good quality (not that Hibernate isn't good). They are highly optimised for commercial databases like Oracle. There are, of course, open source free versions such as JPOX.
And that's a fine argument as long as you don't care about performance.
If on the other hand you have real work to be done, a project that has to be delivered, revenue at stake, and a defined environment then things may not be quite so simple neat and clean.
Java Data Objects is a widely used system that is used for critical applications by large commercial companies. JDO is a standard and there are commercial implementations that compete to give the best performance and scalability and ease of use.
One of the benefits of a good JDO product is that is effectively an expert system which knows in detail how to produce optimised SQL for the underlying database. It will use, for example, specific Oracle features or specific MySQL features to gain performance benefits, while maintaining transactional safety and data integrity, all with no effort from the coder.
There is a lot of skepticism about the performance and practicality of object-relational mappings, but usually from people who have never used a modern product. They really do work, they are simple and effective. There may be the occasional need to drop down to DB-specific SQL or to call legacy SQL stored procedures, but almost all of your code can be portable, and that is a huge business advantage, as it saves both on current development time and the inevitable future migration costs.
I was just having this conversation with someone who felt just like you and developed many applications at a large company here in town. In his words "everyone will talk to my app, which will talk to the database". I then said what about those Microsoft Access and Toad users that are now starting to hit your system? He turned white... well more white :-) He didn't even have any constraints on the system. :-)
.Net. If someone wanted to access more than just the underlying tables and views, I would give them the WSDL describing my SOAP services. I can even use the Java ActiveX Bridge to provide functions to the MS Access users!
You make good points. I hope I can answer them!
Firstly, of course you need constraints and other rules for data integrity at the level of the database. I have seen major problems where developers have not done this, and relied only on the O/R mapping. Fortunately, my JDO product sets all these up for me.
Secondly, I think we are talking about different things when we talk about 'logic'. I am talking about data processing and calculations. I have no objection to simple triggers and other processes being in the database in a way that it makes it suitable for access by other methods. What I am really against is having tens of thousands of lines of PL/SQL code embedded in the database.
Thirdly, there are simple ways to talk to the Java (if that is what is on the server): CORBA is a standard for remote access to functionality that even Microsoft clients can use. Of course, these days there is SOAP, which can painlessly link Java and
I think you make good arguments, but they are mostly out of date.
The practical difference between "write-once-run-anywhere" and "relatively easy to port" is not an overwhelming issue on a bespoke project.
This is a serious mistake that has been made in major projects. I just don't believe in 'relatively easy to port'. I have never seen this work in practise, and when it has failed, it has been disastrous.
Point 1: M:M application:database relationship
The database is not your personal persistent object store. Other people have to use the data.
This is irrelevant. I am responsible for my use of that store. The issue of whether or not I use an O/R mapping has no bearing on how others use the data.
Point 2: You still need to regression test
Even if the application is write-once-run-anywhere portable, you still have to devise, maintain and find the warm bodies to execute a comprehensive regression test.
I fail to see how this is an argument for not having write-once-run-anywhere portability for the software. Testing can be done very effectively with test harnesses and frameworks at the Java level (with systems such as JUnit/Ant). This works fine no matter what the underlying relational store.
Point 3: Java is mostly used for bespoke development
You seem to be making a spurious connection between portability and bespoke software. I write bespoke software, and the ability to have it potentially ported between different operating systems and databases is vital. Bespoke software is expected to last for years or even decades. Having such software tied in to one operating system or one relational store is extremely unwise and potentially disatrous over such a timescale. These days, portability has to be a primary concern.
I am currently involved in a long-term project to port legacy bespoke DOS software to something more modern.
Point 4: O-R mappers are not a substitute for competent developers
If you're developing a database app without reasonable SQL skills on the team you've got bigger problems than portability.
You are right. But competent SQL developers are not a substitute for the code reduction and efficiency obtained by the use of quality O-R mappers as well.
The notion of abstracting away data access into a black box is utopian.
No it isn't. In real, large scale applications this is exactly what is being done.
Data access performance is actually subject to mechanical constraints of disk performance.
That is a matter of fine-tuning the underlying database, is should have no relevance to the Java developer. Nothing about the use of a decent O/R abstraction prevents a good DBA from fine-tuning the underlying system.
For this reason, database independence is just as big a myth as network transparency. Hands up those of us who design EJB components without regard to network round trips. You cannot ignore what the database is doing behind the scenes.
Me. I don't use EJB. I use JDO. It is far more lightweight and flexible. EJB is well known for its performance problems.
An application using POJO's and a reasonably competent data layer gives you flexibility to use database features (and drop out into stored procedures where this is sensible to do). Most work can be done with generic table/record classes. See Fowler's POEA book for deeper discussions of such architectures.
You have just described much of JDO.
Lightweight data access libraries get you most of the practical benefits (reduced need for boilerplate code etc.) that a heavyweight O-R mapping layer does. They also give us more control over this part of our application.
I think you may have been using rather old-fashioned heavyweight O-R mappings rather than the far more lightweight systems (JDO/Hibernate) that are coming into widespread use today. It sounds like you have co
If you cared about licensing issues, you wouldn't be using Oracle. Oracle know this. Therefore they know you're bluffing and will ignore you.
Their customers are self-selected not to care about money or licensing restrictions.
This isn't true. I am an Oracle customer. I use Oracle because of its feature set, and because I have been using Oracle for nearly 20 years and understand it's advantages. Now I am in the process of migrating some of my databases to PostgreSQL because version 8.0 has the features I need for some applications.
If it's written in ANSI SQL then it *can* move from one DB to another without much pain. The upside to using all SQL is that in most cases (not always, I admit) it is MUCH faster and more efficient overall to let the RDBMS manage the data instead of shuffling it off to an app to do the work.
No, it really isn't that much faster. Code embedded in an app server in the database engine (such as that provided with Oracle) can run at very high speed. The idea that all data has to be transferred back and forth between an app in this way is a myth. Good object/relational systems simply don't work that way.
The problem arises when the 'SQL addicts' in question are using platform dependent bonus commands which would certainly cause a problem. Just like all those annoying business website that require MS Explorer because they use special MS extensions.
The problem is that it is not bonus commands that are the problem. With databases such as PostgreSQL and Oracle there are absolutely core differences - porting PL/SQL (Oracle) to pgSQL (PostgreSQL) is a major task. The differences between these and MySQL is huge.
Wouldn't it be nice if either Sun or Microsoft finally got a clue and added persistence to their platform so the whole SQL crap could be left in the garbage dump?
.Net for the same purpose.
They are. Sun already has JDO as a standard extension to Java, and Microsoft are soon to release a standard extension to
It is completely illogical to use a database back end with a different data model to the programming language.
Not necessarily. The object model can map reasonably well to the relational model.
If entity relational is the way to go, then build Java arround it. Since its not build a back end data store that works.
The JDO standard is intended to work with more than just relational - it is general purpose.
Exactly what JDO implementation do you use? You sound like you've had better luck with your implementation than we have.
Versant OpenAccess. I have also heard very good things about Xcalia.
I am certainly not advocating anything as primitive as sticking to a basic SQL dialect and a subset of database features - that is a waste of the database.
.Net that really do make best use of the underlying DB.
Decaff, I think you would be shocked to see how much of that goes on out there....
I'm not surprised at all, and I can see how it would give portability a very bad name! But, it really doesn't have to be like that. There are very high quality object/relational mapping products for Java and
It seems the main problem was that network latency to transfer small bits of data was taking up a quite noticeable amount of time.
This is exactly why I use Java + JDO. My code can be run either on an app server on the same machine as the database, or even in an app server that is part of the database product. There is no network latency. This is the 'middleware' principle.
but there are indeed cases where everything should go in the database.
I would say that in some cases, some small sections of logic may need to be in the database, but I would disagree that there is any situation where everything should go there - there is simply no need.
Saying your not going to take advantage of a databases features is like someone programming in java saying they aren't going to use netbeans cause it won't be as easy to port the code. Ok, probably a bad example but i'm not a developer.
These days, developing portable code does not mean you aren't using the database optimally. For example, I use a Java Data Objects product that has detailed knowledge of certain databases, including Oracle, MySQL, PostgreSQL, SQL Server. My portable code is translated into very optimised SQL that makes good use of the specific database (if there is a way to do a particular action very fast in Oracle, the product will use it).
I am certainly not advocating anything as primitive as sticking to a basic SQL dialect and a subset of database features - that is a waste of the database.
I think it's really a trade off between performance and portability. Sending a message to the DB to transform some data is faster than pulling it out, operating on it and doing an update. You save a netowrk trip and marshaling and unmarshaling your data into objects.
This is true in theory, but I have found it not to be the case in practise, for me. The portable code need not be on a separate machine from the database - it may even run within an embedded app server in the DB engine. There is then no network trip. If you examine the technical details of a technology such as Java Data Objects, you will see that there is actually very little marshalling and unmarshalling. The manipulation of object instances in Java often simply acts as a proxy for database operations - there may not be a need for any data transfer in some cases, simply the generation of SQL.
Whereas for my part I am absolutely sick of dealing with software that does not perform well on ANY platform and cannot be moved rapidly to a new technology.
Me too. That is why I use Java+JDO, and not DB-specific SQL.
Too bad they don't support the neato language where we put the business logic.
Good point. Show me a platform that does not support Java. I would rather have the logic there than in some neato DB language that has to be ported, at great expense.
whereas if the business logic had been stored in the database, reimplemention would be a few weeks work.
A few weeks work? Have you actually worked on such a re-implementation? This is nonsense. A moderate project can take months, and a large scale project years, especially on a live system. I know this from personal experience.
and the result will be 2 systems each faster, more scalable, and more secure than your portable system.
This is simply a statement with no foundation.
There are no security, scalability or speed issues with the system I use - JDO. It is designed to be secure and scalable, to work at high performance on clustered systems and to generate optimal SQL for each version of major databases. Large corporations use it for this purpose.
Which doesn't even touch on the topic of data integrity...
Why should the matter of data integrity be relevant? Systems like JDO and Hibernate and Toplink fully support all aspects of transactions, clustering and cache management. Data integrity is, of course, not an issue. If it were, these products would not be so widely and successfully used in critical projects.
I wish jdbc/odbc had been around in 1997 (on BSDI and Solaris). I'm thinking that maybe I should replace the client interface guts with an odbc abstraction.. Hmm.. I wonder what the performance hit would be?
I have found that in practice there is almost no performance hit, if you are running the Java process on the same machine as the database engine, then most logic placed within Java runs about the same speed as PL/SQL in Oracle.
If you use a quality JDO product, or a good alternative like Hibernate, then that product produces very efficient SQL for your database.
If you have a "high-volume high-turnover commercial project" and you can't cheap-out with PostgreSQL, then pay Oracle its damn money and get on with your business.
In a few years, using PostgreSQL won't be the cheap option, any more than Linux is now the cheap option. There is no point using Oracle if PostgreSQL has all the features and is reliable, and you can get support for it.
PostgreSQL is coming along nicely...
It certainly is, and its the database I use for almost all situations, except...
Would you trust PostgreSQL for a high-volume high-turnover commercial project? I don't yet. In a couple of years, perhaps.
Oracle's stubborness says, time to start looking at DB2.
Absolutely. But how many can easily switch?
For a long time I have had (occasionally heated) arguments with SQL addicts who insist that almost everything about an application should be coded in SQL and stored procedures. Meanwhile I have been moving all my logic away from the database engine, using APIs such as Java Data Objects, which makes my code very rapidly portable between databases. Now I am in a position to switch my code (and data) easily between different database vendors if there is a licensing or price issue.
I strongly believe we should start to think of databases simply as engines for storing and retrieving inter-related objects and not as platforms for writing applications.
XML has a huge share of warts. I'll go so far to say that it sucks
I really don't understand this sort of comment. XML is based on years of research into how to add meaningful markup to textual information in a way that is easy for software to parse and process. It is also designed for internationalisation and extensibility.
Saying things like "hardware is cheap" leads to the performance problems of today. It's extremely bad thinking by a developer and leads to bad code.
The is completely the wrong attitude. Performance problems should be the last worry of a developer. Other concerns like 'does it work right?' and 'Can others make use of this software?' and 'Will this software go out of date because no-one can understand the format of my text files?' are FAR more important than raw speed.
There are plenty of areas where text files are fine but XML is not.
Name one.