Slashdot Mirror
What is Holding SAP-DB Back?
Derek Neighbors queries: "The current story about MySQL 4.0 has erupted into a Postgres vs. MySQL debate. We at GNU Enterprise, who have used about all Free and Propietary databases, would like to know why exactly people arent using SAP-DB? It clearly is on par with Oracle, is GPL and frankly has an awesome support team in SAP AG. There was a PG vs SAP-DB recently. Someone else mentioned that you can get CDROMs for free. So again the question is 'What exactly is hindering a wider acceptance of SAP-DB in Free/Open Software projects?'"
It clearly is on par with Oracle
I think you'll find one of the main strengths of Oracle is it's REPUTATION. People know they can trust it as its been around for years and 'everybody' uses it.
All this talk about free database servers tends to fly in the face of mission-critical nonstop environments. Personally at our company, I have pooh-poohed the free DB's and rather pay a little more for centralized support and two good DB administrators.
.sig
As much as everyone maligns M$, nobody ever puts in MS SQL in the conversation. (We're a DB2 shop not M$)
beware of the trolls
Oracle does replication and hot standby. SAP-DB doesn't. These are pretty important features in the enterprise. Therefore, SAP-DB is not on a par with Oracle. What do you do if you need to work on your primary database machine and you don't have a standby?
SAP-DB used to be the non mainframe version of Adabas, which was being sold by Software AG. SAP bought it (I think because one of their biggest customers was running on Adabas) and open sourced it.
Well, I just learned only a week ago that SAP released (part of) their DB as open-source. Everyone and their dog know that MySQL and Postgres are free, but I guess SAP's DB being free as well is a fact that is not well known enough.
Another thing I forgot: sapdb is unhackable. If you ever wanted
to see what unmaintainable for normal people code looks like, go no further than to sapdb.
It is incredibly bloated and complex, very crufty internally, written in a weird pascal/C++ mix
with an SAP specific format for the files, has a build system that could be
only described as ununderstandable, no comments.
It is what you would expect from a 20+ years old codebase
I'm glad the SAP Berlin guys understand it (they seem to at least), but I see not much chance to do any changes
on your own. This makes it not very useful as a free software project.
Of course it is still nice that they offer it for free, but for all practical purposes it is like a binary only download. To be fair interbase has some of these problems too, but it has still relatively nicer source than sapdb. mysql is much better in this regard.
... and I've taken a look around for other RDBMSs. Maybe the problem is that it's flying a little under radar.
However, I have had the same question in relation to the open source version of Borland's Interbase, the Interbase fork - Firebird, and the hsql Database Engine.
It seems to me that the community has latched on to MySQL and PostgreSQL as -the- database solutions, and this very acceptance places them higher up the food chain. For instance, hunt around for an open source based Content Managemnet Sysytem (ala SlashCode or PostNuke), and almost invariably it has a MySQL backend.
If it's so great, why does SAP normally sit atop a different database, like Oracle or DB2?
Becuase SAP DB doesn't support many of the critical features which are in the larger more respected database systems. The most important feature which is missing is transactions, ie: rollbacks, commits etc etc. At our company, we've had too many half-completed SAP actions messed up to remember! The SAP consultants we hired recommended SAP-DB over Oracle or DB2 because it would save some money, but we regret it now.
Also, SAP-DB doesn't have drivers for some of the more exotic tape systems, unlike DB2 or Oracle. The only tape system we could get working for backups with SAP-DB was an old single drive, manual loader DAT system. I hate to think how many nights I spent manually loading tapes into that thing to try and restore a failed SAP transaction that was messed up by the lack of rollbacks in SAP-DB...go with DB2, Oracle, or even SQL-Server any time. These RDBMS systems have what it takes...SAP-DB is an also-ran.
We are in the process of finishing a large J2EE project with the database end running on SAPDB and we have had no regrets. It runs along smoothly and in fact the only annoyance with it has been a crufty manager application for datamanipulation (for tests), which was remedied by using Access as an ODBC client and a little trouble with the actual creation of a database. The same database has had a 3 months run under load and with the developers hitting it with the weirdest commands and it has only needed one service restart so far. I recommend it.
Plebian: SAP-DB.
BB Man: What is that, I've never heard of it. Why would we use that ratehr (sic) than the industry standard, Oracle?
Plebian: What sort of fucking idiot Boss are you if you've never even heard of SAP for fucks sake? Jesus Christ, why am I even talking with you about this, you're about as clueless as a shit smeared stick!
The PHP comment brings up a good reason why it has not yet taken off. MySQL and other open source DBs have widespread support in applications and more importantly, developer communities. People who are comfortable developing with and even for those packages will continue to develop with and for those packages. As more community resources are available, more people will become comfortable using SAB DB.
http://www.tripuls.de/deut/zg/prod/datenb1.htm
I think that this is what you wanted.
Someone found the original text in a previous discussion
bla
It is...
1. harder to install, with a slightly strange mix of admin tools (combination of old/crufty, and new/experimental)
2. definitely trickier to manage, as you need to learn protocols for setting up, and backing up, databases and their logs, at least. This is true of other RDBMSs of course, but the trend has been toward more self-managing systems.
3. Relies of ODBC as the cli--which is actually fine (eg, compatible with PHP) but still less familiar to Unix/OSS people
4. Still undergoing stabilizing bugfix cycle, seemingly, although I haven't myself ever encountered a problem with it
5. Is, as mentioned, less tolerant of inexpert admins--and more problematic, the error codes are frequently impossible to understand
6. Really is difficult, at present, to hack. In general, the code is VERY challenging to work with (particularly the ugly, custom built build system), although it should be said that the SAP internal developers are steadily improving all aspects of the system, and a time WILL come when external developers can see rewards for their hacking efforts.
Compensating for this is the VERY skilled and responsive SAPDB development team, and a very strong feature set.
Matt
That is not entirely correct. SAP DB is based on what was eventually called ADABAS D, which was a product bought by Software AG and rebranded. The source for this product was the base for SAP DB but it was never a version of ADABAS C.
~~ What's stopping you?
I can't say I've used SAP-DB. However, a quick check of its online documentation reveals that it does NOT do multiversion concurrency control. Oracle does. PostgreSQL does. I believe Interbase/Firebird does. Without it, writing a scalable application is MUCH, MUCH harder because locking keeps getting in the way. Real databases need transactions, but without MVCC, the locking to support them will seriously limit concurrency (and, hence, scalability) in a transactional environment.
If you don't know what MVCC is, read the early chapters in Tom Kyte's book or visit his site. Or read Oracle documentation (search the page for "Data Concurrency and Consistency").
Well, considering PostgreSQL is free, a whole lot less sorry for being picky, but it occurred to that some people might be lost in the irony.
PostgreSQL was never GPL'd. Not even copyleft, but just a plain free software license, can't remember if derived from BSD or MIT X. If one wants copyleft, SAPdb is the only choice now.
Leandro Guimarães Faria Corcete DUTRA
DA, DBA, SysAdmin, Data Modeller
GNU Project, Debian GNU/Lin
Becuase SAP DB doesn't support many of the critical features which are in the larger more respected database systems. The most important feature which is missing is transactions, ie: rollbacks, commits etc etc.
Umm, what are you talking about? Of course SAP DB (nee Adabas) has transactions; it's fully ACID, unlike, say, MySQL.
SAP DB is pretty much equivalent to Oracle 7.3.4 which is to say that it's a solid product for many real-world applications, but lacks many of the features for truly high-end deployment, like clustering, complex replication, guaranteed messaging, etc. I'd take Sapdb over MySQL any day, and probably over Postgres too. Another nice thing about SAP DB is that it can emulate Oracle's system tables, so an Oracle DBA can administer a SAP DB system very easily.
Also, SAP-DB doesn't have drivers for some of the more exotic tape systems, unlike DB2 or Oracle. The only tape system we could get working for backups with SAP-DB was an old single drive, manual loader DAT system.
Again, I'm not sure what you are talking about here. Do you mean you can't get an LSM plugin for SAP DB like you can for Oracle? Because that sort of stuff is really just fluff, you can do anything it can do with your regular storage manager (which may even be Legato) with a few simple scripts. I do agree that SAP DB's backup and recovery is primitive compared with Oracle's RMAN.
More and more scripting languages (PHP, PERL, Ruby, Python, TCL) are working through generalised DB interfaces; there is less and less difference (often none) between backend DB's from an application programmer's PoV.
In some cases the backend DB needn't even be SQL (great news for tiny high-performance web apps), but where the backend DB does stick closely to SQL standards the applications produced with it are more likely to be portable and scalable.
Got time? Spend some of it coding or testing
Does SAP have anything close to Oracle's RMAN? If not, it's not on par with Oracle. It seems like most 'free' databases put backup and recovery on the back-burner and only provide some sort of database dump for backups - which is probably one of the reasons they're not more accepted in high-level professional circles.
I'm trying to not argue, but understand the two reasons you listed.
I'm not against PostgreSQL, Interbase/Firebird, or SAP-DB, I'm just wanting to know why. (I am against MySQL in a medium to large production environment, but we'll wait until 4.1+ materializes.)
First point, "VACUUM" problem. What's wrong with a cron job to do this, nightly? Does it bog the system down too much? Is it unreliable? (I haven't noticed any problems, but my DB is small, right now.) Read about MVCC (I think it's something like multiple version concurrency....) Versioning is better than redo logs, in theory. You need to understand MVCC to understand VACUUM. You can VACUUM live, without killing the DB, and perform hot backups as well, due to MVCC.
Second point, have you tuned it, and are you using PG 7.2+? If you are using older PostgreSQL's (prior to 7.1,) then yes, this is a problem. If you haven't tuned it, there are great docs about tuning. Supposedly, PG 7.1+ is VERY fast at many simultaneous connections, although I know of no 1000+ simultaneous connection benchmarks in existence. (Do you believe benchmarks, anyway?)
If you have tried a newer PG (7.2+,) and have followed all recommendations/tuning guides and still have had these problems, I'd really like to know.
I'm open to PostgreSQL (am using it), Interbase/Firebird, and SAP-DB and want the real scoop on things.
Will PG break under big loads? If so, how big? Does PG have a 24 by 7 problem?
Marketing is of some importance, and the "other guys" have more of it. There are no books on bookstore shelves on SAP-DB. Few web sites "Powered by SAP-DB."
It's quite a different world, with very different build tools, code documented in German, and the likes. It is not something that is easy to hack on.
You can get a tarball, you can get some RPMs that work in some places, but it's not nearly as available as MySQL and PostgreSQL.
Much of the popularity of MySQL stems from there being integrated ISP tools like CPanel that include a DB manager module specifically for MySQL. Similarly, the joint popularity of MySQL and PHP stems from the groups of developers working together closely to ensure that there is good native support for MySQL in PHP.
In contrast, modules for integrating SAP-DB with Perl, Python, PHP, and the like require some degree of effort in "hacking it into place." It's not as simple as "apt-get install python-sapdb sapdb-dbi php-sapdb".
And TOra doesn't include SAP-DB support.
None of these are particularly "technical" matters indicating things that can't work.
It's not a question of "SAP-DB not being an ACID DBMS" (as some idiot claimed in another thread).
It's really largely a question of systemms integration, with a certain amount of "needs more marketing."
If you're not part of the solution, you're part of the precipitate.
PostgreSQL was one of the first kids on the GPL block.
No, it was one of the first on the Open Source block, specifically it has a BSD license. And SAP-DB needs to beat PostgreSQL on several counts for it to be considered:
(a) features, PostgreSQL has them in spades, (b) stability, PostgreSQL is solid, (c) licensing, you can't beat BSD, and most importantly, (d) community, PostgreSQL's user community is just a fanstatic group of fellas.
We used PostgreSQL up to version 7.1.2.
I believe the issue with VACUUM has since been solved in that it no longer holds any table locks.
Regarding scalability, we found that (again, compared to PG 7.1.2) SAP-DB scales better under high load. Of course we tried tuning both as well as we could. We measured transaction throughput and average as well as individual response times while steadily increasing the number of concurrent clients. We did this on a system with plenty of CPU power (quad Xeon). Once we pushed things to over say 150 concurrent connections, SAP-DB showed far less variation in individual response times and better scalability, beyond what PG was able to achieve.
PostgreSQL is a very fast database and I must admit that I haven't tried 7.2. The only reason being that SAP-DB was better than we expected...
Jeroen Boomgaardt
In my experience
- pulling transaction records from a remote database results in a considerably superior solution to that obtainable with any messaging middleware product
- using duplicated "hot-standby" systems is more manageable and efficient than data replication.
So please don't dismiss PostgreSQL and SAP DB on the basis of checkboxes for features you might not want to use.Actually, Oracle *is* fully ACID compliant, *if* you choose to turn on that level of integrity checking. "Out of the box," a default Oracle installation chooses a much lower conflict level for performance reasons, yes. However, you can crank Oracle up (or down) from there. This flexibility of Oracle is part of what gives it such broad market appeal... it takes a Jack-of-all-Trades approach to integrity checking.
The PostgreSQL license is derived from the BSD license, since PostgreSQL was derived from a research project at Berkeley.
This is how it works:
1. Hot backup at 1am Sunday (database is up and running)
2. Nightly backup of archive logs (i.e they contain all the changes to a database.
That will only get you back to the time of your last backup assuming you don't have other realtime copies of your xaction logs and tables. If you're running your db on one big box with box table storage and xaction logs, if some big power surge comes along and crashes the heads of all your drives, all you can do is restore your full and add the xactions up until your last backup. Anything done since that time is gone.
As of Postgres v6.2, time travel is no longer supported.
Q. If it's so great, why does SAP R/3 normally sit atop a different database, like Oracle or DB2?
A. R/3 was originally written for Oracle, so this is a tradition. Up until now, SAP DB hasn't been marketed actively by SAP as not to upset it's many database selling partners. Newer applications are now developed for SAP DB first.
Q. SAP-DB is 20 years old. It has an unmaintainable code base. It is bloated and complex, very crufty internally and written in a weird pascal/C++ mix witha SAP specific format for the files and an uncomprehensible build system.
A. I prefer the word challenging. Parts of it are of course constantly rewritten, so I don't think a single part is actually 20 years old. And I object aliasing '!= make' with 'incomprehensible'. The concepts are actually pretty close to newer build systems like ant. Without it, the criticism would simply read that the make files are incomprehensible.
Q. SAP-DB needs a lot of effort to set up and create a database, sometimes even worse than the magic juju you need to go through with Oracle.
A. Not really true, especially not for a production database. But the entry level documentation could be made better by describing when and how to keep things simple.
Q. Oracle does replication and hot standby. SAP-DB doesn't. These are pretty important features in the enterprise.
A. Replication is not on our agenda (despite the oddly named Replication Manager). Hot standby is currently being implemented.
Q. Not in BSD Ports tree. Probably also goes for Linux, (but the argument there would probably be more "doesn't comes (integrated) with the distribution" If something gets included with distributions, it spreads much faster.
A. True. But our main job is to supply SAP DB for SAP customers. Anything else has to be done by others interested in SAP DB.
Q. harder to install, with a slightly strange mix of admin tools (combination of old/crufty, and new/experimental)
A. Partly a documentation problem. There isn't actually a mix of old and new admin tools. There is a command based API, which is accessible from various programming languages, and there are tools which are implemented on top of this API.
Q. definitely trickier to manage, as you need to learn protocols for setting up, and backing up, databases and their logs, at least. This is true of other RDBMSs of course, but the trend has been toward more self-managing systems.
A. SAP DB is mostly self managing. Some of the tasks were it isn't have severe performance implications (like distribution of data volumes over disks) so simply picking a default is questionable at best.
Q. Relies of ODBC as the cli--which is actually fine (eg, compatible with PHP) but still less familiar to Unix/OSS people
A. There is no standard database API for Unix. But of course anything would be better than a Microsoft API.
Q. Still undergoing stabilizing bugfix cycle, seemingly, although I haven't myself ever encountered a problem with it
A. Insert lame joke about Linux 2.4.
Q. Is, as mentioned, less tolerant of inexpert admins--and more problematic, the error codes are frequently impossible to understand
A. We should probably supply a tool which makes information about the error codes easily accessible from the command line.
Q. Really is difficult, at present, to hack. In general, the code is VERY challenging to work with (particularly the ugly, custom built build system), although it should be said that the SAP internal developers are steadily improving all aspects of the system, and a time WILL come when external developers can see rewards for their hacking efforts.
A. True. Although I still object to the notion that make was presented to mankind on the mount Sinai.
Q. Does SAP have anything close to Oracle's RMAN?
A. SAP DB logs all backups. The DBMGUI can use this log to automate tasks like 'restore from this full backup to this timestamp'. This works also with external backup tools.
Q. Sect Where's the O'Reilly book on SAP-DB?
A. It took some time for PostgreSQL to get books so I guess a SAP DB book is still a year or two away.
Q. Does it have Multi Concurrency Version Control (MVCC)?
A. It is implemented in the undocumented object database part. There are plans to make this also available to the relational database part, but a release date has not been set.
Q. Are there any Free Software success stories of projects using SAP-DB?
A. So far, SAP DB seems to appeal mostly to commercial software vendors for whom Oracle or MS licenses are too expensive/bothersome.
Q. Are there better admin tools?
A. No. Or not yet. But the administration is easily scriptable, so most common tasks can be reduced to a single command.