Evolutionary Database Design

Andre Mermegas writes "Check out this article by everybody's favorite object mentor Martin Fowler on database design. Be sure to take a peek at his wonderful books as well."

Comments

Working Together...

[airrage]

In the projects I've worked, I often find that the DBAs are older men or women and the developers are young. So the friction lies in the fact that the young-guns are doing .NET or Java or XML queing and so the DBA is really at a loss to help "the developer think of things he may have not thought about". Of course, on the table-design side, this maybe true. Secondly, due to the age-difference, "popping over the cube" is also difficult as the DBAs (being more mature shall I say) are less likely to be excited about a new paradigm.

Case in point, when I read in an Oracle PL/SQL book about Nested Tables, the light bulb in my head went off (or lit up, or whatever). Basically, these nested tables were objects with methods (code behind them), however, could be queried like tables. So, instead of selecting say a person's name, birthdate, and calculating an age, I could select name, birthdate, and age (the age column had code behind it automatically calculating the age). Now the beauty of this is for derived quantities that are only used once, but would be burdensome to store, this was a godsend. However, my DBA completely rejected this idea as too untried and new-fangeled.

This may sound very arrogant, but I think the developer should manage the DBA, often the DBA is a lone-wolf with too much power. Often the poor programmer has to submit changes with about as much hope they'll get done as one might have submitting universe changes to God Almighty.

Re:Working Together...

[sql*kitten]

Secondly, due to the age-difference, "popping over the cube" is also difficult as the DBAs (being more mature shall I say) are less likely to be excited about a new paradigm.

I guess you haven't been around the industry too long. You see, this is an industry totally driven by fashions and fads (far more so than even the clothing or entertainment industries). Every year there's a slew of new buzzwords and technologies, each of which promises to be the "silver bullet" and a whole new "paradigm" and none of them ever are. So when some bright-eyed bushed-tailed young hotshot announces that he's discovered the solution to the organization's IT ills, all the "old geezers" just roll their eyes, 'cos they've seen it a dozen times before.

However, my DBA completely rejected this idea as too untried and new-fangeled.

The problem with many developers is that they see a shiny new feature, can't wait to use it, and you end up with an application in which a dozen different people have solved the same problem a dozen different ways.

My attitude is usually that a developer can do anything they want... so long as they're willing to carry a pager that might go off at 3AM, and take responsibility for fixing it before the next business day. Amazing how many times they just wanted to try out a new feature without any real need for it.

In your specific case, you could have done exactly what you wanted to do with a view.

This may sound very arrogant, but I think the developer should manage the DBA, often the DBA is a lone-wolf with too much power. Often the poor programmer has to submit changes with about as much hope they'll get done as one might have submitting universe changes to God Almighty.

Yeah, and the accountants use software, so the developers should manage the accountants! And the salesmen! And the canteen staff! After all, a developer wrote the program that prints their paychecks!

I personally have spent half an hour rewriting a developer's SQL that took the run time down from 15 hours to 9 seconds. Having said that, I don't know all that much about writing, say, MT-safe C++. That's why we have specialists in the first place. I'll bet dollars to donuts that your DBA knows far more about databases than you do, even if you know many more trendy buzzwords than he does.

This article is short and common sense...

[dagg]

But it is great to actually read it. Sometimes common sense things need to be written down just to verify that your techniques really do make sense. There are so many great little tidbits in the article, I'm having trouble picking one out to really comment on. Here's one:

An important part of this approach is iterative development, where you run the entire software life-cycle many times during the life of a project. Agile processes run complete life cycles in each iteration, completing the iteration with working, tested, integrated code for a small subset of the requirements of the final product. These iterations are short, usually running between a week and a couple of months, with a preference towards shorter iterations.

A big issue with iterative development is that the QA folks will quickly fall behind and become very anxious. What's the solution to that? Either embrace the QA person to get closer to the real development environment, or if that is impossible, get a new QA person. That's the only way to succeed.

The big picture

[oliverthered]

While I believe that 'Agile processes' are the best way to develop software, he appears to be advocating anarchy.

When you have a lot of people all making little changed everyone starts to loose sight of the Big picture and you run into a Too many cooks spoil the broth.

I'm sure a lot of people who read this site have seen a lot of code and design, and probably a lot of horrific code and design, well enough said.

Quality assurance

[giel]

Just like the actual users the QA folks should be heavily involved in the actual development cycles. Extreme programming states that every development cycles starts with a functional design and the development of tests for each deliverable. Having these available means that QA is able to keep track of the quality of the deliverables very well.

IMHO if QA cannot keep track of the big picture they fail as QA, because that is just an important part of their job. On the other hand perhaps extreme programming should involve relatively more QA people than 'regular' development methods.

Works for OODB as well.

[bokmann]

I work on a project of about a dozen developers, some os us geographically diverse. We use an Object-Oriented Database with Java (Database is from Versant).

We don't worry about *any* kind of DB administrator. Each developer has their own instance of the database. We don't worry about schema changes that break the database, because we *also* have a way to import/export the database to an XML file. Thus, if the schema radically changes from the deployed version, we just export to XML and re-import, so there is no complex though about 'schema evolution' necessary,

Of course, with everyone having such free ability to make changes that impacts the format of the data store, we need good unit tests to make sure things don't break unintentionally. This is actually one area we need to improve upon. People can make changes that affect the schema easily, and most times, its not an issue... but a lot of times people make changes that would impact the XML format, and they don't always handle it properly.

Unit Tests are Key, but that's nothing new to the concept of refactoring.