Slashdot Mirror
Reuse Code Or Code It Yourself?
eldavojohn writes "I began coding for a project that had simple requirements for my employer — Web services and a test application for them. But requirements have been creeping, as they always do. Initially I had decided to use the Spring Framework with Hibernate. And I re-used a lot of libraries that made things simple and quick for me. The new requests coming in involve capabilities beyond those of the frameworks. Now, I used to be told that good programmers write code and great programmers reuse code. It's starting to look like I would have saved myself a whole lot of time if I had written the database transaction using JDBC instead of Hibernate — now that I'm married to this object model framework, some of this stuff doesn't look doable. So what is better for the majority of software projects out there: reuse code, or code from scratch? What elements or characteristics of a problem point to one option over the other?"
It's not rewriting code or reusing code that makes you a great programmer. It's knowing when to rewrite code and when to reuse code that makes you a great programmer.
Reading code is like reading the dictionary - you have to read half of it before you can go back and understand it.
If you'd tried to write it all yourself from scratch from the beginning you'd still be coding and you wouldn't have gotten the feedback about what needs to change as quickly. Prototype quickly then optimize later.
code from scratch - and reuse your own code - that way you know exactly what it can do and where you have to start from scratch
AFAIK, you can access a DB via both JDBC and Hibernate. Just do most of the job with the frameworks and just the little bit that isn't supported use plain JDBC.
Hibernate makes it very easy to write native queries for things that hibernate doesn't support well (e.g. bulk queries).
I'm really curious where it's come up that you just can't live without JDBC.
Most of the time a rewrite ISN'T the solution and should be avoided - no matter how tempting.
Care to comment more on the problem? Lots of nerds have your attention.
If you think it would be better if you had used JDBC instead of Hibernate when you began coding your application, you are probably doing something wrong.
It's starting to look like I would have saved myself a whole lot of time if I had written the database transaction using JDBC instead of Hibernate
Hibernate is great most of the time, but every Hibernate application I worked on had some JDBC somewhere, and I typically managed my own transactions... With regards to object-hydration, Hibernate (2.x) was an all or nothing, and sometimes I needed something in between for performance reasons.
Obviously, I don't know the problems you face, but I am surprised that a flexible framework like Spring isn't meeting your needs, and that Hibernate is preventing you from using JDBC...
It's better to use the new library. Who know what new features you'll need later. Plus everything you've learned the first time around can be written cleaner and perhaps more modularly the second time around, so substituting pieces of code will be easier.
Assuming your employer allows it, why not take the time you might have spent recoding and contribute and add the features you need to some of the frameworks/toolkits you are using?
You should be asking, "Should I make architectural decisions before or after I collect all the requirements." But you know the answer to that one.
A more experienced engineer would have dug for requirements early, planned for some creep, and would have warned the manager that the risk of starting before the thing is properly speced is that all work might have to be thrown away.
You'll know next time.
-Peter
reuse great code.
Ok first, reusing code is very important. You can get a lot of gains out of code you can borrow/steal from some other place. BUT, code you can't change is rigid (by definition) and will make your life difficult. We used hibernate for one of our projects and I am regretting that decision as well because it brings its own host of bugs that are impossible to fix unless you know how to alter the hibernate code, which means you need to merge with the main branch, but then you must get approval, and the cycle is really awful. If we had built an abstraction between us and hibernate we could swap it out for another ORM technology, possibly even a homegrown one. Would I write that abstraction layer twice, probably not. Would I replace hibernate with something better, absolutely. That may not mean I build it myself, I might buy it from someone and that is a whole different kind of question which is much much harder. So should you reuse code, YES. But only if it is well tested code that you change if you need to. If it isn't, then you should be able to swap out what you grabbed for something better.
If you must "open the black box", and get into the entrails of some rat nest of logic to complete your project, then trying to tweak code you don't understand is a sure-fire time waster, and chances are you'll have to rewrite it later anyway.
Only if the code is clear to you then can go ahead and use it. Only you can answer that question.
If you separate out your database code from the interface code then it would make things a lot simpler when it comes to changing the back end.. it shouldn't matter whether you've written it yourself or been using libraries because you'll only have to make changes in one place, and it should be easy to fire through it without too much hassle (unless you're switching to a new database system that has specific SQL quirks in date handling/binary blob handling or that kind of thing).
Personally I've never gone with frameworks though. With each project I work on I try to learn from past experiences to make my code even more modular. I've also been working on web apps with a lot of feature creep recently.. if you don't manage to keep everything simple it quickly becomes a mess keeping track of the changes.
which is totally what she said
lock in the requirements early, damage control the creep, and make sure your meeting attendance roster doesnt start including more useless management than is really necessary. ive been screwed by managers from departments ive never even heard of before.
Good people go to bed earlier.
A little of both. Portion size is key. I search for functions and classes but not whole frameworks.
For example, as a web programmer, I love jQuery, to simplify my JavaScript. And sometimes I copy a PHP function or class when I need to do something complicated, like translate HTML to wiki mark-up.
But the requirements for each project is so irregular that customizing a one-size-fits-all PHP framework would be the same amount of work yet result in all this extra code, I think.
You have two choices: Read someone else's code and documentation, or write it yourself. Sometimes the act of learning someone else's code is almost as much work as writing the stuff yourself. The nice thing about writing the code yourself is easier to modify, and extend. The act of writing code makes you remember what you did whereas learning someone else's code doesn't always let you remember every nuance. I catch slack from my friend who disagrees with me, but whenever I have the ability to code something that someone made a library for, I typically code it myself.
God spoke to me.
Just duct tape them together with some Perl.
I don't get it. About 2 years ago this post wouldn't have even been front page worthy, and now we have this? If I wanted to use slashdot as a howto forum, I wouldn't be looking here. I just don't get it, why would a post list this make it to front page? It's for nerds, but it doesn't matter except to a small minority, and it's not news.
Or am I missing something?
In general it does depend on what application you re-use. If you reuse a poor piece of software you're building your product on shaky ground.
Now I actually don't like Hibernate and Spring all that much and I use them regularly. Replacing a whole bunch of boiler plate code with a whole bunch of boiler plate XML doesn't actually make your app less error prone. Introducing AOP makes it easy to code cross cutting concerns but can make it hard to debug and understand code as it becomes harder to trace (and instead of a pure stack, you again have to look at XML configuration). Then there's the massive overhead. I'm afraid their popularity is due to software as religion pushed by a culture of marketing, rather than being based soley on techical merit. Hell a few years ago EJB was the word according to Bob, and we all saw how well that went. Try finding a new project actively deploying EJB today.
That said, I've run into the limitations of those products, especially hibernate, and if your scope has crept enough that they were looking like a good solution, but aren't anymore, you need to address the scope creep first. Some creep is expected, and accomodating the business is always a good idea if you can manage it, but people go too far and forget that sufficient scope creep can and will make your project fail. You need to start by talking to the business and ensuring they have an understanding that the more redesign they do after the initial planning the greater the cost and risk. One other thing to watch out for. Make sure you evaluate whether each request is technically possible in the first place, and whether it is practical to attempt what has been asked with the resources you have. (I've often seen business people make requests without understanding the technical effort required. eg. request a change requiring a full blown compiler be written when the component was suppose to be a very simple parser and was scheduled to take 2 weeks to code. That's not entirely their fault. YOU are the technical staff and need to help them understand what is involved in fulfilling their request.).
Once you've addressed the scope creep, look at your application again and re-evaluate the tools. It may be possible to divide the project in such a way that you retain Hibernate and Spring for one part (and let's face it they're the defacto industry standard and are going to be the easiest to support in the short to medium term. Long term is harder to predict, but the less popular an approach is the harder it will be to find someone appropriate to maintain your app). The other part you can do with raw JDBC or another tool. (eg. you might find it's the reporting that Hibernate isn't dealing with adequately, so move to JDBC or a reporting framework that takes RAW SQL queries).
These posts express my own personal views, not those of my employer
Sounds to me like you need to improve your requirements gathering. If you had gathered all the possible requirements, and tried to predict a few also, then you could have likely selected a framework to match your current project's scope, and the scope creep also.
Just what is it that makes ORM a non-starter for this project?
I have now done a few projects now with Hibernate, JSF Netbeans 5.x-6.1 and can feel your pain. I ran across some HUGE bugs and issues that if I would have just done it with JDBC it would never have shown up. Having said all that once I got through the bugs and did some workarounds (Sun support sucks), it is a lot better. I still wouldn't use Hibernate JSF and Netbeans for everything but it doesn't totally suck.
Here are some high level things that might help.
1. If you have a relationship between classes and it only needs to be a one way association do it. Avoid bi-directional associations at all cost.
2. Hibernate can use pure sql and it's own variant of SQL. Use them when you have to. We try hard to avoid every putting SQL in to our code but there are times when we had to do it.
3. Hibernate wasn't made for 40k objects in memory at one time. If you need that then it is possible but you will have to change some caching options.
4. Lazy instantiation. In theory it sounds great, in practice it doesn't. This may start a holy war on Slashdot, but I had to avoid this.
5. Working with Arrays of associated classes and having to make changes to them.
The more I learn about science, the more my faith in God increases.
I tend to turn to the math when this problem presents itself.
I`m a programmer.. not a project manager.. but I can still usually put some kind of approximate number on the time it's going to take me to write from scratch (or re-write) vs. adapt/hack a lib into what I need.
Factor in deadlines, maintainability issues, support issues, sanity, the phase of the moon, the flacid length of my firehose at room temperature .. and I have my decision.
As a side note, I don`t know what specific problems your having.. but I think if JDBC has become preferable over Hibernate.. you're doing it very wrong ;p
This is what I come up against. You're fine with off the shelf stuff as long as the requirements match (and continue to match) the design of the product you're working with. The more it differs, the more you find you have to spend time learning the ins and outs of the product you're working with. This can get to the point where sometimes it'd be quicker to write it from scratch yourself than learn the product inside out.
The balance I try and meet, is to choose lots of smaller off the shelf components rather than one large monstrosity. If the requirements change, the n often you only need to swap out one component or two and continue with relatively little disruption.
The real solution is to make sure the requirements are right from the start. If you succeed in doing this, tell me how!
Basically it comes down to this. If the project started out as a "let's get something going and work from there" project and has now become something of a "this thing works, let's extend it's functionality" then you've got to make a choice.
Either take the concept and build it correctly (either by writing from scratch or extending the classes that you have) or continue to hack it.
I feel that taking a prototype and making it a production quality application needs to have the time spent doing it right! Prototypes fall over, it's their nature, but you don't want the same behaviour from your production apps!
http://www.gibby.net.au
Reuse is only useful *if* you control the libraries that you are reusing. As new requirements come along that exceed the abilities of the libraries you are using, you need to modify those libraries and increase their capacity/use - this will allow for better future reuse.
When those libraries aren't open source, or when modifying them will create issues for other projects, reuse simply leads to stagnation.
Code reuse is a nice goal, but it's hardly the core definition of good software engineering. Code reuse is a benefit that arises from good design; code reuse as a goal unto itself is little more than wasted effort driven by cargo-cult misconceptions.
The thing most people overlook is that for code to be reusable, it must first be ... usable. Code has to be well defined, well designed and well documented in order to be usable and thus reusable. Sure, you can reuse it even if it fails those qualifications, but it will mean more work than not reusing it.
The Internet is full. Go away.
I always write my own bootloader, device drivers, operating system, assembler, compiler, C-interface library, graphics libraries, hardware abstraction layers, collections, and algorithms first. Then I just write a thin layer on top of that to implement the desired functionality. Easy, peasy, japa-neasy!
In my spare time, I grow my own grain, raise my own cattle, remove my own spleen and even generate my own electricity my peddling my home-made bicycle vigorously!
Over-the-top Response Guy! Giving "Over-the-Top Responses" since 1970.
Waaaaaay better!
Over-the-top Response Guy! Giving "Over-the-Top Responses" since 1970.
I know nothing of the scripting languages you are using. What I do know is requirements almost always creep. It is hard to stop creep as the urge is there to please the customer. If you deliver something they wanted a year ago it is unlikely the thing they want today, that's the nature of custom built items. Requirements will likely continue creeping unless you are almost done and then the customer is itching to get their hands on the product before you are comfortable you have thoroughly tested it.
If there is a large ways to go on the project, restart. You seem more comfortable in the scalability of the other language so use that.
Also, a lot of the comments I have read seem to address what he should have done. I agree, but unfortunately, that's water under the bridge.
ORM has it's own problems for database run applications. It adds alot of overhead to the amount of data you have to load when you could be customizing your queries to each method that has to call the database and just have the class instantiate a DB object instead. Is this a little more work? Yes. Is this more scalable? Definitely. Is this 'messier'? Depends. I abstract the sql into a separate database layer so it isn't even included in the functions; that way the DBA can mess with the database queries as much as he wants (as long as he is communicating his changes to the developers) and the developers don't have to see all those queries in their code.
ORM is a solution only if you are fanatic about turning EVERYTHING into an object but a true engineer knows that one tool does not fit all paradigms. And ORM is such a case where objects tend to fall short; good in the short run but the more you try to scale your application up, the more ORM becomes a problem and you end up falling back on hand coding your queries.
This is my sig. There are many like it but this one is mine.
"Now, I used to be told that good programmers write code and great programmers reuse code." is sort of right.... the correct lesson is: "Good programmers borrow, great programmers steal." Same lesson applies to marketers. And to poets (Robert Frost said it first, speaking about poets.) Oh, and to politicians, esp if your name is Joe Biden.
The Spring/Hibernate combination is very flexible. I find it hard to imagine you've come up with something these guys haven't thought of. If you're using the latest versions of these tools you should not have to deal with XML hell.
Understand who to mitigate creep, get sign offs on changes Including time changes and architecture changes.
Creep is fine, just be sure someone signs off on the details.
If they won't, don't change your work.
The Kruger Dunning explains most post on
Check it out: http://ibatis.apache.org/
The more you build on top of something, the more solid the base should be.
If your own stuff is just a few thousand LOC, it does not matter that much if you are locked into a particular framework. You can still do it again with another framwork.
However, if you investment is a major one, you better roll your own or use something rock solid, that even has different implementation with a similiar API or functionality, like a database or script interpreter.
And when in doubt, roll your own or built an abstraction layer.
p.
Without order, nothing can exist. Without chaos, nothing can be created.
"The new requests coming in involve capabilities "
Any project should start with a complete specification of the final product. Yes I know, I also have IT and non-IT partners who DO NOT/CAN NOT/WILL NOT understand this, and then technically pay for every rewrite one way or the other.
Well, just my 2c. When you require at least an outline of WTF they WANT, this happens to you less and less. When you explain it to them for the 50th time, that it will take SERIOUSLY LONGER to modify something, than plan for it in time, they will also start to understand. Slowly though, as they are IDIOTS!!! Did you find it out yet? They are MORONS!
Sorry, don't know what to tell you, I usually write and reuse my own database classes, do not know hibernate and only use ODBC when I really have to (e.g. modifying some ASP crap that already uses it, or SQL 2005 with PHP is easy with it). Just use PDO or mysql_ or mssql_.
I got quite a taste of a few Java frameworks from my last job (Hibernate and Tapestry, to be precise, and given my experience, I'm inclined to agree with the 'hibernatesucks' tag. If you had mentioned Tapestry I would have agreed even more with a 'tapestrysucks' tag, because Tapestry does suck, hardcore. Hibernate I can at least see being semi-useful in some situations).
My view is this: If you're making a simple project, implement it in a simple way. You've got to expect new requirements to come in, so don't go choosing *any* sort of 'framework' right off the bat.
<tangent> Unfortunately you do have to choose a programming language -- I'm not too thrilled with any of the offerings right now. PHP is hosted everywhere, but in 3 different incompatible versions. I use Ruby, but in order to ensure it works on whatever web host, you've got to use it with CGI, so hope your site doesn't get a lot of traffic! If you have a host that lets you run Java servlets and whatnot (despite my old job I'm not especially familiar with how Java applications get hosted), then I'd say Java's a good choice - not my favorite language, but it's fast and stable. </tangent>
Only when your program (/web site) actually needs to do complex things that would take a lot of time to implement (graphics libraries, hardware drivers, complete xml parsers come to mind) should you start bringing in external libraries. And note that I said "libraries", not "frameworks". 'Frameworks', in my view, provide 2 things:
If you can use those libraries that you need without a framework, it's going to be in your best interest not to include the extra frameworky stuff in your project that you don't need (get to the why in a minute). Importing a standard way to structure your code from a widely-used framework can be useful if you're working on a project with several other developers who would otherwise either argue about what made-up in-house standard to use or follow no standards at all, but if you're doing this thing by yourself, you can probably fulfill the needs of your project more simply by doing it yourself.
Even when you're importing functionality that will be controlled by your program (a library), and not the other way around (a framework), I would encourage you to try to implement it yourself, first. You know the specific needs of this project better than the people who wrote any libraries, so you may be in many cases be able to write something vastly simpler that gets the same job done in a way that makes more sense to you (not to mention that many libraries are simply crap, such as the ruby-git wrapper I recently downloaded before writing my own version that bothered to escape arguments to shell commands). If you do this enough times you will find that things you thought were difficult to implement are not so bad, and will wonder why so many people think they need a library.
To summarize, I agree with the guy who said "It's knowing when". Also, for the most part, with Chuck Moore, even though I'm not a Forth programmer ;)
Duct tape, XML, democracy: Not doing the job? Use more.
Coder digs hole, asks for comments on different shovels and other holes; film at eleven.
you should read everything on the internet as if it had "but I'm probably talking out of my ass" appended to it.
Unless your employer pays you to reuse code, you should always code it yourself.
Look, the man is sticking it to you. You know it. I know it.
Let's stick it to da man. Code your BIOS, your OS, your framework, your driver, and your app. It's fun. It's profitable. Your doges will love you.
Your wife will love you. In Alabama, your sister will love you.
Reusing code is for wimps.
When is the last time your review mentioned code reuse?
Come on. We are paid to create things!
God said, be fruitful and multiply. He didn't say be fruitless and reuse other peoples kids.
So obey God and do His will. Create something old.
Actually, the trick is knowing that you _aren't_ a great programmer (honestly what are the odds that you are a great programmer?), and thus choosing to reuse code from better (and hopefully great) programmers.
If you wish to delude yourself, you can believe that you are a great but lazy programmer and thus choose to reuse code from other great (and more hardworking) programmers.
Stuff like Perl + CPAN is good because of all that code you don't have to write. The less code you write, the fewer bugs you make.
More importantly the less code you write, the less code OTHER people have to figure out. If you use popular libraries/modules whenever possible, other people can just go "Ah, the standard wheel", instead of going "He calls it a wheel but is it really a wheel? Better check, the bug might be there". Or they might even go "Ah yes, it's probably that bug in the standard wheel, when are they going to get it fixed already, meanwhile we'll do the recommended workaround".
You can also reuse "code" in other ways. For example - using a popular RDMBS is one way of reusing code. With a good database you don't have to reinvent transactions, row level locking and all that. Lots of smart people have done all that work already. And you can use the DB as a common "interface" for other programs (also written by other people).
A lot of the languages the CS academics heap praises on are powerful for the code you have to write, not the code you don't have to write. Yes it's probably a catch-22 thing, but when it comes to "real world", I'd rather pick the language where I don't have to write so much stuff.
Prefab may be uglier, but it beats spending 10 years carving that perfect sculpture all by yourself, only to have the customer say "erm I want a sculpture of my wife not my ex-wife...".
(Note I am not a great programmer, so feel free to ignore me).
this is the usual hobson's choice. you load up some framework and WOW, 70% of the work is done for you! Whopee! Then the boss wants the buttons to update differently and you find it really hard to do that within the framework, but you manage by adding kludges in 73 places. Then you realize you really need a way to rollback or failover or fooswank and of course it's hard to impossible to wedge that into the framework. And then version 2 of the framework comes out and all your kludges you have to reimplement.
in other words the framework makes the first 70% easy, the next 20% difficult or slow, and the critically needed last 10 percent very difficult to impossible.
I've seen this time after time, with Hypercard, PowerBuilder, Notes, etc, etc, etc...
I prefer to code JDBC directly. Then I at least know what it is doing and can track down and resolve bottle-necks. Plus: there's less xml. For simple, low load applications it's probably ok but even then I find that it is better to get your hands dirty. You're going to need the experience when you tackle a not so simple application.
The benefits of reuse is that you have less code to maintain and costs fall, someone else does the testing and bug fixing of that code so you don't have to. The best thing you can do with a line of code is delete it I say. As long as the code being reused is from a good project (i.e. open source, active, mature, etc), always reuse.
God damn you software engineers.
Sometimes you feel that working from scratch "would have been better". But you might be missing lot of details you have taken for granted or even never knew they were there, and you will have to face them.
There is a great article from Joel related to this... he actually speaks about saying "this sucks, let's rewrite it" so it's not exactly the same, but the points have a lot in common with your case. I recommend you to take a look at it.
Here's a point I haven't seen raised yet: whatever you decide, document the hell out of the why in your code.
Maybe you do some analysis and figure out that Hibernate doesn't work for what you have in mind. You go your own way. If the why for that isn't very explicit in your code, the developer who has to maintain your code two years later will likely look at it and think, "The person who wrote this was a dumbass. They didn't know enough to use Hibernate for this!" Or they didn't understand Hibernate well enough to know how to do it the 'right' way. (And even if you do your research, it's possible this is true.) If you make clear notes of why you made the decisions they did, the person who has to maintain your code down the line will be considerably less inclined to want to murder you.
This is especially important if that person is a psychopath, or you, or both.
Seriously though, new requirements that come in after a project is written are often hard to fit into the framework.
That doesn't necessarily mean always writing from scratch just so modifications can be made more easily. Quite often the advantages of a well written library (good coverage of a domain, a worked over design, tested by multiple users, and quick to incorporate) outweigh the ability to precisely meet each new requirement.
No point in perfectly meeting requirements if the project is never finished (conversely you could argue, no point in finishing if you can't fulfil the core functionality!).
So my advice is reuse as much code as possible. However, as others have pointed out, check what you're basing your work on. There are a lot of shitty libraries and frameworks out there and for those cases you're better off rewriting the whole thing.
When you understand the problem as in project requirements are not creeping up than re-use..otherwise new problems ot you at least, new code..
Fred Grott(aka shareme) http://mobilebytes.wordpress.com
You gain efficiency by understanding a feature set and picking the right tools to do that job.
The problem is, most people/organizations desperately want to build something now, so they feel something's happening, then keep tweaking it all over the place until they're happy, totally changing the spec once it's underway.
In short, they want you to do the job in order for them to then understand it - making it nigh impossible to select the right tools at the beginning.
Most of the really great, really efficient systems are built by someone who had a firm idea, figured out all the aspects, then got on and built just that. Or they played with ideas, building multiple prototypes that likely weren't very efficient, then went back and rebuilt it with what they'd then learned.
Most systems aren't like that. They're designed by committee who won't think through most issues until they see a working version. They essentially force prototyping on you - yet consider what you're building the final system and won't pay for you to do a final rebuild. As a result, they're bloated and suck.
Welcome to the real life of a programmer. Yes, code re-use is something great programmers do. But only if they have great designers who really think everything through first. If you ever find such a mythical beast, let me know and I'll be clamoring to work there too.
I put it this way:
1) You can rewrite/code from scratch and spend your time finding the stupid, trivial bugs that you will invariably code, or
2) You can reuse someone else's well used code and spend your time finding the really nasty, non-obvious bugs that remain in their code. It should be noted that several bugs similar to but probably not exactly the same will remain to be found in your code AFTER you have found your trivial bugs.
The choice is yours.
Cheers,
Dave
They that can give up essential liberty to obtain a little temporary safety deserve neither safety nor liberty.
Ben
I've been involved in many software projects that either forced you to wade through pages of documentation (and we're talking WEEKS - hence loss of productivity!) or required reverse engineering of existing code (and this would take ages, as most of the team had left). If they had left API documentation, then great, no downtime. Otherwise, the quickest option is to re-write code. And the cheapest.
My web domain.
You can either write a straight 1:1 wrapper, or create your own API, customized to be something you're comfortable coding against. You can then replace the technology with something better, when that something better comes along, and all you have to do is change the wrapper. The other reason this is a good choice is you can provide a mock object replacement for your API and run unit tests against it without bringing up an entire web environment.
It sounds like you're describing a DAO layer to encapsulate the use of Hibernate. Again this is a common and accepted practice.
Having a DAO layer enabled me to get my current project up quickly (proof of concept) using Hibernate and then later move to hand written JDBC as Hibernate isn't approved in my shop (grumble, gripe).
In my experience, 7 times out of 10, the functionality you want IS there, you're just not well versed enough in the framework to know how to get at it. 1 time out of 10, you should be writing code, but extending the framework, not abandoning it. 1 time out of 10 you should be writing your own code separate from the library. The remaining 10% of cases are the customer asking for something retarded or overreaching the requirements into implementation.
Jherico
What can the average user can do to ensure his security? "Nothing, you're screwed"
... it the first time a programmer confesses (s)he is married to Hibernate, ..., *that* is new!
Refactor when needed. That way you keep a working project and can expand it as needed (or as quickly as possible).
"Actually, the trick is knowing that you _aren't_ a great programmer (honestly what are the odds that you are a great programmer?), and thus choosing to reuse code from better (and hopefully great) programmers."
This is a similiar flaw to believing that ISO certification means that a company will always create great products. Just as each product should be evaluated on its own (the UL approach), so software should be evaluated on its merits, not on the reputation of the programmer.
There's no such thing as a "great programmer" in the sense that one individual excels in every aspect of software development. Average programmers (whatever that means) are quite capable of producing quality code. Quality code depends more on the dedication of the developers on the project than it does on programming IQ (again, whatever that means).
... was taking a job that involved programming in Java. Your second mistake was listening to people who claim code-reuse makes for great programs.
Great Programmers do the following:
0.) Cultivate understanding their domain through testing of their assumptions
1.) Write the minimal amount of code they need to do the job, and avoid complexifying the problem (in order to make it interesting)
2.) And after testing prototypes they review, revise, and refactor, until they produce the simplest comprehendible code possible.
A great programmer reuses code in the sense that they've learned how to solve certain specific problems, and reapply the same techniques. Successive revisions of the code need not throw everything out each rewrite, but proper refactoring by its nature requires that interfaces are not sacrosanct.
Every time you call out to a library, a system call, or 3rd party framework, you abdicate design responsibility for that feature, accept all of the design decisions of all the developers who were involved in the development of those calls. You're stuck with the tradeoffs they decided were acceptable, as well as, all the assumptions they made about how your application would work. You also assume a minimal amount of complexity for your application in the interface code to the abstraction layer you chose.
If you study the work of the early smalltalkers, the forth programmers, the iconic lispers, you'll see programs that are comprehendible. Entire development systems, operating systems, and applications measured in thousands of lines of code. It isn't as if computers have changed so much that these programs no longer run, or will require considerably more code to run. It is often just that the simplest design decisions we make like what language and platform we wish to support inflicts millions of design decisions upon us.
For some real inspiration:
http://vpri.org/fonc_wiki/index.php/Main_Page
http://colorforth.com/cf.html
http://sourceforge.net/projects/squeaknos/
Unless you like being stuck with Java!
DAOs are a common use of this type of programming but it can be applied to much bigger things:
eg. A GUI layer could switch between Win32 and X11.
A 3D graphics layer could switch between OpenGL and Direct3D. ...etc.
Stuff like this isn't for beginner but I find this is the real key to program portability, rather then "use Java" or whatever (which will usually run into huge obstacles in long-term projects which don't fit the Java mentality).
No sig today...
One of my pet peeves is to see code copied and pasted from other projects or web sites. The naming conventions and formatting are usually inconsistent with the rest of the code and exception handling, logging, instrumentation, etc. are lacking or out of context.
My advice is to use mature libraries whenever possible. I'd rather see my development team contribute to an open source project if we need additional features instead of writing an alternative in-house. The open source code will continue to be improved, tested, and documented, while the in-house solution requires constant investment. Plus, I can find developers familiar with popular open source projects when filling open positions.
For your business-specific code, use "cookbooks" and design patterns instead of reusing code. If you really can't find an open source solution to a general problem, consider starting a new open source project. You'll enjoy all of the benefits I mentioned above if enough people have the same problem.
You can use native sql with Hibernate. I've done some crazy complex queries and it works fine. I just think you need to dig into it's documentation more.
If you want to write from scratch, write your own compiler and build your own system from melted glass.
If you want to make anything from scratch first you have create a new universe by creating another Big Bang....
I write my code from scratch in notepad: standard open source web-esque languages of PHP and Javascript, and sometimes I program Java with notepad.
I have complete control over the program, thank... god.
using appropriate design patterns and coding to interfaces (as spring strongly encourages) would completely prevent you from being 'married' to any particular implementation or persistence mechanism.
It's worth mentioning, if you're going to use other peoples code, always wrap it up and call the wrapper rather than tightly coupling your own code to the foreign code. You never know when you'll want to replace it. The Bridge, Facade and Strategy design patterns are good for this sort of thing.
-1 Uncomfortable Truth
I have learned through experience to always have a door open for those "non standard" requests. I always have them.
So I have sort of my basic library set with functionality that will be always the same (eg debugging, logging, basic DB IO stuff, access rights to pages, basic template engine) and I built whatever I need on top. with this I reuse code, and extend it if there is need, plus I just need to write new code where it is needed.
There is a very small chance that a standard framework will exactly do what you want. Or you just adept your needs to what the framework you choose can do. Both chances are normally very minor ...
"Freiheit ist immer auch die Freiheit des Andersdenkenden" - Rosa Luxemburg, 1871 - 1919
They are not at all related. Seriously. And if you are getting requirements you think Hibernate can't fulfill, you have done something so fundamentally wrong somewhere that if you code the database layer yourself you will be completely screwed. So find out what the real problem is and what you need to change to work with Hibernate.
Stuff like Perl + CPAN is good because of all that code you don't have to write. The less code you write, the fewer bugs you make.
Stuff like Perl + CPAN is also a very mixed bag and has likely caused my employer very much expense towards working around limitations in code therein compared to what has been gained. I know they've spent upwards of 6 figures on my time and others in direct and indirect costs working around the limitations and bugs in Storable, to name one example.
On Wednesday we'll have: "I coded a project, and now it's all done, should I start another project?"
Thursday's topic: "A lot of people around me use the tab key but I like to key in exactly 3 spaces for indentation, who's right?"
And on Friday: "I...uuuuh...well....oh, have you ever hit refresh but the web page said it couldn't?"
to tell them it is a new project. You should also make sure they change the project name so as to prevent confusion with the old and now dead project.
Engineering is the art of compromise.
It sounds like you may have jumped or been pushed into the implementation stage early. The first step is to generate and agree upon the requirements for your application. Once you as the developer and your client know what your application will do (and more importantly what it will not do) you will be able to decide whether or not the code you want to reuse will satisfy your requirements (or can be tweaked to do so) or if you'll be better off building your own.
Using a quote from the article: "It's starting to look like I would have saved myself a whole lot of time if I had written the database transaction using JDBC instead of Hibernate -- now that I'm married to this object model framework, some of this stuff doesn't look doable." If you and the client had sat down at the beginning of the project and described what the application must do, must not do, and what the client would like it to do (or not to do) would you have realized before getting "married to this object model framework" that some of the requirements would be hard or impossible to satisfy?
I whole heartedly believe in code re-use but I have been burned a bit by white box re-use. Be cautious of code you have to do open heart surgery on before it's usable! It's likely you'll be in there again and again.....
Quality code depends more on the dedication of the developers on the project than it does on programming IQ (again, whatever that means).
Programming IQ is defined quite clearly in _The Elements of Programming Style_, Kernighan and Plauger - http://www.amazon.com/Elements-Programming-Style-Brian-Kernighan/dp/0070342075
It's required reading and the advice is timeless.
Reuse:
Homebrew;
The reasons behind reuse should make it your first choice, but at the same time, it's hard to be a great coder without coding. Turn that desire to code loose on the situations where the existing approaches are insufficient.
My rule of thumb is that if it takes you longer to learn to use a tool than writing it your self you should probably just write it.
But really code change is inevitable because reusable code isn't so always. For instance, I made a simple mysql wrapper because the senior programmer thinks ORMs are bloated, now that we have to work with oracle I made a different wrapper for oracle, he said that I should extend the mysql wrapper to handle oracle as well, but, the dialect of sql that oracle and mysql use are so different there is no chance we will ever swap DBMS.
Rewriting is inevitable, I once read (in TDWTF) that any task that is done more than there is worth abstracting out.
But... the future refused to change.
Depends. Not an expert on CPAN's code. But I've read enough PEAR to know how much it sucks. Avoid that crap like the plague. If you can't write code better than that, you shouldn't be writing code.
However, the corollary to that rule of thumb is that if you can write better code than PEAR, you shouldn't be using PHP.
Yet, If you understand the previous rule, then you are sufficiently intelligent to actually use PHP wisely.
What is the problem domain? Is it a rapidly changing complex problem domain? Then you would probably want to "roll your own". If it is a fairly straight forward problem domain that is fairly static, use frameworks and other's code. This is a spectrum of course.
My answer: whenever it feels right.
But the entire problem with asking for advice like this is that without a context, the question is nonsense and so the answers will also be nonsense.
putting the 'B' in LGBTQ+
It depends on whether you are truly using well known libraries or not, and how much extraneous code you pull in, just to avoid writing code. You can end up with huge blocks of unknown, potentially buggy code that you haven't written, haven't read, and which may have unknown side effects, just to save writing a few routines. Using code you haven't written isn't always a clear win, it's a tradeoff, and you have to evaluate the costs and benefits, as you would with any programming decision.
There are some advantages to libraries over frameworks. (Working definition: if you call it, it's a library; if it calls you, it's a framework.) Frameworks are great if your problem fits into the model defined by the framework. Since many web applications are rather standardized, that covers much of web development.
The real problem with frameworks shows up when you need more than one of them in the same program. You can usually use more than one library, but using more than one framework is at best painful and often impossible.
It's annoying when something which could have been implemented as a library is architected as a framework because frameworks are "cooler" than libraries.
This is why I write everything in Assembly. Need a new feature sure I can write that but it will cost ya!
Are you sure you can't use Hibernate? It has a lot of hooks. At least tell me you've read the book by the Hibernate author. I think you might regret bailing on it. Harder requirements need better tools to get the job done, not crappier tools like JDBC.
Using Hibernate does not tie you down to only using the ORM framework. You can still use JDBC if you need it.
While everyone likes to bemoan the inevitable requirement changes that happen in a project, why not embrace that idea from the start? Given that there definitely will be requirements changes, you hope that they will come as soon as possible. After all, your job is to meet their requirements, isn't it? So you work in the way that will best do that. And the sooner you get feedback, the sooner you'll know what needs to change.
So, reuse code if it will bring forward the time that the end user can see a working system and give you that feedback.
And what makes a great developer is also writing the code in a way that minimizes the cost of those inevitable changes.
If you're a good "from scratch" coder, code from scratch. If you're a good "analyze other people's code" coder, reuse their code.
I don't envy the guy who asked this question, regardless of what his capabilities are. He's experiencing the all too common "evolving requirements" problem. Evidently, the framework met requirements when it was chosen. Now it no longer does.
Therefore, his real question is, "how do I throw away this beast with minimal disruption?". I mean, he's either going to do surgery on the beast (reuse) or replace the beast; but one way or another he needs a different animal.
Bottom line? It's a judgement call, and it's based on the talents of you and your team. Me personally, I'd favor a quick-n-dirty full replacement in parallel with the existing code. Spend a few days of intense hacking. I'd know if it's workable after a few days. The mere act of coding the replacement will lead you either to conclude that the problem is not as difficult as you thought, or you will end up splicing the replacement code into the framework and, in essence... reusing!
That's just me though. Then of course there's always the "I got me my 2 years of experience, let's put that on my resume and screw the next coder as I leave for a better job". That option doesn't work so well in a recession.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
While I respect Kernighan and Plauger, I don't accept a definition of Programming IQ as a standard just because some smart and famous guys propose one. Plauger has forgotten more about C libraries than I will ever know, but he's been known to express opinions on aspects of programming that he knows little or nothing about. For example, claiming that there's nothing unique about real-time programming.
All frameworks (versus normal libraries) are problematic to a certain point of view, but ORMs are far worse. To understand why, read this: http://blogs.tedneward.com/2006/06/26/The+Vietnam+Of+Computer+Science.aspx
If you insist on using ORMs, I personally suggest using JPA which abstracts away the underlying implementation and then migrating from Hibernate to EclipseLink. The latter implementation is much more solid and has a much healthier community.
Hibernate's authors are notorious for being rude to their users and their code is notorious for the hard-to-debug exception messages they through. Just my 2 cents.
Maybe you should consider talking to an other programmer about your issue. It's possible that you just don't know how to solve your problem. There are great forums for spring and hibernate questions. And remember there are always many ways to solve a problem. If it isn't 'doable' like you said the one way, just try another approach to solve the problems. Maybe your problem doesn't fir into a technical transaction, try a logical one. Need distributed transaction handling - use JTA, with an XA datasource. But giving an advice without knowing the problem is difficult...
Joel Spolsky wrote a nice article about this a while back. Since non-technical people don't see the code that's behind the UI, they can't really judge the difference between a polished UI with no code behind it and a fully working application. It is very reasonable for them to look at a polished UI and say "let's ship this tomorrow".
The solution is quite simple: make the UI reflect the state of the application! Use sketched buttons for stuff that doesn't work yet, use strike-through text to label stuff that doesn't work, etc. I've been using this technique for years with my customers. It gets the point across every time.
and don't listen to all that stuff about prototypes being proof-of-concepts, that's non-agile blatter from the 70's ;-). If the prototype is attractive enough that the business people would like to use it then you'd be wasting money by throwing it away and starting over.
The fallacy behind starting over is that the prototype is a code mess and the rewrite will be clean. Forget it. If you're no good at refactoring and organizing code then the rewrite will end up a mess too. And if you are good at it, you should apply those skills to the prototype!
Jeeeesus.
Why does everyone always want to take sides? Do what works for you, not something that someone else thinks is right. If reusing code makes sense, reuse it. If you think the existing code doesn't meet the requirements, rewrite it! And if you think you can make the existing code meet the requirements, recycle the code!
Your evaluation period for Productivity 1.0 has ended. Please purchase more coffee to continue using this product.
The fact that the OP's application logic and data access code are tightly coupled reveals he doesn't use Spring Framework to it's full potential. One of the fundamental ideas of Spring is to program by interfaces and decouple the actual implementation from the interface, and configure the implementation declaratively.
If he had designed his application properly, changing the data access implementation would be very unintrusive to the rest of the code. Actually, transactions could be also defined declaratively, further decoupling them from the data access code.
Whether or not you're rolling your own or reusing isn't as important as knowing the tools you're using!
-=- 4ntifa -=-
The libraries that you have used have provided additional value that should be factored in ... documentation, knowlegable community, database independence etc.
This has come at a cost of pushing you into a code structure in which you currently feel restricted.
What I like about Spring is that it acts as kind of an uber-Factory, so if hibernate is not working out for you, you should be able to refactor to another persistence mechanism fairly easily, because the implementation is hidden behind the service that is exposed by Spring. You can then change the implementation using new frameworks, and as long as the signatures / interface dont change then the calling functions will be unaffected.
You may want to consider swapping your hibernate implementation for iBatis see Hibernate vs iBatis
What I dont like about Spring, is that I find it sometimes difficult to piece together a decent mental model when first looking at an existing application from various bits of xml which may or may not have dependencies between one another.
The libraries that have been abstracted at a higher level e.g. Hibernate or iBatis as opposed to jdbc exist because add value by solving common issues. They are successful because of the open communities that spread the knowledge outside of any single organisation. Beware of the danger when writing your own, that you may end up at a later stage having to solve issues that have already been dealt with by existing frameworks.
Good Luck, Neil.
RTFM is not a radio station.
Loosely coupled modules and a solid SOA setup and these problems wouldn't exist.
With solid Data, business etc layers you could just have changed the dbase and dbase code in the bottom without the top caring or even knowing.
... I'd say don't programm anything yourself unless you are abolutely sure it doesn't exist in some form of lib or class. Programming in Java is a PITA as it is, and it's whole point is that you don't have to build anything yourself.
In fact, Java is nearly not at all about programming but about reading docs and integrating libs and classes that provide the code you need. It's OOP Forte with the brakes removed, and one of the reasons it's considered to be so outright boring. That's why there are so many thriving alternatives such as PHP or Python, so that people can get a chance to do some result oriented coding.
You are in Java territory allready and Java now is free (speech) aswell, so continue to play the Java game and learn as much as you can about it by integrating existing code. Java-style research, doc-reading and OOP knitworking is tedious but in the end it'll pay off. Especially if you plan to advance in an suit-style big-bucks IT career. There's a reason any other PL can't get by OOP and/or the Java way of doing things beyond a certain point if they want to be taken for granted.
To go even further, I'd actually look into UML editors and CASE-Tools once you know your way around Java, as these are the most advanced in the Java world and mark the point beyond which Java loses its pain factor. I find JBoss and jBPM BPEL, JPDL and GPD particularly interesting. (If you allready feel sick reading those abrevations, then you know why a large part of me still avoids Java whenever possible :-) )
My 2 cents.
We suffer more in our imagination than in reality. - Seneca
>>Actually, the trick is knowing that you _aren't_ a great programmer (honestly what are the odds that you are a great programmer?), and thus choosing to reuse code from better (and hopefully great) programmers.
Uh, the code you reuse isn't necessarily from great programmers, nearly all the Perl code I've had to maintain was a POS: either it was created by beginners making lots of mistake or it was created by experienced coders who think that TWTOWTDI is a great way to get job security and creates unmaintainable code: I remember one case when a colleague of mine call me because he was trying to understand a Perl script written by an "expert" and he was stuck with two lines of code: it took me 30 minutes and 2 books (and lots of swearing) to rewrite those 'magical' two lines with ... two lines(!) that a beginner in Perl could understand..
What is the UL approach? Links?
hibernate ain't the only persistence layer library ya know. OJB, torque, blah blah blah.
But reuse is still better - just sounds like you need to open your horizons on other alternatives.
Oh yeah, and stop letting the requirements creep.
Then what if you redefine "great programmer" as "a programmer who consistently produces quality code"?
I think you mistakenly equate a "great programmer" with a "smart person" but that is not necessarily the case.
Lots of good advice already, but I want to add one thing:
There is a difference between frameworks and other libraries.
Libraries are a no-brainer; you just don't go and write a
replacement unless you have very weird requirements.
But frameworks tend to dictate how your app is structred,
and to some degree even what it does, and that can be severely
limiting. Frameworks vary in how easy it is to deviate from
the assumed flow of things, but you can indeed find yourself
spending a lot of the time fighting a template that just
doesn't fit your task.
Before you drop it, especially if it is much respected
by other good developers, make sure you really understand it
and what freedoms it does offer you. But if you discover
it simply has a different goal than you, the sooner you
get out the less waste you have to grieve about later.
sudo ergo sum
Sure, that's why I said reuse good code.
;).
BUT where do you start looking for good code?
You are more likely to find good code from a programmer with a track record of producing good stuff, than from a programmer with a track record of producing crap.
Average programmers might be able to eventually produce quality code, but it can take them 10x or even longer to do so. Some programmers will take 3 years to do something that others can do in a day. So if anyone is thinking of assigning a rating to programmers, a log scale might be a good choice.
If you ever come up with a system that somehow produces good code from crap programmers in a timely manner, you may have solved the AI problem or at least be making better progress than most AI researchers
a great programmer can:
1) Evaluate the suitability of existing code for the task
2) Use the existing code appropriately
3) Write excellent readable, maintainable code when needed (and only when needed) to fill the gaps
4) Communicate well with the business to understand the problem and design a solution
5)...
6) Profit!
I wish I was a great programmer then I could cure world hunger, poverty and conflict and walk on water in my spare time ....
I wouldn't necessarily call using a framework code re-use per se; more code generation... I'll admit that it is using someone elses code, but as you have found out, frameworks usually have pretty tight limitations (which is fine if you have a project that you know will not have a requirement change in it's lifetime), either that or they attempt to allow for everything and as such become more complex and messy than simply using the language in a vanilla form. Great programmers build themselves job specific factories, classes, modules, includes (whatever container you use in your specific lingo) and re-use them in other applications... Teams of great programmers build themselves a communcal collection of packaged code and libraries which perform the common business logic or UI elements frequently used in that company or group :)
So let me get this straight: you have used a framework that allows you to mix and match the use of your ORM solution with plain JDBC. It also has a dedicated WS project that isn't bad at all and you happen to need that too. Sounds like a good start to me.
The fact that you now have to write some code yourself is not a shortcoming of the frameworks you used, but a natural step in the evolution of any useful piece of software.
I'm biased because I'm a committer to Spring, but you can't seriously argue that having to extend Spring is an argument against using it.
I would say that the extensibility is one of the main strengths to look for in a framework, for exactly the reasons you're stumbling upon.
It seems like you've written an application that is actually used and now the users want more. Congratulations!
Show a man some news, distract him for an hour. Show a man some mod points, distract him for the rest of his life.
This is a similiar flaw to believing that ISO certification means that a company will always create great products.
It is a common misconception that an ISO certification certifies the actual product quality in any way.
Those standards and a certification received from ISO does not in any way certify the actual, highly subjective product quality perceived by the customer. Rather, it consists of rules and guidelines which your business processes and business process descriptions have to adhere to - e.g. you need to have a process description at all, to begin with, and it has to be accurate, people should "live it" and there should be means of continual improvement, you need to keep track of defects etc.
Generally speaking, you could produce highly defective foobar and still be ISO certified - of course the ultimate goal of all that process yadda yadda has to be to finally improve your products and/or make more profit but in itself ISO 900x certifies the quality of your red tape - you know, those documents nobody would usually touch with a 10ft. pole if it wasn't for making it through the much-loved re-certification.
clicky
(I am assuming, of course, you were refering to the iso 9000 family of standards)
"Only one thing is impossible for God: To find any sense in any copyright law on the planet." - Mark Twain
The problem is, most people/organizations desperately want to build something now, so they feel something's happening, then keep tweaking it all over the place until they're happy, totally changing the spec once it's underway.
Actually, if it's done right, that's the best thing to do. Why? You produce a first release _quickly_ without having to worry about the features that are several versions down the line. That release is adopted and the customer starts actually using it. Whatever happens, they're now actually getting the benefit of having the useful software that you have written for them.
And sure, they might decide that it's wrong in some critical respects. But at least they have _something_ to work with while you're making the changes they requested.
Most systems aren't like that. They're designed by committee who won't think through most issues until they see a working version. They essentially force prototyping on you - yet consider what you're building the final system and won't pay for you to do a final rebuild.
I've never had a client who expects to get free changes to the working software I've delivered that meets their original spec, even when they decide it's not what they wanted. Well, OK, there was one. But the court agreed with me and made them pay up. That's in over ten years of professional software development. I've had clients who ended up with bills nearly double what they originally expected, but they've paid happily because they knew I delivered their original requirements, but those requirements changed. They know I charge per feature developed, and I always make sure they know before I start work on it what each feature will cost. Generally speaking, they're all happy with that.
Yes, code re-use is something great programmers do. But only if they have great designers who really think everything through first.
Some of the best code reusers I've come across are agile developers who espouse not thinking through anything beyond the feature you're working on at the moment. Reuse doesn't require ahead-of-time design, but it does require a lot of attention to detail. In the OO world, it requires a good understanding of design patterns (which means when not to use, as much as it does when to use them). It requires the ability to understand what your code is doing in great detail, and to hold a large chunk of that information in your head at once, to enable you to spot where another piece of code is doing something similar to what you're trying to do now. It requires you to have sufficient confidence to rework that old code to better fit its new application, where necessary (which normally means you'd better have good test coverage of it). But it doesn't seem to require you to identify the opportunities for reuse in advance.
You should get most (preferably all) of the resuests before you start coding. And you should get them clearly. If you start coding and accepting more and more requests you'll end up in trouble. When you start a project, make sure it's been talked about enough, good descisions must have been made etc. Then write down what needs to be done and what the resulting software should have. Whatever is -not- in there ... is too bad. They should have thought of that earlier, or pay you for all the trouble you need to go through.
a hybrid approach: reuse something and rewrite it partially to suit my needs.
It's worth mentioning, if you're going to use other peoples code, always wrap it up and call the wrapper rather than tightly coupling your own code to the foreign code. You never know when you'll want to replace it. The Bridge, Facade and Strategy design patterns are good for this sort of thing.
If wrapping it up means taking a performance hit and if it is a no-go, then make "contact surface" between your and foreign code small, so that you rewrite only minimal part of your code.
Remember that inevitably there will come time of finger-pointing and have mechanisms ready to easily prove that your code works well and who is to blame.
Also, keep in mind that there is no good reason to trust yourself too much, either! When writing part of your code, be suspicious to other parts (you also wrote) like if it was foreign code. At some point, it may become so, some of it may be "offloaded" from you and given to someone else to work on it, and it could become hell for all of you. This attitude will naturally result in clean and functional interfaces between your modules.
Actually, these sorts of creeping-feature problems are best solved in the design process. What we like to do (I am not a programmer, but I work for an IT company that has a team of programmers) is create a complete formal specification of exactly what the app will do, with mockup screen shots, sample reports, etc, listing each and every individual function and feature. Once you have thought out the end product, and gone over this design in excruciating detail with all of the stakeholders, everyone involved will know exactly what you are going to produce, and what it will do. I like to think of good software development like constructing a building. You need to get an architect to produce a detailed and complete set of plans before construction begins. You would not hire a contractor, and begin describing the building to him, and then have him immediately begin building it. The building built in such a way would most likely come in over budget, be a mass of repaired mistakes, and might not even pass the local building code. When you are one person tasked with designing and building a piece of software, put on your "architect hat" and do up a design. If you design properly, you will drastically reduce the amount of time it takes you to actually write the code. Your brain will have already worked out to some extent how the project will be put together, and since you have a good set of blueprints, you will not end up scrapping whole modules and rewriting them because you forgot to account for the consequences of some piece of code you had not yet considered.
On the subject of frameworks, libraries, etc: They are the building materials. It should be fairly evident from the design what kind of materials are appropriate. You do not generally build houses from reinforced concrete and sheets of glass, nor do you try to timber-frame a skyscraper. Use the "materials" which will make the project plan come together well. By the time a good spec is written, you should know what that is, if you have some experience with various "materials".
Also, a good design document is a decent start on some rough and ready documentation. That set of plans will help the next developer who must work on the plan, because the purpose of the software and all of its various operations should be laid out there. If you take to this philosophy, then new modules or features should have their own addendum, or an updated design document should be produced as part of the maintenance process.
I guess I'd ask the OP how good he is at Hibernate.
I can't imagine somehow "outgrowing" it and needing to rip it out alltogether when it manages to fit for some of the largest companies and projects enterprise has to offer. If you come across a query that needs to be hand-coded, fine, do that, Hibernate supports that. But why would you need to write your own ORM tool?
Job security?
My PhD is in software reuse and the main lesson is, never approach a problem with "I am going to reuse something today". The classic mantra is the rule of three, if you find yourself building something for the third time, capture it (and the other two) as a generic component and reuse it in subsequent scenarios. With respect to existing components/frameworks if it does what you want great - otherwise you need to ask yourself the question - "will changing it to meet my needs be quicker than building it from scratch?", if that's the case then reuse, otherwise do it yourself.
Truly great code has logical coherence. It embodies a concept pure in conception and internally consistent in implementation.
To produce great code you need two things. First, you need inspiration. If you don't have that, then methodology doesn't matter. Second, you need to be able to transform the inspiration from the moment of conception to a complete, debugged, tested, and integrated deliverable in a single session. In that session your brain must hold the concept in its entirety without losing track of any details. That means an uninterrupted session of 20 hours plus or minus depending on your age.
Sorry, I can't find the link to the Hacker's Dictionary entry for this. It claimed that all truly great software was the result of giving up and restarting from scratch multiple times. Early iterations serve to teach you what the real requirements are. Also, your recent familiarity makes you much more productive in the second and third iterations. You may be able to produce 5 times more in the second iteration, and 10 times more in the third iteration because you're in the groove.
To be successful at iterations, the time elapsed between iterations has to be really short. Indeed, sometimes inspiration for iteration N+1 comes before N is finished. What they were trying to teach you in the hacker's wisdom was that if that happened, start over rather than trying to alter course.
Well times change. Today we deal with much larger, more complex, and more integrated applications than were typical in the 60s. ALso, development time is stretched over weeks and months, and rapid consecutive iterations are impossible. At the larger scale you are forced into reuse.
Still, there is no conflict of ideas. You can still strive for logical coherence and no-reuse policy at the module/object level, while applying re-use at higher levels. One of the great benefits of object-oriented design was to aid in limiting unintended consequences of complete replacement of the implementation of some objects.
Where possible, you should never expose your choice to use an underlying technology. Instead wrap it in an Interface that exposes the functionality that your application requires and hide the implementation from the rest of your work.
The difference between Canada and the USA is that in Canada healthcare is a right and gun ownership is a privilege.
Of course not, the customer's opinion is (as you say) subjective. If isn't subjective, it's probably unreasonable, uneconomical or downright physically impossible.
You can make lead lifejackets - as long as you write it down.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
That's pretty profound. You should try and make it into a haiku or something.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
"There's no such thing as a "great programmer" in the sense that one individual excels in every aspect of software development. Average programmers (whatever that means) are quite capable of producing quality code." - by ClosedSource (238333) on Tuesday November 04, @09:55PM (#25635839)
Excellent, & accurate, imo @ experience in this art & science/field!
As Hancock (Will Smith) would say?
"GOOD JOB!"
(Perfect reply, & agreed upon here, 110%)...
Heh - 2 days ago in email, after hearing from an old friend online again (he & I have common ground in this field in OS tuning/tweaking for speed + security, etc./et al) & I wrote pretty much the same thing to he!
(He hosts a technical forum in ashentech.com - nice person, good techie too )
In that correspondence, I told him:
"It's all work, there is no 'greatest programmer', & very few of us are 'naturals' at it - we all hit those 'tough spots' in things we haven't done before, OR, where there is no 'turnkey prebuilt solution', such as ActiveX controls, .NET assemblies, VCL controls, or even DLL/lib prebuilt proven API calls that can do the job, by merely calling a prebuilt function or using a prebuilt tool... you have to invent it, yourself, many times... & this? This is work, no questions asked!"
Thus, based on your reply + the thoughts of others here as well... well, it is good to see that others feel (and doubtless KNOW, from hands-on experience) this is the truth of the matter.
----
"Quality code depends more on the dedication of the developers on the project than it does on programming IQ (again, whatever that means)." - by ClosedSource (238333) on Tuesday November 04, @09:55PM (#25635839)
Amen!
Edison said it, best (though I respected Tesla his "rival", a 1,000x more):
"GENIUS IS 1% INSPIRATION, 99% PERSPIRATION!"
APK
P.S.=> Anyhow? Yay Democracy, yea USA!!! OBAMA for the win!
( & the man plans on cutting outsourcing & that means that us guys in this field, especially THIS arena (coding) will be hopefully getting the best paying jobs BACK here on our shores, rather than some exec or project lead outsouring it oveseas (all the while, w/ said exec charging his company, say, $150/hr., paying an Indian, Chinese, or Russian fellow $5 of it, & the exec pocketing the rest himself AND putting security @ risk because who KNOWS what kind of 'backdoor' could have been built into the code & yes, this is a potential danger as well))... apk
I was using Hibernate/Spring on a project up until 6 months ago and found that I was spending a lot of time dealing with memory running out in Hiberate and fixing errors in my Spring xml configuration. I ripped it out and replaced with JDBC and since then my productivity has gone up a lot.
This is NOT suggesting that you do the same. I think it was a good idea in my position for the following reasons:-
1) The amount of data being persisted was small (only one class with a bunch of fields) and is very unlikely to grow much in the future. Most of the logic of the system did not need to interact with the database (it manipulated it and interacted with other systems/files). Hibernate helps by persisting your classes without you having to do much extra work, so doesn't save you much time/code if you already know how to use JDBC and only have one class.
2) I made sure this persistence was done through a java interface (a DAO = Data Access Object) to a class without using any JDBC specific classes to commicate with it, so that the implementing class can be easily replaced with Hibernate or another persistence mechanism without having to change any of the other main system code.
3) Our company is not into training and none of us had training in Hibernate/Spring. This meant that I may have been making mistakes with the use of the framework and I had to spend some weeks learning the hard way about some assumptions hiberate makes. If your company puts you and all your colleagues on a training program for every major thing you use then this is a reason to favour these kinds of frameworks.
3) My application uses a LOT of data and so has to be very good at clearing up memory. Being able to see how memory is allocated and freed in my code (not deep withing the Hibernate Framework) is important. It also needs to be very fast, and so direct JDBC access may be faster than using Hibernate classes.
4) There are only ever one or two developers working on a product like this in my company and so its not as critical to use the de-facto standard that most developers have used before.
5) Hibernate (and all the other frameworks) have had plenty of bugs in their time. True, they mostly get fixed after a bit, but if my small one class to persist my data has a bug or performance issue I can fix it today. With a framework I can spend weeks on the bug fixing forums trying to get them to recreate and look at my edge case problem that blows up my system (not so important to them, but critical to me). My system may have been running out of memory because I wasn't using Hibernate "properly" (maybe if I used a different type of setup parameter it would have worked) or it may have been a genuine bug with hibernate that only occurs under my kind of circumstances (so very few other people see it). By spending 3 days ripping out hibernate and replacing with a small class I saved wasting further weeks on the problem.
In a lot of the replies above people have been sneering at do-it-yourself idea, but in specific situations it really makes sense.
With regard to Spring I can especially see the benefits of Spring on large projects with lots of teams of Spring-trained people who need to link up their code and have Spring xml file editors in their IDE. Or in companies with higher staff turnover of Spring-trained employees, so that a new person can come in and see how a system is wired together.
On a small project with one or two developers who have no previous Spring experience or training and a reasonably straight forward system AND low staff turnover it may not be worth converting to Spring. KISS.
Define "From Scratch".
Remember the quote -- "To make a pie 'From Scratch', first you have to create the Universe."
Each programming tool you use (even assembly language), uses a tool the somebody else wrote. The real question for this article should be -- "What level of abstraction are you comfortable with coding?" - OR- "Which tool most closely addresses your problem space without undo limitations and has the least amount of bugs?"
Personally, I have wandered through the various levels of tools. I have made my own web server (really weak, but functional for ONE purpose), my own web crawler, my own search engine, and then I have used tools that do it all for you -- WSDLs are so darn easy, using any DB beats making your own, embedded browsers, its all a mash-up, but, you get the idea.
Do whatever is best for the specific project, if you find out you were wrong, fix it. You just learned a great lesson, there will be many more, -- and a test at the end.
- I live the greatest adventure anyone could possibly desire. - Tosk the Hunted
I think Great Programmers write reusable and extensible code, it only takes an average programmer to reuse code, I know because I am one. I've seen this from the CMS perspective, there's plenty of them out there and plenty of them can be extended. Its when you start pushing them to the limits that you find which ones are really written to be re-usable and extensible and which ones have the component parts tightly coupled and when you start pulling on one thread (sorry for the pun) you get a whole spagetti of mess.
But if you do happen to make it to my comment, I have a potential solution for you. If you need to extend the framework to meet the new requirements, look into AOP. It will allow you to specify behavior and attach interactive code to anywhere in the call graph. A good example is caching. Say you framework didn't support it, but now they want it... you can point-cut the object hydration calls and (quickly) build a caching system and subvert the original call, or continue the original call flow, all under your precise control.
AOP is honestly, imfho, the best evolutionary leap in software design since OOP.
You can have a biscuit out of a can (or tube) and some of them are good. Or you can have a scratch made biscuit. The best programmers I know either writing from scratch or modify the OSS libraries to do what they need.
Sometimes it is faster to write from scratch rather that learn to use the tool kits out there.
Think Deeply.
You have to have balls and say this is not what you ordered if you want this feature thats a new app and for x amount of money i will be glad to make it for you too.
We had a similar problem in that Hibernate was just not going not work with the data model we inherited (unless we did some major customization). We had successfully used Hibernate & Spring for our proof of concept, and we didn't to throw things away!
Enter iBatis: it let us maintain the framework and rewrite data access layer VERY quickly. It might be worth a look.
FULL DISCLOSURE: We are using the .NET version of Spring, Hibernate and iBatis.
How I develop code: 1) write helper libraries 2) build on top of the libraries 3) extend the libraries when needed not the code 4) the upper layers are very specialized 5) lower layers know only to do a single verb I keep using the same code that I wrote for java 1.2 This rules means that the code is very decoupled and that is one of the most important rules (when you use a framework you are tied to what the framework can do and too many times you need to broke from the framework.
I know both these frameworks very well; I find it hard to understand how a customers/employers requirements can creep in such a way that either of these two frameworks as a solution get in the way of the underlying problem.
Both frameworks are additive to the underlaying standards. Hibernate additive to JDBC so can be used in conjunction with, instead of or even a healthy mix of both. Spring (for web-apps/web-services) is additive to the Servlet specification and can be used in conjunction with, instead of or even a healthy mix of both.
Neither mask, hide or remove the underlying technology features.
There does not seem to be any original article explaining the dead-end you have found yourself at when using either/both of these technologies. Maybe you should blog this so that slashdot fans can better understand your plight. I'm sure the respective authors and maintainers of those frameworks really do want to listen so such cases and improve things for the next iteration.
Unless of course you have a customer/employer who is now asking for your framework to land on the moon (the "moon-on-a-stick" requirement).
I understand your question was attempting to be a general developer question but it does seem you are unsure which avenue to take right now and you are asking the world for help. So please blog your plight.
When you're a good developer the answer is obvious. So lets see where the problem verses solution you created whet wrong so we can help make you a good developer.
If you're asking this question, then you did it wrong.
The whole idea of Spring is that it is agnostic regarding which technologies you use. You should be able to swap out Hibernate and replace it with JDBC without affecting anything beyond the data access layer.
I recommend you just stop everything and read a good book about Spring before you dig this hole any deeper. Or just don't use Spring at all if you don't want to use it properly.
They don't grade fathers, but if your daughter's a stripper, you fucked up. --Chris Rock
Out of curiosity, what exactly are you trying to do that Hibernate isn't solving? Is it some database-level craziness with stored procedures or something?
Hibernate, while not perfect, meets the vast majority of my needs... and for the 0.1% of the time it doesn't, I can always write my own persistence layer.
Code re-use will make you look good when future requirements come up. For example, what happens when you decide to move to MySQL to get away from Oracle, and all of your code is using JDBC with hand-coded SQL? You'll be spending weeks changing that stuff when Hibernate would have abstracted away the vast majority of issues.
Ideally, the best way to avoid bugs is to code smartly and minimize lines of code. Re-using industry standard libraries and frameworks like Hibernate and Spring lets you do that.
rm -rf
Frameworks are great for rapid development, but are bad in the long run (as you explained better than I would because of my broken english).
I think that you should advertise the project developped with a framework as a prototype release (even if it provides the full specification). If the project is successful (many projects are disgarded because the specifications were wrong) and needs to be maintained, the migration to "own made software" can be scheduled and the workload is easier to justified.
People are very reluctant on the idea of "throwable software", but mistakes could be avoided by a prototype before deciding software architecture.
Marketing says... "We want to start building our own cars, you're an experienced car designer, we already have a factor ready so you should be able to design and begin producing in 8 months" and let's say they're right, fairly reasonable.
Then after all that they throw in, maybe 4 months into the project "oh yea, 'green' is in now, so we can't use that gasoline stuff, make it run on solar cells".
If you just want to build a cheap car, reuse parts. If you go for mass production or want to build a Ferrari, start with stock parts and redesign them where necessary.
1. Reuse if a maintained, actively developed component exists that provides the functionality, especially if the component is advertised for reuse - i.e. a library made for linking.
2. Reuse if you know you know less about implementing needed functionality than you plan to spend time and energy learning. Some things do need time to learn, and the mere programming skills do not help much if you are expected to dig into documentation of different standards and hardware specifications.
3. Reuse generally when any solution already exists, because wrapping up a bad component through a flexible interface is better than tightly coupling self-developed component. Later you may replace the mediocre component with a better candidate, but will retain the interface, so minimal changes will be required to your product.
4. Reuse generally, because through wrapping up a piece of foreign code, will give you the benefit of unit-testing, since the chance of even a mediocre foreign component working better than your first attempt to replicate its functionality is fairly high. This gives you two advantages, the one described at 3. and the benefit of said unit-testing.
5. Rewrite if you plan to after-maintain the developed component you otherwise would reuse from somewhere else, especially if you will consider giving it its own life as a standalone generic solution.
6. Runtime linking is teoretically a better idea, however practically, almost every time the version of the linked library changes, the whole product suffers due to degrees of incompatibility with the said library, however promised its version compatibility. This unfortunately happens much to often, one of the reasons many seasoned developers prefer to embed code at compilation time, thus eliminating the runtime incompatibility problem. They make up for it by frequently (or infrequently, pick yours) recompiling and publishing their host product with the updated library version re-compiled into it.
To reuse or not to reuse... that is the question
Fast Federal Court and I.T.C. updates
Actually, the trick is knowing that you _aren't_ a great programmer (honestly what are the odds that you are a great programmer?), and thus choosing to reuse code from better (and hopefully great) programmers.
There's no such thing as a "great programmer" in the sense that one individual excels in every aspect of software development.
1. Being a great programmer is overrated. Producing more than 10 times faster does not result in earning 10 times faster. The pay rate for the great programmer is rarely double the rate for the normal programmer. The benefits of using a great programmer are reluctantly taken by customers. Projects budgeted for one year costing over $100,000 are completed in one month for $20,000, are better aligned with the business, provide more functionality, and are easier to maintain; customers still complain about the higher rate.
2. Great programmers could excel in every aspect of software development. That stated, we are humans with personalities including individual preferences. In the late 1980s, I thought about creating a Unix-like OS, but decided I would dislike creating and maintaining the project. Luckily, others with different preferences had similar plans resulting in Linux and the BSDs. My preference is application platforms, one level below building applications. My chosen specialty requires being expert at building applications and integrating with many systems (operating systems, databases, and clients e.g. browsers.) Does anybody think that RMS or Torvalds would have difficulty programming games? Their specialties derive from personal preferences, not abilities.
3. Great programmers will join existing projects. Using a popular project removes the work of creating and marketing early versions. A great programmer focuses on fixing problems with the software, including redesigning and rewriting whatever is needed to make the project do what the great programmer wants.
4. The OP has discovered that new specifications are difficult with the chosen framework. A normal programmer starts looking for a new framework. A great programmer has more choices:
- Work around the problems without discarding the current framework.
- Fix the current framework to handle the new requirements.
The former is possible because the great programmer does not accept the limitations of the framework. The latter is possible because the great programmer can quickly rewrite the framework without adding much to the project's budget. Normal programmers are more accepting of limitations, less likely to design workarounds, and cannot rewrite the framework in a reasonable timeframe.
5. Almost nobody builds "from scratch." Even if the OP discards the current framework, he will still use existing Java Web servers and operating systems. The choice is not whether to use others' code, just how much.
6. Solutions include:
- Building a new framework.
CONS: High maintenance, less expertise, future requirements may still exceed the capabilities.
- Switching frameworks.
CONS: Future requirements may still exceed current capabilities.
- Fixing the current framework.
CONS: Need to have improvements accepted by framework project to avoid upgrade issues.
- Redesign application to workaround framework.
CONS: Project is less maintainable as different functions use different methods.
No good recommendation is possible without knowing more about the OP's resources and problems.
- Can Hibernate be fixed to handle the new requirements?
- Is the Hibernate team already working to add these abilities?
- Can new functionality use JDBC directly without interfering with the existing code?
I would not discard the proven working code until the new methods have been proven stable and maintaining two systems requires more effort than replacing one.
I spend my life entertaining my brain.
"Any fool can make something complicated; It takes a genius to make something simple"
- F. O. Stanly, co-creator of the Stanley Steamer
You shouldn't design something, let alone start to implement it, until you fully understand the requirements, both present and anticipated for the future.
Anticipating future requirements comes from experience, and shouldn't be something you should rely on coming from your boss. The result of this anticipation (which eventually comes as second nature) is to deisgn flexibility in from the start, and to choose a modular decomposition that lends itself to the types on enhancement you see fit to allow for easily.
Eventually of course your code base may be asked to do something beyond the level of flexibility you've designed in, at which point you will need to rebase/redesign and refactor the code into something that can grow further. Failure to periodically refactor to maintain a clean design will result in the eventual death of the code base due to the 2nd law of thermodynamics.
I like coding my own stuff.
It is easier to to get a 100% requirements, rather than 40-60% if you are lucky.
the "my" is obviously a typo. the incorrect use of "peddling" (selling) instead of "pedaling" (pedal a bike using the pedals) is a genuine error of grammar (or vocabulary, IANALinguist [though i'd like to go back to school for it])
--Justin
Underwriters Laboratories http://www.ul.com/ulprodcert.html
Code better than PEAR?
PHP's unsuitable
But knowing that, safe.
Frameworks solve the easy problems in new and unique ways. why? mgrs hope to hire flunkies in india and somehow the framework will magically make them productive. Also, its more fun to write a framework than to solve real problems and create a reusable library. Also because "architects" want to use somethign nobody else knows so they can feel important. That's frameworks in a nutshell. Great stuff for people who don't know what they are doing. Real boat anchors for those who do. When BEA starts talking about "Managed Servers" and other wordopedia those with real degrees feel like slapping the "architect" and pointing out Java cannot use over 3G user so has to have multiple processes running to immitate the big leagues. If you think spring is bad, trying throwing in Struts 1.x, Tiles, migrating to Struts 2 which is not and upgrade from Struts 1 but a rewrite, an outdated framework from Accenture called Grounds, various outdated frameworks from "platform" teams of people who no longer work for the company, and everything else. It takes 10x to 100x more manpower to use a good handfull of frameworks in your project than to just know what you're doing in the first place. BTW, I'll get your problem in Spring is the join problem. OR mappers never handle more than one table well. that's why we use SQL! It allows Set Calculus. Unless your OR mapper somehow helps with Set Calculus, then it really just gets in the way. Besides, doesn't everyone know how to write SQL by now!
It's worth mentioning, if you're going to use other peoples code, always wrap it up and call the wrapper rather than tightly coupling your own code to the foreign code. You never know when you'll want to replace it. The Bridge, Facade and Strategy design patterns are good for this sort of thing.
I don't know that I would go so far as to say always. It depends on the source of the code (standard vs. non-standard, known vs. unknown, stable vs. unstable) and the likelihood that it might need to be replaced, someday, in the future. If it's never replaced, that extra abstraction layer is a complete waste of time and resources. Either way, it adds an extra layer of complexity, more code to test and debug, more time to write and maintain that code, and might even impact the overall performance, functionality, and maintainability of your code. All of that needs to be balanced against the chance that this extra work now will save you or someone else extra effort in the future.
Overall, I believe in writing the code you need today and refactoring it when you need to change it tomorrow, because, in my experience, the changes I anticipated yesterday rarely match up with the changes that are actually needed today.
I don't think I would call a form of programming that has been pretty-much the dominant form for nearly 20 years now
In 1988, Ada was in its infancy and pretty much junk commercially. I wouldn't call any other OO language (or wanna-bes like Perl5) "dominant" until some years later and Ada was only dominant because it was mandated by the US DOD for all new software and they later backed down.
When you have been around since practically the beginning of time (LISP was one of the first languages and is still around and thriving in various applications), then you can talk about not being a fad.
Functional programming is a discipline (around greater than half a century and counting) and OO is a fad (around about 20 years, but only implemented in decent fashion a decade ago) in the same sense that the difference between a cult and a religion is that a cult is something you join, a religion is something you inherit from your parents.
OB xkcd - http://xkcd.com/297/
Actually it's finding the right balance between reusing and rewriting code that makes a good developer worth his/her weight in gold!
I find that "helper" sub-frameworks to be more flexible than all-encompassing frameworks. If you date your abstractions instead of marry then, then it's easier to dump them for something better. Helper frameworks use a bunch of small functions or classes that work together by convention, not forced. You can skip or rewrite the parts as needed. And they are not over-filled with features in order to keep them simple.
If you need a specific feature that's not there, you add it for that particular project and only borrow it down the road if another project matches that custom need. You don't keep collecting features as you go unless they are actually commonly needed in your org. This way your library doesn't get bloated.
Table-ized A.I.