I'm currently writing up API documentation for a large code branch which was never properly documented (and wasn't written by me), but now needs merging into trunk. I've found several serious bugs in the code as a result, all from trying to explain to the client how to use the API. These bugs were actually blindingly obvious when the behaviour of the code had to be explained.
I've also found some horrible design issues, where various settings the code allowed were contradictory or meaningless, or one setting overrode the behaviour of another, unless something else had been configured, in which case... you get the idea. As soon as you try to explain it, the awkwardness makes the design problems incredibly obvious - and in fact a much better way of doing it is also obvious. I can almost picture the evolutionary process by which this code was written, with each developer adding a new feature without regard to how the whole thing hung together.
So documenting after the fact can definitely detect bugs and design weaknesses - although I don't think this is an ideal way of doing so. Having said that, I'm not sure forcing the developers to document their design beforehand would have helped either, as a lot of developers simply regard documentation as a necessary evil (assuming it is enforced), and will simply write down whatever it is they intend to code, no matter how awkward the result.
I guess you have to have a mentality which loves elegance at all levels, not just in the specific lines of code that are written but how the system as a whole functions. Unfortunately (and surprisingly to me), many developers don't seem to care about the bigger picture, even when they have a deep appreciation of clever or elegant code.
You didn't read the FA. This research is testing the often-repeated claim that passphrases provide more security than passwords by looking at real data from a passphrase system. No one is claiming that this has anything to do with remote login, so you can forget that strawman. Your criticism about knowing whether someone uses a passphrase or not makes no sense either. The whole point of the research was to look at a database which was *already known* to consist of passphrases, and evaluate how much security *such a system* actually provides. Nothing more, nothing less.
So, the research is not worthless - it's actually very interesting to have some real, you know, *evidence* on the subject, rather than just emailing the usual xkcd passphrase cartoon to everyone and claiming "see, passphrases are clearly more secure!!!" (something I have been guilty of, I admit).
If everyone were always perfectly meticulous, focussed, honest, with perfect knowledge of whatever everyone else is doing, their own priorities, skills and experience. then we would need no management at all, or government for that matter.
Stand-ups are not a fix for bad management, or poor communication. They are (one) minimally formal way for a team to communicate with each other. Where I work, we have stand-ups as a team, regardless of whether our manager is present. They help us to organize ourselves.
Funnily enough, people often don't see problems they are dealing with as issues with a wider relevance, and people often have surprising skills or experience you didn't know they had.
Standup meetings are just one way to facilitate team communication, and can be done badly or well, like anything else. They may not be appropriate everywhere, or with every team.
I completely agree that code should be clearly written - but this does not obviate the need for documentation.
Good documentation should explain the why of difficult design decisions. It should also explain the general contract the code is offering - e.g. whether nulls can be returned, or under what circumstances, what errors can be produced under what circumstances, etc. I really don't want to have to read and understand every little bit of someone else's code just to be able to use it productively.
On the flipside, I have seen a lot of completely useless documentation which just re-iterates what is already blindingly clear from the code itself.
Most of your advice is solid, but this bit isn't. Encryption is hard to get right, you have to deal with key management issues, and it can give a false sense of security. For example, many inexperienced people encrypt data because they want to prevent users modifying session state - it's encrypted, so it's secure, right?. This is a mistake, as encryption only serves to protect the confidentiality of information; it does nothing to prevent modification, or replay of earlier information.
Applying encryption "whenever you can" is only a recipe for an insecure application that merely looks secure to novices.
You should design your app so that as little information as possible needs to be encrypted or otherwise protected in the first place. Minimise the amount of information that is actually required to be secured, and minimise the cryptographic secrets you must maintain to achieve security. But get someone who understands cryptography and security to do threat modelling and work out a good security architecture.
OK, I can grok that, thanks for taking the time to explain. Understanding what problem is being solved goes a very long way to making the rest of it comprehensible.
I've been programming about the same amount of time as you, and I can't claim to be much better at explaining my own technologies.:)
I just read through your whole site, but I have no idea what MSS Code Factory is, what problems it solves, what it would do for me or why I would want it. I get that it generates code from a model using a bunch of rules, much faster than a human could type it - but that is rarely the critical problem in writing code.
Now, I've worked with various code generators and high level modelling tools for decades, and none of them have been worth the effort so far. The effort involved in learning the system, working around the inevitable bugs in it (usually involving having to keep patching the output again and again as the model evolves), and interfacing with the bits the model can't handle just make them an exercise in frustration. It's a bit like the old regular expression joke.
I'm not saying yours is the same; for all I know it's totally awesome and doesn't suffer from the same problems as all the other ones I've had to deal with - but you are so into your own project that you've forgotten how to explain why anyone else should care.
Take a small step back, put yourself in our shoes, and tell us why we need it.
I've sifted through hundreds of CVs and interviewed over 50 people. It's hard finding good developers. I don't think puzzles or other forms of intelligence test by proxy are a good way of finding them.
I try to get candidates to talk about the work they've done and communicate what was interesting about it. That one filters out those who don't really care. I also ask about situations where a commonly held way of doing something should NOT be applied. That filters out the ones who've got all the "right" answers by book learning, but haven't actually thought about it themselves. I also ask some ambiguous questions to see how they deal with them.
I've offered jobs to people who got many, many details wrong, but clearly demonstrated they were into writing good software and were easy to work with.
Indeed. Because if the world is warming to dangerous levels but it's not our fault, then of course we shouldn't do anything about it! It's natural, so it must be good.
I just left an organization I'd been in for 8 years, so I was also leaving behind a fair few systems. The guy who was taking over most of them had all the documentation he needed to operate the systems. So I do agree that documentation is really important (although sadly often neglected).
What he didn't understand was why it was like that in the first place. What came out of our discussions before I left was the history - the decisions, compromises, optimizations, and mistakes that shaped them. I could see that afterwards he really got it, and felt confident to take them forwards. It made me realize how almost all documentation presents things as finished products, leaving out how it arrived in that shape.
Of course, if you're hit by a bus, all you have is the documentation.
Retention policy rules have to be fairly simple (in the paper world and the digital world), or they simply don't get applied. But each organisation is supposed to have their own specialised retention policy which spells out exactly what kinds of material should be maintained, archived or destroyed.
Having spent nearly 8 years working for the UK National Archives, I can attest that the main problem with managing digital records is that retention policies are generally not applied at all to digital material. The stuff just builds up in file servers, databases, email systems and other apps. Record managers are still largely stuck in the paper world and don't see the playthings of the IT department as anything to do with them.
In this case, there appears to have been an active internal policy of deleting certain kinds of digital material - when this material should have been retained according to the retention policy agreed with NARA. Anytime I see a department deleting digital material, it's because they really, really don't want to hold on to it. The default position of doing nothing is the norm.
Well, it's good that it isn't looking at a single company then. And that they're not trying to build a starship before it's remotely possible.
The whole problem they're looking at is to try to figure out exactly how investment in and commitment to very long term projects (but ones with profitable spin offs along the way) can be sustained. A starship is a good basis for such a project; it's not doable with current technologies, but we can already imagine ways in which it could be achieved. The problem is probably not primarily technological (without wishing to trivialise those problems). It's more of an organisational, possibly psychological, and certainly economic issue.
The 100 year starship project is supposed to study what it will take to sustain private sector investment into a long range program of building a starship.
I don't think homomorphic encryption provides values for true and false or provides comparison operators - only one operation (e.g. addition) or two operations (e.g. addition & multiplication over a ring).
I do like the idea of arriving at known values through some basic arithmetic though - I'm just not sure the operations provided permit this.
The whole point is that, without the decryption key, you don't know what the result of any processing is. Even were an attacker be able to process the data in some way, they would not know what the result meant. Remember that encryption only deals with confidentiality, not integrity. If you want to be sure that an attacker has not processed the data in some way, you would need to add integrity protection of some sort to your data in addition.
Anyway, since you don't have to decrypt the data first in order to perform operations on it, you can do most of your processing without ever having the data in the clear. Only when you want to learn a result do you need to decrypt. I'm really not sure why you don't see the potential security benefits of such a scheme. The overhead has no bearing on the security, other than its slowness preventing it from being used in the first place.
I totally agree with what you say. Agile is not the absence of planning. You still have to do a lot of requirements gathering and analysis. Some architectural issues need careful spiking to ensure you will meet non-functional requirements.
Agile is not anarchy - it's about controlled and collaborative change in exploring a complex problem space.
That isn't Agile development - it's just poor development. Good agile development doesn't cede control of the development process from developers to customers, any more than good waterfall takes control away from customers and gives it to developers.
In both cases, it requires strong product management - and this is not undertaken by the customers or the developers in either case. Taking scrum as an example, you have the role of product owner, whose job it is to ensure that the right things are being built. Scrum only provides a way for customer feedback to occur and influence the development process during development, as opposed to waterfall where requirements are agreed up-front. One good thing about scrum (over waterfall) is it explicitly has processes for the development team to agree how much can be built, and how it will be built (which can be absent in a waterfall process).
So I strongly disagree with your assertion that agile takes control away from developers. When it works well, it creates a highly collaborative environment in which both customers and developers agree on what will be built, how long it will take, and whether further refinements are necessary. However, this does require a strong product owner to interface between them.
In general, I agree with you. No methodology is a silver bullet that will magically transform a bunch of mediocre people into a tight, intelligent productive team. In fact, I would go so far as to say that most methodologies exist to mitigate the effects of the mediocre. A really good team of people can produce great code no matter what methodology is presented to them (even if they have to work around bits of it sometimes).
I successfully introduced Agile development into my last organisation, which was very much a traditional waterfall style, make-promises-we-can't-keep-but-at-least-there's-a-plan sort of place. We adapted the methodology to fit the organisation to some extent (mostly scrum, with a bit of kanban thrown in, and interfacing it all with traditional Gant chart people). It worked pretty well - mostly because the people we had were really committed to making it work, and I worked double-time on getting the organisation on board, particularly inviting them to sprint-reviews. It created a buzz - and the organisation was sold on it - which helped a lot.
In my new role in a much smaller organisation, I'm not so sure it will work as well - so I'm holding off a bit until I get the measure of the place. I have really talented developers - but communication and visibility is not a big issue due to the small size of the place, and people tend to work individually on tiny projects rather than collaborate as a team on small bits of shared functionality.
Anyway, Agile won't make mediocre people great, and Waterfall won't make great people mediocre. There are advantages and disadvantages to each style of development. The one thing you really do need to make great code is at least a few great (or at least, highly committed) people.
Ted Nelson really pioneered hypertext concepts - there is no being different for the sake of being different here. What he originally proposed is different to the web - it is far more general and powerful (and not really related to the interface - more about the underlying information model).
If anything, the reality is the opposite of what you suggest. The web is really quite poor at doing a whole bunch of things - but it's what we've got, so clever people have spent time adapting the world of computing to it.
He wrote a very entertaining (if grumpy and a somewhat biased) book about all the things which are wrong with computing called "Geeks bearing gifts".
By brute forcing, I assume you mean find another input that hashes to a predetermined value, not finding two inputs which happen to hash to the same value. Without some mathematical attacks, this very unlikely, no matter how fast the hash algorithm is.
A slower hash will make dictionary attacks on salted password tables more difficult - but generally you would simply run the hash many times to increase the computational load for an attacker. Again, the raw speed of the hash algorithm is unlikely to make much of a difference.
Well, I'm currently working on a project that uses Hibernate in Java, and it's really quite hideous. I speak as someone who designed an object-relational mapping system ten years ago. Hibernate is much more powerful than anything I ever hoped to do - and it still sucks. We're talking about an application with 2 core classes, with a one-to-many relationship. Three tables in the underlying database. So it's about as simple a data model as you can get.
To get it to perform, we're having to manually add indexes to the database instead of relying on hibernates automatic index generation. We're having to specify access patterns in queries and annotations. We have queries with sub-selects, which hibernate just doesn't do well. We have the need to return dynamically generated values along with the objects, which is not obvious with Hibernate, but easy in SQL. We have a mixture of HQL (Hibernate Query Language), JP-QL (Java Persistence Query Language), annotations in the code to specify mappings, and some XML mapping configuration.
It's all so bl**dy complicated, it's not obvious what works well and what doesn't, and what to do about it if it doesn't work well. So we're fighting this thing all the way, and increasingly dropping back down to SQL, which is mature and well understood.
The thing I've learned about Java development and developers is they are in love with interfaces, abstraction and frameworks. And its very nice to have standard ways of doing things, and pluggable interfaces, and modularity, and all that goodness.
In many ways, this code has all the standardisation and modularity that I've been pushing as good things for years. But I guess you can have too much of a good thing. Sometimes, it really is better just to write a simpler thing yourself, instead of wrestling with ridiculously powerful APIs which are trying to satisfy everyone, and yet somehow end up never quite doing the "obvious" thing you happen to want to do.
Slashdot Mirror
I'm currently writing up API documentation for a large code branch which was never properly documented (and wasn't written by me), but now needs merging into trunk. I've found several serious bugs in the code as a result, all from trying to explain to the client how to use the API. These bugs were actually blindingly obvious when the behaviour of the code had to be explained.
I've also found some horrible design issues, where various settings the code allowed were contradictory or meaningless, or one setting overrode the behaviour of another, unless something else had been configured, in which case... you get the idea. As soon as you try to explain it, the awkwardness makes the design problems incredibly obvious - and in fact a much better way of doing it is also obvious. I can almost picture the evolutionary process by which this code was written, with each developer adding a new feature without regard to how the whole thing hung together.
So documenting after the fact can definitely detect bugs and design weaknesses - although I don't think this is an ideal way of doing so. Having said that, I'm not sure forcing the developers to document their design beforehand would have helped either, as a lot of developers simply regard documentation as a necessary evil (assuming it is enforced), and will simply write down whatever it is they intend to code, no matter how awkward the result.
I guess you have to have a mentality which loves elegance at all levels, not just in the specific lines of code that are written but how the system as a whole functions. Unfortunately (and surprisingly to me), many developers don't seem to care about the bigger picture, even when they have a deep appreciation of clever or elegant code.
You didn't read the FA. This research is testing the often-repeated claim that passphrases provide more security than passwords by looking at real data from a passphrase system. No one is claiming that this has anything to do with remote login, so you can forget that strawman. Your criticism about knowing whether someone uses a passphrase or not makes no sense either. The whole point of the research was to look at a database which was *already known* to consist of passphrases, and evaluate how much security *such a system* actually provides. Nothing more, nothing less.
So, the research is not worthless - it's actually very interesting to have some real, you know, *evidence* on the subject, rather than just emailing the usual xkcd passphrase cartoon to everyone and claiming "see, passphrases are clearly more secure!!!" (something I have been guilty of, I admit).
If everyone were always perfectly meticulous, focussed, honest, with perfect knowledge of whatever everyone else is doing, their own priorities, skills and experience. then we would need no management at all, or government for that matter.
Stand-ups are not a fix for bad management, or poor communication. They are (one) minimally formal way for a team to communicate with each other. Where I work, we have stand-ups as a team, regardless of whether our manager is present. They help us to organize ourselves.
Funnily enough, people often don't see problems they are dealing with as issues with a wider relevance, and people often have surprising skills or experience you didn't know they had.
Standup meetings are just one way to facilitate team communication, and can be done badly or well, like anything else. They may not be appropriate everywhere, or with every team.
I completely agree that code should be clearly written - but this does not obviate the need for documentation.
Good documentation should explain the why of difficult design decisions. It should also explain the general contract the code is offering - e.g. whether nulls can be returned, or under what circumstances, what errors can be produced under what circumstances, etc. I really don't want to have to read and understand every little bit of someone else's code just to be able to use it productively.
On the flipside, I have seen a lot of completely useless documentation which just re-iterates what is already blindingly clear from the code itself.
Encrypt whatever you can, whenever you can.
Most of your advice is solid, but this bit isn't. Encryption is hard to get right, you have to deal with key management issues, and it can give a false sense of security. For example, many inexperienced people encrypt data because they want to prevent users modifying session state - it's encrypted, so it's secure, right?. This is a mistake, as encryption only serves to protect the confidentiality of information; it does nothing to prevent modification, or replay of earlier information.
Applying encryption "whenever you can" is only a recipe for an insecure application that merely looks secure to novices.
You should design your app so that as little information as possible needs to be encrypted or otherwise protected in the first place. Minimise the amount of information that is actually required to be secured, and minimise the cryptographic secrets you must maintain to achieve security. But get someone who understands cryptography and security to do threat modelling and work out a good security architecture.
... from BSD licensed code, you preserve the copyright notice and the license, right?
Thought not, although that's about all the license requires you to do if you redistribute the code.
People seem to think BSD carries no obligations whatsoever. I guess I've never heard of anyone suing for violation of the license.
OK, I can grok that, thanks for taking the time to explain. Understanding what problem is being solved goes a very long way to making the rest of it comprehensible.
I've been programming about the same amount of time as you, and I can't claim to be much better at explaining my own technologies. :)
Sorry - just like the OP I'm assuming everyone knows what I know! It's a common problem...
"Some people, when confronted with a problem, think
“I know, I'll use regular expressions.” Now they have two problems."
http://regex.info/blog/2006-09-15/247
I just read through your whole site, but I have no idea what MSS Code Factory is, what problems it solves, what it would do for me or why I would want it. I get that it generates code from a model using a bunch of rules, much faster than a human could type it - but that is rarely the critical problem in writing code.
Now, I've worked with various code generators and high level modelling tools for decades, and none of them have been worth the effort so far. The effort involved in learning the system, working around the inevitable bugs in it (usually involving having to keep patching the output again and again as the model evolves), and interfacing with the bits the model can't handle just make them an exercise in frustration. It's a bit like the old regular expression joke.
I'm not saying yours is the same; for all I know it's totally awesome and doesn't suffer from the same problems as all the other ones I've had to deal with - but you are so into your own project that you've forgotten how to explain why anyone else should care.
Take a small step back, put yourself in our shoes, and tell us why we need it.
I've sifted through hundreds of CVs and interviewed over 50 people. It's hard finding good developers. I don't think puzzles or other forms of intelligence test by proxy are a good way of finding them.
I try to get candidates to talk about the work they've done and communicate what was interesting about it. That one filters out those who don't really care. I also ask about situations where a commonly held way of doing something should NOT be applied. That filters out the ones who've got all the "right" answers by book learning, but haven't actually thought about it themselves. I also ask some ambiguous questions to see how they deal with them.
I've offered jobs to people who got many, many details wrong, but clearly demonstrated they were into writing good software and were easy to work with.
Indeed. Because if the world is warming to dangerous levels but it's not our fault, then of course we shouldn't do anything about it! It's natural, so it must be good.
I just left an organization I'd been in for 8 years, so I was also leaving behind a fair few systems. The guy who was taking over most of them had all the documentation he needed to operate the systems. So I do agree that documentation is really important (although sadly often neglected).
What he didn't understand was why it was like that in the first place. What came out of our discussions before I left was the history - the decisions, compromises, optimizations, and mistakes that shaped them. I could see that afterwards he really got it, and felt confident to take them forwards. It made me realize how almost all documentation presents things as finished products, leaving out how it arrived in that shape.
Of course, if you're hit by a bus, all you have is the documentation.
Retention policy rules have to be fairly simple (in the paper world and the digital world), or they simply don't get applied. But each organisation is supposed to have their own specialised retention policy which spells out exactly what kinds of material should be maintained, archived or destroyed.
Having spent nearly 8 years working for the UK National Archives, I can attest that the main problem with managing digital records is that retention policies are generally not applied at all to digital material. The stuff just builds up in file servers, databases, email systems and other apps. Record managers are still largely stuck in the paper world and don't see the playthings of the IT department as anything to do with them.
In this case, there appears to have been an active internal policy of deleting certain kinds of digital material - when this material should have been retained according to the retention policy agreed with NARA. Anytime I see a department deleting digital material, it's because they really, really don't want to hold on to it. The default position of doing nothing is the norm.
Well, it's good that it isn't looking at a single company then. And that they're not trying to build a starship before it's remotely possible.
The whole problem they're looking at is to try to figure out exactly how investment in and commitment to very long term projects (but ones with profitable spin offs along the way) can be sustained. A starship is a good basis for such a project; it's not doable with current technologies, but we can already imagine ways in which it could be achieved. The problem is probably not primarily technological (without wishing to trivialise those problems). It's more of an organisational, possibly psychological, and certainly economic issue.
The 100 year starship project is supposed to study what it will take to sustain private sector investment into a long range program of building a starship.
http://www.100yss.org/about.html
It is not itself a 100 year project to build a starship, or a 100 year project to figure out how to sustain investment...
Also, if you're interested in interstellar research, check out Centauri Dreams:
http://www.centauri-dreams.org/
I don't think homomorphic encryption provides values for true and false or provides comparison operators - only one operation (e.g. addition) or two operations (e.g. addition & multiplication over a ring).
I do like the idea of arriving at known values through some basic arithmetic though - I'm just not sure the operations provided permit this.
The whole point is that, without the decryption key, you don't know what the result of any processing is. Even were an attacker be able to process the data in some way, they would not know what the result meant. Remember that encryption only deals with confidentiality, not integrity. If you want to be sure that an attacker has not processed the data in some way, you would need to add integrity protection of some sort to your data in addition.
Anyway, since you don't have to decrypt the data first in order to perform operations on it, you can do most of your processing without ever having the data in the clear. Only when you want to learn a result do you need to decrypt. I'm really not sure why you don't see the potential security benefits of such a scheme. The overhead has no bearing on the security, other than its slowness preventing it from being used in the first place.
I totally agree with what you say. Agile is not the absence of planning. You still have to do a lot of requirements gathering and analysis. Some architectural issues need careful spiking to ensure you will meet non-functional requirements.
Agile is not anarchy - it's about controlled and collaborative change in exploring a complex problem space.
That isn't Agile development - it's just poor development. Good agile development doesn't cede control of the development process from developers to customers, any more than good waterfall takes control away from customers and gives it to developers.
In both cases, it requires strong product management - and this is not undertaken by the customers or the developers in either case. Taking scrum as an example, you have the role of product owner, whose job it is to ensure that the right things are being built. Scrum only provides a way for customer feedback to occur and influence the development process during development, as opposed to waterfall where requirements are agreed up-front. One good thing about scrum (over waterfall) is it explicitly has processes for the development team to agree how much can be built, and how it will be built (which can be absent in a waterfall process).
So I strongly disagree with your assertion that agile takes control away from developers. When it works well, it creates a highly collaborative environment in which both customers and developers agree on what will be built, how long it will take, and whether further refinements are necessary. However, this does require a strong product owner to interface between them.
In general, I agree with you. No methodology is a silver bullet that will magically transform a bunch of mediocre people into a tight, intelligent productive team. In fact, I would go so far as to say that most methodologies exist to mitigate the effects of the mediocre. A really good team of people can produce great code no matter what methodology is presented to them (even if they have to work around bits of it sometimes).
I successfully introduced Agile development into my last organisation, which was very much a traditional waterfall style, make-promises-we-can't-keep-but-at-least-there's-a-plan sort of place. We adapted the methodology to fit the organisation to some extent (mostly scrum, with a bit of kanban thrown in, and interfacing it all with traditional Gant chart people). It worked pretty well - mostly because the people we had were really committed to making it work, and I worked double-time on getting the organisation on board, particularly inviting them to sprint-reviews. It created a buzz - and the organisation was sold on it - which helped a lot.
In my new role in a much smaller organisation, I'm not so sure it will work as well - so I'm holding off a bit until I get the measure of the place. I have really talented developers - but communication and visibility is not a big issue due to the small size of the place, and people tend to work individually on tiny projects rather than collaborate as a team on small bits of shared functionality.
Anyway, Agile won't make mediocre people great, and Waterfall won't make great people mediocre. There are advantages and disadvantages to each style of development. The one thing you really do need to make great code is at least a few great (or at least, highly committed) people.
Ted Nelson really pioneered hypertext concepts - there is no being different for the sake of being different here. What he originally proposed is different to the web - it is far more general and powerful (and not really related to the interface - more about the underlying information model).
If anything, the reality is the opposite of what you suggest. The web is really quite poor at doing a whole bunch of things - but it's what we've got, so clever people have spent time adapting the world of computing to it.
He wrote a very entertaining (if grumpy and a somewhat biased) book about all the things which are wrong with computing called "Geeks bearing gifts".
By brute forcing, I assume you mean find another input that hashes to a predetermined value, not finding two inputs which happen to hash to the same value. Without some mathematical attacks, this very unlikely, no matter how fast the hash algorithm is.
A slower hash will make dictionary attacks on salted password tables more difficult - but generally you would simply run the hash many times to increase the computational load for an attacker. Again, the raw speed of the hash algorithm is unlikely to make much of a difference.
Yes! I had the same kit. Many happy hours :)
What do you do with the hard drive when you leave?
Well, I'm currently working on a project that uses Hibernate in Java, and it's really quite hideous. I speak as someone who designed an object-relational mapping system ten years ago. Hibernate is much more powerful than anything I ever hoped to do - and it still sucks. We're talking about an application with 2 core classes, with a one-to-many relationship. Three tables in the underlying database. So it's about as simple a data model as you can get.
To get it to perform, we're having to manually add indexes to the database instead of relying on hibernates automatic index generation. We're having to specify access patterns in queries and annotations. We have queries with sub-selects, which hibernate just doesn't do well. We have the need to return dynamically generated values along with the objects, which is not obvious with Hibernate, but easy in SQL. We have a mixture of HQL (Hibernate Query Language), JP-QL (Java Persistence Query Language), annotations in the code to specify mappings, and some XML mapping configuration.
It's all so bl**dy complicated, it's not obvious what works well and what doesn't, and what to do about it if it doesn't work well. So we're fighting this thing all the way, and increasingly dropping back down to SQL, which is mature and well understood.
The thing I've learned about Java development and developers is they are in love with interfaces, abstraction and frameworks. And its very nice to have standard ways of doing things, and pluggable interfaces, and modularity, and all that goodness.
In many ways, this code has all the standardisation and modularity that I've been pushing as good things for years. But I guess you can have too much of a good thing. Sometimes, it really is better just to write a simpler thing yourself, instead of wrestling with ridiculously powerful APIs which are trying to satisfy everyone, and yet somehow end up never quite doing the "obvious" thing you happen to want to do.