Slashdot Mirror

I read the article, and he did touch on the endemic issue I've run into of "that patch didn't come from us, so it's rejected (or ignored)".

This is a bit problematic. There are actual real legal reasons for this in cases (if you don't know who wrote it you don't know for sure you have 100% rights to it) as well as a general wish to get things to work in a particular way. It's much better if, instead of rejection by a person, the maintainer can set up automatic rejection based on a script (for example based on wrong formatting / unwanted includes etc). Debian has specific experience in it's ssh system with the risks of patches and probably wants to increase, not reduce quality. Generally though, a project like Debian is never going to advance if it doesn't find a way to get most patches accepted quicky.

It's better to burn out than fade away.

- J. Shore, N. Young, J. Rotten, M. Fowler

"To truly take advantage of the cloud, software needs to be architected and implemented differently, using microservices instead of monoliths."

You mean convert all your API's into JSON calls and spin up gajillion web services? Why? That increases complexity. Native-app-language-to-native-app-language is much easier than app-language-to-JSON-back-to-native-app-language.

Can't cloud do monolith? If not, what's stopping it? The performance bottleneck usually is and should be the database anyhow for must CRUD apps. Kajillion web services won't solve that. The CAP Theorem (Eric Brewer) limits your options and probably shouldn't be an app-side concern anyhow, but mostly a database side issue.

This kind of hype created a bloated stack in our org that requires dealing with 4x more code than a normal stack would. Nobody can give practical examples of the use of such splitting: they just spit out vague buzzwords stolen from Dilbert's boss, or dreamy shit like "what if we grow to Amazon.com size"? -- Yeah Right. We are more likely to get hit by a meteor while buying a lottery ticket on a unicycle.

Plus, these extra web layers seem a security risk: more doors for hackers to pick the locks of. Who is spreading this microservice rumor/hype? Russians? Microsoft marketers? Wrox? Knock-it-off!

Martin Fowler said:

So my primary guideline would be don't even consider microservices unless you have a system that's too complex to manage as a monolith. The majority of software systems should be built as a single monolithic application. Do pay attention to good modularity within that monolith, but don't try to separate it into separate services.

As far as general IT advice:

1. Data tends to outlive application software, so focus on good data.

2. Be wary of wasteful hype. Let somebody else be the guinea pig. When that somebody else has it running well, THEN borrow the idea.

3. Books are judged by the cover for good or bad, so throw the executives a pretty bone for a few high-visibility parts of a system, but keep most of the regular stuff (grunt screens) in something easy to create and maintain. Don't drag down the entire system chasing eye-candy and UI fads. By the time you're finished, it'll be obsolete anyhow.

4. "Separation of Concerns" is a myth. Most non-trivial concerns inherently inter-weave among each other. You want to manage concerns well, not outright separate them with thick Trump Walls.

Perhaps you want to read this: https://martinfowler.com/bliki... [martinfowler.com]

Oh, I was way ahead of you and had thought of even sending you that link. The problem with Fowler's page is that it doesn't list the kind of weak ownership most companies actually use, namely something between strong and weak code ownership: teams build libraries and apps, anybody within a team can make changes to the code the team is responsible for, but other teams usually go through team members to make changes, get patches approved, etc. You know, pretty much the same way the entire FOSS ecosystem works, they way Github projects work, etc.

I never ever in the last 30 years worked for a company that hat something different than "collective code ownership" ... because as I mentioned before: I don't know what I'm doing or how to build big systems, so only small fly-by-night operations staffed by ex-baristas hire me

FTFY

And maybe you are just an idiot?

Perhaps you want to read this: https://martinfowler.com/bliki...

I never ever in the last 30 years worked for a company that hat something different than "collective code ownership" ... because as I mentioned before: code ownership is a retarded concept.

"What works for Google (or the DOD, or IBM) doesn't work for most other companies, projects, or programmers, because they operate under a completely different set of constraints." - that is VERY true.

I agree that C++ is too complex. The problem is, alternatives are even worse for other reasons. Ruby is HORRIBLE from maintainability and performance points of view. To write maintainable Ruby, one has to use TDD, which is deeply incompatible with how many people think. (See the debates between David Heinemeyer Hanson and Kent beck: http://martinfowler.com/articl...) And I have used Ruby a-lot, so I am not guessing. And Go is really awful because it throws out enormously useful features like virtual methods and exception handling, and Go also has lots of "gotchas" - my favorite is comparing a reference with nil and getting false, then type asserting it, comparing the asserted value with nil, and then getting true. Things like that will result in lots of bugs, and there are many gotchas like that. So I have turned back to C++ because, if used well (i.e., conservatively), one can write very efficient and maintainable programs - IF used well. Efficiency matters at Internet scale because if a Ruby program requires 10,000 VMs but a C++ program requires only 100, that is a very large cost saving.

Integration database vs application database. Martin Fowler (no, not the one in Eastenders).

Know which one you are using and act accordingly.

In practice, at least for internal LOB systems, application databases become integration databases - and most software is still written inside companies for use with internal data, never to see the light of day outside the corporate firewall. I recognize Martin Fowler's contributions, but I think he comes at these things from the wrong perspective. Consider this excerpt (emphasis Martin's):

On the whole integration databases lead to serious problems because the database becomes a point of coupling between the applications that access it. This is usually a deep coupling that significantly increases the risk involved in changing those applications and making it harder to evolve them. As a result most software architects that I respect take the view that integration databases should be avoided.

The problem with this view is that it fails to appreciate that most often the data is the crucial resource and primary focus of maintenance activity, not the several applications which access it. For a few years as a consultant I maintained several applications which all used one of my client's "integration" databases. One of the applications maintained the "meta" portions of the data, such as compliance with differing laws and regulations across many international jurisdictions, attribute management, language translations, and so on. Another maintained the core product data itself. The third maintained specific aspects of the representation of the products within the data and the structure of attributes for publishing to various media. There were also other LOB applications with which I had no interaction, such as for generating content for publishing products onto web sites, brochures, dead-tree catalogs, etc.

For such a database, you have two traditional options (I'll get to the third approach preferred by Martin, bounded contexts, later) for enforcing data integrity: put it in the database itself as sprocs, and so on; build a middleware layer. Of course, one could employ a mix of both traditional approaches. Both of those tacks lead to the coupling Martin warns us to avoid. However, attempting to implement consistent data integrity into all those disparate systems is an obvious recipe for disaster (and of course, Martin doesn't recommend that). What Martin fails to properly appreciate here is that this specific kind of coupling is inherent to the greater precedence of data integrity over the maintenance of multiple distinct applications each accessing some subset of it. More importantly, he does not accept that this is a rare example of coupling being a good thing. IME, agile software proponents tend to be uniformly extremist as to whether a given metric equates to "bad" or "good" outcomes; for example, imagine Mr. Mackey (from South Park) saying "Coupling is bad, mmmkay".

Martin's skewed (IMHO) view is further illuminated by this excerpt:

One the great advantages of an application database is that it is easier to change since all its use is encapsulated by a single application.

No, just no, at least not for most LOB systems. Overwhelmingly, it's the data itself (including its structure) that determines what the applications which access it must do, not the other way around.

Martin asserts that a unified domain tends to develop unmanageable complexity, with domain-driven design and bounded contexts as the preferred mitigation:

Different groups of people will use subtly different vocabularies in different parts of a large organization. The precision of modeling rapidly runs into this, of

Integration database vs application database. Martin Fowler (no, not the one in Eastenders).

Know which one you are using and act accordingly.

In practice, at least for internal LOB systems, application databases become integration databases - and most software is still written inside companies for use with internal data, never to see the light of day outside the corporate firewall. I recognize Martin Fowler's contributions, but I think he comes at these things from the wrong perspective. Consider this excerpt (emphasis Martin's):

On the whole integration databases lead to serious problems because the database becomes a point of coupling between the applications that access it. This is usually a deep coupling that significantly increases the risk involved in changing those applications and making it harder to evolve them. As a result most software architects that I respect take the view that integration databases should be avoided.

The problem with this view is that it fails to appreciate that most often the data is the crucial resource and primary focus of maintenance activity, not the several applications which access it. For a few years as a consultant I maintained several applications which all used one of my client's "integration" databases. One of the applications maintained the "meta" portions of the data, such as compliance with differing laws and regulations across many international jurisdictions, attribute management, language translations, and so on. Another maintained the core product data itself. The third maintained specific aspects of the representation of the products within the data and the structure of attributes for publishing to various media. There were also other LOB applications with which I had no interaction, such as for generating content for publishing products onto web sites, brochures, dead-tree catalogs, etc.

For such a database, you have two traditional options (I'll get to the third approach preferred by Martin, bounded contexts, later) for enforcing data integrity: put it in the database itself as sprocs, and so on; build a middleware layer. Of course, one could employ a mix of both traditional approaches. Both of those tacks lead to the coupling Martin warns us to avoid. However, attempting to implement consistent data integrity into all those disparate systems is an obvious recipe for disaster (and of course, Martin doesn't recommend that). What Martin fails to properly appreciate here is that this specific kind of coupling is inherent to the greater precedence of data integrity over the maintenance of multiple distinct applications each accessing some subset of it. More importantly, he does not accept that this is a rare example of coupling being a good thing. IME, agile software proponents tend to be uniformly extremist as to whether a given metric equates to "bad" or "good" outcomes; for example, imagine Mr. Mackey (from South Park) saying "Coupling is bad, mmmkay".

Martin's skewed (IMHO) view is further illuminated by this excerpt:

One the great advantages of an application database is that it is easier to change since all its use is encapsulated by a single application.

No, just no, at least not for most LOB systems. Overwhelmingly, it's the data itself (including its structure) that determines what the applications which access it must do, not the other way around.

Martin asserts that a unified domain tends to develop unmanageable complexity, with domain-driven design and bounded contexts as the preferred mitigation:

Different groups of people will use subtly different vocabularies in different parts of a large organization. The precision of modeling rapidly runs into this, of

Integration database vs application database. Martin Fowler (no, not the one in Eastenders).

Know which one you are using and act accordingly.

In practice, at least for internal LOB systems, application databases become integration databases - and most software is still written inside companies for use with internal data, never to see the light of day outside the corporate firewall. I recognize Martin Fowler's contributions, but I think he comes at these things from the wrong perspective. Consider this excerpt (emphasis Martin's):

On the whole integration databases lead to serious problems because the database becomes a point of coupling between the applications that access it. This is usually a deep coupling that significantly increases the risk involved in changing those applications and making it harder to evolve them. As a result most software architects that I respect take the view that integration databases should be avoided.

The problem with this view is that it fails to appreciate that most often the data is the crucial resource and primary focus of maintenance activity, not the several applications which access it. For a few years as a consultant I maintained several applications which all used one of my client's "integration" databases. One of the applications maintained the "meta" portions of the data, such as compliance with differing laws and regulations across many international jurisdictions, attribute management, language translations, and so on. Another maintained the core product data itself. The third maintained specific aspects of the representation of the products within the data and the structure of attributes for publishing to various media. There were also other LOB applications with which I had no interaction, such as for generating content for publishing products onto web sites, brochures, dead-tree catalogs, etc.

For such a database, you have two traditional options (I'll get to the third approach preferred by Martin, bounded contexts, later) for enforcing data integrity: put it in the database itself as sprocs, and so on; build a middleware layer. Of course, one could employ a mix of both traditional approaches. Both of those tacks lead to the coupling Martin warns us to avoid. However, attempting to implement consistent data integrity into all those disparate systems is an obvious recipe for disaster (and of course, Martin doesn't recommend that). What Martin fails to properly appreciate here is that this specific kind of coupling is inherent to the greater precedence of data integrity over the maintenance of multiple distinct applications each accessing some subset of it. More importantly, he does not accept that this is a rare example of coupling being a good thing. IME, agile software proponents tend to be uniformly extremist as to whether a given metric equates to "bad" or "good" outcomes; for example, imagine Mr. Mackey (from South Park) saying "Coupling is bad, mmmkay".

Martin's skewed (IMHO) view is further illuminated by this excerpt:

One the great advantages of an application database is that it is easier to change since all its use is encapsulated by a single application.

No, just no, at least not for most LOB systems. Overwhelmingly, it's the data itself (including its structure) that determines what the applications which access it must do, not the other way around.

Martin asserts that a unified domain tends to develop unmanageable complexity, with domain-driven design and bounded contexts as the preferred mitigation:

Different groups of people will use subtly different vocabularies in different parts of a large organization. The precision of modeling rapidly runs into this, of

RancherOS appears to be what people want since it uses Docker itself to boot.

As far as the future Microservices is were everything appears to be headed.

I agree. Something I read 10 years ago really resonated with me, and it refutes this whole idea. From The New Methodology (2005):

Separation of Design and Construction

"The usual inspiration for methodologies is engineering disciplines such as civil or mechanical engineering. . . . Many design decisions, such as how to deal with the load on a bridge, are made as the drawings are produced. The drawings are then handed over to a different group, often a different company, to be built. . . . This allows the construction to be less skilled intellectually, although they are often very skilled manually. . . . So what we see here are two fundamentally different activities. Design which is difficult to predict and requires expensive and creative people, and construction which is easier to predict.

". . . But here lies the crucial question. Can you get a design that is capable of turning the coding into a predictable construction activity? . . . The problem with a UML-like design is that it can look very good on paper, yet be seriously flawed when you actually have to program the thing. . . . This raises an important question about the nature of design in software compared to its role in other branches of engineering.

"These kinds of questions led Jack Reeves to suggest that in fact the source code is a design document and that the construction phase is actually the use of the compiler and linker. Indeed anything that you can treat as construction can and should be automated.

The Web is designed to not need APIs or dedicated documentation. You might have heard of REST - it's a set of constraints on network architecture designs, which are: client-server protocol, layered interface, caching, stateless connections, code-on-demand (e.g. Javascript), and a uniform interface (e.g. HTML). REST is defined by the principal author of HTTP, Roy T. Fielding.

Lots of people call themselves RESTful (Amazon, Twitter) but aren't even close. A RESTful service is pretty much just like a website: You enter at an entry point, then start following hyperlinks and filling out forms to manipulate the state of your client and the server (respectively).

If you have an existing plain-old-HTTP API, you might want to build a hypertext interface on top of it that users can browse and submit documents with. Use a hypermedia data format like JSON-LD, Hydra, or JSON Hyper-schema to expose machine-readable hypertext. HTML is perfectly fine too, and preferable if you want to navigate the API with a Web browser.

See:

http://martinfowler.com/articl...

https://web.archive.org/web/20...

http://www.amazon.com/RESTful-...

There are certainly Agile teams that work that way, but that is not how well run teams operate. Agile has a reputation for not planning, but the opposite is true - there is a-lot of planning in Agile. Well run teams also design. The difference is that Agile teams _allow_ requirements to change as a result of customer feedback, instead of trying to stick to a set of "agreed upfront requirements". Agile teams work at a steady pace, producing working software at regular intervals, instead of waiting for a huge release six months down the road - when it is often discovered that the design is unworkable. Agile projects can go off the rails just like any project, but in an Agile project, you find out very soon, instead of toward the end. There is no magic bullet for this stuff. The intent of Agile is to divide the work up into small items that are deliverable incrementally. It is not rocket science. You still have to do that dividing intelligently, and you still have to practice software design. Some in the Agile community disparage design somewhat, advocating test-driven development, but that point of view has been largely discredited. See http://martinfowler.com/articl...

Testability is critical, but it's pointless without actual tests. By tests, I do not mean a single test case verifying only the "normal" path where everything goes exactly as expected. I mean testing every boundary case, varying degrees of complexity where applicable, and failure modes, ensuring that it fails in the way that it is intended to fail and in every case where it is intended to fail.

(the truth)
All this might be wonderful, but very expensive. I'm not here to tell you how you should do your job, or how other developers should do theirs. but I'm just saying that in my experience, where you have unhappy developers, you might have some unhappy code, and that means the agility of the project to be able to move forwards with new features is in question, as I'd predict new features taking exponentially more time to implement, as it's likely the code if suffering from a lack of separation of concerns.

The warning it there from the developer, but the stakeholders are happy. Case closed, move on... until that change request comes in, and the new developer says it's going to take 10x longer than the estimate! Hence, the "just get it working" approach - which further kicks the can down the road called "technical debt". Sooner or later the company pays for it, and often it's with a complete re-write in a few years!

Well said.

This is indeed a worst case scenario, which unfortunately play out too often. It is most often caused by not having enough developers on the "His side" of the argument. I for instance am a senior level developer who is very involved in the business side of my employer's decisions. This probably comes from my time as a contractor where I have also seen these his side, her side, and the truth arguments play out time and again. A company that does not have enough developer minded people in CTO, VP of Software Development, Director of Application Architecture, etc. positions is going to find themselves in your worst case scenario quite often. Most well run companies I have worked for and contracted for have CS/EE majors with MBAs (or equivalently minded people) in the positions I listed above, so "His Side" in these companies have many of the best arguments from "Her Side" already included.

Technical debt is a necessity in any product. Being aware of this debt and estimating its impact going forward is very much a part of any quality change management process. Martin Fowler has a great blog post describing the relationship between prudent, reckless, deliberate, and inadvertent technical debt that is worth a read. I particularly like his reasoning behind the inadvertent / prudent quadrant.

If you don't take proper care of your technical debt, it will ruin any software product. But properly managing your technical debt is not the same thing as just taking "Her Side" every time she complains about technical debt.

If these are as critical services as you say, I would assume you have some sort of redundancy, at least a 2nd server somewhere. If so, treat each as "throw away", build out what you need on the alternative server, swing DNS and be done. Rinse and repeat for the next 'upgrade'. Then do your work in the middle of the day. See Immutable Servers: http://martinfowler.com/bliki/...

Another approach rather than in-place same-tech modernization is the Strangler Pattern, coined by Martin Fowler.

This involves building a new application (or applications, utilities can be important) "around the edges" of the existing one, rewriting functionality aspects into a new system; chipping away at the old system (give the users a better solution for some things and they will not mind working with two systems).

Here's Mr. Fowler's original article about such:
http://martinfowler.com/bliki/...

Read this: http://martinfowler.com/books/... Then realise that all organisations are sitting on top of mountains of crappy legacy code, and developers that can deal with it are the most genuinely valuable people in the marketplace.

This was eloquently described by Phil Karlton, and extended by Martin Fowler:

There are only 2 hard things in computer science: Cache invalidation, the naming of things, and off-by-one errors.

http://martinfowler.com/bliki/TwoHardThings.html

Nowhere on the list did I see cache invalidation. What gives?

I guess it's the third item on the list at work, only the other way.

To all the posters here who seem to have no real idea about Java http://martinfowler.com/articles/lmax.html I work in the City of London and although it is only a recent development many low latency systems are being written in Java, due to the faster time to market, the G1 GC and deterministic garbage collectors. You may be interested to know that the world's largest investment banks have most of their trading, risk and flow systems in Java so all these comments about Java being unstable, crap or slow just aren't true.

He says it quite nicely:

http://martinfowler.com/bliki/FlaccidScrum.html

Of course that was in 2009. Nothing has changed, and I've long past the point of being fed up with the non-technical fuck-tards that think they can sprinkle Scrum-dust on a mountain of technical debt and it'll go away. This is usually done in the presence of a stable of bad developers who lack the discipline to do the actual hard work of the XP practices that deliver good products in the first place.

The parent article author can STFU already. It just reeks of, "Wah! My agile hurts me because I won't do the hard stuff".

Oh, and while your at it agile wimps: stop fucking trying to do "distributed agile" with fucking China and fucking India in order to save 30% on what's already a crap-pile due to communication problems. It's not going to help one bit.

Also, get off my lawn...

You might consider joining ThoughtWorks (if you can get in.)
http://www.thoughtworks.com/mission-and-values
http://martinfowler.com/bliki/ThreePillars.html

They are a global IT consulting/custom software development company with offices in 10 countries.

ThoughtWorks organizes their business around three pillars, the third being "Advocate passionately for social and economic justice." That means that the company generally avoids the same sort of work you would like to avoid AND seeks out work that will make the world a better place.

They also do a lot of cutting edge stuff... they were pioneers in agile and continuous integration 10 years ago when both of those were considered kooky by the mainstream.

I don't know how much --if any-- "mathematically ambitious" work they've had using CUDA but you could be the guy who brings that capability to their customers.

If you don't like dynamic languages, fine, but implying that folks who choose to embrace FP and dynamic languages are "hobbyists" serves no purpose other than to expose your lack of depth as a developer.

Read this and take a long hard look at yourself.

I inject a little (apparently very well targeted) teasing into a thread full of uninformed anti-Java ragefacing, and you're judging *me*?

My implication served my purpose just fine (hi there!), and may or may not reflect my actual feelings on dynamic languages.

It's that some of us have put up with statically typed languages for 15 years, and have had enough of that crap.

I've had enough of this crap.

Let's be friends! Nay, will you be my pair programming partner? Imagine the insights and perspectives we could share! Imagine the paradigms we could shift, the behavior-driven mocking we could exchange while rebasing our git trees and eating Mongos. Yes, Mongos!

WILL YOU ACCEPT MY PULL REQUEST?

So hot.

It's not that we're missing out. It's that some of us have put up with statically typed languages for 15 years, and have had enough of that crap.

If you don't like dynamic languages, fine, but implying that folks who choose to embrace FP and dynamic languages are "hobbyists" serves no purpose other than to expose your lack of depth as a developer.

Read this and take a long hard look at yourself.

Don't define functions inline of your statement.

Why? What is the negative consequence of doing so. If the function only needs to be referenced once, what is wrong with defining it where it will be referenced? It is more direct, and makes the action of the call to click() clearer.

Don't perform evaluations in your return statements.

Why? In what way is this:

var newElement = $("#showhidecontrol").clone().click(function () { hideableElement.toggleClass("hidden"); })});
return newElement;

clearer than

return $("#showhidecontrol").clone().click(function () { hideableElement.toggleClass("hidden"); })});

?

Avoid calling methods/functions on objects via the return of another evaluator.

Why? The API is a fluent API, specifically designed to allow this. Why not use this feature? Are you saying that respected programming writers like (e.g.) Joshua Bloch and Martin Fowler are wrong to recommend this style?

The code sample he provided is counter to reusability

I'd say it's very reusable: I can simply include this code fragment on any page I want hideable elements on, and it's reused. If necessary, I can wrap it in a function. If I wanted to make it more reusable, it wouldn't be hard to turn it into a jquery plugin, so then all I'd need to do is $(".hideable").addHideControl();

But in the end, how reusable do you need a 3-line code fragment that was written in less than 5 minutes to be?

readability

There are a lot of people who'd disagree with you there. Yes, you need to be vaguely familiar with jquery to understand what I wrote, but if you aren't familiar with the tools I'm using, what are you doing trying to maintain my code?

unsafe in that it doesn't evaluate possible nulls

There are no possible nulls. $(selector) is defined to always return a result set (which is possibly empty). The clone() method always returns a copy of the object set it was called on. The click() method always returns the object it was called on.

and arguably impossible to debug

I was debugging very similar code only yesterday, and didn't find it even remotely difficult.

That was the whole point of this question: he wants to learn from master practitioners before making mistakes i.e. drowning.

camservo: I don't know of a particular program to learn, but I found great pleasure reading the JVM's source code. I do recommend to be suspicious of everything you read. You may find code that doesn't look right, e.g. excessive use of Header Interfaces. The best thing to do is ask other people why they used it there.

There's an interesting writeup of a trading system implemented in java:

"The system is built on the JVM platform and centers on a Business Logic Processor that can handle 6 million orders per second on a single thread. The Business Logic Processor runs entirely in-memory using event sourcing."

The LMAX Architecture

If I'm getting this right, scientists view software as nothing more than a specialized calculator.

I can certainly confirm that as an engineer, that's certainly how I view it. Almost literally in most cases.

But I have never deluded myself that our really complicated calculator programs would be appropriate for some sort of deliverable product for general use. In fact I repeatedly try to make that point and people still don't get it. And usually "Software engineers" expect a bunch of their absurd rituals to be followed anyway, even if it's never going to go anywhere or be used by anyone but me. So I would contend that the professional programmers and particularly the software process types are the ones who don't grasp the concept.

Brett

Brett

Here's a tool that may be helpful to you in explaining the issue to various people: "Technical Debt". Your quick-n-dirty code that does what you need it to do carries a great deal of technical debt, but the nature of its limited use means the interest payments are small, so there's no value in paying down the debt. Start trying to turn it into a product to be used by many people, however, and you'll quickly find that the interest payments are unbearable, and that it costs a great deal of effort to pay off the technical debt, to turn the code into something that is maintainable.

You can thank the software engineers for that elegant and powerful notion :-)

What do you mean by technical debt?

This is the definitive article: http://www.martinfowler.com/bliki/TechnicalDebt.html

I'd personally rather spend more of my time designing applications and less time hammering out the plumbing.

This can already be achieved through use of proper abstractions, design patterns, and Inversion of Control containers. I would argue that this is a more modern and elegant solution than the sorts of "code generation" proposed by the CASE tools. IMHO, automatic code generation is often a brute force solution to problems where better abstractions would result in a more elegant and simple solution. Personally, I prefer elegance in code rather than ugly brute force tool use, but maybe that is just me.

Virtualization can be valuable in this context yes, but with proper Test Driven Development, Inversion of Control containers, and object mocking frameworks it should be possible to do most of the testing without having to spin up an entire virtual simulation of the execution environment. Combine this with build and deployment scripting and continuous integration for a really first-class development operation.

Virtualization can be valuable in this context yes, but with proper Test Driven Development, Inversion of Control containers, and object mocking frameworks it should be possible to do most of the testing without having to spin up an entire virtual simulation of the execution environment. Combine this with build and deployment scripting and continuous integration for a really first-class development operation.

Every piece of data that comes from the user must be editable in the future

This is not always as easy as it first sounds. Often, the state of the object as it existed at some point in the past must be preserved, even if that is not the present state of the object as of that date in the past. If you are confused, then take a look at Martin Fowler's article on Temporal Objects for an introduction to how deep the rabbit hole goes with "effective dating".

Any interface that allows a user to create a new database entry MUST provide a method to merge duplicate entries.

Again, combine this with effective dating and you have an interface that is so advanced that it will probably confuse most users even if it is technically implemented correctly.

computer assisted ERP is one of those ideas that sounds good in principle, but often falls short in practice. Not everyone using the ERP system is going to be an engineer after all; so it is unreasonable to expect that level of sophistication and understanding, even though that may be necessary in order to get real value out of an ERP system.

I have to disagree with you here. The influence of sexism is not just about discrimination. You don't have to actively discriminate against women to drive them out of FOSS. When there's a 99% male population (not uncommon on many projects) every single comment is enough to make a woman feel unwelcome. More importantly, with that many men around, you'll always find plenty who just don't see the problem. So it happens that in a lot of instances like this, the guy in question doesn't get called out by others. The end result is that it seems to any casual observer to be okay to hate on women in general and on their technical skills in particular.

Now there are projects that are particularly welcome to newcomers and projects that are not. But when women can't find any projects where they can feel welcome, we have a problem. When they feel compelled to start their own women's groups because they feel slighted against in the main population, there is a problem. All the flamebait in this thread and the summary notwhitstanding, this is a real problem that goes beyond simple discrimination.

BTW, I agree with the above poster that Mark Shuttleworth's comments had nothing to do with sexism as such. They were just taken out of context and then misinterpreted. But if you don't see anything wrong with the RoR presentation, then I think some introspection really is in order. Martin fowler did a great post exploring the issues surrounding it and explains it a lot better than I can, as usual.

A re-write from scratch is nearly always the wrong approach.

I'm tired of hearing Joel Spolsky's opinion touted as an irrefutable fact.

"Never rewrite anything" is great advice for those who have no hope of ever understanding what their code actually does, or---more importantly---what it should do. It's not surprising that Joel, an ex-employee of Microsoft, would give that advice, but that doesn't make it universally applicable.

The reality is that it depends on a number of factors, including on how much technical debt the project has accrued and how this impacts the organization's flexibility. Having simple feature additions blow up in your face because nobody really understands the current code base is an excellent reason to ditch that code base at the earliest opportunity.

There are many other factors to consider, but what I've written suffices to show that a blanket statement like "rewrites considered harmful" is simply wrong.

You are looking at it from the wrong standpoint (speed, accuracy, security, etc) with incidental metrics. The real value of LINQ is to improve the design of your code by making generic queries first class objects supported in the language and providing an abstract interface by which any type of data source, including other objects, can be queried without having to know the source at design or compile time. As I have already said there are fringe benefits to the LINQ implementation as well including extension methods, lambda expressions, anonymous data types, and partial methods. The benefits lie in the abstraction of the concept of "query". It sounds like you are looking at the benefits of LINQ at too low of a level to fully appreciate what all of that brings to C# and .NET going forward. Let me ask you this, are you the type of person that hacks together the first thing that comes to mind or do you use Design Patterns and Dependency Injection and carefully consider abstractions in your code? If you are the former rather and not the later then I am not surprised that LINQ didn't have much impact on how you approach software development and programming. Try reading through the LINQ articles on MSDN, Channel9, and maybe pick up a book on LINQ and give it a chance. It is tough to explain the full benefit and power of LINQ to someone who isn't used to thinking abstractly, but have faith in the experience and judgement of other more experienced developers, LINQ has value going forward.

I am curious about integrating with git. I know that there is plenty of support to integrate with Subversion. What does git provide in terms of integration? I am not talking about IDE integration, I am talking about continuous integration. Does Cruise Control work with git?

LINQ is a great addition to the language to be sure, but the Microsoft's present implementation of it with regard to ORM functionality still leaves some things to be desired when compared against a more robust data access framework like Hibernate (in Java or NHibernate for .NET). Many of the LINQ to SQL examples also continue to perpetuate mixing of the UI and data access layers along with many other OO design and style faux pas which encourage and perpetuate bad programming practices. LINQ is useful in some cases, but the Query objects with Repositories that generally follow from a proper separation of concerns between data access and business objects tend to encapsulate the LINQ anyway so really it becomes a matter of what syntax would you prefer to use in your data access layer and not how clients be calling into that layer. Of course, Microsoft implies with their examples that there is nothing wrong with using LINQ to SQL directly in your code behinds or UI control event handlers...sigh. I really like .NET and I think that Scott Guthrie and the .NET division are really doing a great job with the platform, but I also wish that Microsoft would stop promoting the "everyone can be a developer" concept with McDonalds style programming examples that ignore quality and good design while simply asking, "would you like fries with that?" as the code wizard steps you through the creation of a poorly designed unscalable pre-packaged app. At the very least the could include the caveat with their examples that the sample code is for instructional purposes only and not meant for use, at least not as-is, in non-trivial or serious application development.

LINQ is a great addition to the language to be sure, but the Microsoft's present implementation of it with regard to ORM functionality still leaves some things to be desired when compared against a more robust data access framework like Hibernate (in Java or NHibernate for .NET). Many of the LINQ to SQL examples also continue to perpetuate mixing of the UI and data access layers along with many other OO design and style faux pas which encourage and perpetuate bad programming practices. LINQ is useful in some cases, but the Query objects with Repositories that generally follow from a proper separation of concerns between data access and business objects tend to encapsulate the LINQ anyway so really it becomes a matter of what syntax would you prefer to use in your data access layer and not how clients be calling into that layer. Of course, Microsoft implies with their examples that there is nothing wrong with using LINQ to SQL directly in your code behinds or UI control event handlers...sigh. I really like .NET and I think that Scott Guthrie and the .NET division are really doing a great job with the platform, but I also wish that Microsoft would stop promoting the "everyone can be a developer" concept with McDonalds style programming examples that ignore quality and good design while simply asking, "would you like fries with that?" as the code wizard steps you through the creation of a poorly designed unscalable pre-packaged app. At the very least the could include the caveat with their examples that the sample code is for instructional purposes only and not meant for use, at least not as-is, in non-trivial or serious application development.

You can still write good code without having a CS background. Sure, if you designing an operating system, designing a compiler, or delving into some low-level code, more engineering know-how is required. However, I've found my Math/CS background only to be mildly useful when writing database apps...

Here are some things you can do to become a better programmer...

Mentorship
Find someone (preferable a group of people) you can talk shop with. Nobody loves solving someone elses problems, but most programmers like to discuss best practices. Pick apart other people's code. It could be an open source project or maybe just the source code to some guy's game you found online. Maintaining code allows you to see other people's mistakes. Writing good code is hard, writing maintainable code is even harder. You can learn a lot about design by being on the other end of the development process (i.e. maintanance). Sometimes mentorship is just reading good code.

Read, read, read...
All the coding in the world will get you only so far. You need to consult the wisdom of experts. Some other good books are:
Object-Oriented Software Engineering: Practical Software Development using UML and Java by Timothy Lethbridge and Robert Laganiere
2nd edition
website for book
They even have a DVD with an entire semester's worth of lectures (based on the book) available for $50 online.

Applying UML and Patterns
by Craig Larman
3rd edition
sample chapters here
There's a series of videos (about 3 hours in length) that describe many of the details in the book.

Anything by Martin Fowler
Martin Fowler's website

You may wish to also check out:
A seminar on the Object Oriented Design with a focus on the Unified Process (or something close to it)

Methodology, methodology, methodology

The above books will introduce you to many of the existing methodologies in software engineering. It's not so important what methodology you choose; what is important is how you implement it. While many people may disagree with me on this. Engineering methods can be taught, problem solving skills can be acquired, and math skills can be learned. You might not be the next Edsger Dijkstra, but you can learn to write very good code on schedule and under budget. It may be hard to write code at home when you've been writing code at work all day, but try working through a project from beginning to end with perfect documentation (yes, make use cases...) utilizing all the right tools (i.e. version control, bug tracking, make tools). Doing this will make you a better programmer. Learn to take a few minutes each day and write out what your going to do. I often start writing code for the day by opening up notepad and figuring out what I'm going to do in psuedo-code. After, I just flesh out the notes and spend the rest of my time testing. Some guys like to do this on paper. Many professional organizations start out a days worth of coding drawing UML (or some close approximation to it) on a white board. The key is to plan before you code. Don't just jump in front of your IDE.

Your system has been halted in order to prevent a loss of data.
Ummm, shouldn't that read "Your system has been halted in order to guarantee a loss of data"?, since I was never given a chance to save anything before the system halt.

The philosophy is fail fast: the idea being that it's better to cut your losses at losing unsaved work, than to potentially cause widespread data corruption, security breaches etc. that could occur if the system continues working when the kernel is in an unknown or unstable state. It's the same philosophy that underlies kernel panics.

Having used the Model-View-Controller (MVC) Design Pattern for years, I know that many people (even programmers) are not aware of what it actually is.

I recommend skimming through Martin Fowlers excellent description.

You can also get additional links to PHP, .NET, Ruby, Python and Java examples from the main Wiki article on MVC.

If you have never heard about it (and many of you haven't), you are missing out on a great design pattern.

- Jesper

That's a complex problem that most programs, even modern ones, probably are not designed to consider.

They may not be implemented to consider it "as is", but the best modern object oriented (OO) software systems would be designed to allow this type of functionality to be injected into the existing implementation at runtime on a future date. This is a great advantage of modern OO and modular systems, you don't have to know about all possible requirements or uses up front if you are able to develop and inject new dependencies as the circumstances change. Unfortunately, this type of advanced understanding is not yet widely grokked among those who identify themselves with the software development profession, but that will change as more people recognize the advantages of the powerful design patterns enabled by OO design and analysis.

I can't tell if you're making fun of Extreme Programming proper, the way it is defined here: Fowler tract.

It strikes me as pretty cool... Just an unfortunate name.

I find UML very useful when I'm thinking about some classes I'm about to write. I can draw out a few rough boxes to represent classes, and get a view of how my various classes can interact. The way I do this is a very quick processes, but it helps get a view of the way that some software components can fit together before I jump into coding. The sketches can often help initiate design discussions. In this way, I'm a using UML as a sketching tool.

At the opposite end of the spectrum, you can buy some very expensive tools that let you try to capture every single nuance of the software in the UML diagram itself, and the code is generated directory from the UML model. This Model Driven Architecture (MDA) approach tries to treat UML as a programming language, and I think it fails horribly. I think writing code by manipulating boxes and arrows in an MDA tool is a terribly inefficient way to develop software, though there are many vendors who will try and tell you otherwise.

In summary, I think using UML as a rough way to sketch out software design is still a good way to go. Using UML as a programming language has never been a good idea, and probably should die.

I find UML very useful when I'm thinking about some classes I'm about to write. I can draw out a few rough boxes to represent classes, and get a view of how my various classes can interact. The way I do this is a very quick processes, but it helps get a view of the way that some software components can fit together before I jump into coding. The sketches can often help initiate design discussions. In this way, I'm a using UML as a sketching tool.

At the opposite end of the spectrum, you can buy some very expensive tools that let you try to capture every single nuance of the software in the UML diagram itself, and the code is generated directory from the UML model. This Model Driven Architecture (MDA) approach tries to treat UML as a programming language, and I think it fails horribly. I think writing code by manipulating boxes and arrows in an MDA tool is a terribly inefficient way to develop software, though there are many vendors who will try and tell you otherwise.

In summary, I think using UML as a rough way to sketch out software design is still a good way to go. Using UML as a programming language has never been a good idea, and probably should die.

It's easy to add complexity. It's hard to simplify.

Something where Refactoring's author, Martin Fowler, agrees strongly with you.

A lot of what you described isn't refactoring, which is defined as making design improvements that don't change behavior. Instead, those guys were just doing developer goldplating.

I think if you read the book, you'll see how a lot of the techniques in Refactoring are intended to be used in the service of simplification.

On what grounds is the claim made that ruby supports DSLs better than Perl?

Uh, read the links given.

The whole "internal DSL" moniker just sounds like flexible and intuitive syntax.

"Internal" or "Language-powered" DSLs are, indeed, an application of flexible syntax to produce intuitive constructs.

That's a good thing, but does not constitute a DSL as any other definition I've seen describes them.

This article claims to have coined the terminology, and certainly using "Domain Specific Language" to refer to such languages implemented through, e.g., Lisp macro systems which operated in the manner of what Fowler calls an "internal" DSL is not new.

Perhaps it does need terminology, but you might find that with regular expressions, source filters (considered questionable practice, but still), formats, an interesting prototype system, eval, and the above modules that Perl has pretty darn good "external DSL" and "internal DSL" support.

I might, indeed. So?

So do Lisp, Scheme, Haskell, and many other languages.

Not familiar with Haskell, but Lisp/Scheme certainly does, as does REBOL and, yes, many other languages.

Forth is built on your idea of "internal DSLs", BTW.

Its not my idea, but, sure. So?

Data stacks, postfix syntax, and an executable library are the three basics idea of the language.

Yes, they are. So?

Is it a point-by-point comparison or just the author's unqualified opinion? Which points does the author consider, on which of those does Ruby score better than Perl, and why?

If you want answers to those questions, read the links provided. Since I'm not endorsing or recommending the author's position (I don't know much Perl, and don't really have a position in the Ruby v. Perl argument -- I'm only pointing out that what the author did in the links that chromatic provided was make a claim about Ruby v. Perl regarding DSL, not claim that Ruby invented any of the features involved in the discussion) I don't really feel the need to regurgitate everything he writes in detail for you instead of allowing you to read it for yourself if you care. If you are just looking for a Ruby v. Perl argument here, well, sorry to disappoint you.

If you're wanting Perl programmers to concede that Ruby is better based on a simple opinion

I don't really give a flying flip whether Perl programmers concede that Ruby is better, either generally or in terms of DSL support, since I don't even have an opinion on those issues myself.

If I want some variation in languages, there are ones out there that vary much more than Perl, Python, and Ruby.

Probably. So?

Ruby might be worthwhile for its specific blending of them, but so far all I hear is how cool Rails is and how much better of a Perl Ruby is supposed to than Perl 5.

while certainly I've heard both of those, if that's all you've heard, you haven't really been listening to very much. But then again, I'm not trying to sell you on Ruby, so it doesn't really bother me.

Well, we have Catalyst, Mason, CGI::Application, and a hundred other web toolkits.

No doubt you do. So?

Perl 5 is still being developed, and Perl 6 has useful stuff being written in it even though it's not completely finished just yet.

Okay. So?

Perl is much more mature, performs generally better from what I've seen, is much more widely supported, and has a much broader base of publicly available modules.

No doubt. Though having more modules doesn't mean that it has better

Alright, I know this is going to be flame fodder, but I'm genuinely curious: does Ruby have anything to offer for someone who's already very proficient in Python (and Django, so Rails is already covered)?

If you look at it from a features and libraries perspective, then you really won't find much different. But you shouldn't look at it that way.

There's a significant programming style difference between the Python and Ruby communities. Python programmers, as a general rule, tend to be more inclined toward imperative programming, while Ruby has much more of the feel of a multi-paradigm language. To understand a lot of Ruby code out there, you need to wrap your head around concepts like functional programming, metaprogramming, and domain-specific languages. Ruby code tends favor higher-order functional combinators over primitive looping syntax (an influence from functional programming), extending the system base classes extensively (an influence from Smalltalk), and a lot of it has the flavor of trying to extend the language to look more like the problem space itself (domain-specific languages; look at something like rake, for example).

Python aims to be "clean" in a very contentious sense of "clean." To put it in a way that will be taken very partisanly: Python is the language that you get when "clean and easy to understand" is taken to mean "only promotes the concepts that are easy to understand to a mediocre imperative programmer." For example, Guido is on the record that higher-order functions like reduce are supposedly hard to understand.

To be fair, yeah, Python is a reasonably clean imperative/OOP dynamic language, which is no mean feat, but many of us folks really would rather have a language that supports a broader range of programming paradigms. (Though I'm not as great of a fan of Ruby as I used to be before I got going on Scheme, I must say...)