Slashdot Mirror

It's the customers' fault. The RFPs always call for off-the-shelf solutions. So instead of paying $100k for a couple of guys to write exactly what the customer needs (the catch: it won't be ready for a few months), they pay $7 Million (*) and get something immediately.. that will never do quite what they want.

(*) Yes, Seven Million US Dollars: that's how much Albuquerque Public Schools paid for their accounting package. You know, the one that has been featured on the front page of the local newspapers several times, because APS's Accounts Payable was so screwed up that they were unable to pay their bills and some local businesses that they owed money to, had to take out lines of credit just because of the screwed up cashflow... Holy shit, for a tenth of $7M I could have written a hell of a system, and it would have worked too. But even at $700k I would have been viciously underbid by a competitor, if there's any justice in this world. So maybe I would have to do it for one percent of $7M. ;-)

The money spent on business software is just appalling. Oh, some of the waste I've seen! Alas, I never have credibility when I rant about this, because it always involves sour grapes on my part. *sigh* People will never know...

I found the way out!

What was ASP is now Perl.(look at the link before you click, then look at the address bar after you arrive). What was SQL Server is now MySQL. And what was IIS is now Apache.

I'm sleeping much better these days now that I don't have to scramble every week there is another hideous security flaw announced. Not to mention they(MS) recently stated if they opened their source, even worse flaws would be revealed.

As the new Rush song(Secret Touch) says, "The way out is the way in".

if you are looking for XML data, feel free to use this page(asp at the moment, but it will soon be redone in perl).

The important thing is it outputs XML, so if you want to build an interface to it for your own application, you can. Its not a 100% complete database, but it should give you basic information on any book available.

I wrote this specifically for external search engines back when XML was the new hot thing. Funny thing is, the sites that search us usually want an FTP data feed, so this doesn't really get used much. But again, feel free(be reasonable if you use a bot - maybe limit your bot to a search every 5-10 seconds, please).

The online bargains were another symptom of the dot-coms that are mostly gone now. They were there because these people would do just about anything to bring people to their web sites. In other words, they lost money on every sale but made up for it in volume.

The death of these businesses has been a good thing for my online bookstore, which is an addition to a business that has been around for twenty years. Now we can reasonably compete with fair prices, whereas before it was hard because of all the damn giveaways and businesses selling merchandise at a loss.

Its not a good thing for the economy in general if you have a bunch of businesses blowing through venture capital by selling their merchandise at a loss or giving it away. But that was the business plan for a great many merchants trying to establish themselves online. Good riddance, I say.

My programming team is considering making some sweeping changes to our code base (150+ perl CGIs, over a meg of code

First of all, I think its important to realize that you have a medium-sized website and not a big software project. Therefore, some of the above comments recommending refactoring, UML, and eXtreme programming may be a bit overkill.

Web programming != software development! Its usually done at a much faster pace. Even if an object-oriented approach is taken, you are still probably talking about simple function libraries rather than complex C++ or Java classes. Again, overkill.

150 files is still a small enough project to be managed by one or two decent coders. Actually, I just looked at the amount of stuff I've written over the years for my online bookstore and its more like 500 files and over 4 megs of code. I don't feel like its too much of a job to manage this codebase by myself.

So, here are my recommendations.

You probably have gotten better at programming since the time you started your project. Take a few of the most recent CGIs you have written and compare them to the first ones you wrote. You just might notice a glaring difference in the quality. Also, the first pages you wrote are likely to be among the most important in your project, yet they are also likely the worst quality-wise.

Regardless of what language you program in, I think its important that you can tell whats going on in the program by reading the comments. If a manager can understand what a program does by reading the English bit, there's a good chance other programmers will be able to jump in and help as well. One specific rule I also follow: if you do regexes, say IN ENGLISH what those regexes do. I say this because regexes are one of the hardest things to read.

Look for any code that can be "factored out" of your scripts and put those into function libraries. Then include those in your program. The only problem with this occurs when you have huge function libraries that slow down your scripts when you include them. In that case you would logically separate your functions into different files. I have included very common functions in different include files, so I can make the actual code compiled or interpreted as small as possible.

Consider using a flowcharting tool as an aid to programming and/or documenting your code.

Standardize how you name variables and functions, write comments, identation, and spacing.

Be sure and include the date you write your scripts in the comments, in case the filesystem wipes this out.

I'm sure theres other things I've left out, but following the above guidelines have helped me do exactly what you are trying to do: manage a growing codebase. But don't forget, this is web programming, not rocket science, and some of the above suggestions may be more trouble than they are worth. Keep it simple.

OK, call me thick, but the company that produces the book Writing Solid Code denied you a job...because your coding was too solid?! I was under the impression they were always looking for talented developers.

Or was that another joke?

All shopping carts that I have ever seen specifically allow the user to update data. They almost universally allow you to change the quantities and to add and remove items to a cart. This involves an update to the database usually(exceptions being a file-based cart or a cart stored entirely in memory).

Additionally, HTML does not keep state - meaning all data values in the cart must be passed somehow - usually either a cookie, a hidden form element, or in the query string(everything you see after the ?).

Compounding all this is that many dot-com sites were rushed to market. Speed was the ultimate requirement dot-com developers had, not security, not soundness of algorithms used. Yes its stupid, but add all this up and you have a lot of insecure websites. (Mine however only allows price input to the cart from values in the database itself and is additionally protected by an OpenBSD firewall :) )

if you're doing something besides an index search on a large table, stop
...
6.5 and below is a stability nightmare

As a side note, I came up with a program to create a full text index to search a SQL Server 6.5 database, because it didn't have one and doing keyword searches entailed using LIKE %keyword% clauses, forcing a table scan. Then, two weeks after I did that MS came out with SQL 7. I still like my way better because I can do things like stripping single quotes out of the keywords or whatever to match my search function.

I appreciate your comments, you gave me a few ideas. If you go to my bookstore and do a search there are currently five joins being performed for each search. Its still fast, but I'm also always looking at improving things. Also, I've never really fully studied what tuning page-level & row level locking could do for me - I just use the defaults. Unfortunately for me, I've never had a DB guru to turn to - just books, trial & error, and the occassional online epiphany such as yours.

I work for an independent online bookstore, Page One. We have a profitable internet business, and the majority of our customers have found us through comparison sites that we partner with. We have never spent money on internet advertising, so partner sites are very important to us.

Personally, I welcome Amazon not working with comparison sites. It will be their loss. There are already over 50 other bookstores that are set up to sell books online. If one single company decides it doesn't want to be compared with the rest, it wont have a great effect on comparison searches.

Folks, there are a great variety of stores out there to choose from. The internet lends itself to variety, not monopolies that dominate an entire area of business. If one company decides to go against the nature of the internet(which lends itself perfectly to lots of businesses and comparison shopping) they are only going to be hurting themselves.

...and I'll see to it that people in the publishing industry see the /. comments.

(Damn, this always happens...the stories that are the best ones for me to post to inevitably occur on the weekend when I'm doing non-computer related activities.)

I run a online bookstore and thus try to pay attention to this sort of thing. We are a smallish independent place that has watched the number of independent bookstores dwindle from over 5,000 bookstores to just over 3,000 in the last 10 years, a 40% drop.

If you really care about where you get your information, do what you can to ensure that your local independents survive and try to diversify your online spending rather than just going to amazon or wherever. From my standpoint, these digital books are just going to proliferate. I combined my 3 "O'Reilly Bookshelves" into 1 CD and I love it. Computer books and travel books in particular are really useful to put into digital form. It is happening and it will continue to happen. Even though I sell books I want to see this happen. Being able to carry 100 books in my handheld reader would be awesome. Within 5 years, the display will be as readable as the printed page, and of course searching, indexing and linking are vastly superior to the printed form.

In this business, we survive by keeping up with technology, and my company knows based on past experience as well as having an inside ear to the publishing companies that this is going to proliferate. The King ebook promotion was very successful for Simon & Schuster, and I just got through emailing my agreement to promote several other upcoming ebooks. We're going to be there when it happens.

I have somewhat of a positive view about this, and I'm pretty sure that authors and publishers together will work toward an amicable solution for readers. They have to. If for example Random House finds that it sells 3 times as many ebooks that are say, encrypted text that you could transfer from computer to computer after decrypting than it does by distributing it in a closed program, guess which way is going to win? The fact that King could not read his own book was very telling.

You can ensure that the publishers of ebooks play fair by demanding certain things from them before you buy. Simon & Schuster is the publisher doing the most stuff now, why not email them your concerns? I was able to get some positive comments through to them and so can you. Trust me, publishers are *far* more concerned with the popularity of the publication than they are with the technology used. It should be easy to convince them that using proprietary readers is not the way to go, and that good old plain text is.

Suggestions for the future sale of e-books.

- Insist that publishers publish in text. Encrypted text would be the best way to do this. Don't buy anything stored in a "glassbook reader" or anything else that you cannot transfer your property to and from. The Glassbook reader was very buggy, not only that what happens in a few years when I can't use the program at all because I upgraded my OS?

- Write some publishers and show how using proprietary technology is limiting their sales. They are there to make money and not only that would likely listen to your intelligent comment because the book industry is notoriously non-technical while at the same time relying a great deal on technology.

- Encourage the creation of a digital library. Write your congressman and let him/her know that our society needs one. We should have a library where works can be "checked out" for a period of time by those who can't afford to pay for them. This is absolutely vital.

Remember, the book industry is driven by user demand. We are all here trying to compete with amazon and the chains. Innovation rules in this industry, that's why I don't see knowledge being tightly controlled by the publishers. They are really just trying to keep up with the changes brought by the digital age.

And support your independent bookstores! Most of us are deeply concerned about freedom of speech and will defend the 1st amendment with our careers if not our lives.

I run one of the online bookstores that is selling the eBook. This was suppossed to be a fair method of distribution, with each bookstore participating in Simon & Schusters promotion having an equal opportunity to sell the book. However, the moneyed players(Amazon, Barnes & Noble, et. al.) have been allowed to give the book away, effectively shutting us little guys out of the promotion.

I feel that this is a bad sign for the first major fiction publication that is strictly electronic. If you care about the future of ecommerce, don't just give all your money to the big guys! Ensure that smaller companies can survive on the internet by patronizing their businesses as well. Otherwise we will end up with an internet that is totally dominated by big firms.

For those of you going to Barnes and Noble thinking you are doing the right thing in boycotting Amazon - you're not. In the bookstore business, B&N are widely seen as the real enemy - they are the ones going around and causing all the independents to close down. There were over 5,500 independent bookstores in the U.S. in 1990 - today there are less than 3,500. We can thank the chains, and especially B&N for our new lack of diversity of information sources.