Load List Values for Improved Efficiency

An anonymous reader writes "Reduce the number of database hits and improve your Web application's efficiency when you load common shared list values only once. In this code-filled article, learn to load the values for drop-down lists when your Web application starts and then to share these loaded list values among all the users of your application."

Comments

PHP-ADODB Caches these queries

[displague]

With ADODB for PHP (and perl http://adodb.sf.net/, you can call CacheGetAll, CacheExecute, etc... The query resultset is saved to a temporary file. This avoids having to create the cache within the same function you would normally call without having to write extra code.

[first useful post?]

Re:Changes to the lists?

[AndrewStephens]

An excellent point, the article assumes that the data will not change very often, if at all. However, if the list data doesn't change very often then there is little point storing it in the database in the first place.

Not to say that the article was actually bad or anything, its just a little light on when you would want to use this, and what some of the problems with this approach are.

Is This Consultant on YOUR Payroll?

[rimu guy]

If you have yet to read the 25-page FA, may I present the precis:

Database hits are expensive. Reduce them where possible. For example, cache static lookup data.

The simplicity of the point however is lost in the complexity of the article. It covers web.xml settings, servlet classes, list value loaders, persistence backends for said loaders, data source 'helpers' for said loaders, custom object classes for the loaders, several subclasses for said object classes, and a jsp page (to boot). Phew.

The author refers to this design as a quick and easy approach. It is not. If you are not familiar with Java and read this article, please do not be put off. He could have demonstrated the point with a far simpler example. E.g. static variable, sql statement, jsp code, done.

[The author] has worked with IBM Global Services for one year, and has five years of experience in J2EE-related technologies. And it shows. I dread to think how much a fully realized IBM Global Services project would cost should all its consultants apply this sized sledgehammer to each small task. Hopefully the article was not written up on the client's dime as well.

--
Java Hosting

Memcached

[captainclever]

Or you could use something like Memcached. Works with pretty much any language, and tonnes less hassle. (Thanks LJ ;))

Re:Java Servlet init question ..

[Vengeance]

The individual list types are all implemented as singletons.

That is, they have static 'getInstance' methods, maintain private static references to their own solitary instance, and have private constructors.

This is a very standard OO pattern, and seems to be one thing the article got essentially right.

Re:Hungarian & long var names

[Anonymous Coward]

Vector obj_VectorOptions = new Vector();

What's with the mixing of Hungarian and long variable names?

I've been programming Java for 8 months and even I can tell this is a waste of space.

Re:Changes to the lists?

[Anonymous Coward]

Before making assumptions about your database, check with the DBA or research the architecture of the database. Many DBMS's will cache frequently accessed static query results.

Re:Changes to the lists?

[ahmusch]

Of course, the article leaves one key point unsaid but implied...

Understand the nature of your data.

In any system, there's basically three kinds of data:

-- Static: This is the stuff that changes at a glacial pace, such as state codes, currency codes, and so forth. (for bonus points, put all the static code/description values in a single table with a type identifier for an even larger performance increase at the cost of slightly more complicated code.)

-- Configuration: This is the data that drives the logic of the application. (Go ahead, put it all in code. Good luck with the maintenance.)

-- Data: The actual records that you process or interrogate at some level to do something.

If the JDBC monkeys don't understand the nature of the data at this fundamental level, then telling them to cache data is a recipe for concurrency and consistency nightmares. Not because they mean to, but because they don't know any better.

Caching the static data is probably safe, but the configuration data is only *somewhat* safe to cache. It's too expensive to continually round-trip for everything, so what my project has done is implement a warm-boot process. For every transactional record it attempts to process, it checks a warm-boot status table. If the time has changed, the app flushes and repopulates its caches of both static and configuration data. Sure, it's a little kludgy, but it gives us at least parts of the best of both worlds.

Re:You know what's great

[Anonymous Coward]

The figure of 100,000 has been thoroughly debunked and, as a result, you can pretty much ignore anything said by someone who chooses to use it (would you pay attention to an author whose sig says "The Earth is flat!"?).

Anyway, you can google for the debunkings (http://www.google.com/search?hl=en&q=bush+killed+ 100%2C000+debunked) yourself but one page that talks about it is at http://www.papillonsartpalace.com/dveathby.htm.

And, before anyone gets all upset and accuses me of being pro-Bush, I'll state here that I am an Objectivist and a Libertarian. The deluge of lies and gross manipulations from both sides of the GOPDem monster make me sick.

Re:Externalize Picklists...

[philipborlin]

There are very good reasons not to use Vector. The main one is that it internally synchronizes all calls. Any algorithms 101 class will show you why that is a false sense of security.

The classic example is:
vector.size();
vector.get(0);

Each one of those calls is synchronized internally but the JVM can still switch threads inbetween the two calls causing a race condition. To make that code thead safe you need to synchronize externally:

synchronized(vector) {
vector.size();
vector.get(0);
}

Now the code is thread safe but there were three synchronization calls (our explicit call and one for size() and one for get()). Very inefficient.

It's a lot worse than just the Vectors

[attonitus]

There's the use of Hashtable; the non-thread-safe singletons; the traversal of the entire lists in ValidValuesTable.getOption; the unnecessary storing of the options and values string-arrays; the failure to pre-size the ArrayList; the fact that DropDownItems aren't immutable and, my favourite (repeated several times throughout the code):

private void loadStatusList() {
Hashtable obj_Hashtable = new Hashtable();
ListValuesDAO listValuesDAO = new ListValuesDAO();
obj_Hashtable = listValuesDAO.getListValues("STATUS");
...

I hope that IBM Global Services' QA for products is better than their QA for developerworks articles.

Re:Changes to the lists?

[willCode4Beer.com]

I'm going to have to argue that storing data in static member variables is generally a bad idea. (Of course, so is using scriptlets in the JSP) Excessive use of static variables can lead to unpredictable behavior, and can be hard to debug. Also, syncing the data becomes more complex because of multi-threaded access to the variable.

While working on high performance web apps where we want to cache the data and prevent having it become stale we genereally to store it in the application context ( ServletContext.setAttribute(name,object) ) with date information.
So, you create a class to hold a date object representing the moment the data was cahed, and the data you wish to cache. Have your servlet or action class (you do have some kind of MVC right?) check that the date is not beyond your pre-dermined max life, if it is, re-fetch the data, otherwise, continue on. Some apps would continue on if another process was busy updating. Depends on requirements and load.

This would also mean not fetching the data at application startup. Which, depending on your app, can be a good thing. If your app is deployed in a cluster, and different boxes may get different types of requests you may not actually need all of the data cached on every box.

We also have a servlet (security controlled) that can be called to flush the cache. So, when an editor is using a content management tool and hits publish, the last step and updating the DB is to flush the cache.

Of course, when I see a *java* programmer using the old *Hashtable* and Vector classes, I'm instantly prejudiced againt his code.