Data Management In Collaborative Software Applications?

nfreier asks: "I am a developer working on a research project at the University of Washington. The project is called UrbanSim. We are developing a metropolitan land use and transportation modeling system along with a package for data manipulation and visualization. The data required as input to this modeling system is of significant size and complexity, originating from multiple sources and in multiple formats. We are looking for a method to track data changes at a central repository, much akin to the notion of tracking changes in source code à la CVS. We thought of using CVS initially but quickly realized that it does not handle large files and binaries well, nor is it well suited to tabular data. Other options include hand-rolling our own data server or using a database. Obviously, the drawbacks to hand-rolling our own are time and effort. The drawback to a database system is that it generally restricts us to using basic relational data formats only. Another constraint is that our entire project is under the GNU GPL and anything that we rely upon should also be freely available."

"Any solution we use must be able to:

• track multiple versions of data
• be accessed via network from multiple disparate geographical sites
• support local modifications without affecting the repository
• handle large binary and ASCII data

The data management problem must be a common issue in other projects and I am hoping that someone can advise me of a possible solution."