Slashdot Mirror

Well if it's just plain text they could open the file in an editor, highlight the text, and use copy/paste to drop it in a textarea field in an HTML form.

So let's make the problem harder. How would I have enabled uploading of a binary object in 2001? I'd do the same thing I do today; use "forms-based file uploading."

RFC 1867 defining "forms-based file uploads" appeared in 1995, some months before Berners-Lee and company released RFC 1945 proposing HTTP 1.0. HTML 3.2 (1997) defined the "file" input type that Web applications use for file uploads. That element is usually displayed as a text box into which a file path can be entered, along with a "Browse" button that opens the local file browser.

Of course, there were many other ways to upload a file to a web server by 2001. I wrote a PHP-based application for a legal foundation that enabled subscribed attorneys to share documents in an online archive. Members could register a document online. They'd then receive a mail message to which they would reply after attaching the file. The message included a unique code that so the file could be matched to the registration entry in the database. This worked especially well with people on services like AOL whose interfaces at the time were not friendly to RFC-1867 uploads. (Nowadays I'd probably use either WEBDAV or dspace depending on how much structure I'd need.)

Sometimes I'd use Samba to export directories from a Linux application server to the Windows desktops. Then staff members could simply drop HTML files into pre-determined folders and have them appear within the context of the website. Nowadays, though, I handle pretty much all the content editing on the browser side with forms and TinyMCE. That way I get reasonably clean HTML code as opposed to what you get from someone copying and pasting from Word into a text box.

I wouldn't have thought uploading files was a key obstacle in the way of platform-independent development in 2001. Surely there must have been some bigger roadblocks.

If you're talking about an application that periodically collects files from a Windows workstation, in 2001 I'd have used an automated shell script running smbclient to connect to the workstation using SMB/Netbios and collect the file(s). What happens next depends on the application. I often run PHP scripts from the command line to process text files because I know it well, because of the enormous array of functions it offers, and, frankly, because I've always found Perl's syntax rather intimidating. grep, sed, and awk go a long way, too. Automatically manipulate a graphic image? ImageMagick.

Have you tried it? It's quite powerful and free. They have a good tour video here: http://www.dspace.org/about-dspace/DSpace-Video.html

There's also DSPACE.

I'm surprised that there were quite a few programs not mentions on the DMS wikipedia page -- People might consider them to be more as repository software than DMS (or RMS), but some other ones to mention that would be useful to managing already existing documents:

• FedoraCommons
• DSpace

And if you're looking for librarians with an IT background, in the libraries they're called "Systems Librarians". You might also check out the oss4lib and code4lib communities.

Checkout http://www.dspace.org/ Cheers m

Very true. I'd take a look at DSpace or Open Library for examples of software designed to handle gigantic numbers of documents and maintain sensible indexes for them.

DSpace ? http://www.dspace.org/

Where to start, first try http://pisdcoalition.org/
as the 'alternative' site to prism (they forgot that wanting to share you knowledge is the work of communists).

Background:
Researchers at Universities do research.
They are paid by the University, and they (well the University) may have received a grant to carry out the research (from nsf in the US or the research councils in the UK for example).

Once they have done their research they write it up, normally in a paper (in the arts it can be a dance!).

They send the paper to our journal. The journal's editorial board receive it and will the have it peer reviewed by other researchers in the same field to ensure it meets a level of quality and is suitable for the journal. This is the crucial part of the process. But the peer reviews do not get paid for this, and the VAST majority of editors do not get paid either.

The publishers then sell the journal to the very Universities who supplied the articles for free and allowed their academics to peer review and edit for free (on university time normally).

The publisher will normally demand they own the copyright.

The price they sell journals to Universities have gone up far more than inflation year after year after year, which means unis cancel journal subs. Plus the contracts are complex with huge tie-ins and 'if you buy x you must by z' clauses.

All publishers to is take the work of the academic (for free), get the editors and peers to review (for free) and then demand they own it, all for basically doing little more than formatting the document, proof reading and putting on a website (and, rarer now-a-days, in print). These are basic clerical jobs, not something which means they should own the copyright.

As noted, Universities and academics often do not have access to their own work.

There are changes afoot.

The Open Access movement is taking off (either through freely available journals, or by making the articles available on University websites). The latter are referred to as Institutional Repositories (unsexy name!) and I happen to run one. The software they use is either http://www.dspace.org/ or http://www.eprints.org/ both are free and open source.

Chris

Sisyphean task for a large government agency or library,

Moodle is used by around 21000 organizations (URL:http://moodle.org/stats) while Dokeos main page reports "over 1000". Number of users (students or organizations) is of course not an clear indicator of how good the systems are, but it most definitely is the indicator of success.

There are many worthy Open Source tools and systems for education. Some of them probably technically better than Moodle (I work with Moodle daily), but in terms of success I don't know anything that can be mentioned in the same sentience. For example, the much heralded Sakai is used by some tens organizations ("over 70" according to the Sakai Wikipedia page).

Some of my favorites:
Elgg, http://www.elgg.org/
LAMS, http://www.lamsinternational.com/
DSpace, http://www.dspace.org/

Btw, I fail to see how my original post merits a Flamebait, Score: 0. Its strongly worded, sure, but it is my honest assessment of the article linked to. The assessment is based on a single issue, I'll grant even that, but it is still correct. If you don't know enough about Open Source in education to know to include Moodle into your list of successes in that field then you don't know enough to write an article about Open Source in education.

Somebody mentioned Microsoft. They actually participate in such a project, namely PLANETS, where they do in fact work towards open formats and preservation of our ability to access past, current and future file formats.

It's exciting work. Relevant too ;)

Thibodeau and the rest of the people at NARA have been thinking about this problem for awhile, as have other researchers around the world. If you're interested in such things, there are a few places to start looking.

CAMiLEON http://www.si.umich.edu/CAMILEON/
Cedars http://www.leeds.ac.uk/cedars/
InterPARES http://www.interpares.org/
DSPace http://www.dspace.org/

Lockheed Martin won the NARA contract to develop the Electronic Records Archives.
http://www.archives.gov/era/acquisition/option-awa rd.html

After hearing them talk about it at the Managing Electronic Records conference (http://www.merconference.com/), I'd say they have a few things to work out yet... but these are important questions for the preservation of history, culture, and more. These questions also involve authenticity, the value of evidence, and more...

HP and MIT have been working on this same issue with the DSpace Project.

$308M would sure go far if doned to this open source federation!

I read the problem as the Uni wanting to give students easily accessible space so they could store presentations and the like for easy retrieval in the classroom, dorm, or cluster.

For students, it's the same thing. Each student gets a home directory and in their, along side the standards like a drop box and a shared folder, would be a folder serving as the student's home directory on the web server. This is very easy to set up with Apache, it just needs read access to the directory.

Now if you're trying to archive or permanently store the material, then maybe something like dspace would be more appropriate. Note that dpsace is for archiving and later retrieving finished works not a collaborative tool.

How about DSpace? From their website: "A groundbreaking digital repository system, DSpace captures, stores, indexes, preserves and redistributes an organization's research material in digital formats."

If your going to continue to add to your archive, then how about d-Space. Its an open source project over at http://www.dspace.org/

Might be overkill maybe not. Theres a good article over at MIT Technology Review this month.

http://www.technologyreview.com/articles/05/07/iss ue/feature_mit.asp

One of the big things in Universities at the moment are Institutional Repositories. Bascially a web accessible database containg records and FULL TEXT of all articles published by academics at the organisation.

Have a look at a list of a few here (from the eprints website). Many of them are in there early days, but in a few years will have grown to be quite a collection. The next step will be to cross search them, perhaps using Z39.50.

There are two software applications to run these sites. eprints from Southampton University (UK) and dspace from MIT/HP.

Some other links can be found here.

Chris
--

• I learned about DSpace
• Learned about versatile ways in which people manage their music collection
• ... let alone the list of scripts for mp3/ogg/whatever tag manipulation, put in one place, with some kind of opinion, so I do not have to freaking google it

http://dspace.org/introduction/index.html

"What Kinds of Content Does DSpace Accept?

DSpace accepts all forms of digital materials including text, images, video, and audio files. Possible content includes the following:

* Articles and preprints
* Technical reports
* Working papers
* Conference papers
* E-theses
* Datasets: statistical, geospatial, matlab, etc.
* Images: visual, scientific, etc.
* Audio files
* Video files
* Learning objects
* Reformatted digital library collections
"

http://www.dspace.org/

"A groundbreaking digital repository system, DSpace captures, stores, indexes, preserves and redistributes an organization's research material in digital formats."

http://helium.knownspace.org/whyknownspace.html

"KnownSpace is a data manager---something that can help users build, organize, reorganize, annotate, search, mine, visualize, and navigate large, heterogeneous, dynamic data spaces. The aim is to provide a uniform platform for researchers around the world to develop and disseminate software to provide better interfaces, more intelligent applications, and more sophisticated and uniform networking---all for free, with source code easily changeable and available to anyone."

A possible front-end.

http://haystack.lcs.mit.edu/index.html

An Institutional Repository (IR). Save yourself from rolling your own! Check this out mate: DSpace

Disregard this if you already have knowledge of the project, or, if it doesn't suit your needs. This is a very powerful and mature development of peer-review, content management workflow and academic submission from MIT. It is an IR, NOT a content management system!

Your friend,

Raj.

http://www.dspace.org/

"A groundbreaking digital repository system, DSpace captures, stores, indexes, preserves and redistributes an organization's research material in digital formats."

Dspace would be a part of any solution. What makes it complicated is that HR isn't a single discipline, but several under one tagline.

An Institutional Repository (IR).Since this is slashdot and many of us tend to appreciate software somewhat, may I present this: DSpace

Disregard this if you already have knowledge of the project. This is a very powerful and mature development of peer-review, content management workflow and academic submission from MIT. It is an IR, NOT a content management system!

We are currently trying to build an institutional repository using DSpace. Open Source and pretty cool:

http://www.dspace.org/

In the Library we are testing LOCKSS. It looks like it could go far if it catches on:

http://lockss.stanford.edu/

Yes indeed - one wonders why MIT didn't use its own open-source repository platform, DSpace, into which they have put significant money (along with HP) and which is built entirely with open-source components. They've released under the BSD license.

MIT recently released the source code for DSpace. We are currently evaluating it for use by our company.

where have you seen libraries using open-source technologies (like Linux) to solve problems?

UC (University of California) San Diego has started moving towards open-source software. We were a Solaris/Sybase/Netscape shop a few years back. But the cost just isn't practical, especially for places with tight budgets like libraries. So we've been moving (albeit slowly) to commodity hardware, Linux, Apache, Tomcat, etc. for our server-side stuff. Some of the developers use Linux and/or MacOSX for their desktops, too. There are links on my homepage, or respond to this if you're interested in more info.

In terms of software written specifically for libraries, Greenstone and OSS4Lib that other posters mentioned are good. Also check out D-Space, NC State's MyLibrary, and if you want to handle MARC data in Java/XML, MARC4J. Of course, all the standard open-source software works for libraries, too.

-Esme

The collaborative effort of the institutions mentioned, and the stories posted, are not primarily focused on courseware (although they are explicitly intended to support long-term storage and access to courseware materials). The goal of these efforts, which in these stories surround the DSpace project specifically, is to extend the range of services provided by these institutions, more specifically their libraries, to incorporate a scaleable model of digital shelf space. In other words, these are infrastructure efforts (so if you really are impressed by that part, don't bother reading on!).

At MIT Libraries, for instance, the main focus of their DSpace implementation is to capture the digital products of research conducted within the MIT community. This includes articles, books, technical reports, theses, datasets, audio files, videos, images, maps, and so on. Much like the existing physical library buildings and collections, these are to be organized according to how they can best serve the departments, labs, schools, and research centers at the Institute, which the new exception being that at first DSpace will focus on capturing materials generated locally, rather than selecting and collecting materials produced externally. Or worse, research materials that are generated locally by people at MIT, then given to publishers, and then sold back to the libraries at great cost. So from an infrastructure perspective, what they are trying to achieve is to extend the range of what libraries provide in terms of collections and services to now also include all kinds of digital materials, starting especially with digital materials created at MIT.

A few examples illustrate this best: first, consider the junior faculty member with her own articles on her department web page. We've all seen such web pages disappear within 1-3 years. What happens to her colleagues at other institutions who lose access to her articles, which maybe never got published in traditional outlets, but are nonetheless vital to their own work, and thereafter are reduced to so many broken bookmarks? At MIT, DSpace will take stewardship of those materials, giving them a persistent url and carefully recording descriptive, technical, and preservation metadata about the files and their formats. So in this case, DSpace takes that 1-3 year period of unreliable access and extends it to a minimum of 3-7 years of predictably reliable access. At this point in the web's history, you can't really get that anywhere else, and there's every reason to hope that number will really reach into the decades; it just can't be promised reasonably today.

A second example: an interactive, multimedia, experiential web resource administered by some professor on an aging redhat 6.0 machine under their desk. It's rich in data, it demonstrates a breakthrough in the state of the art, or the idea, in some nascent discipline, and it's widely used by scholars of that discipline, and it _can't_ be "just printed out". What happens when that machine blows a partition, or is comprimised because its amateur sysadmin is really a scholar, not a wizard?

Obviously, as indicated in the story, a good third example is courseware materials. If you look closely at OCW or the other well-known examples thereof, you'll see that in many ways, they are (IMHO) foremost publishing ventures serving the educational process. Getting the materials into standard form, getting them delivered by a deadline, keeping them viable during their relevant terms. Doing this so openly, and freely, is indeed very exciting. But every term that comes up introduces new classes, new upkeep, etc., and you have to have an answer for where the materials from the previous semesters' courses are going to land. There has to be infrastructure support for that, and having a service in the libraries providing long term persistent storage and access to do just that is an awfully good answer, if the tools, policies, and budget are in place to do that.

These examples were much better articulated by several of the excellent speakers at yesterday's launch event (sorry, couldn't find a link), and are increasingly recognized as very common and very troubling scenarios across academia. Once you think about what the technological requirements of providing that infrastructure are, it quickly comes clear that such initiative require solid, reliable software with lucid, maintainable designs, and no magic. After all, you could do it with just a filesystem, right? :) To get the services delivered properly, and in a way scholars can trust, however, you have to focus on developing policy, procuring budget, and delivering on an mission-driven focus of getting the service right and keeping it running. In other words, what you don't see behind the systems is the amount of non-technical work behind getting these things going, and making them sustainable.

The focus of the multi-institutional efforts is to expand, replicate, and formalize approaches to doing the same at many other large institutions where the impact can be equally significant. Seeing the level of public and private support of these efforts, and that there's a line in the sand now drawn with a software release marking a reliable starting point to answering the technical question, is quite exciting, and indeed is a breakthrough. If you really still think nothing new is being offered, and that DSpace isn't more than a stripped down sourceforge thinking like a card catalog, send me email and I'll direct your attention to a few folks at MIT and HP who will blow your mind with how well they've thought through and planned for these problems. :)

OpenCourseWare is MIT's initiative to share course materials via the web. Dspace is an attempt to solve the long-term storage problems associated with born-digital research materials.

It will be possible to put things into a Dspace archive that will not be accessible to certain people; OCW materials are by nature meant to be universally accessible.

They do have a Business Plan to pay for the site and I quote from the introduction below:

Operations Funding Model MIT Libraries plan to transition DSpace from its reliance on outside funding to a more sustainable funding model. Consistent with the Libraries mission, Core Services will be offered free of charge to all registered members of the MIT community. In keeping with MIT s mission, content will be offered as freely as possible via the Web to the public. This service strategy precludes seeking user or subscription fees for means to support the ongoing operations of DSpace. The proposed funding model will rely upon a number of potential resources including, but not necessarily limited to, support from the Institute, revenue from Premium Services, and support from corporate and federation partners. Support may take the form of financial support or in-kind assistance. Collectively, these contributions will cover the operational costs of DSpace, as well as some future development needs.