I would think that Cocoon would pretty much totally fit the bill here. It is a very strongly XSL-based content delivery engine that is being used by several major sites.
I think Apache's projects are way beyond what anyone else is doing in all these areas. The XML project and the Jakarta project are both even more exciting to me than the HTTPD itself. Both are pushing technologies that have applications both with Apache HTTPD and with other servers on the market (IIS included).
Before you get excited about anything else, do yourself a favor and check out what they have to offer. I think you will find powerful, real-world ready environments.
I just cannot believe that this represents the best efforts of UI PhDs. I have yet to see an innovative change in GUI's since 1995. The Explorer integration in Win98 was interesting, but ultimately lame (but at least they tried something somewhat new).
Everytime I think of innovation in UI, I think of the demo I got of Netscape 5.0. No, not mozilla... the 5.0 that existed purely in concepts given to the Netscape marketing droids to come and show us (I was at Ford Motor at the time). They did a demo with Mozilla as this omni-app, basically a shell replacement... it had pop-up windows, tabs all over the place, etc. It was totally different from the Windows UI, and it had a lot of interesting features.
Ford shot it down in flames. The meeting was a disaster. Why?
"That doesn't look like Windows."
Grrr... how sad to be trapped in a "worst is best" situation.
(And, as you can see, the Mozilla-as-your-desktop concept never went anywhere. Too bad, it could have at least offered as much as litestep does)
> Using this logic, the 95% of business PC
> users who are using Windows should
> never switch to *NIX.
I would never switch a user without a convincing business case for doing so, so I agree -- that 95% shouldn't change unless there's a driving business case to do so.
I consider eliminating BSOD's and lowering support costs to be a driving business case well worth the inconvenience of a move:-).
Switching to Exchange or Outlook without a driving reason (like the need for a good calendar system, for instance) would *not* be a good reason for a switch. There *is* a difference!
> The great thing about Exchange is that the mail > stays on the server
Sure, kinda like oh, say *IMAP*? Exchange doesn't offer much of a value add in this area. Actually, when it ends up corrupting the mail store it's less of a value add:-).
(A friend of mine has an email message that he could drop into the mail queue of any Exchange server in his environment and *poof* -- corrupted data store! He never figured out why, but it was very consistent in his environment)
Wow, I can't believe you thought Outlook was one of the best mail clients... I just totally *hated* it, and not just because I harbor a distaste for MS in general (well, OK, that may have colored my impressions). It just seemed clunky and hard to configure. Every time I said to myself "Oh, I wish I could change *that*", I had to search through endless options menus to find it (and sometimes failed utterly).
It suffers the same lack other MS apps do: there are so many options, appealing to so many disjoint sets of users, that finding the set of options *you* want is nigh unto impossible. Take a good look as MS Word the next time you start it up (especially Word 2000, yikes!) and you'll see what I mean.
I still use Netscape Mail. Far from the best, but it is what I'm used to, and it works quite well with Exchange as the server (once I got them to enable IMAP, the rest was easy).
Well, first of all, as others pointed out, you have to make them prove that their solution solves some sort of problem that the current solution does not. The ball is in their court to prove their case, and it better be good.
That having been said, is Exchange suitable? Well, up to somewhere around 3000 clients I have heard it is OK. Above that, there are definite scalability problems. From my time at a major auto manufacturer whose name begins with a "F", I know that Exchange is flaky as hell unless it is kept at a very low number of users per-server. So, expect to have to purchase numerous NT servers and task an admin to each one (oh, and of course in each case you need at least two machines -- a PDC and a BDC, so double that machine count).
I don't know the hard details; the last time I looked on the Net for more supporting data to back up what I knew from those in the trenches of the Ford Exchange project, I couldn't find anything detailed. I assume MS must pressure people not to disclose details when things go bad.
I do know a local office that has only 300 people and is moving off of Exchange because it is too flaky.
I would say it categorically *does*, considering that Netscape/AOL still pays the salaries of many of the major mozilla developers. So again, the slashdot tempest-in-a-teacup reaction.
Not that I support this banner-mania at all. I think it's pretty ridiculous that *this* is the value-add Netscape is providing in their rebundling of Mozilla. There are plenty of other features they could have been supporting that would address the core markets for a browser:
- Customizability for corporate / ISP deployment
- Fine-grained security features (again, for corporate/ISP deployment)
- Bundling templates for Composer, to assist with the beginning web-developer
- An actual useful set of default bookmarks, not just a set of links to Netscape. Maybe even several "bookmark" packs that you can pick at install ("Do you want to buy stuff off the web?" "Do you need to do academic research?", etc.)
I can't say I agree with this assertion (that the worse the censorware works, the better for us). As you acknowledge, the false positives (Mona Lisa == pron, Dick Armey == naughty site) are what burns us. I would have to assume, given previous censorware policies, that these services are always going to err on the side of over-censoring. If they under-censor and junior gets to see a naked woman, they end up looking pretty stupid given their claims of accuracy. No one is going to buy something that 90% of the time screens goatse.cx...:-).
No, I think we need to continue to point out that the claims these people make are not accurate, and that software is not the answer (and probably will never be the answer). The censorware makers need to be challenged -- I'm honestly shocked that they can continue to make the claims they do, given that Peacefire has proven countless times that these claims are categorically not true!
With all the notes about how hard it is to prosecute this sort of activity, I really have to wonder... how long will it be before someone in a maddening situation like this cracks and puts a *real-life* hit on the luser?
I mean, sure, he's 31337 and all... but someone out there probably knows who he really is. And could be persuaded to go over there with a baseball bat and DoS his head.
Not that I advocate this at all:-). I just know I was sorely tempted at times when as an undergrad, high-school kids kept hacking into our school network. These were known to be local kids; hell, sometimes they would just walk in to the lab and shoulder-surf until they got a password, then sit down and log in. Our head admin chased one haX0r all the way out of the building and onto the dirt bike the kid had sitting outside the door...
Anyway, I predict the development of Black ICE soon:-).
Oh, I realize I focused purely on Java technologies. Sorry, that's my current focus. Other choices that make sense could be:
Zope
Custom module in Perl (we did this before going to Java, using XML::Parser, XML::Grove, and a custom template->HTML converter. It took about 2 weeks to write).
Basically, what you want is the following:
1) HTML-like files with "expanding macros" (so the designer puts in something like )
2) A processor that takes that file and performs substitutions on it. This means you either use a strict XML syntax (so you can use any XML parser as your base) or you write a customer HTML+macro processor. I prefer using XML, but then you may have to pre-process HTML from designers into XML (and cache that result).
It's hard to believe this didn't get much
traffic or replies... it's a good topic.
My recommendation is to not focus on the "application server" buzzword; just focus on technologies. If you are using Java, there are several packages out there that do this "templating" service. At my last company, we wrote a home-grown solution that (IMHO) worked even better than any of these, so home-grown is good too.
Look at these packages and evaluate them for yourself. I think any of them could work fairly well for your situation. Cocoon uses XML/XSL, but can be very well factored so that (in the end) the designer still only has to know some very simple expandible macros he can use on a page:
Cocoon
http://xml.apache.org/cocoon
This is from the Apache project, and is very
high quality and (they claim) fast. You'll
need to master XSL to have your programmers provide the "services" for the designers.
WebMacro
http://www.webmacro.org/
This is less XML-centric. It has several nice little operators. In my opinion, though, it's not as powerful as Cocoon can be. It may, however, be more simple to integrate for a smaller-scale project.
Resin
http://www.caucho.com/products/resin/index.xtp
Resin is a servlet engine + XSL processor. It
seems very powerful, and I may use it on a future project. It's licensing terms are a little strange (definately not Open Source), but it could be a legitimate choice.
This, more than anything else, just points out this: know your rights. And keep your mouth shut, stupid. Say as little as possible, and talk to a lawyer ASAP.
If you ask them if you need to tell them something, and they say "no", THEN SHUT UP! They've already decided you're a suspect, don't give them anything to help prove or encourage that. Just say "I'd like to speak to a lawyer."
Here's a link to a helpful pamphlet on this. Don't try to beat the cops at their own game, just don't play it:
http://www.shadeslanding.com/firearms/cops.no.ht ml
I have no objection to the money. I love money, I use it all the time:-).
The issue I was pointing out is that this is a big enough problem that cooperative efforts would seem to be useful. Posting a single monetary reward would seem to discourage that effort; people will "hoard" their ideas in hope of being the only one to figure out the solution.
I'm not saying it might not be successful. I just thought it was interesting in a psychological way to ponder what this might lead to.
Posting a reward is only going to lessen the chance that people will put their heads together and find a provable instance of prior art. Isn't the greed to be the only one to find it going to make it unlikely that people will collaborate on this?
I mean, how un-open source can you get? Think about it...
That's a nice idea, but Code Warrior on Linux is a front-end to GCC/EGCS, so I believe the same problem still exists... GCC won't do what the poster needs.
> Of course you could always just stop it with the drugs.
Or tell them to piss off. Drugs tests are bullshit. Never took one, *never will*.
Tests have no measure of how I perform on the job. If you think otherwise, please prove how finding out I was high on Saturday night in any way helps you as an employer assess whether or not I'm able to do my job Monday morning.
This article is so ridiculous. It brings out what has always been a "bright lights, big city" sort of problem and assumes that just because another segment of population of LA and NY have found drugs, that something is wrong with high-tech culture.
It's a fact. People with money in NY and LA sometimes do coke, speed, and (rarely, I think) heroin. No big revelation there. The fact that the dot.com world intersects with entertainment and high finance (two worlds very immersed in the drug scenes for a *long* time) just means that it is more likely that the top-notch folks are exposed to that lifestyle.
In my experience in drug-rampant Detroit / Ann Arbor, no one I've met in high-tech has done anything other than booze, pot, and LSD/shrooms. The first is legal, the second is a relaxant that probably won't hurt you in small doses, and the third is attractive to creative people who want to expand their minds. No crisis here, folks, move on.
It's not a problem to break out the pure MARC data into a database. It's three tables:
MARC Record
Marc Tag
Marc Subfield
Each table is linked to the table above via foreign keys.
It's the other problem you point out -- using that data with some *context* to get something meaningful. I don't know how Web OPACs do it, but for the digital library work I do, we end up producing very specific tables for books, serial articles, etc with specific fields populated from parsing the MARC record. In our case, this is static, but a dynamic approach would be better; this would require the MARC data and it's representation to be linked -- edit the MARC record, the representation changes too (with a rebuild of the derived tables).
Dublin Core is expressable in RDF; I believe www.purl.org goes into some detail on this. DC is pretty neat for true digital library content, but if you have lesser aspirations, this can be grafted on at a later point (it's just going to end up being another classification scheme for your existing data -- digital library work pretty much *is* classification schemes).
Yes, I work with MARC records all the time on the Digital Library apps I write. The LOC web site is an essential aid, but I basically have had to develop lookup tables of my own for all the language codes, country codes, etc.
A further problem is that these codes have changed over time; MARC records created long ago (how old is MARC?) have incorrect codes that I find I have to deduce and add to my lookup tables.
But yes, MARC is the essential data format. I have code in Perl and Java that can parse MARC fairly well. That's easy -- dealing with the data in some way that makes sense for your collection is harder.
That said, proprietary solutions to this problem (usually referred to as "Web OPACs") run you big bucks. If you think about it, you need a data structure, an interface to populate and edit the data, an interface to query it... for 5000 records, this might not be hard, but it's not a weekend's job.
It's not naive. I know it for a fact. UMI is my biggest client, and I know Digital Dissertations and it's policies pretty well. On this point, it's all crystal clear.
This isn't an opinion, in other words. I'm stating the facts as I know them, at least for *this* content on ContentVille (I am aware there are issues with other material they sell).
> They have no right to publish those theses unless they sign contracts with the people who wrote them. Plain and simple.
Which they did. Because UMI bought the publishing rights to the thesis from the author, and ContentVille is just reselling content for UMI.
So, the poster is wrong. He did give permission, many years ago when he signed a bunch of papers. I'm not sure if he can rescind permission now, but UMI would be the folks to contact about that one.
BTW -- UMI posts abstracts for many dissertations they don't sell, and the rules they have for what they can and cannot sell (and to whom) are really complex. Even if ContentVille lists something, it may not be available for sale to you (depends on the rights UMI has and where you live).
Contentville is buying this data, either from UMI or from the University directly (I know they buy from UMI, not sure if they have other sources).
So (again) THEY PAID FOR THIS. You, the author, gave the university the right to do this. I understand it's fairly clear that the Univ. reserves the right to sell it (and they contract through UMI to do this, and UMI sells to ContentVille).
That's exactly the case. I can tell you for a fact that ContentVille gets at least some portion of it's dissertation content from UMI.
UMI's Digital Dissertations has (I believe) the largest set of dissertation abstracts in the world. They have every right to the content they have (although mistakes are made sometimes, so UMI will remove a thesis if they in fact determine they don't have rights to sell it).
So please moderate this up to +5, this is another Slashdot tempest in a teacup. If they have your dissertation, somewhere along the line *you* gave permission for it to be used.
I'm about to embark on a multi-six-figure project to *fix* a bad image archiving system for a major vendor. What they did was this:
- Content is manufactured via scanning of microfilm to TIFF images
- The TIFFs are put through a QA process. Bad scans are re-done. All of this is done by *people* (the scanning too).
- Once out of QA, the images are burned to CD
- The CD storage is via several jukeboxes of CD reader/writers (2 read heads, 1 RW head). Each jukebox holds 200 CDs, I think. These are combined into CD towers with multiple jukeboxes in them.
- Access to the images is then managed by a piece of C++ code that knows how to drive the CD jukebox (it's huge, and not like a normal CD drive) and get data off of it.
This turns out to completely suck. Getting images off the drives is slow and buggy. The media chosen was far too cheap, for one thing. Springing for the good stuff, however, suddenly makes CDs much less attractive in terms of cost. We often see read errors, stuck drives, etc.
My company contracted to put this data out onto the Web. This turned out to be a nightmare. With unreliable and very slow hardware (remember, the CDs have to be moved into a reader from their position within the jukebox!), real-time delivery is just impossible.
By contrast, spinning disk is constantly decreasing in price. Sure, you pay a lot of money for an EMC unit, but it offers much more in terms of flexibility and expansion.
My client took the wrong road. Now they are faced with several months of I/O to get the CD data (1600 CDs worth) off the buggy drives and onto an EMC unit. Then they have to pay us to update our Web server software.
CDs are great for archives. They are very very bad for any sort of access to the data. I wouldn't recommend a CD solution. Another point to mention is no one seems to know what the shelf life of CD-Rs is... be a pity to have those turn into coasters in 5-20 years.
Slashdot Mirror
I would think that Cocoon would pretty much totally fit the bill here. It is a very strongly XSL-based content delivery engine that is being used by several major sites. I think Apache's projects are way beyond what anyone else is doing in all these areas. The XML project and the Jakarta project are both even more exciting to me than the HTTPD itself. Both are pushing technologies that have applications both with Apache HTTPD and with other servers on the market (IIS included). Before you get excited about anything else, do yourself a favor and check out what they have to offer. I think you will find powerful, real-world ready environments.
I just cannot believe that this represents the best efforts of UI PhDs. I have yet to see an innovative change in GUI's since 1995. The Explorer integration in Win98 was interesting, but ultimately lame (but at least they tried something somewhat new).
Everytime I think of innovation in UI, I think of the demo I got of Netscape 5.0. No, not mozilla... the 5.0 that existed purely in concepts given to the Netscape marketing droids to come and show us (I was at Ford Motor at the time). They did a demo with Mozilla as this omni-app, basically a shell replacement... it had pop-up windows, tabs all over the place, etc. It was totally different from the Windows UI, and it had a lot of interesting features.
Ford shot it down in flames. The meeting was a disaster. Why?
"That doesn't look like Windows."
Grrr... how sad to be trapped in a "worst is best" situation.
(And, as you can see, the Mozilla-as-your-desktop concept never went anywhere. Too bad, it could have at least offered as much as litestep does)
> Using this logic, the 95% of business PC
:-).
> users who are using Windows should
> never switch to *NIX.
I would never switch a user without a convincing business case for doing so, so I agree -- that 95% shouldn't change unless there's a driving business case to do so.
I consider eliminating BSOD's and lowering support costs to be a driving business case well worth the inconvenience of a move
Switching to Exchange or Outlook without a driving reason (like the need for a good calendar system, for instance) would *not* be a good reason for a switch. There *is* a difference!
> The great thing about Exchange is that the mail > stays on the server
:-).
Sure, kinda like oh, say *IMAP*? Exchange doesn't offer much of a value add in this area. Actually, when it ends up corrupting the mail store it's less of a value add
(A friend of mine has an email message that he could drop into the mail queue of any Exchange server in his environment and *poof* -- corrupted data store! He never figured out why, but it was very consistent in his environment)
Wow, I can't believe you thought Outlook was one of the best mail clients... I just totally *hated* it, and not just because I harbor a distaste for MS in general (well, OK, that may have colored my impressions). It just seemed clunky and hard to configure. Every time I said to myself "Oh, I wish I could change *that*", I had to search through endless options menus to find it (and sometimes failed utterly).
It suffers the same lack other MS apps do: there are so many options, appealing to so many disjoint sets of users, that finding the set of options *you* want is nigh unto impossible. Take a good look as MS Word the next time you start it up (especially Word 2000, yikes!) and you'll see what I mean.
I still use Netscape Mail. Far from the best, but it is what I'm used to, and it works quite well with Exchange as the server (once I got them to enable IMAP, the rest was easy).
Well, first of all, as others pointed out, you have to make them prove that their solution solves some sort of problem that the current solution does not. The ball is in their court to prove their case, and it better be good.
That having been said, is Exchange suitable? Well, up to somewhere around 3000 clients I have heard it is OK. Above that, there are definite scalability problems. From my time at a major auto manufacturer whose name begins with a "F", I know that Exchange is flaky as hell unless it is kept at a very low number of users per-server. So, expect to have to purchase numerous NT servers and task an admin to each one (oh, and of course in each case you need at least two machines -- a PDC and a BDC, so double that machine count).
I don't know the hard details; the last time I looked on the Net for more supporting data to back up what I knew from those in the trenches of the Ford Exchange project, I couldn't find anything detailed. I assume MS must pressure people not to disclose details when things go bad.
I do know a local office that has only 300 people and is moving off of Exchange because it is too flaky.
I would say it categorically *does*, considering that Netscape/AOL still pays the salaries of many of the major mozilla developers. So again, the slashdot tempest-in-a-teacup reaction.
/ISP deployment)
Not that I support this banner-mania at all. I think it's pretty ridiculous that *this* is the value-add Netscape is providing in their rebundling of Mozilla. There are plenty of other features they could have been supporting that would address the core markets for a browser:
- Customizability for corporate / ISP deployment
- Fine-grained security features (again, for corporate
- Bundling templates for Composer, to assist with the beginning web-developer
- An actual useful set of default bookmarks, not just a set of links to Netscape. Maybe even several "bookmark" packs that you can pick at install ("Do you want to buy stuff off the web?" "Do you need to do academic research?", etc.)
I can't say I agree with this assertion (that the worse the censorware works, the better for us). As you acknowledge, the false positives (Mona Lisa == pron, Dick Armey == naughty site) are what burns us. I would have to assume, given previous censorware policies, that these services are always going to err on the side of over-censoring. If they under-censor and junior gets to see a naked woman, they end up looking pretty stupid given their claims of accuracy. No one is going to buy something that 90% of the time screens goatse.cx... :-).
No, I think we need to continue to point out that the claims these people make are not accurate, and that software is not the answer (and probably will never be the answer). The censorware makers need to be challenged -- I'm honestly shocked that they can continue to make the claims they do, given that Peacefire has proven countless times that these claims are categorically not true!
With all the notes about how hard it is to prosecute this sort of activity, I really have to wonder... how long will it be before someone in a maddening situation like this cracks and puts a *real-life* hit on the luser?
:-). I just know I was sorely tempted at times when as an undergrad, high-school kids kept hacking into our school network. These were known to be local kids; hell, sometimes they would just walk in to the lab and shoulder-surf until they got a password, then sit down and log in. Our head admin chased one haX0r all the way out of the building and onto the dirt bike the kid had sitting outside the door...
:-).
I mean, sure, he's 31337 and all... but someone out there probably knows who he really is. And could be persuaded to go over there with a baseball bat and DoS his head.
Not that I advocate this at all
Anyway, I predict the development of Black ICE soon
Oh, I realize I focused purely on Java technologies. Sorry, that's my current focus. Other choices that make sense could be:
Zope
Custom module in Perl (we did this before going to Java, using XML::Parser, XML::Grove, and a custom template->HTML converter. It took about 2 weeks to write).
Basically, what you want is the following:
1) HTML-like files with "expanding macros" (so the designer puts in something like )
2) A processor that takes that file and performs substitutions on it. This means you either use a strict XML syntax (so you can use any XML parser as your base) or you write a customer HTML+macro processor. I prefer using XML, but then you may have to pre-process HTML from designers into XML (and cache that result).
It's hard to believe this didn't get much
traffic or replies... it's a good topic.
My recommendation is to not focus on the "application server" buzzword; just focus on technologies. If you are using Java, there are several packages out there that do this "templating" service. At my last company, we wrote a home-grown solution that (IMHO) worked even better than any of these, so home-grown is good too.
Look at these packages and evaluate them for yourself. I think any of them could work fairly well for your situation. Cocoon uses XML/XSL, but can be very well factored so that (in the end) the designer still only has to know some very simple expandible macros he can use on a page:
Cocoon
http://xml.apache.org/cocoon
This is from the Apache project, and is very
high quality and (they claim) fast. You'll
need to master XSL to have your programmers provide the "services" for the designers.
WebMacro
http://www.webmacro.org/
This is less XML-centric. It has several nice little operators. In my opinion, though, it's not as powerful as Cocoon can be. It may, however, be more simple to integrate for a smaller-scale project.
Resin
http://www.caucho.com/products/resin/index.xtp
Resin is a servlet engine + XSL processor. It
seems very powerful, and I may use it on a future project. It's licensing terms are a little strange (definately not Open Source), but it could be a legitimate choice.
This, more than anything else, just points out this: know your rights. And keep your mouth shut, stupid. Say as little as possible, and talk to a lawyer ASAP.
t ml
If you ask them if you need to tell them something, and they say "no", THEN SHUT UP! They've already decided you're a suspect, don't give them anything to help prove or encourage that. Just say "I'd like to speak to a lawyer."
Here's a link to a helpful pamphlet on this. Don't try to beat the cops at their own game, just don't play it:
http://www.shadeslanding.com/firearms/cops.no.h
I have no objection to the money. I love money, I use it all the time :-).
The issue I was pointing out is that this is a big enough problem that cooperative efforts would seem to be useful. Posting a single monetary reward would seem to discourage that effort; people will "hoard" their ideas in hope of being the only one to figure out the solution.
I'm not saying it might not be successful. I just thought it was interesting in a psychological way to ponder what this might lead to.
Posting a reward is only going to lessen the chance that people will put their heads together and find a provable instance of prior art. Isn't the greed to be the only one to find it going to make it unlikely that people will collaborate on this?
I mean, how un-open source can you get? Think about it...
That's a nice idea, but Code Warrior on Linux is a front-end to GCC/EGCS, so I believe the same problem still exists... GCC won't do what the poster needs.
> Of course you could always just stop it with the drugs.
Or tell them to piss off. Drugs tests are bullshit. Never took one, *never will*.
Tests have no measure of how I perform on the job. If you think otherwise, please prove how finding out I was high on Saturday night in any way helps you as an employer assess whether or not I'm able to do my job Monday morning.
This article is so ridiculous. It brings out what has always been a "bright lights, big city" sort of problem and assumes that just because another segment of population of LA and NY have found drugs, that something is wrong with high-tech culture.
It's a fact. People with money in NY and LA sometimes do coke, speed, and (rarely, I think) heroin. No big revelation there. The fact that the dot.com world intersects with entertainment and high finance (two worlds very immersed in the drug scenes for a *long* time) just means that it is more likely that the top-notch folks are exposed to that lifestyle.
In my experience in drug-rampant Detroit / Ann Arbor, no one I've met in high-tech has done anything other than booze, pot, and LSD/shrooms. The first is legal, the second is a relaxant that probably won't hurt you in small doses, and the third is attractive to creative people who want to expand their minds. No crisis here, folks, move on.
It's not a problem to break out the pure MARC data into a database. It's three tables:
MARC Record
Marc Tag
Marc Subfield
Each table is linked to the table above via foreign keys.
It's the other problem you point out -- using that data with some *context* to get something meaningful. I don't know how Web OPACs do it, but for the digital library work I do, we end up producing very specific tables for books, serial articles, etc with specific fields populated from parsing the MARC record. In our case, this is static, but a dynamic approach would be better; this would require the MARC data and it's representation to be linked -- edit the MARC record, the representation changes too (with a rebuild of the derived tables).
Dublin Core is expressable in RDF; I believe www.purl.org goes into some detail on this. DC is pretty neat for true digital library content, but if you have lesser aspirations, this can be grafted on at a later point (it's just going to end up being another classification scheme for your existing data -- digital library work pretty much *is* classification schemes).
Yes, I work with MARC records all the time on the Digital Library apps I write. The LOC web site is an essential aid, but I basically have had to develop lookup tables of my own for all the language codes, country codes, etc.
A further problem is that these codes have changed over time; MARC records created long ago (how old is MARC?) have incorrect codes that I find I have to deduce and add to my lookup tables.
But yes, MARC is the essential data format. I have code in Perl and Java that can parse MARC fairly well. That's easy -- dealing with the data in some way that makes sense for your collection is harder.
That said, proprietary solutions to this problem (usually referred to as "Web OPACs") run you big bucks. If you think about it, you need a data structure, an interface to populate and edit the data, an interface to query it... for 5000 records, this might not be hard, but it's not a weekend's job.
It's not naive. I know it for a fact. UMI is my biggest client, and I know Digital Dissertations and it's policies pretty well. On this point, it's all crystal clear.
This isn't an opinion, in other words. I'm stating the facts as I know them, at least for *this* content on ContentVille (I am aware there are issues with other material they sell).
> They have no right to publish those theses unless they sign contracts with the people who wrote them. Plain and simple.
Which they did. Because UMI bought the publishing rights to the thesis from the author, and ContentVille is just reselling content for UMI.
So, the poster is wrong. He did give permission, many years ago when he signed a bunch of papers. I'm not sure if he can rescind permission now, but UMI would be the folks to contact about that one.
BTW -- UMI posts abstracts for many dissertations they don't sell, and the rules they have for what they can and cannot sell (and to whom) are really complex. Even if ContentVille lists something, it may not be available for sale to you (depends on the rights UMI has and where you live).
Contentville is buying this data, either from UMI or from the University directly (I know they buy from UMI, not sure if they have other sources).
So (again) THEY PAID FOR THIS. You, the author, gave the university the right to do this. I understand it's fairly clear that the Univ. reserves the right to sell it (and they contract through UMI to do this, and UMI sells to ContentVille).
That's exactly the case. I can tell you for a fact that ContentVille gets at least some portion of it's dissertation content from UMI.
UMI's Digital Dissertations has (I believe) the largest set of dissertation abstracts in the world. They have every right to the content they have (although mistakes are made sometimes, so UMI will remove a thesis if they in fact determine they don't have rights to sell it).
So please moderate this up to +5, this is another Slashdot tempest in a teacup. If they have your dissertation, somewhere along the line *you* gave permission for it to be used.
Oh boy.
I'm about to embark on a multi-six-figure project to *fix* a bad image archiving system for a major vendor. What they did was this:
- Content is manufactured via scanning of microfilm to TIFF images
- The TIFFs are put through a QA process. Bad scans are re-done. All of this is done by *people* (the scanning too).
- Once out of QA, the images are burned to CD
- The CD storage is via several jukeboxes of CD reader/writers (2 read heads, 1 RW head). Each jukebox holds 200 CDs, I think. These are combined into CD towers with multiple jukeboxes in them.
- Access to the images is then managed by a piece of C++ code that knows how to drive the CD jukebox (it's huge, and not like a normal CD drive) and get data off of it.
This turns out to completely suck. Getting images off the drives is slow and buggy. The media chosen was far too cheap, for one thing. Springing for the good stuff, however, suddenly makes CDs much less attractive in terms of cost. We often see read errors, stuck drives, etc.
My company contracted to put this data out onto the Web. This turned out to be a nightmare. With unreliable and very slow hardware (remember, the CDs have to be moved into a reader from their position within the jukebox!), real-time delivery is just impossible.
By contrast, spinning disk is constantly decreasing in price. Sure, you pay a lot of money for an EMC unit, but it offers much more in terms of flexibility and expansion.
My client took the wrong road. Now they are faced with several months of I/O to get the CD data (1600 CDs worth) off the buggy drives and onto an EMC unit. Then they have to pay us to update our Web server software.
CDs are great for archives. They are very very bad for any sort of access to the data. I wouldn't recommend a CD solution. Another point to mention is no one seems to know what the shelf life of CD-Rs is... be a pity to have those turn into coasters in 5-20 years.