Netscape Dumps Critical File, Breaks RSS 0.9 Feeds

An anonymous reader writes "In the standard definition of RSS 0.91, there are a couple of lines referring to 'DOCTYPE' and referencing a 'dtd' spec hosted on Netscape's website. According to an article on DeviceForge.com quite a few RSS feeds around the web probably stopped working properly over the past few weeks because Netscape recently stopped hosting the critical rss-0.91.dtd file. Probably someone over at netscape.com simply thought he was cleaning up some insignificant cruft." Some explanation has been offered by a Netscape employee.

Comments

Ack

[Cygfrydd]

I would've seen this post sooner, but my RSS feed was broken... something about a 404?

Re:Ack

[Anonymous Coward]

I for one am glad that RSS is being deleted. Everyone needs to switch to Atom. It doesn't have multiple incompatible standards that make it impossible to know how to handle a feed, plus it's cleaner and supports everything the Retarded Stupid Shit supports and more.

Why would this break RSS readers?

[eurleif]

I don't see how this would break RSS readers. DTDs pretty much never get read except by validators. Normal SGML and XML parsers just treat the DTD URL as an opaque string, not as something that can be retrieved.

Re:Why would this break RSS readers?

[acroyear]

Actually the DTD is loaded up by pretty much every proper XML library even if validation is "off".

The DTD contains more than just the element definitions and hierarchy. Its also used to define entities (&...;) that are non-standard to XML but may be expected in the file. HTML has tons of pre-defined entities but XML only has the core 4. All others are defined in DTDs and loaded on the fly as part of the processing.

There are ways to turn it off at the lowest levels, but higher-level abstractions/libraries might not give access to that. For example, with JAXP + SAX you can turn off DTD loading, but Jakarta Commons Digester doesn't give a setting where you can trigger that, so Digester tries to load the dtd, and even with validation off you can't change that. My only recourse is to take the DTD lines out of the various config files. (Reason: My JBoss server is deployed in private networks where the server can't reach the internet).

Re:Why would this break RSS readers?

[tomhudson]

My only recourse is to take the DTD lines out of the various config files. (Reason: My JBoss server is deployed in private networks where the server can't reach the internet).

why not change your hosts file so that my.netscape.com/publish/formats/rss-0.91.dtd points to a copy on your_local_network/publish/formats/rss-0.91.dtd ... (for example, 192.168.1.96 my.netscape.com)

Re:Why would this break RSS readers?

[acroyear]

which "hosts file"?

we have licensed customers with this (in fact, they're the problem - internally we don't see any problem with having outbound internet access for our servers and trust the firewalls we put up); i can't go telling their IT departments (who are sometimes completely inept) that we need to go mucking around with OS configuration files, especially in Windows.

Re:Why would this break RSS readers?

[smurfsurf]

Why don't these tools fetch the file only once and cache it? They are not suppose to change, are they? Even if the caching is just for performance reasons.

Re:Why would this break RSS readers?

[Anonymous Coward]

So find a copy in the wayback machine and host the DTD yourself.

Re:Why would this break RSS readers?

[Anonymous Coward]

You're saying that it's impossible to correctly parse XML unless you can get a port 80 connection to a random internet host while doing it?

Wow, XML is even more fucked up than I thought.

Re:Why would this break RSS readers?

[x_MeRLiN_x]

http://en.wikipedia.org/wiki/Hosts_file

Re:Why would this break RSS readers?

[acroyear]

In general, you're right. The various standard languages built in XML do basically require and expect that you're able to look at files on the internet. XML wasn't designed for many of the private-network uses its being used for today, and neither were many of the tools that reference public DTDs.

However, with most parsers there is a way to set up a mapping where it can look on the local box for a file matching a specific URL. JBoss, Hibernate, and Struts all do that for DTD files they have in their respective Jars, loading them as resources. However, I've never seen a clear example of how to do that for other files. It's "black box magic" that I've never found a reasonable, simple example of how to implement in my own systems.

Google doesn't help because "Java" "XML" "Resource File" still brings up thousands of entries I don't have time to nail down. And trying to dig through the thousands of lines of source on those apps is just as much a search I don't have time for.

Re:Why would this break RSS readers?

[acroyear]

Caching it requires having access to get it in the first place, doesn't it? I can't ask them to open their system up to the 'net just to grab some file every time they restart their servers, can I?

See my comment elsewhere in this thread - there's some "magic" that Java can do to load a local file as a resource given an external URL but I've never seen a decent example of how to set that up for my own apps.

Re:Why would this break RSS readers?

[Ozan]

I always thought about having abstraction and redundancy introduced in a way that critical resources like the DTD get a real URN like urn:netscape.com:rss:0.91 and then parsers fetch the resource by looking up the URL in a special service that translates a URN to one or multiple URLs. With multiple ones, if the first one doesn't work another one can be tried.

Basicly this system would do to URNs and URLs what DNS does to domain names and IP numbers.

Re:Why would this break RSS readers?

[acroyear]

well, as long as they're talking about an XML "2.0", maybe you should write to a committee member and submit the idea?

Re:Why would this break RSS readers?

[KrisWithAK]

You are right. I wish I would have seen this article earlier so that I could have posted sooner -- and others to get to see the "solution"!

Ever since I started developing on a laptop during my commute, I discovered that XML-based programs like J2EE servers would simply stop working. I experienced the same thing at work where, by default, your desktop applications (namely Eclipse) do not have access to the internet, and the servers will never have access to the "Internet".

The proper thing to do is for your application to use an XML catalog for resolving entities/URIs. There is a good article at http://xml.apache.org/commons/components/resolver/ resolver-article.html that helped me out. In addition, if you are using Eclipse with the web tools platform, you can customize the catalog so it resolves DTDs and entities locally. See http://wiki.eclipse.org/index.php/Using_the_XML_Ca talog.

Re:Why would this break RSS readers?

[enosys]

Using the hosts file to redirect accesses to the DTD would block access to my.netscape.com. Maybe they deserve it for breaking RSS and still not having fixed it, but I'm sure some users would complain.

Re:Why would this break RSS readers?

[tomhudson]

True, but if all they're doing is grabbing a dtd from there, who will notice ? It's offline right now anyway ...

My.Netscape is undergoing changes!

We are currently working to launch an updated My.Netscape complete with all the functionality you've enjoyed in the past built upon an entirely new technical foundation. This will come with additional functionality and a much overdue face-lift. Unfortunately, such an endeavor requires us to take the current My.Netscape off line for the time being. We're sorry for this inconvenience.

(my bold - not in original)

Re:Why would this break RSS readers?

[Mithrandir]

What you need to implement is org.xml.sax.EntityResolver. There's several methods that need to be implemented that are the different ways the SAX parser could query for stuff. Basically it will give you the Public ID and/or System ID and ask you to return a stream to what that resolves to. Then, in your code, all you do is run a hashmap that maps a given ID to a local resource (eg file or database BLOB) and then do your own stream opening/processing from there. I attempted to post some example code but seems like that trips the lameness filter :( So, just have a look at the interface. The code required is pretty trivial to implement. If that fails, you should be able to work out my email address from the website address under my profile - send me an email and I'll send you the code we use in one of our projects.

Re:Why would this break RSS readers?

[Sir Pallas]

According to the Internet Archive, the DTD was last changed in February 2003. Here's the latest copy of RSS 0.91. Perhaps someone should set up a redirect at PURL.

Re:Why would this break RSS readers?

[Anonymous Coward]

The DTD contains more than just the element definitions and hierarchy. Its also used to define entities (&...;) that are non-standard to XML but may be expected in the file. HTML has tons of pre-defined entities but XML only has the core 4. All others are defined in DTDs and loaded on the fly as part of the processing.

If you're not validating documents, you don't need to know what the list of entities are. Oh, sure, it'd be nice to have them, but it shouldn't fundamentally break any application to be fed garbage due to missing entity definitions. Documents still parse, structurally.

Of course, the correct thing to do is to resolve the DTD URL to a local file. The W3C standards have been absolutely clear about this for a long time. If URLs are actually being fetched, that's the fault of lazy programmers that assume they know what they're doing without actually understanding the specification or its best practices.

Re:Why would this break RSS readers?

[acroyear]

actually, I pin it to lousy programmers writing lousy libraries. the preferred (don't fetch it) mechanism should be the *easiest* to implement, rather than being an inconsistent case of obscurity (that the sax system created).

but really, it's 'cause most programmers actually suck at writing libraries, but their libraries get used (and thus, have backwards-compatibility maintenance issues) before they can get them refined into the right way to do things.

Re:Why would this break RSS readers?

[nuzak]

Wow, XML is even more fucked up than I thought.

Yeah, especially when you don't think at all. It has to do with validation, nothing to do with parsing.

As for validation, should every single possible doctype anyone might think of be hardwired into the XML spec? Here's a free clue: DTDs and XSDs don't have to use http urls, and DTDs even have something called a "system catalog".

What kind of idiot would delete that?

[Anonymous Coward]

It's almost as if the team retooling netscapes web presence has no idea about what a DTD is. I'd say it begs the question but in this industry reckless stupidity from clueless morons is the norm. Glad we switched to atom.

Re:What kind of idiot would delete that?

[Anonymous Coward]

I'm not suprised they removed the file from their server, it's costing them bandwidth for someone else's website. Maybe the question should be "what kind of idiot writes a program using a dtd which isn't cached locally". The performance hit on your program by failing to do this is massive.

the probably web

[Anonymous Coward]

what is the probably web?

Re:the probably web

[Cygfrydd]

That's the browsing experience you get when your DTDs are moving targets.

Ugh

[SuperBanana]

According to an article on DeviceForge.com quite a few RSS feeds around the probably web stopped working properly over the past few weeks because Netscape recently stopped hosting the critical rss-0.91.dtd file.

STOP, Grammar time. Ooooh whoooaaa oh oh...

Probably someone over at netscape.com simply thought he was cleaning up some insignificant cruft."

Or Netscape got tired of people using their bandwidth. Regardless of the reasons: if you reference a file on someone's site, it's hardly their fault if they move/change/delete it, and it breaks your stuff.

Re:Ugh

[fatphil]

But DTD's were designed to be precisely that. Likewise class paths in java.
Unnecessary hard coding of something that's not necessarily non-ephemeral.
I never liked the idea, I'm glad to see that some of my worries are well-founded.

Re:Ugh

[aiken_d]

Well, you're doing pretty good spotting grammar problems, but you're pretty ignorant when it comes to the content of the story.

RSS is a spec that was created and promoted by Netscape. For proper validation, this remote (Netscape-hosted) file is needed and was supplied by Netscape to facilitate implementation of RSS. Removing it breaks a standard that they promoted. That is, they encouraged people to use this file that they hosted as part of the RSS spec. To reiterate, they told people to use this file and then (presumably accidentally) removed it, thereby breaking some RSS feeds and readers. Got it?

Maybe you should stick to the grammar nazi stuff on stories you actually understand, or at least leave out the technically ignorant comments and only complain about syntax problems in the story.

Re:Ugh

[Anonymous Coward]

Regardless of the reasons: if you reference a file on someone's site, it's hardly their fault if they move/change/delete it, and it breaks your stuff.

That's not what Tim Berners-Lee thinks.

Dont blame Netscape

[Anonymous Coward]

blame the people (programmers/management) who based their business operation and website programming solely on the security of a third parties business and infrastructure being available

DTD's should be held and referenced locally within your infrastructure and your control

Obligatory Lamport quote on distibuted systems

[Programmer_Errant]

"A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable."

Bull

[Anonymous Coward]

RSS readers don't do anything with the DTD or try to access it at all. This didn't break anything.

Re:Bull

[dmorelli]

"RSS readers don't do anything with the DTD or try to access it at all. This didn't break anything."

You, sir, are wrong.

XML parsers (which RSS readers use to... you guessed it, parse the XML) will try to look at things referenced in the DOCTYPE declarations.

I guess I can vaguely understand the desire to have the 'authoritative' provider of the DTD in one place, but it's always made me uneasy to have this single point of failure for the potentially many consumers of a DTD.

It's apparently also made many of my employers over the past 10 years uneasy. I've had to take control of this situation many times at many jobs and change it so we host copies of DTDs ourselves (or remove the DOCTYPE altogether from documents).

There have also been times I've worked on software that otherwise did not need to have an internet connection available. This DOCTYPE business is a problem in that case.

Re:Bull

[KrisWithAK]

There is no need to host the DTDs on an actual server. I usually copy all the DTDs I need into a subdirectory of my application's installation path. See my original post at http://slashdot.org/comments.pl?sid=216818&cid=176 03580, or a great article on entity and DTD resolving at http://xml.apache.org/commons/components/resolver/ resolver-article.html.

Re:Bull

[Anonymous Coward]

You, sir, are wrong.

You, sir, are an idiot. Any application that relies on loading DTDs from the given URL is broken. Granted, there are plenty of broken applications, written by incompetent (or probably just lazy) programmers, but that doesn't mean it's XML/Netscape/whatever's fault when they break.

I wrote an XML-based application in C once, using Expat to do the heavy lifting. It fully conformed to the XML spec, but it didn't use a validating XML parser. 90% of the XML spec seems to deal with the issue of validating XML documents, which is all well and good, I suppose. However, 99% of the time, you don't need a validating parser; it's good enough to simply verify that the document is well-formed. Furthermore, it's good practice not to use validating parsers everywhere, and this sort of situation is precisely the reason why. If you have an XML application running on a cell phone or a dial-up link, do you really want the application to be calling out to some remote site every time it turns on (or heaven forbid, parses a document)? Not to mention the latency, bandwidth cost, and processing time, especially if it's a lengthy DTD.

I'll grant you that it can make life easier to rely on a generic parser that pulls the spec down from the Web automatically and assigns types, but there are so, so many things wrong with that, it's hard to know where to start. The whole point of URLs in DTDs is that they're supposed to be fixed and unique identifiers, not that they actually point to an actual resource. That's just a convenience. If the URN specs were finished, they'd probably have used those instead, precisely to avoid this sort of confusion. In any case, you take the string and use it as an opaque value to look up into your local DTD store, or whatever. A basic application of hash tables.

DTDs specified as URLs are just a convenient way of dividing the name space, just like how Java (and Apple, sorta) uses inverted domain names like com.sun. for packages/classes specific to particular organizations.

Did you think this through?

[Anonymous Coward]

Normal SGML and XML parsers just treat the DTD URL as an opaque string, not as something that can be retrieved.

Of course they retrieve it - unless they already have a local or cached copy. How else would they be able to parse a document marked up using a custom DTD?

Don't answer - go hang your head in shame.

Re:Did you think this through?

[TheThiefMaster]

A lot of rss readers can't parse a custom dtd, they assume that rss is pretty much fixed, and ignore the dtd line completely.

Re:Did you think this through?

[Thuktun]

Of course they retrieve it - unless they already have a local or cached copy. How else would they be able to parse a document marked up using a custom DTD?

A DTD is used when the parser wants to validate the document at parse time. It isn't needed for parsing. Having a DTD present won't necessarily allow your XML application to interpret random XML you didn't expect.

SPOF?

[basketcase]

Or maybe some smart person at Netscape decided to teach some people a lesson about using a 3rd party as a single point of failure?

Hmm....

[Anonymous Coward]

a few RSS feeds around the probably web stopped working properly.

*blinks*

Re:Hmm....

[tomhudson]

Welcome to the new Quantum Web Computing Experience. Blink again, and there's a 50/50 chance it'll be back to normal - or your computer will disappear.

Doesn't AOL own Netscape?

[gravesb]

And if so, why would anyone rely on AOL to make something on the web work?

Re:Doesn't AOL own Netscape?

[TheSkyIsPurple]

It's pretty much a separate entity now as far as AOL's concerned, I think. Ever since Mozilla got spun off.

HAHA

[eMbry00s]

Suck that, Web 2.0!

Hey dickhead!

[Anonymous Coward]

Or Netscape got tired of people using their bandwidth. Regardless of the reasons: if you reference a file on someone's site, it's hardly their fault if they move/change/delete it, and it breaks your stuff.

If you had even the slightest understanding of what a DTD is your opinion would be different and may even be valid.

The point...

[Junta]

Is to have a common component shared among many documents without replication.

Class paths is java are the perfect example to say how it *should* work. Java CLASSPATHs in every application/installation I have seen are site-local, all paths accessible without going over the internet to another site to get classes.

To be similar, an RSS site should copy this DTD to their local server, or to a server with which they have a concrete understanding of the relationship. Either a commercial agreement with a peer or at least using a server from an organization who explicitly defines the purpose of hosting to be a common place to promote it as a standard.

Did netscape promise itself to be an organization sharing that DTD explicitly, or did site developers get in the practice because 'it just always was there'?

Re:The point...

[KrisWithAK]

Resolving DTDs and entities in XML parsing does work like CLASSPATHs in Java. Applications need to properly set up an XML catalog which tells the parser to look in a local store before the Internet for certain URIs. Please see my earlier post at http://slashdot.org/comments.pl?sid=216818&thresho ld=0&commentsort=0&mode=nested&cid=17603480. Or jump straight to Norman Walsh's informative paper at http://xml.apache.org/commons/components/resolver/ resolver-article.html.

Re:The point...

[fatphil]

If you take a copy, no-one can guarantee the two are identical.
If you don't take a copy, no-one can guarantee that the remote one is always accessible.

Catch-22.

Host it yourself!

[nascarguy27]

This is the precise reason why I host everything myself including my own series of tubes, dubbed the Internets. I host not only every file that my site uses, but I also have a program that regularly crawls the entire Internet and compresses it onto my own distributed system. That way I can browse the Internet by myself without worrying if someone else's system will fail. Although I do need to replace systems every now and then. But that's not a problem, b/c the distributed system has 3-5 copies of the Internet, each copy in a different place. Wait, isn't their some other company that does that? I can't quite place the name.

Seriously though, relying on some other system so your site will work is a recipe for disaster. It's similar to relying on someone to take you to work everyday. After a while, you get used to that fact that someone else is driving you that you don't even think about it. Then your driver gets deleted somehow. And you're stuck with no way to work.

Re:Host it yourself!

[ScrewMaster]

Then your driver gets deleted somehow. And you're stuck with no way to work.

It's even worse if your driver gets somehow deleted while you're on the road. Happens to my XP box all the time.

So let me get this straight

[Anonymous Coward]

Whenever someone accesses a RSS file, Netscape would know the IP for every access? How stupid can that be? Why don't the readers just cache the DTDs and fetch only if there's a problem?

Re:So let me get this straight

[XMyth]

Because of abstraction (the lower level XML library is what would retrieve the DTD, but it'd only receive the raw XML data to be parsed/validated) the request for the DTD almost certainly doesn't contain the referrer. Even if it COULD contain the referrer, there's no real reason why it should IMHO.

Re:So let me get this straight

[Anonymous Coward]

In a word, no. Please read the RSS spec before calling it "stupid."

Re:So let me get this straight

[Schraegstrichpunkt]

Exactly. The problem is that you're supposed to have an XML catalog that lists the location of the DTDs on the local filesystem (at least, that's how 4suite works). If you don't set that up, a validating parser can either fail or fetch the DTD from the URL specified in the document (personally, I prefer that it just fail so that I can fix the problem, but it's not the default).

Re:So let me get this straight

[FlunkedFlank]

Yes, this is a known issue with XML/SGML. In fact external DTDs can do more than just log the usage, they can also be cause of various security issues. See:

http://www-128.ibm.com/developerworks/xml/library/ x-tipcfsx.html

Re:So let me get this straight

[Talchas]

Isn't that what PUBLIC identifiers are for?

DTDs are different

[pikine]

A DTD (document type definition) is a file that describes how an SGML document is structured. In this case, the DTD that went missing defines RSS 0.91, which is used by Navigator 4 for "channel" subscription.

It is expected that DTDs are hotlinked. For example, if you ever look at html source of a web page, you would see:

<!DOCTYPE ...>

on the top, and the hotlink goes to somewhere on w3.org. That is because W3 is the authority body that defines the html.

Since Netscape is the authority body that defines RSS 0.91, it is a bit strange how they stopped hosting the definition.

In any case, the missing definition won't affect software that processes RSS feeds. It only affects software that checks whether a SGML document is structured properly according to that missing DTD.

The main interest to this article seems to be the speculation how a deprecated web 1.0 company could end up hiring a clueless webmaster who deletes important files without recognizing its importance.

Re:DTDs are different

[Anonymous Coward]

The main interest to this article seems to be the speculation how a deprecated web 1.0 company could end up hiring a clueless webmaster who deletes important files without recognizing its importance. No, not all.

The main interest is that many organizations have foolishly tied the availability of their services to Netscape's services, despite that Netscape has been in flux for nearly a decade.

Re:DTDs are different

[mrchaotica]

Since Netscape is the authority body that defines RSS 0.91, it is a bit strange how they stopped hosting the definition.

It's not strange; it's just their way of telling everyone that RSS 0.91 is dead and people should stop using it, apparently.

Then they're broken!

[Anonymous Coward]

A lot of rss readers can't parse a custom dtd, they assume that rss is pretty much fixed, and ignore the dtd line completely.

That's just wrong, you should try software authored by folk who know what they're doing. atom is XML, even Microsoft MUST parse it as an XML doc before internally converting to RSS for IE7, right?

Re:Then they're broken!

[r00t]

Fetching the spec is idiotic.

First of all, that's a needless operation. It can take time; don't forget the DNS lookup and all.

Second of all, it's not as if you could handle any random DTD. Software doesn't work that way. (this is one of the reasons why XML itself is a mostly-lame idea) If the XML doesn't match expectations, you can't convert it to your own internal representation. You probably have a C struct that you need to fill in. Even in some wild interpreted language like perl, you just won't have any use for unexpected data structures and you damn well need the expected data structures.

Re:Then they're broken!

[funfail]

Your rant is a bit overgeneralization but I agree that there is no point in downloading the spec for RSS readers.

Re:Then they're broken!

[Bogtha]

That's just wrong, you should try software authored by folk who know what they're doing.

It's called a non-validating processor and it's totally compliant with the XML 1.0 specification.

Re:Then they're broken!

[Bogtha]

it's not as if you could handle any random DTD.

Yes, that's totally feasible. You're mistaking the semantics of document types with the external DTD subset.

It's true that inventing new element types and putting them in your DTD isn't going to magically make software understand what those element types mean. But DTDs provide other information - for instance, what entity references expand to, which attributes are IDs, and so on. This is useful information and can be processed in a generic fashion.

Re:Then they're broken!

[Columcille]

There's a reason these things are called standards, they follow a standard method. A dtd might be good for someone doing something unique, but if a person is following the atom standard or one of the rss standards, retrieving the dtd should be unnecessary.

Re:Then they're broken!

[ultranova]

Second of all, it's not as if you could handle any random DTD. Software doesn't work that way. (this is one of the reasons why XML itself is a mostly-lame idea)

Sure you can. Just show it to the user as-is. It's just a hieararchical tree of text nodes, he should be able to make at least some sense of it. That's one of the reasons why XML is such a cool idea :).

As a side note, that's what Firefox does when it encounters an XML file it doesn't have a stylesheet for.

Re:Then they're broken!

[Anonymous Coward]

> As a side note, that's what Firefox does when it encounters an XML file it doesn't have a stylesheet for.

You mean unless nsFeedSniffer.cpp decides otherwise?

Incomplete Quote

[multipartmixed]

I believe the quote you're looking for is:

Damn you, Bill Gates! Damn you all to DRM!

This is a major problem,

[Morky]

This could seriously affect both of the guys using Netscape.

Simple misspelling

[pionzypher]

What? You've never heard of a probability web before? ...Holds the improbability drive at NS in place?

Did you read the XML specification?

[Anonymous Coward]

Non-validating processors are not required to read any external DTD subset.

AOL?!

[Anonymous Coward]

Isn't w3.org supposed to host DTDs? Sigh.. what is this, I don't understand? Every "doctype" there is links to w3.org. Seriously, NETSCAPE?! A commercial organization!

W3C doctype

[pizzach]

How is this any different than the W3C doctypes ala

? Most web tutorials tell you to use the remote file....but if it ever goes down what happens? Is there a reason for it being taught this way?

Re:W3C doctype

[DavidTC]

It's not any different, except the w3c is run by intelligent people, and Netscape, apparently, is not.

I've always thought the full paths were a bit stupid too, and they should have some sort of shortcut standard, one that says "Use w3c's HTML4.0 standard", and the web browser knows how to contruct a path to find w3c standards. That way, when "Use netscape's RSS0.91" standard stopped working, web browsers could have a trivial update, or their config could even be changed manually, to tell them where to find netscape's standards.

Granted, they already have something like this in the DOCTYPE, that's what '-//W3C//DTD HTML 4.01//EN' is, but then they blow it by then including the path after that. The parser should, instead, have to look at W3C and go 'Hey, I know where that is, that's w3c.org' and construct a standardized path using 'DTD HTML 4.01', like 'http://w3c.org/doctype/HTML4.01.dtd'. (And I just realized that string mysteriously doesn't include 'strict' or whatever in it, so now I'm slightly confused as to what good it's for.)

That way, when something happened to a server, the standard can be trivally updated to say 'W3C now means this domain, instead of w3c.org', and every damn page in existence doesn't have to change. Mandate that every parser should expose these locations to be reconfigured manually if needed, although obviously some sort of automatic updating is a good idea. (Notice, in general, application software doesn't need to be updated, because application software doesn't try to download the stuff in the first place.)

Now someone's going to host the DTD at some random place, and everyone will manually update everything to load the wrong URL when someone asks for "http://netscape.com/publish/formats/rss-0.91.dtd" then Netscape will move it back, and some applications will change back, and some won't, and it will be a big mess.

I understand the point of paths, in that, in theory, everyone can produce their own format and publish their own DTD. This has not, and probably is not, going to happen, and at this point all browsers interpet DOCTYPE strings as unparsable strings, and the only ones who actually read the things are the validators.

Re:W3C doctype

[SteveAyre]

Granted, they already have something like this in the DOCTYPE, that's what '-//W3C//DTD HTML 4.01//EN' is, but then they blow it by then including the path after that. The parser should, instead, have to look at W3C and go 'Hey, I know where that is, that's w3c.org' and construct a standardized path using 'DTD HTML 4.01', like 'http://w3c.org/doctype/HTML4.01.dtd'. (And I just realized that string mysteriously doesn't include 'strict' or whatever in it, so now I'm slightly confused as to what good it's for.)

Just a few of the recognised ones are:

"-//W3C//DTD HTML 4.01 Strict//EN"
"-//W3C//DTD HTML 4.01 Frameset//EN"
"-//W3C//DTD HTML 4.01 Transitional//EN" ...which do include the strict etc. information.

"-//W3C//DTD HTML 4.01//EN" which you asked about is equivalent to "-//W3C//DTD HTML 4.01 Strict//EN"

Given that I've seen some websites using relative paths (ick) I'm guessing that they are used, by some parsers anyway, as an alternative to the path. I suspect it's up to the parser which order it tries them in or if it will even try more than one.

Re:W3C doctype

[DavidTC]

Ah, okay, I just picked a sample one I had somewhere because I, like most human beings, just pick whatever DOCTYPE our HTML editor gives us for the version of HTML we choose. ;)

And, yes, the entire thing is a mess. The point of paths was a noble one, but the end result is that absolutely nothing uses them except validators, and validators should be able to go and look up 'The w3c's standard for HTML 4.01' without have an explicit path.

Netscape Says No RSS 0.91 For You

[gastropod]

From April 2001, "Netscape removed the RSS 0.91 DTD from their website. This means that all RSS feeds which depend on the RSS 0.91 (many, MANY news sites) cannot be used with a validating parser."

It seems as though it just took them 5+ years to follow up on the threat? Primary links are broken, but of course the lively /. discussion (which, um, I haven't read) remains.

Re:Netscape Says No RSS 0.91 For You

[Bright Apollo]

mod parent +1 informative, this is the explanation necessary for the thread.

-BA

Context helps

[Anonymous Coward]

Non-validating parsers are still required to check documents against an internal DTD subset. Who builds an RSS reader using a non-validating parser without support for all 9 incompatible RSS versions? Obviously a reader attempting to retrieve this RSS DTD is built on a validating parser.

Sorry about that

[christopherfinke]

my.netscape.com is undergoing a redesign, and when we announced the redesign about 10 days ago, the DNS entry for my.netscape.com was changed to point to the new server where My Netscape will be living. This had the effect of making anything under the old my.netscape.com unavailable, since the only thing public on the new server is a splash page. (Nobody on the team was especially aware of this DTD file since all of the old Netscape employees were let go last year around the time Netscape.com was redeveloped; anybody working at Netscape now was hired since then.)

Now, why this file was living under my.netscape.com is anybody's guess, but we'll have it restored ASAP. I only wish that someone had brought it to our attention so that I didn't have to find out about it from Slashdot.

Christopher Finke
Netscape Developer

Re:Sorry about that

[mmurphy000]

What's the official way to let you know about this sort of thing? I'm not trolling -- the better you can inform folk like us about how to interact with you, the more likely it is you'll get a response when you need it. For example, a quick scan of the Help and FAQ pages linked to off of the Netscape home page shows no mention of how to contact folk like you.

Re:Sorry about that

[christopherfinke]

What's the official way to let you know about this sort of thing?

You're correct that contact information appears to be MIA in the Netscape Help pages; I'll make sure to remedy that ASAP.

For something as serious as this, a user could have checked the profile of one of the Netscape Anchors or developers, where many of them list their screennames or websites, and subsequently, their e-mail addresses. (At least, I know I do.) Alternatively, any Netscape.com member could use Netscape sitemail to contact any of the staff members. Obviously, these are unacceptable for normal circumstances, but I wouldn't call this situation a normal circumstance.

Re:Sorry about that

[martyb]

(Nobody on the team was especially aware of this DTD file since all of the old Netscape employees were let go last year around the time Netscape.com was redeveloped; anybody working at Netscape now was hired since then.)

Now, why this file was living under my.netscape.com is anybody's guess, but we'll have it restored ASAP. I only wish that someone had brought it to our attention so that I didn't have to find out about it from Slashdot.

Ummm, maybe I'm mising something here, but I would think that your web log would show a spike in 404 errors for this file, right? In my experience, it is helpful to assume that I do not know what I don't know, and to put procedures in place to help make those omissions stick out. So, a scan of your log files not only for this file, but for any others that also have a high number of 404's (especially from a multitude of referers) would be worth investigating.

BTW, best of luck on the redesign!

Re:Sorry about that

[Anonymous Coward]

any Netscape.com member could use Netscape sitemail to contact any of the staff members. Obviously, these are unacceptable for normal circumstances Why is it unacceptable for members to contact the site, in normal circumstances? That's anything but obvious.

Re:Sorry about that

[gjuk]

1) Isn't it great that the guy comes on and is open and helpful? There are plenty of organisations could learn from this

2) I found it amusing that the /. summary states "quite a few RSS feeds around the web probably stopped working properly" - what; so perhaps none stopped working? Would be great to see a list of ones which actually did.... anyont?

Re:Sorry about that

[dubl-u]

Hi, Christopher. First off, full marks for stepping up and explaining things honestly. You have done more good for Netscape than a dozen PR people. I'm sure you'll take a lot of crap from my fellow Slashdotters, but don't let it throw you. Listen to and acknowledge their legitimate complaints and you'll do fine.

I only wish that someone had brought it to our attention so that I didn't have to find out about it from Slashdot.

If you are looking to learn a lesson from this, how about this one: URLs are forever!

Whenever I make a change to a live server, my biggest concern is to not break existing usage. If I ever change an URL, I make sure to redirect old usage to new usage that's just as good. And if I'm ever not sure something is used, I generally look back at least three months in the logs. Especially if you've inherited a pile full of mystery, good analytical tools for your server logs are vital. Trying to run even a modestly-sized site without them is like running a large store without tracking your inventory: your life will become a series of unfortunate surprises.

Re:Sorry about that

[christopherfinke]

Sorry, I meant that it was unacceptable that a member would have to take all of those steps to actually contact someone, not that it was unacceptable for people to contact the emplpoyees.

Re:Sorry about that

[christopherfinke]

URLs are forever!

Indeed, words to live by. I wouldn't pin this mistake on one person not checking the right logfile though; in a company as large as AOL, when an entire 150-person workforce is laid off and a new (much smaller) team is brought in to manage the old properties, things sometimes get lost in the shuffle. The entire my.netscape.com service happened to be one of those things. I'm sure that this incident will act as a reminder to never let this type of thing happen again.

And BTW, it appears that the DTD file will be restored early tomorrow morning at the latest.

Re:Sorry about that

[VGPowerlord]

They removed every file, causing a spike in 404s for all of them.

Re:Sorry about that

[martyb]

Doh! <wipes egg from face>. Point made. I would still love to get my hands on their logs to see what other important pages may be failing.

Re:Sorry about that

[Anonymous Coward]

The file was living on my.netscape.com because that's what RSS was originally used for. The My Netscape portal was the first site to use RSS, and hosted both the DTD and the format documentation on that host.

FWIW, Netscape has done this before, and they only found out about it when RSS users and developers started screaming at them on blogs.

-Someone who's been around awhile.

Re:Sorry about that

[ggvaidya]

The feeds, I think, are working fine: they just spit out the DOCTYPE line normally. The problem occurs when feed readers, which use the DOCTYPE line to get the DTD to make sure the document is formatted just like the DTD says it is, try to download the DTD from Netscape's website. Since they get a 404, they assume something is wrong with the feed, and tell you as much.

According to the article, Google and Firefox's feed readers don't mind that the DTD's gone missing, but Microsoft Live's RSS Feed Gadget does.

Hope that helps.

Re:Sorry about that

[stevesliva]

an entire 150-person workforce is laid off and a new (much smaller) team is brought in to manage the old properties, things sometimes get lost in the shuffle.

I wonder if your beancounters would realize that a web property's value rapidly approaches zero when it is not maintained. Probably not. I guess they think netscape is a hole in the internet that you throw money into.

Re:Sorry about that

[GURU Meditation 8000]

don't your web guys check their access.log file or have stats for the site? ;-)

Re:Sorry about that

[Sirpete]

How about doing a Google link:my.netscape.com search before removing it? (76800 hits, all broken since...)

Huh?

[Digital Dharma]

Netscape? What's a Netscape?

Seriously bad programming

[owlstead]

If I would create a reader that was dependent on version 0.91 of the distribution, it sure as hell would include the DTD in local storage. It makes no sense to create a reader that can also use, say, version 0.92 since you would not know what had changed (and there is no such thing as inheritence between versions of a DTD afaik). Actually, as other readers noted, it would be terribly stupid to make your web-server or client rely on a third party computer for which you cannot guarantee the uptime.

These URL's are mainly there for their Uniqueness, not so much as for a place of quaranteed storage. Of course, they are also a nice place to look for the actual definition, but after that you would need a local repository. This is the first thing an XML library should support, and the first thing a moderately intelligent programmer should look at. I get *very* annoyed if this kind of basic rules are ignored. And I've even seen them ignored by people pointing to the XML digital signature definitions, where security and reliability should be the first requirements in the design.

Also, what would happen if w3c.org or netscape.com go the way of the Gopher? If they go bust? It's a quickly changing world out there.

Re:Seriously bad programming

[voidptr]

The DTD is spelled out in the incomming feed file though. Even if I had a local copy in my application, the RSS feed I just grabbed from Joe's Random Blog probably references the netscape copy, and there historically hasn't been a good mechanism to redirect that to a local DTD store.

The URLs there were initially intended to be used as unique namespaces since each domain had someone that could guarantee uniqueness inside that domain, not really as actual retrievable URLs, although they were usually valid and contained the official reference copy of that DTD.

Then people started using validating parsers that used the URL, and then people started using validating parsers in situations they didn't need them without realizing the mess they just created the first time someone tries to run an app without an internet connection.

Once that happened, anyone who published a DTD as a standard was taking on the responsibility of maintaining that url for decades to come, which is obviously an impossible situation in a large company.

Re:Seriously bad programming

[A beautiful mind]

From the article:

Apparently, a lot of feed readers and services (e.g. Firefox and Google) don't bother following links to dtd files, or may have their own cached versions available. But others -- Microsoft's Live.com RSS feed gadget is one example -- do check for the files, and refuse to load feeds if the referenced dtd file can't be located.

Microsoft software at it's best again. Don't look suprised

Re:Seriously bad programming

[DavidTC]

If the w3c collapses web standards are screwed anyway.

Re:Seriously bad programming

[Anonymous Coward]

hasn't been a good mechanism to redirect that to a local DTD store.

Yet somehow browsers manage to store certain files locally, even if they were originally from a remote server. We should have physicists look into this defiance of the laws of physics, perhaps even a grant from the government for a few billion dollars, in the event that this alien technology can be converted into a weapon somehow, or it can at least be taxed.

Re:Seriously bad programming

[owlstead]

"... and there historically hasn't been a good mechanism to redirect that to a local DTD store."

You mean like an EntityResolver? Of course I agree on the last statements.

Re:Seriously bad programming

[voidptr]

I said "historically". I've seen a half dozen Java XML parser toolkits recently, and not one mentioned setting that up anywhere in their docs. The hooks are there, but nobody knows to use it.

Re:Seriously bad programming

[owlstead]

Ok, I'll have to agree on that. There are some seriously bad Java Docs out there. The XML digitital signature is another example. A programmer just following the docs would simply check if the signature would match, and think the document was signed. If you would send a message that signed signed something completely different, the program would still say that the signature was valid. That's just the URL that points to an incorrect locations, there are about a dozen other checks that need to be performed that are just not mentioned.

If the documentation is lacking, even experienced programmers could go wrong. What if someone substituted a different URL for the scheme? Would it go to blackmagic.eu because it is was not found in the local repository? Why do these libraries start and go on the internet without asking anyway? Why not make the local storage the default? Oh well.

You gotta be kidding me...

[Sleepy]

This blast is not squarely aimed at you, but you triggered it. Treat this in the spirit it is meant please (if I didn't give a crap at all, I wouldn't comment. Show this to your insulated bosses who don't know the first thing about community and transparency. Kudos to you BTW for showing initiative and acting on a Slashdot post. Honestly, I would not have given the "new Netscape" that much credit.).

>I only wish that someone had brought it to our attention so that I didn't have to find out about it from Slashdot.

This rankles.

Have you EVER tried contacting Netscape from the outside world? Seriously, I can count the number of times:

*) When my.netscape.com locked out Konqueror (1998?)
*) When my.netscape.com WITHDREW the ability to embed RSS feed on your "my" page -- actually this was PRE-RSS if I recall. Way before it was commonplace, you could embed Slashdot and Linux Today feeds. Then they killed it, presumably because they got enough users or some pointy haired reason. 1999.
*) When my.netscape.com adopted a shitty policy of DELETING all your mail if you don't login for 30 days. This did not seem to be publicised by an actual email. They don't seem to delete the mailbox itself, which violates RFCs I'm sure and basically insinuates the mailbox is active. I lost tons of mail from 1996-2003 (yeah yeah backups. Some things I didn't think I would need later). ?? Happened in 2003. Note that mailboxes were only 5MB still, so I quickly bailed for a 100MB Yahoo account.
*) The 2001 deletion of Netscape Developer. This lost a ton of Netscape copyrighted Javascript documentation.

Just TRY contacting Netscape from their page. The best you can do is use the WRONG FORM to submit to some contracter who won't forward it. Or, oh yeah - there's a 900 number for by the minute Support.

Back when it mattered, there was no 'Google Guy' for Netscape, who would act as an unofficial liason. After Jamie Z left, no one internally tried to fill the shoes of a community facing employee.

While I'll be eternally grateful for Netscape's open sourcing of their browser. What a different world it is now. Too bad that step is something the current management would never have allowed (that's the perception). I can't think of a more opaque Internet company than today's Netscape. I'm sure there are people who disagree or wish it could be changed (you're here..) but that and a $1 gets you a cup of black coffee. Show this to your boss - there are suggestions here :-)

Re:You gotta be kidding me...

[christopherfinke]

You make several good points that I want to respond to more fully, but I've got to run out, so I'll have to do that later. In the meantime, I'll put this out there: my e-mail address is chris@newnetscape.com; my screenname and other contact information is available at my website. Anyone who wishes to do so can contact me regarding issues with any of the Netscape websites or the Netscape browser; if I can't solve your problem, I can definitely get you connected with the right person.

Re:You gotta be kidding me...

[Alphab.fr]

) The 2001 deletion of Netscape Developer. This lost a ton of Netscape copyrighted Javascript documentation. Unless I'm mistaken, this has been (quite some time afterwards) transfered to the mozilla fundation, and can be accessed at http://developer.mozilla.org/en/docs/JavaScript Cheers,

Re:You gotta be kidding me...

[christopherfinke]

I can't speak for the pre-2006 Netscape, but as far as the current Netscape organization is concerned, I have to disagree with you regarding transparency.

* The former GM, Jason Calacanis, blogged extensively about Netscape and encouraged other employees to do the same. He also called out other industry big-wigs for "not having time" to blog about their product.
* Many Netscape Anchors, Navigators, and developers have an active blog where they write about Netscape and/or are available to discuss it.
* There is the official Netscape blog where we detail new and/or upcoming features and ask for discussion and feedback.
* Netscape.com holds a weekly chat when there is guaranteed to be an Anchor in attendance.
* As developers, we are encouraged to be active in the community; we haven't been given any explicit "gag orders," so we're free to speak for Netscape (as I'm doing now) if the need arises.

Can the above be said to be true for Digg, del.icio.us, reddit, etc.? I'd have to say that of all the current players in the social news space, Netscape is one of the most (if not the most) forthcoming and transparent about their operations.

Re:You gotta be kidding me...

[poot_rootbeer]

Again, I appreciate your openness and availability, but if I have a problem with a Netscape website, the first place I'm going to look is not going to be the website of an individual developer, hosted over at efinke.com. I'm going to go to the Netscape Help page, and if I can't find a simple way to report a problem there I'm going to leave.

Your entire company needs to make a commitment to open dialogue, not just you individually.

Re:You gotta be kidding me...

[Anonymous Coward]

> Your entire company needs to make a commitment to open dialogue, not just you individually.

Yeah, but his entire company probably can't respond in time to a Slashdot article, whereas he can.

Why should anyone contact you?

[Anonymous Coward]

It is not the first time that Netscape dropped important files and Netscape didn't care in the past. E.g. the RDF schemata http://home.netscape.com/WEB-rdf and http://home.netscape.com/NC-rdf bite the dust some time ago and Netscape gave a fucking fart. Other format specs are also gone for years, but I am to lazy to look up what whent missing years ago.

But hey, from where I sit Netscape is anyhow run by a bunch of liberal arts graduates and doesn't have any technical competence left.

Logs?

[hhawk]

not trying to be a troll here.. but.. one would think that that file would have been accessed quite often and that would have shown up in the logs...

If I was a new hire at some old company where everyone else had been let go, I'd at least check out the logs and see what is being used? and then if some file is being hit 1,000's of times a day.. maybe ask a few questions..

Who's...

[Anonymous Coward]

..the other one ?

IE7 is already broken anyways

[The_Real_Deuce]

Last time I looked, if the RSS 0.91 feed references the DTD, IE7 refuses to display it anyways... it's only of the DTD references is removed that IE7 "works" "properly"

A rewording...

[Shaltenn]

Once upon a midnight dreary, while I websurfed, weak and weary, Over many a strange and spurious bookmark of 'free news galore', While I clicked my fav'rite feed, suddenly there came a warning, And my heart was filled with mourning, mourning for my dear amour. "'Tis not possible," I muttered, "give me back my free news source!" Quoth the server, "404".

Re:A rewording...

[Anonymous Coward]

Quoting without attribution or even any indication that it is a quote is plagiarism. 404.

127.0.0.1, anyone?

[AtariDatacenter]

Anyone remember the incident, maybe 8 years ago, when the root DNS servers dropped the entry for localhost, 127.0.0.1? We had a lot of random code break because of that.

Deja Vu (Super Dupe?)

[evilviper]

Wow.

I'd swear today I was looking at the /. frontpage from 4+ years ago...

After a little searching, I've found the exact same story, from April 2001:

http://slashdot.org/article.pl?sid=01/04/28/211921 1

Those who forget history are doomed to repeat it.

I look forward to reading this story on /. again in 2013.

Content based addressing

[Omnifarious]

The web needs some scheme for content based addressing. Like the urn:sha1 scheme used in gnutella. This (and some sort of reasonable caching scheme) would do a lot to alleviate problems like this. It could also help a lot with the Slashdot effect.

why don't...

[DaSH Alpha]

RSS readers just cache the DTDs and refer to them locally instead? It's not like they're going to change or anything...

This Explains Why People Hate the IT Department

[Master of Transhuman]

Referencing the other topic today...

You mean to tell me that every RSS reader references - and actually tries to FIND and DOWNLOAD - a specific SPECIFICATION hosted on ONE SITE ON THE PLANET?

Are you people utter fucking morons or what?

I can't believe design decisions like this.

I'm especially irritated because I have just spent the last week trying to find an rdiff-backup or rsync that functions on Windows WITHOUT A FUCKING 2GB or 4GB FILE SIZE LIMITATION! Even the Cygwin people could only tell me to "try it"...How about putting it in the fucking FAQ what the file size limits are?!

We HAVE LARGE FUCKING FILES THESE DAYS, MORONS!! We've HAD THEM FOR FIVE YEARS!!

Oh, nobody can be bothered to support librsync anymore...right...

Only Fedora Core 5 bothered to patch it so rdiff-backup can handle large files...

Backup isn't "sexy" like 3D wobbly windows, so nobody cares...

Chimpanzees. Jesus Baron von Fucking Christ!

Re:This Explains Why People Hate the IT Department

[vga_init]

Are you OK, man? :-/

Re:This Explains Why People Hate the IT Department

[Slashcrap]

I'm especially irritated because I have just spent the last week trying to find an rdiff-backup or rsync that functions on Windows WITHOUT A FUCKING 2GB or 4GB FILE SIZE LIMITATION! Even the Cygwin people could only tell me to "try it"...How about putting it in the fucking FAQ what the file size limits are?!

It really fills me with regret to hear that those free software developers didn't write their free software to meet your exact needs.

I suggest you use commercial backup software instead.

You can make the money to buy the software through male prostitution. It won't be a great stretch for you, since you're clearly already a cocksucker.

re:netscape rss

[haggus71]

OMGZ! Netscape is still around?

How critical is cache coherency in this case?

[tepples]

How critical is the guarantee that the local copy is identical to the remote copy in this case? If not, then why not cache a known good copy locally and then use the cached copy when the remote one is not accessible?

Re:How critical is cache coherency in this case?

[fatphil]

Caching indeed is the sensible thing to do. But what do you do when the original disappears? Should you remove your local copy? Caching simply moves the problem through time.

Re:How critical is cache coherency in this case?

[tepples]

But what do you do when the original disappears? Should you remove your local copy? Caching simply moves the problem through time.

Which is why you have to define at the time of designing your application what would best suit the end user. Is there a situation where the user would prefer that his or her documents become unusable?

sacked

[v1]

so what was the inspiration to sack the entire netscape team?

And this is not the first time it has happened. Every time I install netscape on an old iMac, it stalls out for like 90 seconds on initia launch because something else on your server the browser depends on has been deleted. A few times I had a very hard time installing Netscape because it refused to quit digging for that page on launch. I never did figure out exactly what it was it was looking for. (something to do with first run registration I would guess) This happened about what, a year ago.

So, how many other things in Netscape will blow up if netscape.com goes away? Since that would mean netscape was out of business, does anyone at netscape care? I've had to throw out software I paid for because the author's web site went away and prevented me from reinstalling the app, and that's REALLY annoying.

Catalog files...

[adageable]

I'm no RSS expert, and I don't spend that much time dealing with it, honestly.

However, I do spend a LOT of time dealing with XML. And many environments offer quite an elegant way of dealing with this: the catalog file.

The catalog file provides an alternative mapping between the public identifier and a local, system referenced file SO THIS SORT OF THING DOESN'T happen.

If the RSS standard is to be dependant on external doctype definitions, the RSS readers should either:
(1) Include the doctype hosted on their own system and change all of their doctype public URI's or (2) The RSS readers should include the requisite files locally, and contain a catalog that maps the public URI's to the locally deployed model.

Like I said before, I'm not so into RSS, so I'm not sure which is applicable, but certainly, I don't think that Netscape should have to host this...

XML has 5 entities

[jonasj]

HTML has tons of pre-defined entities but XML only has the core 4.

Small correction: It actually has 5 (< > & " '). The last one wasn't in HTML4.

Grrrrr

[jonasj]

I thought slash would escape the entities. Anyway, here they are: &lt; &gt; &amp; &quot; &apos;

Resolved

[christopherfinke]

The file is now restored, but it will not be available forever. See this post at the Netscape blog for the full details.