I don't think that scraping html pages is the point of this project. They scraped web documents in order to construct an RDF encoded knowledge base that is searchable as a semantic web.
A semantic web relies on document authors to encode machine readable meaning into their documents. This codifies meaning and removes a burden of audience (read search tool) inference. It allows authors to apply another layer of clarification. This can be considered a burden or an opportunity by the author.
The issue of trust can be addressed by the semantic web adding more meaning to the original document. This is how the shared experience of web users can be leveraged to handle untrustworthy sources.
Nothing can replace human, tacit knowledge based filters as to what is useful and what is BS, but this is a new methodology that is better than what we've got.
Slashdot Mirror
I don't think that scraping html pages is the point of this project. They scraped web documents in order to construct an RDF encoded knowledge base that is searchable as a semantic web.
A semantic web relies on document authors to encode machine readable meaning into their documents. This codifies meaning and removes a burden of audience (read search tool) inference. It allows authors to apply another layer of clarification. This can be considered a burden or an opportunity by the author.
The issue of trust can be addressed by the semantic web adding more meaning to the original document. This is how the shared experience of web users can be leveraged to handle untrustworthy sources.
Nothing can replace human, tacit knowledge based filters as to what is useful and what is BS, but this is a new methodology that is better than what we've got.