Slashdot Mirror
Web-Crawling Program Spots Disease Outbreaks
no1home writes "There is a story at Discovery Channel's site about a new utility for mapping disease. The premise is to have bots crawl the web looking for stories about disease outbreaks and log them onto a map. '"We were originally thinking about how we could expand disease surveillance and pick up outbreaks earlier than traditional methods," said John Brownstein of Harvard Medical School and Children's Hospital Boston, who created HealthMap in September of 2006 with Clark Friefeld, a software developer at Harvard Medical School.' But then it was noticed by Google.org and has since grown into its own website, HealthMap Global disease alert map, and claims to be able to identify 95% of all disease outbreaks, some of them before WHO or CDC."
Now all the hypochondriac hyper-nerds have another reason to sit home on their computers, cloistered from the outside world. :-)
Part of the hardcore faithful who believed in Apple long before it was cool again to do so
CLOSE THE PORTS
Fascinating TED Talk on a similar (the same?) project? As I recall, some of video was a bit unpleasant to watch, but (IMHO) very worthwhile.
I'm a nature photographer.
a facebook or other social app for people to self report symptoms is a great idea that no one has uilt yet. one could even "out" symptoms of their friends or speculate which friends made them sick. lots of issues with it, but a different data source for inf disease folks, even if the data was not completely accurate, would be helpful in predictions.
too busy to do it myself now...
> one could even "out" symptoms of their friends or speculate which friends made them sick. lots of issues with it, but a different data source for inf disease folks, even if the data was not completely accurate, would be helpful in predictions.
Yeah, right, just what we need, an inaccurate resource for the insurance companies to data-mine. Your premium has now increased by a factor of 5, just because someone with your name (Mike Smith) allegedly made someone else sick. Great.
No thanks.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
Let's apply this system to detecting suspected election fraud across the US! Have you seen a CEO personally applying firmware patches to voting machines? Put a pin on the map! You have evidence that someone's vote has been bought? Put a pin on the map! Soon we will know which states' votes can be trusted and which ones can't.
On the other hand, perhaps we should apply pins in cases when we see something that is actually GOOD in politics. Surely this would require far fewer pins?
as far as I can make out. It relies heavily on human reporting. And sometimes it takes a while for news on disease outbreaks to make the news.
Unless there is some way to report directly TO this crawler, I seriously doubt the claim that a web crawler can know of outbreaks before the WHO does.
hmm... I just referenced The Who - a band...
Seven Days with Ubuntu Unity
This is neat - although how useful it is I don't know... It'll be pretty obvious that communitites not tied to the www 24/7 will be sorely under-represented. Also, the disease categories seem a bit narrow - it would be cool to have stuff like murder, violence and alcohol related-disease in there. Ok, not transmissible diseases in their own right but they still have some pretty profound health-related consequences.
The CDC, and local and state health departments all have a list of "reportable" diseases. (Things from TB to gonnorhea to ebola to SARS) If a doctor encounters them, they are supposed to notify the health authorities. That is for biostatistics and epidemiology purposes.
If they have to look these cases up in the news instead of getting notified by hospitals and clinics, then the system is in a really bad shape.
What the designers expect from this will highly color my opinion on it. The article linked isn't exactly clear on this.
Do you want to track and try and predict disease breakouts in first world areas then probably decent, track world wide stuff then terrible. Outside of the obvious (self reporting) there is the whole issue of how much of the world is on the internet? While much of the first and even quite a bit of the second world countries are on the vast majority of the population doesn't have computers, let alone internet access.
I can easily see many many great uses for this and I expect all of them to be explored at some point - I can also clearly see many not so great uses and I fully expect them to be used too. As the old saying goes, there are lies, damn lies, and statistics.
In fact we can already see the maps being posted and used by people who have little to no understanding (if we are generous, I'm sure some understand and use them to further their own aims) to say things the data *can not say* and it isn't even mainstream yet. *sigh* It's like many things we have today - the greater amount of good it can do the greater amount of abuse one can use it for too.
------- Sorry about the spelling, I suffer from two problems. Dyslexia makes it difficult to spell well, lazy makes it
Apparently news reporting is still good for something. I never would have guessed.
I was looking forward to viewing the "erectile dysfunction" map based on viagra posts.
Seems that the only outbreak in Iraq is rabies. Figures. Must be Al Qaeda.
why would anyone rely on reports from the media on what outbreaks are going around when you have trained professionals with lab equipment diagnosing these illnesses to begin with?
If you mod me down, I will become more powerful than you can imagine....
I've been playing Pandemic 2 all night and this is really freaking me out.
I think I'm moving to Madagascar...
The New York City Dept of Health monitors sales records of certain medications gathered from drugstore chains to detect disease outbreaks and biological attacks.
Airplane Photos, Airline News, Planespotting Guides
Another report of the same HealthMap thing was on /. not too long ago:
http://science.slashdot.org/article.pl?sid=08/07/09/1424247
and what I said then still stands - the plural of "anecdote" is not "data". I defy anyone to come up with useful statistical models and tests on actual disease incidence based on web-crawling for disease names.
Well, it's not really a "map of disease"
breakouts. In fact the map part is rather just
a shiny pony?
A list could have done just the same amount
of good. Since for the most part each area has
one pushpin that just sums up the area.
[FWIW, I only looked at US pins.]
I was expecting a cluster map, like you see on...
Wunderground Wundermaps
or on...
http://www.housingmaps.com/
At least if it was a cluster map I could
look at an area and think, "I sure as heck
ain't traveling there for work this week."
I think if public interaction would be
allowed, that would turn up the dial to
a more 'fine' resolution rather than the
grainy "Cryptosporidium in local pools"
that I already know about cause I read
the local paper. Or that the measles
outbreak is almost contained. I can get
that from the 10pm news.
That further detracts from the usefulness
of this website as it stands, because I
doubt someone that reads the news less
than I do, would be more likely to go to
a website and search what new diseases
popped up this week. [All hypochondriacs
aside]
It's a good seed/foundation as long as
they have the financial stamina to keep
it going.
-AI
For me, it is far better to grasp the Universe as it really is than to persist in delusion
If your purpose is to scientifically track and deal with real diseases, reporting from the lab is useful.
If your purpose is to dig up scare stories as headlines to sell newspapers/websites/etc, the which ever way digs up more scare stories (regardless of being true or not) is useful.
Oliver.
What if you're too sick to blog about it?
I believe we just saw this. A search on this site for the single word "disease" shows this link 3rd:
http://science.slashdot.org/article.pl?sid=08/07/09/1424247
No google foo required to find this dupe.
jeff
Is this the opposite of the "Manly Joe" who never lets on that he is sick and takes computer lessons from the Amish?
My first Journal Entry ever, in 8 years! http://slashdot.org/journal/365947/aphelion-scifi-fantasy-horror-poetry-webzine
I think the day has come where hospitals need to maintain reliable Blogs considering they have the resources to do so. If it proves efficient, outsourcing to small IT service providers to maintain these blogs/websites will also be good for businesses.
At least if it was a cluster map I could look at an area and think, "I sure as heck ain't traveling there for work this week.
On a global scale, check out the RSOE EDIS (Emergency and Disaster Information Service).
It aggregates all sorts of disasters, from short-time events such as automobile accidents, and current tropical storms, to longer term ones, such as epidemics and forest fires.
It is not that I don't like the idea but it is essentially flawed.
A) It still requires human input. No one reporting the disease does not mean that it is not there.
Looking at former Yugoslavia and seeing only 1 case of meningitis while here in Bosnia everyone knows about (and it is on TV, radio and in the papers) the brucellosis epidemic that has been going on for months or even years maybe.
B) That input must be made over the internet.
Look at Africa. It is practically squeaky clean. There is one case diarrhea in the entire Botswana. And everyone is completely healthy up in the North.
Could it possibly be due to the lack of internet-based inputs instead of due to the lack of diseases?
Check out UK or the East Coast of USA. They are crawling with diseases.
C) It should preferably be in English. Can the crawler read any of these articles:
http://www.zzjzfbih.ba/content/view/66/13/
http://www.dw-world.de/dw/article/0,,3259389,00.html?maca=bos-rss-bos-all-1475-rdf
http://www.slobodnadalmacija.hr/BiH/tabid/68/articleType/ArticleView/articleId/14733/Default.aspx
http://www.dnevniavaz.ba/dogadjaji/panorame/bruceloza-prepolovila-prodaju-livanjskog-sira-
http://www.blic.co.yu/repsrpska.php?id=44508
Basically, what they come to is that there is a SHITLOAD of cases of brucellosis among the various cattle in Bosnia.
And that it is going to stay that way for a long time, cause nobody is really doing anything about it.
It is a fine idea, but unless you have every square kilometer of Earth covered with internet access and people who will report it in a language that the crawler understands - it is beyond useless.
Even dangerous.
Zoom out over Asia and turn on the Google in Chinese under Feeds. China's disease count jumps from around 40 to around 140.
No. You can't fix all the problems by "putting it on the internet".
Mit der Dummheit kämpfen Götter selbst vergebens
Uh ... they better distinguish between the human-to-human diseases and the ones that are spread to humans from elsewhere. Yellow Fever, for instance, is always present in the jungle wildlife and only occasionally spreads to humans. I'd hate to have some unsuspecting tourist become a datapoint on the map in Paraguay. And just because Arizona has not reported any bubonic plague cases recently doesn't make it safe to play with the prairie dogs.
Did anybody else read the title of this story and wonder if Spider-Man was dabbling in epidemiology?
Here's one to mull over...
Let's say for a moment that you come home after a long day and start to notice you have a bit of a cough, a stomach ache or even joint/muscle pain. You then hop onto your computer and post an innocent message to either a web forum you visit regularly or to a personal blog that you aren't feeling well with a brief description of what's bothering you.
Now, let's say this is a common habit you have where you make such posts every few days, particularly with days you don't feel well.
What's to stop someone with a vested interest in knowing your baseline health cycle started archiving this information? Depending on the circumstances and who is given access to this information, it could be quite easy for someone to profit directly off of your well-being.
If you have a habit of complaining about your health online on a regular basis, your little cries for sympathy could be turned against you should someone like your insurance company decide to use such a system to secretly track you in this manner. And, combined with the near free-reign various industries now have to track your daily activity without notifying you first, they could probably get away with such tactics quite easily.
8==8 Bones 8==8
Time to put Bob on the case
http://en.wikipedia.org/wiki/ReGenesis
A computer once beat me at chess, but it was no match for me at kick boxing. Emo Philips
Flawed doesn't mean it's a bad idea or shouldn't be done; much the opposite in fact.
A) It still requires human input. No one reporting the disease does not mean that it is not there.
If no one is reporting the disease, it doesn't matter whether you don't report it online or by ACME 15-second express to their doorstop - it's not being reported.
B) That input must be made over the internet.
Duh. Some places will clearly benefit more than others... so what? If I have the ability to increase the literacy rate in five nations, I'll do it even if I can't do it in all the rest yet.
C) It should preferably be in English.
Unavoidable. Except it's not too hard to write in an ability to pick up on any of a multitude of languages. Hell, just use Google's page translator for a fugly hack.
This isn't meant to solve all the problems, but I'm sorry - if implementing this concept means only one life is saved over the entire planet, it is entirely worthwhile. And if it increases reaction time to any single outbreak, it'll be irreplaceable. Nothing ventured, nothing gained and there is no harm in trying something simple and easy using technology we clearly already have.
I live in constant fear of the Coming of the Red Spiders.
I wonder if search engine requests could be accumulated in a similar useful way...
This isn't meant to solve all the problems, but I'm sorry - if implementing this concept means only one life is saved over the entire planet, it is entirely worthwhile. And if it increases reaction time to any single outbreak, it'll be irreplaceable.
No - it isn't.
Did you bother to read what I said at all?
Or look at the map they are presenting?
Example I made of China should point you out just how many cases are unseen if you miss a single language.
And Google still speaks only a handful of them - badly.
While the lack of internet infrastructure has just made Africa free of disease. WO-FUKIN-HOO!
US is choking with disease and epidemics but Africa is clean.
Disease control costs money and resources. Both human and material.
A tool like this is nothing more than a noise generator at this point. And a faulty one at that.
It will create requests to send in medical research teams to a bunch of restaurants on Manhattan cause someone had an upset stomach, but it will ignore almost entire continents because it does not speak their language or because they don't have "the internets".
Duh. Some places will clearly benefit more than others... so what? If I have the ability to increase the literacy rate in five nations, I'll do it even if I can't do it in all the rest yet.
Yeah... sure...
And if we had the ability to increase the size of tits in developed nations, who really cares about some Africans dieing of cholera, right?
Both of those things are a "doctor matter".
Epidemics are not about "one life is saved over the entire planet".
Epidemic means that it is a threat to a entire society of people. One life, or five or ten or even hundred can be meaningless in such situations.
Right now, this is a unreliable and therefor pointless and dangerous tool that can only increase the dispersion of available resources without actual benefit to anyone.
In 5-10 years, as Google ads more languages and actually learns to use properly the ones it lists now - maybe there will be SOME use for it.
Even so - it is still flawed unless you start feeding it medical records from every medical center and emergency room in the world.
But then - the privacy concerns raise its ugly head. So we will probably never see that part.
So, unless we all turn into compulsive hypochondriacs who blog all day about their diseases - there will be not be enough input even in developed nations.
And even in a world of compulsive hypochondriacs who have no privacy concerns - Africa and a great part of the rest of the world will remain out of the system because they lack the internet connection.
And compulsive hypochondriacs who blog all day about their diseases.
Mit der Dummheit kämpfen Götter selbst vergebens
While the lack of internet infrastructure has just made Africa free of disease.
Not in the least! I don't think anybody is suggesting closing up shop and making this the be all and end all of epidemiological tools. All it is is just another way of discovering or verifying an outbreak. You don't have to choose between tits and cholera; you're lucky enough to be able to get your tits as an extra bonus to halting the spread of cholera.
I live in constant fear of the Coming of the Red Spiders.
- this system does not DETECT outbreaks (nor does it claim to): it presents a map of already reported outbreaks
- no disease outbreak EVER ever been initially detected by an automated system before an alert doctor or other healthcare provider: a human has ALWAYS been the "sensor" that detects disease outbreaks on the ground.
- the WHO and the CDC are not SUPPOSED to identify every disease outbreak in the world, just like a building maintenance staff isn't supposed to identify every coffee spill in the building: most coffee spills can be handled locally, and the maintenance staff never needs to be notified at all. Likewise with disease outbreaks.
It is very interesting, and sometimes useful in generating hypotheses, to view a map of news coverage of outbreaks, but a system like this can never, ever take the place of alert clinicians who, with brain computing power that dwarfs the fastest processor Google can field, are able to spot trends and unusual cases better than any computerized system.
Better to also spend money improving the education and capabilities of those clinicians, and improving the systems by which they can notify a larger world about their unusual cases -- maybe those notifications could feed into a system like the one described and finally give it the ability to truly detect outbreaks.
Joel Selanikio, MD
DataDyne.org
Washington, DC
Right now it is far less useful than for example googling for news about diseases.
For fuck's sake... look at the map. Half the diseases it lists are what could otherwise be described as "tummy aches". And most of them across US.
It does not discover or verify anything. It is only generating noise at one end and ignoring everything at the other.
It is a blind man holding an elephant's dick.
Mit der Dummheit kämpfen Götter selbst vergebens
One thing that the article should mention, but doesn't, is the system's false positive rate.