Re:VistA released under FOIA
on
Using the FOIA
·
· Score: 1
I believe that the requests were made by a doctor working with the VA.
Some parts of the code have not been released, because the whole process clammed up after September 11 2001.
I Don't know names, but someone on the hardhars or openhealth mailing lists should be able to give more detail.
VistA released under FOIA
on
Using the FOIA
·
· Score: 4, Interesting
One of the interesting uses of FOIA that I have seen has been the release of VistA, the software that runs the hospitals for the department of Veterans Affairs.
VistA is a huge suite of programs, and something that would not normally otherwise make it into the public domain. Billions of dollars of investment have gone into this, and there is hope that it can be used more widely.
VistA is written in "M" (sometimes called "Mumps" and with the GPL of the Linux version of GT.M, a compiler and database server for "M" by Sanchez, the whole shooting match is doable on an open-source platform.
The level of functionality available from this project is incredible, and in some areas is unmatched by commercial offerings costing hundreds of millions of dollars.
Firstly, we were able to make some substantial assumptions about our database updating.
Because the system is not directly interacting with registrants, we were able to define a limited set of messages in XML.
we ensured that the processing of those messages that effect change includes all change as a component of the response.
we ensured that the record ordering in all responses is controlled.
we ensured that the content of the response returned from any back-end application server will be identical to any other back-end application server (except for the cryptographic signature).
Finally, we put a layer on the front-end webserver, which feeds transactions to all of the backends in parallel. For transactions effecting change it waits for answers to be in agreement before passing the response on to the client. For enquiry transactions it just feeds the first response back as soon as it gets it.
Basically, we worked around the fact that there is no two-phase commit in PostgreSQL. There are a few gotchas in there that we had to workaround, (e.g. updates to any given domain are serialised) but the above was the plan, and it has worked out just fine in practice.
There is also a backend process to catch up the replication on any server that has to be taken out of the loop for a period of time. Since the backend servers are geographically distributed (with a VPN holding it all together) it is fairly easy for one to drop out of action for a short period.
All in all, the replication model we used took around 4 weeks effort to implement, which was probably a reasonable price to pay. Even with support for two-phase commit we would still have had a significant amount of effort to use it, and to do the automatic catch-up processing.
I was a designer of the system that runs.nz (New Zealand), which is also based around PostgreSQL, running on three replicated back-end application servers.
The system was developed in mod Perl and went live on October 14th 2002.
The plan is to release this (including client software) under the GPL after a stabilisation period.
The fact that the spam didn't bounce might validate your e-mail address, but it doesn't do that for the spammer because in almost all cases they fake the From address, and they send the e-mail through an unattended open relay.
They never see the bounce message.
I have had a spammer regularly using our companies domain as the RHS of their fake address, and I know exactly who receives these bounces:-(
Personally I get maybe 200-300 spam per week because I post on mailing lists, and my e-mail address is on the internet in a variety of places. Even running both spamassassin and bmf over my incoming e-mail I get around 30/week (but no false positives).
I still prefer to read and reply to my mail on my computer.
3G means that when I'm not on a LAN I can get it quicker though:-)
I can't honestly see that changing for the meantime. I filter my e-mail into around 70-odd mailboxes, and it helps me visually to handle the 1000+ e-mails I get each day.
If you have no past projects to judge on, then you are seriously up shit creek.
In my company we have developed for RS-232, DOS, Windows, Client/Server, WEB, WAP, PDA and Phone, and it usually ends up being my job to estimate it, or to estimate the bits nobody else has a clue about.
Estimating is seriously hard. People have tried all sorts of ways to simplify it (function points, for example) and it is still hard.
The first problem is that complex programming projects that are called "complete" in a reasonable time are actually compromises. Everyone starts with the idea of building something which works according to some (possibly vague) specification, but what gets implemented varies from that ideal by some amount of compromise.
The second problem is feedback loops. If I have 'X' functionality that I want to implement, sometimes I will think of a method of implementing 'X' which makes 'Y' additional functionality easy (Yay!), and sometimes my way will make 'Y' hard (boo!). This problem is somewhat behind the "we need a specification" camp, because if we write everything down first, then we can work on making our compromises in a structured way, choosing the 'X' implementation (that we think of) which makes 'Y' easiest. Alas, the more complicated (and hence more difficult to estimate) projects inevitably introduce requirement 'Z' after this just to screw it up. Or they are specified by bozo 'B' who just can't think crookedly enough. All of this feeds back (as good feedback should) into the first problem, of course.
Amusingly, our company was recently reviewed by a client for several projects and found to have a puzzlingly high development rate for each function point. It is my considered opinion that we achieve this partly by minimising the bureacratic overhead involved in specifying the system. This doesn't mean not doing a specification, it means simplifying it as far as is practicable for the project. Then the other part, is that we use talented programmers who deserve such specifications can independently make sensible implementation decisions during the project. Common sense is remarkably rare, but it is findable.
So you are up shit creek, because you don't know how to evaluate all of these side-effects. What that means is that you have to be conservative.
Conservative estimation is much easier. It takes a boring analyst around 2 days to design a screen (whateverthehell that is!), and it takes a boring programmer around 4 days to implement it. If it's a complex screen (lots of fields, say), double it. If it's very complex (lots of fields with lots of complex validation), double it again. If you want something that works reliably, factor another 2 days (times comlexity factor) in as well. Add 20% for project management. Add another 30% for implementation related tasks. Finally, add 30% at the end for fixing the stuff that you missed earlier.
Conversion to a 3G handset from a Palm OS device will take at least as much programmer and tester time as the original. Especially since Palm OS is mature and understood, and I don't know WTF a "3G handset" has running on it: Symbian? Java? Brew? Wince? PalmOS? You choose.
We name all of our servers on a theme, and this works well, in terms of isolating the names of the servers from their function. Our current theme is "philosophers" which can actually go on for a while.
If you pick a theme for your server names at the same time, the name for the server room can be quite straightforward. Name them after fish and a roomful becomes a school. Name 'em after birds and it becomes a flock. There are lots of collective nouns out there just waiting to be used.
So my suggestion would be to name them after orchestra instruments, so you could justify calling the place "The Pit":-)
Stuff also has daily coverage. This is the site for the major Wellington newspaper, The Dominion Post, (among others) and Wellington's really where the major LOTR action has been in New Zealand, especially since it's where Peter Jackson lives.
There is no external auditing of P3P that I can see.
I set this up on one website I built, but why? I was able to say whatever I wanted. If my browser acted on this sort of information I would be forced to disable it, since it is not, and cannot be trusted without an external verification.
I have worked as an application developer / designer with DBMS backed applications for the last 17 years. There are reasons for not choosing a database, but not usually very good ones.
When you want speed and flexibility and scalability and reliability and extendability and particularly developer productivity you will undoubtedly end up shooting yourself in the foot later if you avoid some form of DBMS up front.
Where you have a particularly well-defined, narrow functionality, and performance in a small footprint is a requirement, an RDBMS may not be such a good choice, but DB libraries like berkeley db can still be very useful.
And with PostgreSQL, Firebird , MySQL and so many other free, open-source projects out there covering such a broad spectrum of needs for a database, why would you not use that expert work?
Many here have spoken of the "insane" "horrendous" "crazy" amounts of money spent on IIS. How many think that this money was spent *mostly* to make sure that no one died?
Damn but ain't it a shame that all that money spent on IIS didn't go to improve Apache instead...
Please explain how something can be 20x quieter...
Decibels is a logarithmic scale.
So 10db is 10x (power), 20db is 100x and so on.
This might seem like they are understating the case, but in general 10db is considered to sound like "a doubling in volume" so a difference of 40db is roughly 2 to the 4th or 16x.
Fiddle with a few db more and you can easily justify "20x quieter."
Is Sub-selects and foreign keys. These are probably the two biggest features I've constantly found myself needing / wanting.
Then you probably should be using PostgreSQL, which has had these features for some time. It also has UNION, functional languages, extensible types, full transaction support, SQL standard syntax, unlimited row length...
Stick with MySQL when performance is critical and you want to do straight SELECTs from simple tables with few joins and very few updates.
Go with PostgreSQL if you want an RDBMS that supports transactions, that scales when writers and readers both use the DB at the same time, if you want to support complex queries or if robustness, programmability and configurability are important to you.
If it is abusive, then there are likely to be recognisable phrases or other patterns in the text. This will likely be particularly true if it is driven from one particular source.
My personal approach would be to write a quick perl program, and alias those e-mail addresses to that. If the program detected abuse, then the program could automatically reply expressing your point of view.
If no abuse was detected then you could just pass the e-mail through to the intended recipient at their new e-mail address, if you want.
This should take only a few hours to implement, max.
I've been playing with Gnome 2 for a while now, and I must admit it is starting to feel like a stable release.
There are a few things that I have mixed feelings about though. The default WM is switching to Metacity, which doesn't have the power and configurability of Sawfish, and that is symptomatic of the general reduction in configurability.
Someone, somewhere has decided that configurability === complexity, and that a bewildering array of choices is too many for a newbie. This appears to have been translated into a general 'dumbing down' of the interface.
I can no longer tell Sawfish to remember my window sizes. The Gnome Panel can no longer swallow applications, so all of those WM applets are now useless to me. I can't run the Afterstep clock applet!
I guess it is the applications job to remember what window size I last used, and to remember it without me telling the WM to do so, but they don't - not even Nautilus2 remembers it's window sizes - every time it opens a new window which is slightly less than 1/4 of my screen size.
Overall, this is probably a good thing. People should be writing their applications to remember UI things, and having the WM remember those probably makes them lazy, but I can see a bit of pain in my future with Gnome 2, until these issues are solved and Gnome 3 is released, perhaps.
At least Gnome 2 does seem somewhat snappier than Gnome 1.4, and the styling is better, especially with anti-aliasing available throughout.
Just because you have bandwidth to your house doesn't change the fact that you have an ISP who is providing that bandwidth to you and you have to abide by their rules.
Hosting elsewhere provides broader choice, allowing me to host anywhere in the world, and to pick a hosting ISP who doesn't restrict me.
Hosting elsewhere also provides much greater possibilities for anonymity.
However the commercial reality is that the available bandwidth in a connection is the total upstream + downstream, and that's what drivers the provisioning costs. The pattern of usage for most consumers is that they pull down more than they push. Fact.
P2P applications change that a lot, as does hosting, but individuals don't usually want to host stuff at home.
Sure, it's an interesting thing to say, and it challenges us to think about it, but rather than assume some vast conspiracy to defraud the consumer, consider what happens when the consumer gets what they ask for.
People can be producers without aving upstream bandwidth. Hell, many companies don't host their own websites, for a bunch of really good reasons. Likewise many ISP accounts I see allow for hosting along with your e-mail, so they can still be producers. It ain't hard.
I have a Fuji that uses SmartMedia but it's only 1.3 megapixels and I am in the process of looking for a 3.3 megapixel (or so) replacement.
I'm not planning on using SmartMedia though - from my reading around I have come to the conclusion that CompactFlash is probably the 'best' solution.
SmartMedia is a pretty raw interface to the memory (i.e. it ain't smart at all), and the camera has to support a particular size specifically, much like those older PCs would only take a 16M SIMM and not a 64M one. OTOH CompactFlash has more smarts on the device and requires correspondingly fewer of those little details to be built in. That's why all the really big ones (512M cards, 1G & 4G microdrives) are on CompactFlash.
And of course you would only get something with a memory stick in it if you were addicted to Sony. I ain't.
You might also see so-called 'MultiMedia' Cards appear to be similar to SmartMedia. Possibly they are crappier, since the website suggests that you won't get 'em larger than 64MB (and I can't find any on sale larger than that), and they have been around since 1997.
Slashdot Mirror
I believe that the requests were made by a doctor working with the VA.
Some parts of the code have not been released, because the whole process clammed up after September 11 2001.
I Don't know names, but someone on the hardhars or openhealth mailing lists should be able to give more detail.
One of the interesting uses of FOIA that I have seen has been the release of VistA, the software that runs the hospitals for the department of Veterans Affairs.
VistA is a huge suite of programs, and something that would not normally otherwise make it into the public domain. Billions of dollars of investment have gone into this, and there is hope that it can be used more widely.
VistA is written in "M" (sometimes called "Mumps" and with the GPL of the Linux version of GT.M, a compiler and database server for "M" by Sanchez, the whole shooting match is doable on an open-source platform.
More information about VistA hardhats.org and sourceforge and VistA Documentation Library
The level of functionality available from this project is incredible, and in some areas is unmatched by commercial offerings costing hundreds of millions of dollars.
Firstly, we were able to make some substantial assumptions about our database updating.
Finally, we put a layer on the front-end webserver, which feeds transactions to all of the backends in parallel. For transactions effecting change it waits for answers to be in agreement before passing the response on to the client. For enquiry transactions it just feeds the first response back as soon as it gets it.
Basically, we worked around the fact that there is no two-phase commit in PostgreSQL. There are a few gotchas in there that we had to workaround, (e.g. updates to any given domain are serialised) but the above was the plan, and it has worked out just fine in practice.
There is also a backend process to catch up the replication on any server that has to be taken out of the loop for a period of time. Since the backend servers are geographically distributed (with a VPN holding it all together) it is fairly easy for one to drop out of action for a short period.
All in all, the replication model we used took around 4 weeks effort to implement, which was probably a reasonable price to pay. Even with support for two-phase commit we would still have had a significant amount of effort to use it, and to do the automatic catch-up processing.
There's more information (a full technical architecture and prototype findings document) at the NZ Domain Name Commissioner's website
Andrew McMillan.
I was a designer of the system that runs .nz (New Zealand), which is also based around PostgreSQL, running on three replicated back-end application servers.
The system was developed in mod Perl and went live on October 14th 2002.
The plan is to release this (including client software) under the GPL after a stabilisation period.
The fact that the spam didn't bounce might validate your e-mail address, but it doesn't do that for the spammer because in almost all cases they fake the From address, and they send the e-mail through an unattended open relay.
They never see the bounce message.
I have had a spammer regularly using our companies domain as the RHS of their fake address, and I know exactly who receives these bounces :-(
Personally I get maybe 200-300 spam per week because I post on mailing lists, and my e-mail address is on the internet in a variety of places. Even running both spamassassin and bmf over my incoming e-mail I get around 30/week (but no false positives).
I still prefer to read and reply to my mail on my computer.
3G means that when I'm not on a LAN I can get it quicker though :-)
I can't honestly see that changing for the meantime. I filter my e-mail into around 70-odd mailboxes, and it helps me visually to handle the 1000+ e-mails I get each day.
He is soliciting suggestions for changes, but isn't giving out the source :-(
I know the change I would like is to make it run on non-Windows operating systems.
In my company we have developed for RS-232, DOS, Windows, Client/Server, WEB, WAP, PDA and Phone, and it usually ends up being my job to estimate it, or to estimate the bits nobody else has a clue about.
Estimating is seriously hard. People have tried all sorts of ways to simplify it (function points, for example) and it is still hard.
The first problem is that complex programming projects that are called "complete" in a reasonable time are actually compromises. Everyone starts with the idea of building something which works according to some (possibly vague) specification, but what gets implemented varies from that ideal by some amount of compromise.
The second problem is feedback loops. If I have 'X' functionality that I want to implement, sometimes I will think of a method of implementing 'X' which makes 'Y' additional functionality easy (Yay!), and sometimes my way will make 'Y' hard (boo!). This problem is somewhat behind the "we need a specification" camp, because if we write everything down first, then we can work on making our compromises in a structured way, choosing the 'X' implementation (that we think of) which makes 'Y' easiest. Alas, the more complicated (and hence more difficult to estimate) projects inevitably introduce requirement 'Z' after this just to screw it up. Or they are specified by bozo 'B' who just can't think crookedly enough. All of this feeds back (as good feedback should) into the first problem, of course.
Amusingly, our company was recently reviewed by a client for several projects and found to have a puzzlingly high development rate for each function point. It is my considered opinion that we achieve this partly by minimising the bureacratic overhead involved in specifying the system. This doesn't mean not doing a specification, it means simplifying it as far as is practicable for the project. Then the other part, is that we use talented programmers who deserve such specifications can independently make sensible implementation decisions during the project. Common sense is remarkably rare, but it is findable.
So you are up shit creek, because you don't know how to evaluate all of these side-effects. What that means is that you have to be conservative.
Conservative estimation is much easier. It takes a boring analyst around 2 days to design a screen (whateverthehell that is!), and it takes a boring programmer around 4 days to implement it. If it's a complex screen (lots of fields, say), double it. If it's very complex (lots of fields with lots of complex validation), double it again. If you want something that works reliably, factor another 2 days (times comlexity factor) in as well. Add 20% for project management. Add another 30% for implementation related tasks. Finally, add 30% at the end for fixing the stuff that you missed earlier.
Conversion to a 3G handset from a Palm OS device will take at least as much programmer and tester time as the original. Especially since Palm OS is mature and understood, and I don't know WTF a "3G handset" has running on it: Symbian? Java? Brew? Wince? PalmOS? You choose.
Hope this is some help!
If you pick a theme for your server names at the same time, the name for the server room can be quite straightforward. Name them after fish and a roomful becomes a school. Name 'em after birds and it becomes a flock. There are lots of collective nouns out there just waiting to be used.
So my suggestion would be to name them after orchestra instruments, so you could justify calling the place "The Pit" :-)
Check out this cool map :-)
Heh! I never realised how forward-compatible perl was until you pointed this out!
:-)
There is no external auditing of P3P that I can see.
I set this up on one website I built, but why? I was able to say whatever I wanted. If my browser acted on this sort of information I would be forced to disable it, since it is not, and cannot be trusted without an external verification.
See Debian Jr. for more information.
Surely the British Empire spanned more than that - it's 12000 miles or so from England to New Zealand...
When you want speed and flexibility and scalability and reliability and extendability and particularly developer productivity you will undoubtedly end up shooting yourself in the foot later if you avoid some form of DBMS up front.
Where you have a particularly well-defined, narrow functionality, and performance in a small footprint is a requirement, an RDBMS may not be such a good choice, but DB libraries like berkeley db can still be very useful.
And with PostgreSQL, Firebird , MySQL and so many other free, open-source projects out there covering such a broad spectrum of needs for a database, why would you not use that expert work?
Damn but ain't it a shame that all that money spent on IIS didn't go to improve Apache instead...
:-)
Decibels is a logarithmic scale.
So 10db is 10x (power), 20db is 100x and so on.
This might seem like they are understating the case, but in general 10db is considered to sound like "a doubling in volume" so a difference of 40db is roughly 2 to the 4th or 16x.
Fiddle with a few db more and you can easily justify "20x quieter."
Then you probably should be using PostgreSQL, which has had these features for some time. It also has UNION, functional languages, extensible types, full transaction support, SQL standard syntax, unlimited row length...
Stick with MySQL when performance is critical and you want to do straight SELECTs from simple tables with few joins and very few updates.
Go with PostgreSQL if you want an RDBMS that supports transactions, that scales when writers and readers both use the DB at the same time, if you want to support complex queries or if robustness, programmability and configurability are important to you.
If it is abusive, then there are likely to be recognisable phrases or other patterns in the text. This will likely be particularly true if it is driven from one particular source.
My personal approach would be to write a quick perl program, and alias those e-mail addresses to that. If the program detected abuse, then the program could automatically reply expressing your point of view.
If no abuse was detected then you could just pass the e-mail through to the intended recipient at their new e-mail address, if you want.
This should take only a few hours to implement, max.
Gnome2 will hit Debian unstable fairly soon. In fact it is mostly in there already - there are just a few things in experimental really.
I would say that in your case you probably will be better off with Gnome2, because it does seem snappier than 1.4.
I've been playing with Gnome 2 for a while now, and I must admit it is starting to feel like a stable release.
There are a few things that I have mixed feelings about though. The default WM is switching to Metacity, which doesn't have the power and configurability of Sawfish, and that is symptomatic of the general reduction in configurability.
Someone, somewhere has decided that configurability === complexity, and that a bewildering array of choices is too many for a newbie. This appears to have been translated into a general 'dumbing down' of the interface.
I can no longer tell Sawfish to remember my window sizes. The Gnome Panel can no longer swallow applications, so all of those WM applets are now useless to me. I can't run the Afterstep clock applet!
I guess it is the applications job to remember what window size I last used, and to remember it without me telling the WM to do so, but they don't - not even Nautilus2 remembers it's window sizes - every time it opens a new window which is slightly less than 1/4 of my screen size.
Overall, this is probably a good thing. People should be writing their applications to remember UI things, and having the WM remember those probably makes them lazy, but I can see a bit of pain in my future with Gnome 2, until these issues are solved and Gnome 3 is released, perhaps.
At least Gnome 2 does seem somewhat snappier than Gnome 1.4, and the styling is better, especially with anti-aliasing available throughout.
Just because you have bandwidth to your house doesn't change the fact that you have an ISP who is providing that bandwidth to you and you have to abide by their rules.
Hosting elsewhere provides broader choice, allowing me to host anywhere in the world, and to pick a hosting ISP who doesn't restrict me.
Hosting elsewhere also provides much greater possibilities for anonymity.
However the commercial reality is that the available bandwidth in a connection is the total upstream + downstream, and that's what drivers the provisioning costs. The pattern of usage for most consumers is that they pull down more than they push. Fact.
P2P applications change that a lot, as does hosting, but individuals don't usually want to host stuff at home.
Sure, it's an interesting thing to say, and it challenges us to think about it, but rather than assume some vast conspiracy to defraud the consumer, consider what happens when the consumer gets what they ask for.
People can be producers without aving upstream bandwidth. Hell, many companies don't host their own websites, for a bunch of really good reasons. Likewise many ISP accounts I see allow for hosting along with your e-mail, so they can still be producers. It ain't hard.
I have a Fuji that uses SmartMedia but it's only 1.3 megapixels and I am in the process of looking for a 3.3 megapixel (or so) replacement.
I'm not planning on using SmartMedia though - from my reading around I have come to the conclusion that CompactFlash is probably the 'best' solution.
SmartMedia is a pretty raw interface to the memory (i.e. it ain't smart at all), and the camera has to support a particular size specifically, much like those older PCs would only take a 16M SIMM and not a 64M one. OTOH CompactFlash has more smarts on the device and requires correspondingly fewer of those little details to be built in. That's why all the really big ones (512M cards, 1G & 4G microdrives) are on CompactFlash.
And of course you would only get something with a memory stick in it if you were addicted to Sony. I ain't.
You might also see so-called 'MultiMedia' Cards appear to be similar to SmartMedia. Possibly they are crappier, since the website suggests that you won't get 'em larger than 64MB (and I can't find any on sale larger than that), and they have been around since 1997.
Damn right we should. Stupidity is no defence against illegality, and nor is short-sightedness.
They broke the law.
Sure, lot's of companies would like to ignore the law: these guys did.