3-D Software for 'Virtual Surgery'

Roland Piquepaille writes "Computer scientists at Brigham Young University (BYU) have developed a new software tool to perform 'virtual surgery'. This tool, dubbed 'Live Surface,' will allow surgeons to visualize in 3-D any part of a patient's anatomy with just a few clicks of a mouse. Similar software already exists, but according to the Deseret Morning News, Live Surface is interactive and fast. This software can be used for better diagnosis by physicians, but it might even suppress the need for some exploratory surgeries. The researchers add that Live Surface might even been used for special-effects in movies or games by extracting an actor's performance from a video clip."

Comments

A pioneer

[Lindsay Lohan]

I've been visualizing human anatomy in 3-D for many years.

Re:A pioneer

[Lindsay Lohan]

Sorry to disappoint, but I am in reality a pathetic, overweight middle-aged man, a hairy stretchmarked gut resting my greasy keyboard as I write this, chewing Nicorette and choking down a few hotpockets.

But if you want, we can still trade emails?

Re:A pioneer

[andrewman327]

Dad?!?

Yeah, its us.

[engagebot]

Yeah, this already does exist. Because we make it. www.meti.com

We have a laproscopic surgery simulator for a mere $40k that will totally blow your mind. You can learn to stitch, tie knots, remove gall bladders, the works.

So far today...

[Apocalypse111]

Ok, so today we've had stories on how to do 3D Virtual Reconstructions of places or environments and now 3D visualization of people's internals. All we're missing is an article on holograms.

I want my freakin' holodeck!

So we'll soon have photoshop for the body.

[w33t]

I can see Doctors of the future using the magenetic-lasso to extract tumors more easily.

We can only imagine, however, what the clone tool will be used for.

If only...

[Trouvist]

If only they truly had the technology they claim, they would have quickly been bought up by the GE's or Microsoft's of today. Does anyone here have any idea of the worth of an alogrithm that would automatically segment the entire human body for virtual exploratory surgery within reasonable timeframes?

Machine shop for the body?

[The Great Pretender]

Here we go: You scan an MRI, feed it in to the computer. Some Dr. on his sail-boat looks at the MRI identifies the area to be removed, and does a virtual surgery. The virtual surgery goes into the computer. The patient gets prepped, goes into surgery, a robot surgen following the 'virtual surgery' removes the offending piece.

It all sounds so nice and efficient, but I can see so many things were this could go horribly wrong. I for one will be sticking with the over-worked, stim-taking resident who will be standing by my body. I don't feel comfortable with the medical industry moving in the same direction as the car manufacturing industry.

BYU press release link with more media and info

[Pvt_Waldo]

There is a press release right here from BYU that has links to various videos and other media. Can't seem to find any papers or articles about the process, though I noticed it's being patented so there may not be a lot available (?)

Re:BYU press release link with more media and info

[grammar fascist]

Can't seem to find any papers or articles about the process, though I noticed it's being patented so there may not be a lot available (?)

Having taken a vision class from Dr. Barrett (CS 750 at BYU), I can fill in some details. I might be able to dig up the paper later. I think you can find it in the latest SIGGRAPH proceedings - dunno if Citeseer has indexed it yet.

It's a segmentation algorithm that works well and fast in 3D images. It uses a graph-cut algorithm to classify voxels as inside or outside whatever you're trying to isolate. You (the doctor) lay down "seed" voxels with a mouse, clicky-clicky, and a few seconds later, the algorithm has isolated the structure. For example, say you want to isolate bone. Hold down the mouse button and move it over the bone. Hold down the other and move it over non-bone. If the algorithm makes a mistake, make some more seed voxels.

This is nothing new so far - the CV folks have been segmenting with graph-cut for ages. The problem is that it's very, very slow - minutes for a single segmentation. Barrett and Armstrong have developed a hierarchical version of the algorithm that uses watershed regions to presegment, and merges them as it runs. Doing graph-cut on large regions is a lot faster than doing it on single voxels. Their stuff is the first interactive speed, seeded 3D segmentation algorithm that produces quality results.

I saw the demo in class. It was really rather impressive, if you're familiar with the subject area.

Does it have Clippy?

[Rob T Firefly]

Hey, it looks like you're perfoming a coronary artery bypass operation!

Would you like to:

• Obtain a suitable transplant from the greater saphenous vein?
• Overcharge the patient's laughable insurance?
• Get in a quick few strokes on the back nine before the operation?
• Play golf instead?

BSOD

[TheAngryMob]

A system crash would give new meaning to "Blue Screen of Death."

Been there, done that....

[Zarian]

I've been doing this for a while now. http://www.atlus.com/trauma_center/ ;)

old ideas, new clothes

[noneme]

Virtural surgery is not a new concept...I remember being in Junior High School and having the option in science class to disect a frog or to use a program on a 386 to move around .bmp's of a frog's anatomy.

The 3D model is an interesting way to put the MRI / CAT data on a computer screen (and far better than the .bmp's of a frog's organs) but what advantage (besides eye-candy) does this offer over looking at the raw MRI or CAT results?

One thing that could make this a great learning tool is an interesting interface that would help one practice a surgery with something more than a mouse or touch screen. Nintendo and Altus have already created a toy that does this, a far more intricate and realisitic version could be of use: http://ds.ign.com/objects/695/695152.html

Re:old ideas, new clothes

[VoidEngineer]

The 3D model is an interesting way to put the MRI / CAT data on a computer screen (and far better than the .bmp's of a frog's organs) but what advantage (besides eye-candy) does this offer over looking at the raw MRI or CAT results?

The answer is pretty simple. Doctors have to deal with information overload, and 3D models are an effective way of managing huge amounts of data. Consider: A typical MRI exam contains 60 to 90 slices. Looking at a single 3D image is much more efficient than looking through 60 to 90 2D slices and trying to form a mental 3D image in your imagination.

I work as a PACS Administrator for a Department of Radiology at a community hospital in NYC (PACS stands for 'Picture Archiving and Communication Systems'... I'm the network, systems, database, and applications administrator all roled into one). We have two 16 slice CT scanners and a 1.5 tesla MRI scanner... we average about 2,000 CT and MRI cases per month, and each one of those cases requires about 20 to 30 minutes of time for a radiologist to read. When you do the math, we have 4 to 6 full time radiologists on staff (at $350,000 yearly salary each), meaning that we're spending about $1.5M to $2M just on salary to get through those images.

Storage wise, each CT or MRI slice is generally 512x512 pixels, and works out to be something like 60 to 100kb, if I recall. So, a typical MRI series tends to be about 5MB or so. Different scanners average different sizes depending on how you want the images to spit out, but each series tends to range between 1 and 10 megabytes. The thing is, you might have up to a dozen series taken during your exam (those of you who have layed in an MRI scanner for an hour know what I'm talking about). A rule of thumb that I use is that an hour of our MRI scanner's time is roughly equal to 100MB of data. Each day, we generate an average of 1GB of new clinical data from our MRI scanner that has to be read by a radiologist.

So, consider it this way... each day, we have 3GB of data generated (MRI scanner and 2 CT scanners), at 365 days a year, equals a cool terabyte of clinical data each year that has to be read by the doctors. At $2M per year in doctor's salaries, we're spending approximately $1.50 to $2.00 per MB to get the cases read.

Anyhow... thought you might be interested to know some of the economics involved in reading MRI and CT results. The 3D models takes approximately 10 minutes to read, rather than 20 to 30 minutes to sort through the 2D images. The net result is that doctors who are comfortable with the 3D images can sift through 2x to 3x more cases. And because the doctors are paid on a per-read basis, that's the difference between making $350,000 per year and $700,000 per year.

All I can tell you is that when you start dealing with terabytes of MRI and CT images, they start all looking the same after awhile... and the doctors use any tricks they can to manage the data, simplify it, and process it. It's a clasic case of information overload. The 3D images help to simplify the 60 to 90 slices and to organize it into a coherent 3D object.

The other thing is that the brain has a whole section of it devoted to spatial and 3D analysis; so the 3D images allow the doctors to tap into another area of the brain when they're reading the exams... it allows them to mix up their daily workflow, think about things with a different part of their brain, and to literally get a different perspective on the clinical problem at hand.

my own 3D anatomy

[deathcow]

I went through the stages to donate part of my liver to my infant daughter in 2005. Washington University medical took a full torso cat scan of me, and then gave me a copy of the CD on the way out the door! (I did have to ask for it.)

So I take the CD, and find it has 3D visualization software on it. I ran it and told it to load all the cat scan slices. After it thought about things for a minute, Pow! Full 3D rotatable torso, I could dive in/out up/down whatever. I could change various colors and such to help see embedded structures like biliary tracts of the liver, or the tracts inside the kidneys.

Having been so close to a high end medical operation like a liver transplant for several months, I saw some wicked imaging tools. The ultrasounds they use to monitor my daughters new liver actually colors all the blood flow in blue and red (i.e. venous and arterial, though it is arbitrarily selected I understand) and you can move a trackball around to measure the instantaneous velocity of bloodflow in various veins or arteries in cm/sec with the click of a button.

You can bet that in 20-30 years this stuff is going to be VERY high end and we're going to stand a lot better chance at surviving some bad stuff. "Watch now! The nanobots are just reaching the clogged vessel as we speak, and you can see the bloodflow is already up by 1%, yes look here they have begun to expel the media into the colon!"

Re:my own 3D anatomy

[Chitlenz]

Isn't it cool?

I design the 3D diagnotic interfaces to these systems and I love my job =)

We just got a GE 3T Mri magnet put in at our flagship clinic in Greensboro, and it indeed has a magnet resolution of apparently 90nm (we were trippin on this when they fired it up for the first time...

The color ultrasounds are kind of a pain in the ass to deal with btw, and can get out of manageable control rather quickly. We had an cardio tech generate a dataset on a cardio ultrasound station spanning a 30GB resultset (and no... noone's software would open it ... )

Which package was on the disk? Lots of places use e-film, which is cheap and has a basic 3d modeller function built in. Thats a really cool app, and its been slowly taking over the low end of the PACS market of late.

Recently we've started to see ppl venture slightly into practice automation with stuff like autoinjectors for contrast, but AFAIK noone doing any actual sugery via MR or CT in the US just yet (though I understand the Aussies of all ppl have some early prototypes for working on sick ppl in the outback).

I think my favorite modality flavor is the CT, damn if those color 3d studies arent pretty =) We dropped in arterial tracing recently in our package, and its really fascinating to dig down to the rock bottom of the tech behind some of this kind of equipment. 64-slice CT is such a high resolution that you can now generate an almost photographic quality picture of someones face from the console our of a brain scan. The other day, I noted that on the 64-slice, we not only see everything in the subject, we can get the thread count of the shirt they're wearing =P Oh yeah... if you think CT techs don't make jokes about you naked .. think again =)

--chitlenz

Already done for non-invasive surgery

[mustafap]

A company I used to work for, haptica, developed this for key-hole surgery about 7 years ago

www.haptica.com

What was nice was that they used the Havoc physics engine - the Havoc boys were just round the corner from us in Dublin.
Those havoc boys, they knew how to party!