3-D Software for 'Virtual Surgery'

Roland Piquepaille writes "Computer scientists at Brigham Young University (BYU) have developed a new software tool to perform 'virtual surgery'. This tool, dubbed 'Live Surface,' will allow surgeons to visualize in 3-D any part of a patient's anatomy with just a few clicks of a mouse. Similar software already exists, but according to the Deseret Morning News, Live Surface is interactive and fast. This software can be used for better diagnosis by physicians, but it might even suppress the need for some exploratory surgeries. The researchers add that Live Surface might even been used for special-effects in movies or games by extracting an actor's performance from a video clip."

Comments

A pioneer

[Lindsay Lohan]

I've been visualizing human anatomy in 3-D for many years.

Re:A pioneer

[Anonymous Coward]

I've been visualizing your anatomy, that is until you ruined it w/your recent nipple slip.

Re:A pioneer

[Lindsay Lohan]

Sorry to disappoint, but I am in reality a pathetic, overweight middle-aged man, a hairy stretchmarked gut resting my greasy keyboard as I write this, chewing Nicorette and choking down a few hotpockets.

But if you want, we can still trade emails?

Re:A pioneer

[Steve Embalmer]

that honestly is the funniest thing I've ever seen on /.

Re:A pioneer

[andrewman327]

Dad?!?

Re:A pioneer

[thrillseeker]

Mom?

Yeah, its us.

[engagebot]

Yeah, this already does exist. Because we make it. www.meti.com

We have a laproscopic surgery simulator for a mere $40k that will totally blow your mind. You can learn to stitch, tie knots, remove gall bladders, the works.

Re:Yeah, its us.

[jpardey]

This is a Ronald article. Of course he is going to hype up whatever he is talking about.

Re:Yeah, its us.

[gramji]

I have played with your hardware/software. Your rep stopped by our lab for a demo. I must say I was impressed but a lot of work still needs to be done especially with using models other than mass-spring so the organs don't feel like water filled balloons and response among others. The work suggested here is definitely needed though I don't know how complete/advanced it is.

Re:Yeah, its us.

[Anonymous Coward]

His name is "Roland", not "Ronald". "Roland Pickypail". Get it right, please.

So far today...

[Apocalypse111]

Ok, so today we've had stories on how to do 3D Virtual Reconstructions of places or environments and now 3D visualization of people's internals. All we're missing is an article on holograms.

I want my freakin' holodeck!

Re:So far today...

[geekoid]

You have some wierd desire for 'holo' people to take over your ship when it runs amok?
It seems to be inevitable.

So we'll soon have photoshop for the body.

[w33t]

I can see Doctors of the future using the magenetic-lasso to extract tumors more easily.

We can only imagine, however, what the clone tool will be used for.

Re:So we'll soon have photoshop for the body.

[stunt_penguin]

Superfluous third nipples. The fashion statement of 2019.

But will it run Linux?

[neonprimetime]

The software, Live Surface

Somebody research it for me. It's academic, so it has potential???

Integrate into Doom 3

[harrkev]

Can this be integrated into Doom 3 for more realistic body parts flying everywhere?

If only...

[Trouvist]

If only they truly had the technology they claim, they would have quickly been bought up by the GE's or Microsoft's of today. Does anyone here have any idea of the worth of an alogrithm that would automatically segment the entire human body for virtual exploratory surgery within reasonable timeframes?

Re:If only...

[grammar fascist]

Does anyone here have any idea of the worth of an alogrithm that would automatically segment the entire human body for virtual exploratory surgery within reasonable timeframes?

Ha ha, conspiracy theory. Great stuff.

They have the technology. I saw a demo of it in class and had the segmentation algorithm explained to me. Yes, I'm a graduate student at BYU.

Nice try baiting the moderators, though.

well now,

[to_kallon]

The researchers add that Live Surface might even been used for special-effects in movies or games by extracting an actor's performance from a video clip.

sure sounds like progress to me...

Machine shop for the body?

[The Great Pretender]

Here we go: You scan an MRI, feed it in to the computer. Some Dr. on his sail-boat looks at the MRI identifies the area to be removed, and does a virtual surgery. The virtual surgery goes into the computer. The patient gets prepped, goes into surgery, a robot surgen following the 'virtual surgery' removes the offending piece.

It all sounds so nice and efficient, but I can see so many things were this could go horribly wrong. I for one will be sticking with the over-worked, stim-taking resident who will be standing by my body. I don't feel comfortable with the medical industry moving in the same direction as the car manufacturing industry.

Re:Machine shop for the body?

[denis-The-menace]

Today in 2015, a mix-up at the DC General Hospital caused a Presidential hopeful to get a Sex-change operation instead of a triple-bypass heart surgery. The makers of Robo-surgeon disavow any responsibility since it is the Hospital's underpaid orderlies that roll in the patients in to the wrong room. The now Congress-woman was unavailable for comment. Doctors give the patient a few days to live due to the excitement of not getting enough of herself.

Re:Machine shop for the body?

[shawn(at)fsu]

I'm confused if your trying to be humorous or not. I'm not good with humor....
In any case I don't see how it can make things much worse. It's not like people have gone in to have the left leg amputaed and woke up with the right one gone. See one story here

Re:Machine shop for the body?

[The Great Pretender]

That's the scarey thing, is this humor?

Re:Machine shop for the body?

[aardvarkjoe]

It all sounds so nice and efficient, but I can see so many things were this could go horribly wrong. I for one will be sticking with the over-worked, stim-taking resident who will be standing by my body. I don't feel comfortable with the medical industry moving in the same direction as the car manufacturing industry.

That's one way to look at it. But on the other hand, I doubt that the car manufacturing industry makes mistakes half as often as the medical industry does.

BYU press release link with more media and info

[Pvt_Waldo]

There is a press release right here from BYU that has links to various videos and other media. Can't seem to find any papers or articles about the process, though I noticed it's being patented so there may not be a lot available (?)

Re:BYU press release link with more media and info

[grammar fascist]

Can't seem to find any papers or articles about the process, though I noticed it's being patented so there may not be a lot available (?)

Having taken a vision class from Dr. Barrett (CS 750 at BYU), I can fill in some details. I might be able to dig up the paper later. I think you can find it in the latest SIGGRAPH proceedings - dunno if Citeseer has indexed it yet.

It's a segmentation algorithm that works well and fast in 3D images. It uses a graph-cut algorithm to classify voxels as inside or outside whatever you're trying to isolate. You (the doctor) lay down "seed" voxels with a mouse, clicky-clicky, and a few seconds later, the algorithm has isolated the structure. For example, say you want to isolate bone. Hold down the mouse button and move it over the bone. Hold down the other and move it over non-bone. If the algorithm makes a mistake, make some more seed voxels.

This is nothing new so far - the CV folks have been segmenting with graph-cut for ages. The problem is that it's very, very slow - minutes for a single segmentation. Barrett and Armstrong have developed a hierarchical version of the algorithm that uses watershed regions to presegment, and merges them as it runs. Doing graph-cut on large regions is a lot faster than doing it on single voxels. Their stuff is the first interactive speed, seeded 3D segmentation algorithm that produces quality results.

I saw the demo in class. It was really rather impressive, if you're familiar with the subject area.

Re:BYU press release link with more media and info

[Chitlenz]

3D is becoming very much of both diagnostics and surgery, but lest we forget this technology is ONLY as good as the doctors that use it. That, and voxel data seems like a rather incomplete dataset compared to DICOM out from say a CT modality. When we code diagnostic systems, we make a point to include the new in-progress review specs for CT and MR that are going to be the foundation of future surgical platforms to come (DICOM btw, is the medical image standard that was developed to allow for things like 3d volume sets to have uniform storage requirements even though they contain more than flat data). In progress review allows the doctor to see you on the CT skid remotely and in realtime. CT autogenerates the matter densities (Houndsfeld units) as part of the 'stack' or volume set (from this you can seperate bone and muscle tissue, etc. in the 3d volume).

We've been seting up to do this commercially for 8 years now, and have several commercial patents in the area at this point, but I've been waiting to see who could successfully integrate surgical tools into the modalities themselves. The problem I have with this guy's approach is that its hardly revolutionary (see Vitrea or Terarecon for the true state of the art in this field), and while its a good start I don't know that its all that newsworthy? As for the fast rendering thing, ummm we discovered an AMAZING thing in the corse of our dev cycle......

You see the average radiologist in the US makes a LOT of money (somewhere in the 600k range), but many hospitals to save money give them the 300$ Dell workstation of the month deal. When you use real machines (we have a deal with VoodooPC) you can do neat things like not only render the 3d volume set quickly, but also use it for things like an image map of the flat study sequences (cross-sectional). In fact, you'd probably be quite surprised at what a real 3d diagnostic workstation can do.

So yeah, this guy crossed Dicom and DirectX, welcome to 3 years ago.

Chitlenz

Special effects...

[Anonymous Coward]

"...researchers add that Live Surface might even been used for special-effects in movies or games by extracting an actor's performance from a video clip..."

I suppose if a doctor accidentally pops a virtual blood vessel, it would show off some particle system effects.

Does it have Clippy?

[Rob T Firefly]

Hey, it looks like you're perfoming a coronary artery bypass operation!

Would you like to:

• Obtain a suitable transplant from the greater saphenous vein?
• Overcharge the patient's laughable insurance?
• Get in a quick few strokes on the back nine before the operation?
• Play golf instead?

too easy

[Anonymous Coward]

This tool, dubbed 'Live Surface,' will allow surgeons to visualize in 3-D any part of a patient's anatomy with just a few clicks of a mouse.

Sometimes the gags just write themselves

BSOD

[TheAngryMob]

A system crash would give new meaning to "Blue Screen of Death."

Re:BSOD

[__aaclcg7560]

That's why they would use a Mac instead. Of course, the doctor might find it ignoring if the Mac keeps saying, "Uh, no."

Re:BSOD

[Anonymous Coward]

Not really. It has been involved in airplane crashes (never count on a laptop to give you a map when you are IFR; likewise, avoid Airbus who make heavy use of Windows there), it has been responsible for making the USN Ronald Regan run in circles (real life recapitulating real life), and even has crash a nuclear power plant (never allow an outside windows box into a plant).

So, no, it is just the same old.

Been there, done that....

[Zarian]

I've been doing this for a while now. http://www.atlus.com/trauma_center/ ;)

Re:Been there, done that....

[MS-06FZ]

See also: The Touch

CS Webpage and Videos

[Anonymous Coward]

http://cs.byu.edu/feature/?section=display&id=168

But does it run on Linux?

[Harmonious Botch]

I really, really hope so. I'd hate to have my last sight, as the gas takes effect, be the surgeon wheeling in a PC with Windows.

Grits anybody?

[drenehtsral]

Muhahahahaha!

old ideas, new clothes

[noneme]

Virtural surgery is not a new concept...I remember being in Junior High School and having the option in science class to disect a frog or to use a program on a 386 to move around .bmp's of a frog's anatomy.

The 3D model is an interesting way to put the MRI / CAT data on a computer screen (and far better than the .bmp's of a frog's organs) but what advantage (besides eye-candy) does this offer over looking at the raw MRI or CAT results?

One thing that could make this a great learning tool is an interesting interface that would help one practice a surgery with something more than a mouse or touch screen. Nintendo and Altus have already created a toy that does this, a far more intricate and realisitic version could be of use: http://ds.ign.com/objects/695/695152.html

Re:old ideas, new clothes

[VoidEngineer]

The 3D model is an interesting way to put the MRI / CAT data on a computer screen (and far better than the .bmp's of a frog's organs) but what advantage (besides eye-candy) does this offer over looking at the raw MRI or CAT results?

The answer is pretty simple. Doctors have to deal with information overload, and 3D models are an effective way of managing huge amounts of data. Consider: A typical MRI exam contains 60 to 90 slices. Looking at a single 3D image is much more efficient than looking through 60 to 90 2D slices and trying to form a mental 3D image in your imagination.

I work as a PACS Administrator for a Department of Radiology at a community hospital in NYC (PACS stands for 'Picture Archiving and Communication Systems'... I'm the network, systems, database, and applications administrator all roled into one). We have two 16 slice CT scanners and a 1.5 tesla MRI scanner... we average about 2,000 CT and MRI cases per month, and each one of those cases requires about 20 to 30 minutes of time for a radiologist to read. When you do the math, we have 4 to 6 full time radiologists on staff (at $350,000 yearly salary each), meaning that we're spending about $1.5M to $2M just on salary to get through those images.

Storage wise, each CT or MRI slice is generally 512x512 pixels, and works out to be something like 60 to 100kb, if I recall. So, a typical MRI series tends to be about 5MB or so. Different scanners average different sizes depending on how you want the images to spit out, but each series tends to range between 1 and 10 megabytes. The thing is, you might have up to a dozen series taken during your exam (those of you who have layed in an MRI scanner for an hour know what I'm talking about). A rule of thumb that I use is that an hour of our MRI scanner's time is roughly equal to 100MB of data. Each day, we generate an average of 1GB of new clinical data from our MRI scanner that has to be read by a radiologist.

So, consider it this way... each day, we have 3GB of data generated (MRI scanner and 2 CT scanners), at 365 days a year, equals a cool terabyte of clinical data each year that has to be read by the doctors. At $2M per year in doctor's salaries, we're spending approximately $1.50 to $2.00 per MB to get the cases read.

Anyhow... thought you might be interested to know some of the economics involved in reading MRI and CT results. The 3D models takes approximately 10 minutes to read, rather than 20 to 30 minutes to sort through the 2D images. The net result is that doctors who are comfortable with the 3D images can sift through 2x to 3x more cases. And because the doctors are paid on a per-read basis, that's the difference between making $350,000 per year and $700,000 per year.

All I can tell you is that when you start dealing with terabytes of MRI and CT images, they start all looking the same after awhile... and the doctors use any tricks they can to manage the data, simplify it, and process it. It's a clasic case of information overload. The 3D images help to simplify the 60 to 90 slices and to organize it into a coherent 3D object.

The other thing is that the brain has a whole section of it devoted to spatial and 3D analysis; so the 3D images allow the doctors to tap into another area of the brain when they're reading the exams... it allows them to mix up their daily workflow, think about things with a different part of their brain, and to literally get a different perspective on the clinical problem at hand.

And the Biggest Investors are?

[TheRequiem13]

The porn industry.

Re:And the Biggest Investors are?

[Anonymous Coward]

Of course, the Mormon Church (BYU) has long been known to be in business with the Pron Industry... In fact rumor has it, that the new trend in adult video is going to be virtual surgery followed by a hot steamy sex scene???

You DOLT!

my own 3D anatomy

[deathcow]

I went through the stages to donate part of my liver to my infant daughter in 2005. Washington University medical took a full torso cat scan of me, and then gave me a copy of the CD on the way out the door! (I did have to ask for it.)

So I take the CD, and find it has 3D visualization software on it. I ran it and told it to load all the cat scan slices. After it thought about things for a minute, Pow! Full 3D rotatable torso, I could dive in/out up/down whatever. I could change various colors and such to help see embedded structures like biliary tracts of the liver, or the tracts inside the kidneys.

Having been so close to a high end medical operation like a liver transplant for several months, I saw some wicked imaging tools. The ultrasounds they use to monitor my daughters new liver actually colors all the blood flow in blue and red (i.e. venous and arterial, though it is arbitrarily selected I understand) and you can move a trackball around to measure the instantaneous velocity of bloodflow in various veins or arteries in cm/sec with the click of a button.

You can bet that in 20-30 years this stuff is going to be VERY high end and we're going to stand a lot better chance at surviving some bad stuff. "Watch now! The nanobots are just reaching the clogged vessel as we speak, and you can see the bloodflow is already up by 1%, yes look here they have begun to expel the media into the colon!"

Re:my own 3D anatomy

[Anonymous Coward]

Dont leave us hanging, what happened to your daughter????????

Re:my own 3D anatomy

[Chitlenz]

Isn't it cool?

I design the 3D diagnotic interfaces to these systems and I love my job =)

We just got a GE 3T Mri magnet put in at our flagship clinic in Greensboro, and it indeed has a magnet resolution of apparently 90nm (we were trippin on this when they fired it up for the first time...

The color ultrasounds are kind of a pain in the ass to deal with btw, and can get out of manageable control rather quickly. We had an cardio tech generate a dataset on a cardio ultrasound station spanning a 30GB resultset (and no... noone's software would open it ... )

Which package was on the disk? Lots of places use e-film, which is cheap and has a basic 3d modeller function built in. Thats a really cool app, and its been slowly taking over the low end of the PACS market of late.

Recently we've started to see ppl venture slightly into practice automation with stuff like autoinjectors for contrast, but AFAIK noone doing any actual sugery via MR or CT in the US just yet (though I understand the Aussies of all ppl have some early prototypes for working on sick ppl in the outback).

I think my favorite modality flavor is the CT, damn if those color 3d studies arent pretty =) We dropped in arterial tracing recently in our package, and its really fascinating to dig down to the rock bottom of the tech behind some of this kind of equipment. 64-slice CT is such a high resolution that you can now generate an almost photographic quality picture of someones face from the console our of a brain scan. The other day, I noted that on the 64-slice, we not only see everything in the subject, we can get the thread count of the shirt they're wearing =P Oh yeah... if you think CT techs don't make jokes about you naked .. think again =)

--chitlenz

Before the BYU jokes...

[ChePibe]

Well, before the Mormon/BYU jokes start coming out or someone makes a strange accusation about Mormons and the medical sciences regarding this article, it should be pointed out that Russel M. Nelson, a member of the second highest governing body in the Mormon church, is a noteworthy heart surgeon, who has served as chairman of the Council on Cardiovascular Surgery for the American Heart Association and received a Citation for International Service from that same organization.

Re:Before the BYU jokes...

[Anonymous Coward]

Well, before the Mormon/BYU jokes start coming out or someone makes a strange accusation about Mormons and the medical sciences regarding this article,

Now there's a self-fulfilling prophecy if e'er I saw one.

Already done for non-invasive surgery

[mustafap]

A company I used to work for, haptica, developed this for key-hole surgery about 7 years ago

www.haptica.com

What was nice was that they used the Havoc physics engine - the Havoc boys were just round the corner from us in Dublin.
Those havoc boys, they knew how to party!

2 things

[geekoid]

1) are thise images online?

2) How is your daughter?

Re:2 things

[deathcow]

1) are these images online?

Probably not on the Internet for general access, one might hope. I don't doubt that they are online for Washington University doctors to access. I had ~just~ gotten a chest X ray as well and found a copy of it on the CD as well, so I think they are pure digital with instant access in that regard.

Blood lab monitoring was another nice aspect of being at Childrens Hospital. When you are having a liver transplant, your blood is checked over and over for liver function levels, blood clotting levels, arterial blood gasses, etc. MANY tests throughout the day. These lab numbers were all available on the Windows computer in the ICU and we could freely graph them on screen, compare graphs against each other, etc. and keep watch with everyone else as to what was going on.

2) How is my daughter?

My daughter rcvd liver transplant at age 8 months (due to biliary atresia) and she is now 2, bright happy and beautiful. That said, she is a transplant patient, so she is immunosuppressed and has difficulties associated with that. Right now the challenge is food allergies, I assume as a resut of her immunosuppression therapy.

Re:2 things

[Schraegstrichpunkt]

How long will she have to be immunosuppressed?

Not too far from the truth...

[fahrbot-bot]

You scan an MRI, feed it in to the computer. Some Dr. on his sail-boat looks at the MRI identifies the area to be removed, and does a virtual surgery.

When my wife had a brain biopsy for her GBM, they did indeed get an MRI image and feed it into a computer in the operating room. The computer generated a 3-D image using the scan and aligned it to her head. The image was accurate to 0.06mm (I believe) and could even generate views "looking through the needle" so the Dr. (actually in the room) could avoid blood vessels and such. The system also tracked the surgical instruments within a field and displayed them on/in the image.

It's called a Stereotaxic Biopsy.

Won't somebody think of the CHILDREN!?!?

[Anonymous Coward]

Terrorists could use this software to learn how to do do appendectomies and coronary bypass grafts! It's a training tool for the enemies of freedom!!

Is this the same BYU ?

[permaculture]

Is this the same BYU where Prof Steven Jones has written a white paper about 911?

http://www.physics.byu.edu/research/energy/htm7.ht ml

"ABSTRACT
In this paper, I call for a serious investigation of the hypothesis that WTC 7 and the Twin Towers were brought down, not just by impact damage and fires, but through the use of pre-positioned cutter-charges. I consider the official FEMA, NIST, and 9-11 Commission reports that fires plus impact damage alone caused complete collapses of all three buildings. And I present evidence for the controlled-demolition hypothesis, which is suggested by the available data, testable and falsifiable, and yet has not been analyzed in any of the reports funded by the US government."

extremely simple interface

[peter303]

I saw this software demoed at SIGGRAPH this week. I was impressive by the extremely simple user interface.
It only has three basic commands, unlikes the dozens to hundreds I see on comparable medical-CAD software.
The commands are rotate view, add a chuck here, delete a chuck here. There sophisticated segmentation technques
mostly guess your intention right, i.e. do you want to display this bone or muscle, but you have to do some final adjustments.

Virtual Surgery = Operation by Milton Bradley

[Jakuta]

Will the patients nose light up red when the proceedure goes wrong?

Another Silo of Innovation

[tr0tt3r]

From the article:
BYU has applied for a patent on Live Surface, and Adobe will have nonexclusive licensing rights to the product, Barrett said.

This software is supposed to do great things, but now if you want to do these great things you have to either use a patent or buy proprietary software from Adobe. I wonder if the graduate student who helped write this program had government sponsorship of any kind.

This kind of innovation silo is immoral. In my opinion far more immoral than proprietary software in general. Take a look at www.gplmedicine.org for fully developed version of this article. No one should be celebrating this or anything like this.

Fred Trotter