Microsoft Demos "Deep Zoom" Technology

Barence writes "Yesterday, during a presentation for this year's Imagine Cup, Microsoft's Mark Taylor demonstrated the company's Deep Zoom technology to appreciative gasps of admiration from the computing students present. It's pretty impressive stuff, and you can try 'deep zooming' for yourself at the Hard Rock Memorabilia Site." Unfortunately the demo requires the Silverlight plugin and the story is pretty thin on technical details. I would be interested to see how they captured the image data to that level without massive pixelation.

Comments

Imagine Cup

[suso]

When I read Imagine Cup, I did a double take. Back in the 90s, Impulse, the company that made the popular 3D software Imagine, had a program called "Imagine CUP", which stood for Imagine Constant Upgrade Program. It allowed users to pay for the upgrade to Imagine up front and they could receive all the minor versions inbetween the major versions.

So is this digital zoom stuff like the software that they "download off the internet in CSI: Miami" *Snicker*

Re:Imagine Cup

[dave420]

No, this is the software they use on CSI (NY, at least). You can read a few articles about it starting here.

Re:Imagine Cup

[electromaggot]

...how they captured the image data to that level without massive pixelation. It's not that impressive. You zoom in extensively and it just gets fuzzy. So big deal: they just interpolate the color values between each pixel "point" instead of drawing huge square pixels.

I was much more impressed with PicLens.

Re:Imagine Cup

[mark72005]

it's refreshing to get an inside look microsoft's plans for silverlight and the internet

Re:Imagine Cup

[ozmanjusri]

they just interpolate the color values between each pixel "point" instead of drawing huge square pixels.

It's not a new interpolation algorithm.

It's a live version of the The shift-and-add method or image-stacking technique used by astronomers for decades. It's just that now computer hardware is fast enough do it seamlessly.

Basically, the zoom is made from hundreds of still photographs taken from different vantage points. There was something similar being done with tourist destinations, if I remember correctly.

It's an interesting toy, but the practical applications are limited by the lengthy production process.

Haven't you ever..

[katterjohn]

seen CSI? This technology is so passe.

Re:Haven't you ever..

[dave420]

It's the actual software they use on CSI. Read more here.

Re:Haven't you ever..

[Eudial]

It's the actual software they use on CSI. Read more here. I think he refers to the software in which they miraculously rotate a single two dimensional image to see stuff from other angles, or enhance gritty 320x200 CCTV images into uber-high resolution with no artifacts or fuzziness.

(Might have been in some other forensics/cop show they did that, though.)

Re:Haven't you ever..

[billcopc]

(Blade Runner)

Load photo image.

Enhance.

(zooms in)

Enhance.

(pans around some obstacle)

Enhance.

(pans to the back door, opens the door?, reads license plate from some car a half-block away)

Enhance.

(finds intelligent life in Arkansas)

Re:Haven't you ever..

[MobileTatsu-NJG]

"...or enhance gritty 320x200 CCTV images into uber-high resolution with no artifacts or fuzziness."

Depending on the footage, this is semi-possible. There's software out there that can watch the motion of an object and determine what the sub pixels were. It's not ideal in every scenario (even less likely slow in the case of a blurry face on a security cam...), and it won't be as snazzy as CSI, but it is possible in a general sense. It only works, though if it can get actual motion vectors from the footage.

DeepZoom

[digitalgiblet]

My understanding is that you use different resolutions of the photo. The original photo is obviously the highest res you can have, but you can make successively lower res copies. More or less just bring up a a higher res version when the user clicks.

I saw this demoed at the Atlanta Code Camp back in March. Very cool to watch.

Re:DeepZoom

[Bozzio]

You sir, are right.

Well, at least that's what it looks like when you use their silverlight app.
You can actually spot the "seams" when zooming. Some of them aren't even superimposed correctly, leading me to believe that they are using a series a pictures taken with different cameras, instead of just storing lower resolution copies of the master image.

oh lordy...

[nuzak]

I would be interested to see how they captured the image data to that level without massive pixelation.

You don't ... you don't actually think that the image data came from one photo ... do you?

*slaps forehead*

Re:oh lordy...

[Tarlus]

You don't ... you don't actually think that the image data came from one photo ... do you? Nope.
If you can find them, zoom in on those Beatles bobble heads that the article describes. They're very highly defined. Then zoom out a bit and scroll around to (for example) the surrounding Hard Rock Cafe frame. Wonderfully blurry with respect to the bobble heads.

As you zoom out further, you'll notice how the "container" holding those bobble heads antialiases itself differently from the surrounding different-res artwork.

If you move amongst the different images of guitars and clothes (etc) you'll notice in the lower right that it identifies who the centered item belongs to.

So it appears to me that this is a number of different graphical objects that can be zoomed at relatively different distances at the same time. And it looks like they can be embedded within each other.

Unfortunately?

[bigdanmoody]

Unfortunately the demo requires the Silverlight plugin...

A Microsoft tech demo requires the installation of new Microsoft software to view? Who would have though?

While Silverlight might never be as widely-supported as Flash, I hope that perhaps the competition might force Adobe to do something about the CPU hog that is Flash.

Re:Unfortunately?

[SlashWombat]

Your joking, right?

It's Intel/AMD that are expected to fix this issue, by making huge leaps in processing grunt.

I mean, you can not expect the current crop of programmers to actually write (or even just optimise) fast code.

We now have many touted languages that are actually interpreters, not compilers. The argument from programmers is that today's CPU's are fast enough, and that these "new" languages are much easier to debug. They may as well be written in Visual Basic in my opinion!

Re:Unfortunately?

[dodgedodge]

The beauty of Silverlight is the number of languages you can use. Even Ruby is coming to it.

Silverlight is a browser plugin. It takes all of about 10 seconds to install.

Flash needs to just go away.

Re:Unfortunately?

[Tweenk]

Hm, apparently that should have been "Windows browser plugin".
(the Linux stuff is called Moonlight, and isn't functional yet)

Re:Unfortunately?

[prockcore]

Amazing how it works on firefox in OSX then.

Maybe not CSI

[decowboy]

But how is this different different from google maps (or live maps, or WHATEVER allows you to zoom out a lot)..

Re:Maybe not CSI

[Anonymous Coward]

A major difference is the inclusion high resolution collections, which are not fixed at runtime and can be rearranged programmatically. I know this because that is what we did on the Hard Rock Memorabilia project.

Aside from that, it is another form of a "tile server" application... Just one that happens to be rather easy to use from a development perspective, and one that has been done really well (Zoomify/AJAX-based solutions don't hold a candle to the tile stitching and easing effects built into the MultiScaleImage control, IMHO).

Re:Maybe not CSI

[decowboy]

How about...that depends...on the source material? That Hard Rock site does have some pieces with amazing detail, but the stamp (that the article refers to) has clearly been added to the image, because even normal print doesn't have that detail.

Multiple resolutions

[clarkn0va]

My understanding is that you use different resolutions of the photo. Just speculating here (I don't anticipate installing Silverlight for another 24 years or so), but I think you're on the money. It should work something like Google Earth, where the resolution is improved progressively as you zoom in.

db

SeaDragon

[Dragonshed]

Silverlight's MultiScaleImage control (aka deep zoom) is a version of the SeaDragon renderer. The image format it uses is a custom tree structure that contains pixel details relevant to both it's position in the tree and relative to it's peers. Essentially, it's a hierarchical image with very smooth transitions.

Silverlight: silverlight.net
SeaDragon: http://labs.live.com/seadragon.aspx

Re:SeaDragon

[CODiNE]

Thanks for the link, with it I'm starting to see MS' new strategy to compete with Apple stealing their "cool".

The Seadragon team is currently tuning its DirectX implementation, making the most of the new Windows Media Photo format, and cranking on the Photosynth Technology Preview.

So they're essentially recreating Apple's Quartz + OpenGL + standard image formats with Photosynth + DirectX + WMPF.

Re:SeaDragon

[Dragonshed]

So they're essentially recreating Apple's Quartz + OpenGL + standard image formats with Photosynth + DirectX + WMPF. Simply put, apple does an incredible job visually representing itself, it's technology and providing a user experience that is very hard to match.

That said, I disagree that microsoft is recreating any preexisting technology. You could argue that DirectX is just like OpenGL, but that's likely grossly oversimplified.

Photosynth and Seadragon are demoed here: http://www.youtube.com/watch?v=vHsYnkLnepk

Neither of those are similar to things that apple has done.

layered bitmaps

[Brit_in_the_USA]

The Beatles models and signatures pear to be the highest level of detail unless there are other "Easter eggs". That level of zoom on any surrounding areas is pixelated. They have stacked multiple high res photos at various scales in this particular area.

Re:layered bitmaps

[notaprguy]

Check out the fingerprint on Bo Diddley's guitar. That's pretty cool too.

Yet another Deep Zoom

[MythMoth]

Ian Griffiths implemented a deep zoom for the BBC in their Big Weekend festival. Rather pleasingly they chose to call it the "Big Zoomy Thing" in a nice bit of anti-jargon.

Re:Yet another Deep Zoom

[harlows_monkeys]

That one is also using Silverlight. Except they botched the version check, and so it won't work if you have a later version than the one they coded for. Oops.

Sounds like GigaPan

[higgins]

The folks at CMU have a similar thing:

http://gigapan.org/

It uses a (cheap) commodity digital camera, combined with a smart tripod, good photo stitching software, and a nice Flash UI to give you highly zoomable panoramas. The CMU thing has been around for a while --- over a year at least, plus I'm pretty sure you can get one of the tripod mounts if you participate in the beta and create your own.

Deep Ream

[Foofoobar]

There next product for stealing your checkbook while Windows does a colonoscopy

Re:As seen on Super Troopers!

[Anonymous Coward]

Super Troopers? I hope Decker shoves a unicorn horn up your ass.

Deep Zoom? More like Quick Load.

[Itninja]

I admit the demo is neat and all, but they are not really zooming into the same image. They have just developed a way to quickly load the high resolution image on the fly. Kind of like how Google Maps will deliver a higher res map when you zoom in; but this is happening much faster.

Re:Deep Zoom? More like Quick Load.

[xtracto]

Kind of like how Google Maps will deliver a higher res map when you zoom in; but this is happening much faster.

Kind of like what happens when you use Google earth very close (i.e., in-situ) to where the servers with the data are stored.

Comment removed

[account_deleted]

Comment removed based on user account deletion

WTF?

[cptdondo]

If you're like me and a bunch of very smart students, you can't fail to be impressed. I must be dumb.... Stiching together an image of higher-res photos might be a technical wow, but sorry, I'm not really impressed. This sort of thing I might expect from a college lab, but for a multi-billion dollar company to present this as some sort of earth-shaking innovation?

Re:WTF?

[Dragonshed]

The earth-shaking innovation is in the form including deep zoom as part of a plugin featuring a fast 2d compositor with video decoding and animation support, common RIA application components and controls using a small .NET Runtime, packaged in a 4.3mb download, "installed in 20 seconds or less", and all of it designed to run on multiple platforms.

MS Devs have done some amazing things within their allotted size quotas. /perspective-and-koolaid

Uses gigapixel imagery as source

[prakslash]

There is a bit of a misdirection in articles and other material about Deep Zoom.

Most people go ooh and aah because they (wrongly) assume that it zooms into normal resolution photos .

It doesnt (because as you and I know, it physically can't).

Deep Zoom does NOT perform CSI/CIA-style photo enhancement. If you dig deeper, you will find that what Deep Zoom is intended for is to enable one to focus on a smaller portion of a giga-pixel photograph so you do not have to download the whole photograph.

Think of it like a hierarchical smooth slicing of a large high resolution photograph and only downloading those "planes" and "sections within a plane" that the user is interested in seeing.

Interesting technology but not magic.

Re:Uses gigapixel imagery as source

[dave420]

They actually use SeaDragon (the name of the technology) on CSI, for those sections you're talking about. Obviously they lie about what it's doing, but that's the software you see.

Re:Uses gigapixel imagery as source

[markana]

And exactly how is this different from the wavelet-compressed MrSid format? LizardTech was doing this sort of "download-what-your're-focused-on" multi-resolution zoom *years* ago. Six years ago I could zoom in smoothly and deeply to an area of a multi-GB image, and the plugin would grab only those pixels needed to show that area at that resolution.

So what exactly is new here, except for the use of Silverblight?

Re:Installing Silverlight

[Dragonshed]

You may find reason to install it when it reaches RTM and companies start using it for production work. Right now it's beta1 (beta2 is going to be released sometime in the next couple weeks), and it's mostly for customers/developers wanting to experiment with it.

What becomes of silverlight content, whether it's all eye candy or not, is anyones guess. What I can say is, developing for Silverlight 2 kicks ass.

No free lunch

[icebike]

There is two ways to get this level of zoom to work:

1) have the pixels in the first place
2) having more pixels in the first place.

Anything else is a fundamental violation of the laws of physics and math. You simply can not fake what you don't have without it being exactly that: a fake. There is no storage printing technology which could accomplish this level of zooming, and they carefully do not say that this is actually a continuous zoom of a picture on a stamp.

Deep Zoom works by letting you meld several images in such a way as pretend its one image.

Basically, its a con-job of transitioning several different images, where one is a re-photograph of sub portion of the original.

The implication of the article is that this is all one image containing a nearly infinite level of detail, which it most emphatically is NOT.

The author is probably equally impressed by street corner magic tricks.

Re:No free lunch

[vux984]

Deep Zoom works by letting you meld several images in such a way as pretend its one image.

That's still very useful.

Basically, its a con job of transitioning several different images, where one is a re-photograph of sub portion of the original.

'con job' has needless connotations of an intent to deceive.

The implication of the article is that this is all one image containing a nearly infinite level of detail, which it most emphatically is NOT.

No. The implication of the article is that you can provide this as a user interface, which is very cool. Google Earth isn't interesting because its a 'con job' to let us think we can zoom in and out of a single monster image of the planet. Its interesting because its a natural and convenient UI to use.

And we don't have to download every single pixel of every single higher res image of a tree in Nigeria to have a closeup look at a parking lot in London. Detail is loaded on the fly, as needed, while the user gets a 'seamless' and comparatively low bandwidth experience.

Its not particularly new as an idea. Or even as an implementation. But maybe Microsoft's tools make setting it up substantially easier, and that alone would be a nice bit of progress.

The author is probably equally impressed by street corner magic tricks.

I am impressed by street corner magicians too. Not because I think they're magical, but because I am impressed at their showmanship, sleight of hand, dexterity, and general ability to appear magical.

Re:This is not new...

[iang]

Typical Slashdot... they post a snarky anti-Microsoft comment with a pretentious air of superiority but get the details wrong.

Photosynth is not Deep Zoomm. Photosynth reconstructs 3D models from collections of 2D photos of the scene acquired from different positions and angles. And as far as I know, Photosynth wasn't an acquisition - it was produced by Microsoft Research.

Deep Zoom was an acquisition, but it was the technology formerly known as Seadragon. It's completely unrelated - Deep Zoom/Seadragon is a 2D thing.

And it's an acquisition, but so what? Ooh, naughty Microsoft - how dare they take exciting technology developed by a startup and put it in the hands of millions of users? Shocking! Clearly it they should have left it to sink in obscurity.

I don't get it

[Mike1024]

Don't we already have the ability to process multi-resolution images in, for example, Google Maps? You know, zooming in and out images with large total resolution?

It would be impressive if the photo they demonstrated on was anything but a photoshop, but given that the 428x134 signature is 52x11 in the 350x237 statuette picture which is 29x26 in the 428x350 hard rock picture which is 87x87 in the 428x399 stamp picture, for the stamp to be real would require a 33 gigapixel stamp (which, at 1 inch square, would be printed at 33,000,000,000 DPI).

To me zooming in and displaying a different image isn't really as exciting at the article author makes it sound? Maybe I'm missing something because the journalist sounds pretty damn excited about it.

Re:I don't get it

[prockcore]

what you're missing is that seadragon constructs these things analytically from a collection of photos.

Basically I can run around taking random photos some zoomed in, some not and seadragon will automatically stitch it all together.

The image is only 21K

[statemachine]

But the viewer is 126G.

Re:they stole it from blade runner and csi

[Drathos]

I'm far more interested in the tech that allowed them to change the camera angle of the photographs in Bladerunner. When's that coming?

Re:Installing Silverlight

[Bozzio]

Sibelius is a popular music notation software package.
It has become pretty popular in the past 5ish years since its learning curve isn't nearly as steep as its main competitor Finale.

People criticize Sibelius since, typically (at least for the versions I've used), its output isn't exactly professional quality.
It is, however, a great tool for music students.

Back in the day, Finale was the only option for amateur composers to produce professional looking manuscripts.
I'm not sure how far Sibelius has come in the last few years, so things might have changed.

Crashed FF 3.0 on my Mac

[oborseth]

It crashed Firefox 3.0 on my Mac Book after installing the plug in and viewing the demo.

Re:Crashed FF 3.0 on my Mac

[plover]

It consistently crashes Firefox 3.0 RC2 on Vista 64, too, after installing the Silverlight plug-in. I disabled the plug-in and no crashes.

Of course the Silverlight and the zooming works as advertised in IE 7.0.6

Deep Ripoff

[jdb2]

This is a lame "embraced" and "extended" version of an old demo effect that was first demonstrated in the early 90's, if not earlier. Obviously the entire "zoomed-out" image is not stored. The "zoomed-in" images are stored, but to make the effect work a series of intermediate images has to be stored between each "zoom stage". For example, in one implementation, when "zooming" through the intermediate stage between a "larger" and "smaller" image, for each series of frames an "outside" or "boundary" image is stored in full resolution and that image is zoomed ( and clipped against the view port boundaries ) until it is outside the view port while at the same time the "internal" image is enlarged until it fills the viewport and then the process is repeated again with the "internal image" now consisting of the next "boundary" image surrounding another "internal image".

Go to Pouet and you'll find many demonstrations of this effect.

jdb2

This has been done. Better.

[Animats]

See Charles and Ray Eames' Powers of Ten. Now that's a zoom.

As for doing it in real time, Keyhole (bought by Google and renamed Google Earth) was doing this on PCs five years ago. Any decent GPU can do this today, and you can download Google Earth to see it.

I saw one of the first systems able to do this in real time about 25 years ago. It was inside a classified tank at a major aerospace firm, and required a rack of special-purpose hardware. The user interface was beautifully simple - a big trackball (for pan), a lever (for zoom), and a knob (for rotation).

Even Microsoft's little film isn't original. That technique has been used a few times in commercials.

So Silverlight doing this isn't exactly a big "wow" development.

No, but I've seen GigaPan

[awtbfb]

I don't watch much TV, but the functionality is awfully similar to GigaPan.

Re:Installing Silverlight

[lilfields]

I would like to see Silverlight content able to be indexed on search engines...that is one HUGE disadvantage that Flash has...it would really help push this product with web developers. Otherwise you have to create two versions of the site, one for search engines and one for users...though I think 100% flash sites are stupid...but people use them, and like I said it could help Microsoft boost it's market share quite substantially. Silverlight does look pretty impressive...

Prior art

[joeslugg]

As I'm reading the descriptions and seeing it on YouTube, I'm thinking I've SEEN something like this before.
And I finally remembered; Jef Raskin's "Humane Interface".
Zooming demo from several years ago that runs in Flash here.

Quite similar, IMHO. Hmm?

Re:Installing Silverlight

[billcopc]

Sibelius Scorch, which is what Ucklak was probably referring to, is a browser plugin to display music notation. It's basically a DRM-encumbered midi/pdf hybrid. It's used almost on almost all sites selling sheet music, because it can restrict printing and saving.

You can still take screenshots and stitch them back together, but that's obviously a pain in the ass.

The plugin itself tends to be unreliable, it often bombs without delivering the goods, while still counting as a print/view and thus often locking you out of the product you paid for, which then requires much dicking about with the site staff to get it reset.

Re:Installing Silverlight

[Dragonshed]

Similar hurdles exist for indexing silverlight content as they exist with flash. Silverlight is mainly for media and data/info visualization.

It's technically possible to index silverlight 1 content, because it's content is "loose Xaml files", which means the site has xml files alongside html/js/etc, that is rendered by the silverlight 1 engine.

Silverlight 2 has the same capabilities, but noone will use them, because using C# for application/interaction logic is way more productive than using Javascript. Silverlight 2 sites using C# have the following structure

SomeSite.XAP (zip file containing all code and assets)
- AppManifest.xml
- ApplicationCode.dll (.NET Assembly containing Entrypoint and embedded assets)
- SomeResources/ (compressed folder)
- SomeResources/SomeImage.jpg (...)

AdditionalContent.XAP (supplemental resources and code)
- AppManifest.xml
- SupplementalCode.dll

This makes silverlight 2 apps and content updates really easy to, but are a barrier to extract information.

In both cases the information gained isn't nearly as useful as textual html content, and completely different heuristics would be necessary to analyze the importance of one unit of textual content vs another. Indeed, nearly all the visual cues (The relative position, color, highlights, animations, and reactions to the user) would likely be lost in the process. Perhaps the search engine that can index flash and silverlight content is one that analyzes both visual and textual content.

Re:This is not new...

[reg]

Photosynth was acquired from the University of Washington... The original was in Java and called photo tourism. http://phototour.cs.washington.edu/

Re:Installing Silverlight

[afidel]

Why not use Rosegarden and Lilypond, fairly easy to use and great professional quality output. Awesome for students since it's you know free =)

Re:Installing Silverlight

[Bozzio]

The main reason is that most Music School "computer admins" won't want to fudge around with Linux.
Yes, I know, Lilypond works in Windows.
You try teaching 120 computer illiterate musicians how to use it :)

Sibelius is popular because it's relatively easy, and it runs on Windows (so it's relatively easy to install/manage for its user base).

Pre-cursor video and related GUI designs

[FredThompson]

You can watch a Seadragon presentation from TED at http://www.ted.com/index.php/talks/view/id/129
focus-plus context screens are similar http://www.patrickbaudisch.com/projects/focuspluscontextscreens/index.html

Re:Installing Silverlight

[benwaggoner]

It's technically possible to index silverlight 1 content, because it's content is "loose Xaml files", which means the site has xml files alongside html/js/etc, that is rendered by the silverlight 1 engine.

Silverlight 2 has the same capabilities, but noone will use them, because using C# for application/interaction logic is way more productive than using Javascript. Silverlight 2 sites using C# have the following structure Well, the XAML (markup GUI, and what's probably interesting to index) and code are still in different files. A developer can choose to put the XAML outside the .xap and the code in it. And if it's an unencrypted .xap (as most would be), it'd be easy enough for a search engine to look inside the .xap to find the XAML.

Searchability of XAML is definitely something we're working on, and have guidelines for how to develop apps that are easily searched and index.

Silverlight 2 Beta 2 released today

[benwaggoner]

Silverlight 2 Beta was actually released today.

Runtime and SDK downloads and lots of other info about it here http://silverlight.net/GetStarted/