Slashdot Mirror
Upscaling Retro 8-Bit Pixel Art To Vector Graphics
An anonymous reader writes "Two researchers — Johannes Kopf from Microsoft, and Dani Lischinski from The Hebrew University — have successfully created an algorithm that depixelizes and upscales low-resolution 8-bit 'pixel art' into lush vector graphics. The algorithm identifies pixel-level details (original paper — PDF) to accurately shade the new image — but more importantly, the algorithm can create smooth, curved contour lines from only-connected-on-the-diagonal single pixels. At long last, we might be able to play Super Mario Bros. on a big screen without stretching our beloved plumber's pixels to breaking point. You really must look at the sample images." Scroll down in the paper to see how their technique stacks up against some others, including Adobe's Live Trace.
I've always thought that this would be possible. Now we just need to create a translator that will let us play our retro games with vector graphics.
Sig: I stole this sig.
Because we haven't strip-mined the past enough. I'm impressed by the results but weary of its implications.
http://johanneskopf.de.nyud.net/publications/pixelart/paper/pixel.pdf
Don't tell me people have never heard of the coral cache
EvilHom3r over on Reddit seems to have mirrored the paper (as images) here.
My verdict: Yeah, it looks all nice and smooth, but with all upscaling, it's basically interpolating data. The original just didn't have that much detail, so you can only get so much out of it. Sometimes it works, sometimes it doesn't.
(Oh, and it makes all text look pretty bad. Kinda Comic Sans-y, if I can say that without invoking instant hate.)
Since I can't read the PDF either, I'm forced to infer from the dolphin picture that it's raining cheese and the dalai lama is coming over for lunch.
WTF original sketches? whose ass did you pull that from?
SUPER SAI is similar IIRC
Snowden and Manning are heroes.
No, not really. From the paper:
Various commercial tools, such as Adobe Live Trace [Adobe, Inc.2010] and Vector Magic [Vector Magic, Inc. 2010], perform automatic vectorization of raster images. The exact nature of the underlying algorithms is not disclosed, however, they generally do not perform well on pixel art images, as is evidenced by the comparisons in this paper and in our supplementary material.
EvilHom3r over on Reddit seems to have mirrored the paper (as images) here.
My verdict: Yeah, it looks all nice and smooth, but with all upscaling, it's basically interpolating data. The original just didn't have that much detail, so you can only get so much out of it. Sometimes it works, sometimes it doesn't.
(Oh, and it makes all text look pretty bad. Kinda Comic Sans-y, if I can say that without invoking instant hate.)
I know what you mean but it's pretty impressive nonetheless. The flaw I found interesting was in the keyboard image. I'm sure the intention here is square keys, but their algorithm made all the keys round. Some things in a pixelated image should not be smoothed but without human context that's a very hard call to make.
These posts express my own personal views, not those of my employer
RTFP - they list the various existing upscaling algorithms, and why they're not perfect:
Several later algorithms are based on the same idea, but use more sophisticated logic to determine the colors of the 2x2 block. The best known ones are Eagle [Unknown 1997], 2xSaI [Ohannessian 1999], and Scale2x [Mazzoleni 2001], which use larger causal neighborhoods and blend colors. Several slightly different implementations exist under different names, such as SuperEagle and Super2xSaI. An inherent limitation of all these algorithms is that they only allow upscaling by a factor of two. Larger magnication canbe achieved by applying the algorithm multiple times, each time doubling the resolution. This strategy, however, signicantly reduces quality at larger upscaling factors, because the methods assume non-antialiased input, while producing antialiased output.
The latest and most sophisticated evolution of this type of algorithms is the hqx family [Stepin 2003]. This algorithm examines 3x3 pixel blocks at a time and compares the center pixel to its 8 neighbors. Each neighbor is classied as being either similar or dissimilar in color, which leads to 256 possible combinations. The algorithm uses a lookup table to apply a custom interpolation scheme for each combination. This enables it to produce various shapes, such as sharp corners, etc. The quality of the results is high. However, due to its strictly local nature, the algorithm cannot resolve certain ambiguous patterns and is still prone to produce staircasing artifacts. Lookup tables exist only for 2, 3, and 4 magnication factors.
whereas what they did here is:
Our goal in this work is to convert pixel art images to a resolution-independent vector representation, where regions with smoothly
varying shading are crisply separated by piecewise-smooth contour curves.
Seriously, just look at the whale image linked from TFS.
I'm going to steal it. Next time I see something I'm both apprehensive, and tired of, I shall be /we?ary/ of it.
IIRC, Inkscape had a nice raster to vector function that I had good luck with. I seem to remember it allowed for a bit of tweaking too, to achieve best results.
Last time I checked, these filters you speak of in Zsnes and other emulators simply extrapolate the pixels without paying any special attention to shapes and colors. TFA picture looks quite a bit more advanced.
Yes, this is news. Most scientific and technological advancement is iterative improvement. This paper describes an improvement in the state-of-the-art for vectorizing pixel art (which is not the same as pixel-smoothing, though a vectorized image is usually re-rasterized for display).
Seriously, read the paper. The algorithm described will require some basic understanding of splines and graph algorithms, but it's surprisingly accessible for a graphics/vision paper. Also, shiny pictures.
Those are all post-processing up-scaling effects. But they do have limitations in that even the pixels themselves can still be made out. Obfuscating them only goes so far. But with this new technique, they're pure vectors. They will scale infinitely with splines to match your screen resolution, much like fonts.
Life is not for the lazy.
Mirror: http://easy-iphone-unlocking.com/download/pixel.pdf
I play in Znes with the Hq3x filter. It might be good, but it's nowhere near the quality of what these people have accomplished. Znes having this technique implemented would be amazing,
Grammar nazis are to this community what excrements are to gold.
It looked like it was on the verge of a Slashdotting, so I CORALized it.
http://johanneskopf.de.nyud.net/publications/pixelart/paper/pixel.pdf
Very interesting, and quite effective.
Willie...
You can't understand the difference between SCALING and VECTORIZING? Really? Scaling algorithms increase the number of pixels, but you're fundamentally still dealing with a raster image. A vectorized drawing is a whole different beast.
That said, I would like to compare the final results against a best of breed scalar like hq4x for the same final output resolution.
Which is exactly what TFPDF says. Your rating : 10/10 on reading skills, 1/10 on creativity ...
"DRM is like the Ford Pinto: it's a smooth ride, right up the point at which it explodes and ruins your day."-C.Doctorow
I redraw bitmaps as vector art as a little side business, and I have to say this is the best "livetrace" automated program I've come across. Shameless plug: Vectify.com. Still no substitute for hand drawn (rounded corners that should be sharp; variable, wobbly line widths; but very good for an automated process). I've scanned through the PDF paper, and it looks like it's too CPU intensive to be used in retro games in realtime: from page 6, "There are many avenues for future work. Obviously, it would be nice to optimize the performance of the algorithm so that it can be applied in real-time in an emulator." As someone who used to play old games via dosbox and SNES9x, having something like this as a selectable filter option one day would be welcome. Now that I think about it, this kind of thing would be useful for app developers who want to scale up low-res artwork to work on higher res displays (like going from the original iPhone to the iPad). At least, it could be used as a starting point to further manual refinement.
Corel PhotoTRACE was out in 1993, and produced results comparable to the Live Trace examples in the PDF. Such results are far inferior to this algorithm in most of the provided examples.
You do not have a moral or legal right to do absolutely anything you want.
The real test for the Kopf-Lischinski algorithm will be how it can handle Nethack.
I can see the fnords!
This is news? Anyone else run a NES on those dozen or so emulators that already has pixel-smoothing options?
One suspects that if /. posted an article about a new engineering method that will allow us to finally build a cheap, working space elevator, you'd post, "This is news? Haven't we had ways of making cables for centuries?"
"Convictions are more dangerous enemies of truth than lies."
Have a look at the paper, if you haven't yet. Their algorithm is impressive, but even in the samples they chose, some things get better results (IMO) with hq4x (Fig. 10, Mario Bros. scene, perhaps?) or Photo Zoom 4 (Fig.9, That warrior with the sword).
That we like the way pixel art looks in the first place!
This project is technically interesting but as art it completely misses the point. Not every piece of human creation needs to be updated, upscaled, "improved" and redone.
I know the originals make more sense to me.
Zoom in on Mario a bit... Enhance...
NVIDIA and AMD should implement this and related algorithms for their LCD upscaling!
The algorithm looks like it could be parallelized onto the GPU easily enough, and doing it transparently at the video card driver level would enable PC gamers to play classic games on large LCDs without the thumbnail sized pixels.
Just imagine what this scaling algorithm could do for classic low-res games like Diablo.
This is why closed-source sucks... I'd have done it myself by now if I had access to the AMD GPU Windows driver source code!
Commodore 64 Porn arrives in high-res
Also, the pins of the chip was a failure, becoming sawtooths instead of parallel pins.
The problem with this algorithm, like many others, is that it's subject to an extension of the Pareto principle. In 80% of cases, it may be an improvement, but that doesn't help if it makes the result worse in 20%. Cause it's the failures we notice, and judge quality by.
Algorithms like this also don't work well for animated sprites. Our imagination can fill in the gap between Mario holding the hammer above his head and the hammer hitting a barrel, precisely because of the technical limitations. But the same abrupt transition with high res splines would be jarring. (And a morph would be even worse)
Speaking of failures, I must say the greatest result was Adobe's version of the fish. They were the only one that managed to make the fish more scary!
Yeah, it's from Super Mario World, i.e. the SNES - and on the SNES "Sprites can be 8 × 8, 16 × 16, 32 × 32, or 64 × 64 pixels, each using one of eight 16-color palettes", which would make the dolphin a 4-bit sprite.
"I'm not anti-anything, I'm anti-everything, it fits better." - Sole
Too late, you already did.
-dZ.
Carol vs. Ghost