Slashdot Mirror
Pixel Inventor Goes Back To the Drawing Board
lawpoop writes "Russell Kirsch, inventor of the square pixel, goes back to the drawing board. In the 1950s, he was part of a team that developed the square pixel. '"Squares was the logical thing to do," Kirsch says. "Of course, the logical thing was not the only possibility but we used squares. It was something very foolish that everyone in the world has been suffering from ever since.' Now retired and living in Portland, Oregon, Kirsch recently set out to make amends. Inspired by the mosaic builders of antiquity who constructed scenes of stunning detail with bits of tile, Kirsch has written a program that turns the chunky, clunky squares of a digital image into a smoother picture made of variably shaped pixels.'"
Just exactly what the fuck is wrong with square pixels? They are the easiest to manipulate algorithmically.
I've abandoned my search for truth; now I'm just looking for some useful delusions.
I mean seriously..
In my early days of computing, pixels werent square.. they were rectangles (4:3 display, 320x200 resolution.. do the math)
Now who wants to write a rasterizer for non-rectangular pixels... any takers?
"His name was James Damore."
Somebody addressed this problem a long time ago, I see it on CSI every week.
RTFA.
Whale
Why are we suffering from it since so ?
I did not read the article, so I don't know if it's answered there.
+1 blatant
0 = 1 + e^(Alt something)
Someone forgot to credit the Onion for this article.
Really, square pixels are an invention? Meh. And he might want to credit Georges Seurat for his new direction.
"National Security is the chief cause of national insecurity." - Celine's First Law
Jeebus needed a single-pixel transparent GIF to do the walking on water miracle.
Here's a relevant article about it: http://alvyray.com/memos/6_pixel.pdf
Why are we suffering from it since so ?
I did not read the article, so I don't know if it's answered there.
It is now apparent that for a comparable amount of information stored, a more complex algorithm (with maybe even N passes required) could be employed to produce better results to the human eye. To me, this article seems to miss the beauty of keeping it simple and going with the square. I would also bet that all of his examples are done by starting out on a square based pixel image. How would one scan an image in one pass with his new suggested method? This might become a better standard but I would wager it would make a lot of things computationally more expensive and displaying the images more complex. Not to mention manipulation of the image gets a bit trickier and probably throws a monkey wrench in a lot of our widely implemented compression technologies that already produce this sort of "creative blocks" of multiple pixels.
I'm not an expert in this field and I find his further research neat and mildly innovative but I would bet that when it comes down to weighing the practicality of implementation that squares remain.
My work here is dung.
He just "invented" JPEG too!
Now who wants to write a rasterizer for non-rectangular pixels
From the article: The pixels are still square; they're just cut into two pieces along a line through the pixel, and each piece has a color. (It sort of reminds me of S3TC.) The edge of a polygon would have one piece for the front and one for the back, and any other points along it would have one piece for each of two texture samples.
Jeebus!! Invented the pixel. I'll be damned. :-P
That's nothing. What about the guy back in the 40s that invented the color blue!
No, no... They didn't have color in the 1940s. Just look at the movies from back then...
Bow-ties are cool.
...but I couldn't get past all the square pixels on the page.
There's no -1 for "I don't get it."
He says that it was the logical thing to do, it still is! Keeping the pixels square makes sense in almost every computer science aspect I can imagine, and this guy have had 50 years of regret and came up with something that's comparably very hard to implement in scanners, memory, screens and software. Triangles and hexagons are two other ways he might have gone that's comparably simple, but squares are more intuitive. I think his contribution in the past was brilliant. He really should have no regrets.
- Henrik
- when the Shadows descend -
I would never have known that the Japanese have blurry genitals.
Eloi are stupid, throw morlocks at them!
"Why are we suffering from it since so ?
I did not read the article, so.."
You couldn't have read it with your square pixel screen, anyway.
This sounds like the ongoing debate between analog and digital audio. Everyone likes using images like these during the debate, but given enough resolution (bits), the closer the digital audio will be to its original analogue (electrical) source.
I'm god, but it's a bit of a drag really...
Actually, you jest, but I remember the first time I saw footage from WWII that was in colour and being stunned, because it was so vivid.
And, then there was the Russian guy who created colour photos in 1909 using techniques he created himself.
There are more things in heaven and earth, Horatio, Than are dreamt of in your philosophy.
Lost at C:>. Found at C.
And that's not just because Civ5 is coming out soon...
Blar.
Square pixels are easy to manipulate to be sure, so are single-core CPUs easier to code against, but the world is not perfectly right-angled. I'm now waiting for my flying car; which will use nano-morphing, variable shaped pixel in-car & HUD displays, and all controlled by a hella-core processor system running Hellabuntu. The sky's the limit! Or something like that.
This is the NSA, we're gonna geet U h@x0r5! Also, what is a h@x0r5?
But maybe that's because that's what we used in early games.
It does translate fairly well to an interlace design for scans, though.
-- Tigger warning: This post may contain tiggers! --
That's what it sounds like to me.
Square? Not on my old Trash-80! Heady days indeed, populating that 128x48 grid of monochromatic pixels. The aspect ratio really screwed with attempts to generate graphics using trigonometry.
to fit a larger baby image on his 1957 computer?
From what I get from the article for him it is like a day has not passed since then and things like image compression were never invented.
Oh, wait, that is what Alzheimer's does to you... I see...
Anyway, we should thank him for the pixel (and for it being square and not something ridiculous that would give us problems for years to come...).
Violence is the last refuge of the incompetent. Polar Scope Align for iOS
Working in the web division of a semi-fine jewelry retailer.
graphic artist: Do you know computers?
me: I should hope so. Do you have a question?
graphic artist: Yes. It's with Photoshop.
me: Ok, I might be able to help. What's the problem?
graphic artist: Ok, let me zoom in here. You see what I have here? (zoomed in so that the pixels were big blocks on the screen.) Everything is really blocky.
me: Understandable at this view level.
graphic artist: Well, it's not working for me. I need to be able to get a smaller shape in here but it's all too blocky.
me: Let me get this straight. You want to get a shape in the image smaller than a pixel?
graphic artist: *beaming* Exactly! How do I do that?
me: I'll look into it. *slowly edged away*
And she was getting paid three times what I was. Things like this make me want to lock myself in the server room, trip the halon and wait for the blackness to take me.
Kwisatz Haderach
Sell the spice to CHOAM
This Mahdi took Shaddam's Throne
The configuration of the variable-sized "pixels" depends on the image, so you're not going to get a new screen with more detail -- this is for storing images. From what I can tell, he's doing a basic form of an old and well known compression technique ("macroblocks" in JPEG, H.264, and others) and calling it a new form of pixel.
in one, a seam divides the mask into two rough triangles, and in the other a seam creates two rough rectangles
My digital alarm clock already invented that.
I'm sure if the picture on the left was resized to match the pixel density on the left it would look just as good..
That's not water, it's just single-pixel #00C PNGs.
Most of the great artists were mad.
Answer: Aunt Flo was rigid.
Remember to maintain your supply of
1) A pixel isn't "invented" by anyone. A pixel is just a concept that is so straightforward, like the wheel, language and adding numbers. It's not a question of which single person "invented" it. It's just a question of, once the technology is there, it WILL be used, no matter what.
2) What kind of screen are you going to use for that? Each pixel can have different types of pixel sizes so no screen could fit that. A square grid is the most uniform division of 2D space into units.
3) If this would have been about hexagonal pixels, I'd have found this cool.
4) At best, this is a new compression scheme for storing pictures - but certainly not a way to display them (see 2))
5) Non square pixels are not a new idea, see for example sensors of cameras.
Rigid, or frigid? :-P
Lost at C:>. Found at C.
Perhaps it is a clever algorithm, but the summary and article make it sound like he is re-inventing the pixel. I don't think that is correct. The example shown starts with a square-pixel image, and outputs another square-pixel image, just at a finer resolution with less blockiness.
While it does appear to work, it isn't clear to me how much information can be inferred correctly. Furthermore, cameras often don't use square sensors to begin with, so this isn't directly applicable to the raw image format.
Obligatory paper on the subject of pixels: A Pixel Is Not A Little Square, A Pixel Is Not A Little Square, A Pixel Is Not A Little Square! (And a Voxel is Not a Little Cube).
Your imagination is your own... ;)
Remember to maintain your supply of
blatant as it may be, I read the article three times now - and Soilworker, you did well not to bother. I'm pretty sure the answer is not in there.
This doesn't seem to be about square pixels in terms of display technology (where hexagonal pixels may indeed be superior).
It also doesn't seem to be about picture acquisition.
On the face of it, it seems to be talking about mapping rudimentary shapes to pixels so that they conform to a most-likely contrast-matching scenario with regard to surrounding pixels. Which some other posters here already pointed out with posts about JPEG and the like - but it's not really comparable to that either. Not in technique and not in performance.
At best, as far as I can take away from it, it could be a different way to display an image when zoomed in / a technique that could be used when enlarging an image to provide greater apparent detail (although you wouldn't want to enlarge it - you'd want to store the masks found with the original image for display).
The results in the news blurb look pretty decent and if nothing else 'different' from other 'smart scaling' methods, so it's worth exploring. But what this has to do with square pixels as we're mostly familiar with them, I have no idea.
Now, about those hexagonal display pixels...
The current problem is that on an LCD display, the Red, Green, and Blue pixels are adjacent to each other, not co-located. Coming up with a scheme to make all 3 colors appear to emanate from the exact same point would be a useful development.
I've abandoned my search for truth; now I'm just looking for some useful delusions.
Or fractal images could be used to produce even better pictures that would scale well. Sounds like he is trying to invent a half-assed fractal (which admittedly would take less processing to produce).
I've abandoned my search for truth; now I'm just looking for some useful delusions.
While it describes the algorithm, does anyone know of a reference implementation that I can play with? The results are quite compelling!
Slashdot's rate-of-post filter: Preventing you from posting too many great ideas at once.
...and not to mention that in nearly every case the final image will be rendered on or captured with a device that uses uniformly sized pixels.
=Smidge=
From TFA: "...So one thing you can do is say, I’m going to assume the next pixel is like this one. Don’t talk to me, don’t tell me anything about the image, until you get something different..."
Brilliant! He's reinvented Run-Length Encoding!
"Remember when I said I would never lie? Well, that was the first time."
We just got the first 300 plus dpi screen in iPhone 4, and obviously that will come to other devices. At that point, you don't care about the pixels anymore because you can't see them. What is onscreen does not appear to be made out of pixels, it appears to be made out of curves and lines. Most photographic prints are less than 300 dpi. Most people have never seen an image with higher resolution than an iPhone 4 screen. We don't talk about the shape of the pixels in a chemical photo print, we talk about the image that is shown there. We're entering that time in onscreen images now also. As great as pixels are, they are better when you can't see them.
Nope, he's just described a limited form of the commonly used image (and video) compression technique known as a Macroblock.
Actually, you jest, but I remember the first time I saw footage from WWII that was in colour and being stunned, because it was so vivid.
And, then there was the Russian guy who created colour photos in 1909 using techniques he created himself.
There are more things in heaven and earth, Horatio, Than are dreamt of in your philosophy.
Nonsense!
I looked at the embedded exif data of the pictures on the site you link to.
The images are from early 2010.
If you have non-square pixels and want to create horizontal or vertical lines, you will get the exact same problem as we currently have with square pixels and diagonal lines.
The pixel element simply can't have a shape that naturally supports lines in all different directions.
Actually, you jest, but I remember the first time I saw footage from WWII that was in colour and being stunned, because it was so vivid.
Yeah, I know there actually was color photography in the 1940s. I mean Wizard of Oz came out in the 1930s... I thought about whether to add a footnote indicating that the joke wasn't factually correct, but decided against it.
Bow-ties are cool.
In 1880~1900, black/white photographs could be sent to Germany (the China of the time) to be hand colorized. Of course, they didn't always get the colors right due to cultural differences.
Do you even lift?
These aren't the 'roids you're looking for.
Many image-enhancement techniques exist that do just this, and this is not really new. In fact this proves that square pixels work just fine to transmit the information, but the image can be enhanced to a larger resolution by non-linear techniques that work better than simple [traditional] upsampling.
Help! I am a self-aware entity trapped in an abstract function!
Bravo,
my favorite C&H ever.
Yeah, but that was hand tinting, which was (mostly) more like painting.
The Russian guy actually did it with light filters and three separate images. Way ahead of his time.
Lost at C:>. Found at C.
The printing industry has dealt with this sort of thing for a while. Read up on Stochastic Screening. Not quite the same, but it gives you a sort of idea of the problem of mapping continuous tone data in a non-continous space.
Are easy to grasp, and have all sorts of uses. You can produce them on a CRT simply using a scanline technique, they're simple to address logically, and sections can be moved around (albeit by discrete amounts) trivially, and you can apply some simple but useful filters very easily.
Other representations in addition are also useful. Some people find fourier transforms of the data helpful. Others find it useful to identify lines and discrete shapes.
It seems that Kirsch has been playing with another representation. Seems to do a rather good job of enhancing certain types of typical real world image. Good stuff, but not a reinvention of the pixel.
http://nvl.nist.gov/pub/nistpubs/jres/115/3/V115.N03.A03.pdf
Sometimes they fool you by walking upright.
*laugh* No worries, I knew you were joking, and acknowledged it in my reply.
But some of the kids on the lawn might not have known that. ;-)
Lost at C:>. Found at C.
First, here's the actual paper, since it clarifies what exactly he's suggesting and doesn't seem to be linked anywhere in the article.
It's not a suggestion that we start using non-square pixels for displays or cameras or scanners or what not, though he's certainly not being very clear about anything and the reporting on this is just making matters worse. What the paper proposes is a method where:
1) The image is split into 6x6 blocks
2) For each block, you go over the four rotations of the two following two-section masks:
The triangular mask:
ABBBBB
AABBBB
AAABBB
AAAABB
AAAAAB
AAAAAA
The rectangular(ish) mask:
BBBBBB
BBBBBB
BBBAAA
AAAAAA
AAAAAA
AAAAAA
for a total of eight effective masks, and average the values under each section, resulting in two values, A and B.
3) For the mask and rotation that has the largest difference between A and B, you output the mask, the rotation, and the A and B values, resulting in 19 bits from a 6x6 (288 bits) block.
Though he talks of non-square pixels and whatnot, it's really just a compression algorithm. A really stupid one. Basically it's a bad variation of vector quantization, with lots of baffling details. Why 6x6 blocks? Why those specific masks? Why are you maximizing contrast instead of minimizing error like any sane person would do, WHY? There's no rationale given for any of these choices, not theoretical, not empirical, not even subjective.
The same sort of rigor extends to his comparison, where he compares his compression algorithm to, instead of, say, another compression algorithm, the image apparently simply downscaled and then scaled back up. And not even with a halfway decent resampling algorithm, but with nearest neighbour. Not to mention that the "non-square pixels" version has 2.375 times as many bits to work with. If he'd done a comparison to a reasonably modern compression algorithm like JPEG, the results would be much less favorable to him.
tl;dr Some old guy put together his My First Compression Algorithm kit and it's being treated like a revolution in graphics by ignorant reporters. Nothing to see here, move along.
For a more detailed explanation of why there wasn't color in the old days just listen to Calvin's dad: http://multifamilyinvestor.com/wp-content/uploads/2010/03/calvin-hobbes-world-black-white-color.jpg
So he just drew pixels with even smaller pixels? Can we make something to redraw the pixel pixels with even smaller pixels that split them up. Like some sort of megapixel.
I know this is getting off topic but the summary was really offtopic for the article...
Anyway after reading the summary I had the same thought about hexagonal pixels possibly being superior for some applications. However I think the biggest drawback or issue to contend with would be for text display. At best fonts would need to be recreated with a hex display in mind and at worst dithered. Maybe with pixel size approaching the eyes visual limits it won't be as much of an issue.
TODO create witty sig.
Actually, you jest, but I remember the first time I saw footage from WWII that was in colour and being stunned, because it was so vivid.
Any link to color WWII footage because I would like to see this. I've seen color pics from Civil War era but not video from before 50's... Just don't tell me it's in Disney's vault.
This post was generated by a Cadre of Uber Monkeys for Monkey-Man2000 (603495).
Another world just stored vector graphics instead of bitmaps, takes less space, makes it much easier to animate too.
The algorithm he created looks a lot like HQX which is used mostly to scale old video games. His algorithm seems generalized to work on high-color images while the HQX algorithms expect something closer to 16-color or 256-color images. HQX probably deals with dithering better.
The whole thing about "square pixels" is just the media angle.
I'm surprised you've never seen "The Color of War" on History Channel or PBS. Anyway youtube has lots of footage. And oh yeah, they didn't use video. It was film
"I disapprove of what you say, but I will defend to the death your right to say it." - historian Evelyn Beatrice Hall
You're still a square, and the Summer of Love is over!
I wonder if anyone at Pixar actually uses a "drawing board", and if they do if there's any benefit to going back to it.
You are welcome on my lawn.
I mean Wizard of Oz came out in the 1930s...
True. And you'll notice that all of the parts of the story that were filmed on Earth are in black and white. It's only after Dorothy drops through the wormhole and they start filming the parallel world called the Land of Oz that the film shows color.
Calvin's dad explained it all.
When our name is on the back of your car, we're behind you all the way!
There are plenty of screens in consumer devices this very day that only give an effective pixel measurement. Many times they are actually made up of tiny dots or rectangles layered offset from each other. A cluster of 3 or 4 different colored dots could be considered 1 pixel. The commonly used method of sub pixel rendering for font smoothing uses this sub structure of a pixel to produce a better edge.
If this guy's format of storing color information takes off, we could use the data within his files to create a better image across the substructure of a screen. I don't see what the problem is. With the proper software, photographs could be rendered better on almost any modern LCD by using the substructure of a screen pixel combined with his variably shaped pixel format.
The improvement may not be all that great, but new screen technologies are using effective pixel measurements more and more. We could see benefits on todays technology and lay the software ground work for display manufacturers to stop cramming their technology into some square box which can only ever be an effective measurement.
I think what Kirsch is trying to do is create a file format which can store a better approximation of what's going on in any 1 grid coordinate. Using this new form of digitally recording an image, you can render it more accurately on different forms of displays. This is an improvement in recording visual information so that it is separate from the display technology it will end up on. I think.
Most font's are vectors. An equation representing a line. They would be rendered just as smooth, if not smoother. The rendering algorithm would be changed to graph the line across a hexagonal grid. I imagine that sub pixel rendering would still be used as well.
As a graphic artist, I recognize that bitmap/pixel fonts are niche use formats and serve a limited need. They are a supplemental format that are only beneficial in a few circumstances. 99% of the fonts you see are vector paths.
mad at what?
rewriting history since 2109
Just to elaborate on this, you need to distinguish between a square sampling grid, and a square (box) reconstruction filter. An image is really just a set of samples, the color at a grid of points in the image, not a grid of the average color over a square area (not usually, at least). It's very similar to a PCM sound file, where it's a recording of the instantaneous amplitude at each point, not the average over the sampling period. If you reconstruct the sound wave with a box filter, you get a crappy result. The same with images; a gaussian reconstruction filter is usually better than a box filter. LCDs are actually a step backwards in this regard, at least for images, since they have much more boxy elements than a CRT.
Showing my ancient history, I remember an early presentation inside DEC for what became the DEC 1M pixel terminal that became the basis for most of X11's design, from perhaps 1982. The pixels were only 1 bit each (ie black or white). They were debating a design where the scan lines were .717 as far apart vertically as the pixels were horizontally and each odd numbered line was shifted horizontally (by the CRT scanning circuitry) by 1/2 pixel. The result was a hexagonal layout that also nicely filled the 4x3 screen while the source was a 1024x1024 array of bits.
Inability to nicely draw vertical lines is what killed this idea. In the end it was more practical to lose 256 lines of there expensive display memory off the bottom and make square pixels. This extra display memory ended up being used for off-screen images, it's availability but limited size also greatly affected the design of X11.
They also had ethernet cables that were the really thick yellow things with the vampire tap that you screwed onto them to add a new machine to the network. All of this was in a the Dec research department showing the future of computers...
That is what it looked like to me as well, but I found the actual paper, and he is creating his "non square-pixel" image from a larger image, not upscaling it from a smaller one. In other words, it is basically just a form of poor-man's compression where you replace each 6x6 block with one of 8 decompositions containing two coefficients each.
Really, back to basics is bottomless if you chose to look beyond convenient hype.
``Tension, apprehension & dissension have begun!'' - Duffy Wyg&, in Alfred Bester's _The Demolished Man_
I did this with a b/w scanner had in the school newspaper back in '89. Got some cyan/magenta/yellow gels from the theater department and was churning out color seps for the newspaper. Got an A in intro to desktop publishing. Dropped my intro to DOS class as we were still learning about moving files around and such and decided to get a Mac at that point.
I drank what? -- Socrates
If it's frigid enough, it'll be rigid.
But my Rigid tools are orange!
I drank what? -- Socrates
Okay. Minus five points from the "graphic artist" for not knowing how to resample the image. Plus one point for trying to improve her knowledge instead of suffering in silence.
Minus one point from you for not knowing something that's not directly in your field.
Minus ten points from you for not even trying to help.
Minus fifteen more points from you for being a jerkass about it on Slashdot.
So she's down four, and you're down twenty-six.
81 according to the article, that's how old he is.
Most of you youngsters in here maybe don't get it, but as I have now blown through my teens, my twenties, and now most of my thirties I have definitely begun to acquire more and more of a distaste for that which I do to make money.
If I make it to 81 and I still give two craps about technology in general, much less have enough enthusiasm left to try and contribute, then I'll consider myself an extremely lucky man.
Kudos to this guy for working in tech SINCE THE FIFTIES and not burning out.
How many of us will be able to say the same?
Genuine fractals already does this. The results are amazing, but you aren't going to make a 100x100 pixel image look good on a billboard.
zosxavius photography
mad at how reality isn't real enough to represent the real reality.
My $50 eyeClops projector seems to have hexagonal pixels. It seems to be incredibly bad for fonts... but the projector itself is so bad at everything it does that it's hard to tell which of its many failings is most to blame.
Blurry, hexagonal pixels are also not optimal for structured light experiments.
Youtube is a great place to start. High Quality WW2 Color Footage
Man blir trött av att gå och göra ingenting.
"An image is really just a set of samples, the color at a grid of points in the image, not a grid of the average color over a square area (not usually, at least)"
No sample, whether a pixel or a sound sample, is EVER a point sample. It's NEVER an instantaneous value. That would require that your sampling function be a delta, and delta functions have nice little features like taking infinite values and having infinite bandwidth.
A real sample is always an average over some finite time or space (or other dimension), weighted by the sampling function. In many cases that sampling function is indeed designed to be as close as possible to a rectangle (or boxcar) function.
Big deal.
September 2011: Looking for Cocoa/iOS work in Boston area Cocoa Programmer Quincy, MA
The approach looks very much like wedgelet approximations see theory and images at http://ibb.gsf.de/homepage/laurent.demaret/wedgelet/docu/wedgelet_short_guide/node1.html
He didn't claim the photos were digitized in 1909, he claimed the photos were colorized in 1909.
"In 1909 a remarkable project was initiated by Russian photographer Sergey Mikhaylovich Prokudin-Gorsky. His mission was to record – in full and vibrant color – the vast and diverse Russian Empire."
Unless you are saying the comment was tongue-in-cheek. Even then, it would be lame tongue-in-cheek and not haha.
Non square pixels are not a new idea, see for example sensors of cameras.
Alvy Ray Smith, computer graphics researcher and co-founder of Pixar, will tell you that A Pixel Is Not A Little Square, A Pixel Is Not A Little Square, A Pixel Is Not A Little Square! (And a Voxel is Not a Little Cube). I don't necessarily agree with everything in the article (e.g. coordinate systems) but it is worth a read.
It is probably worth referencing Alvy Ray Smith's (co-founder of Pixar) enlightening memo from 1995 entitled: A Pixel Is Not A Little Square, A Pixel Is Not A Little Square, A Pixel Is Not A Little Square! (And a Voxel is Not a Little Cube)"
First world problem...
Agreed, but the voronoi diagram of those sample points gives the squares.
By the way, I forgot to mention the article is very interesting, thanks for the link!