Slashdot Mirror
The iPad As a Shape-Recognition System
An anonymous reader writes with an interesting use for the iPad: "The guys over at the Volumique blog have a different idea as to how to tackle apps for Apple's devices. They aren't just thinking about a digital activity on such devices, they are experimenting with using physical objects through Apple's multi-touch screens. Imagine being able to buy the playing pieces for a board game, but then loading up an app on your iPad for the actual board. The pieces would be recognized when placed on the iPad's screen, it would even recognize which direction they were facing. This may sound like an impossible feat unless you use a much more expensive device like Microsoft Surface, but Bertrand Duplat and Etienne Mineur at Volumique already have it working."
Soon the thing will take your fingerprints and surreptitiously send them off...
For justice, we must go to Don Corleone
Comments attached to TFA indicate it may be a hoax, after watching the videos carefully.
Without more details or having this app actually for download it's hard to say whether real or fake. Indeed it sounds too good to be true, so it probably is.
The last choice (“jeux vidéo”) clearly selects itself a moment before the token is actually set down onto it in the first video on their blog.
Alexander Peter Kristopeit bought his basement from his mommy for one dollar.
Microsoft Surface is also much larger (the size of a coffee table or so) where it might make sense to put objects on it. Board gaming on a Surface would be quite an experience, but with an iPad you'd be covering most of the display unless you stuck to just a few playing pieces. You also have to fit the entire board on the screen, as it won't be able to scroll with pieces on it. The example given in the article, Monopoly, just wouldn't wouldn't work at all.
"A week in the lab saves an hour in the library"
Could be a trick, of course. Could be not.
Looking at example with two figurines and iphone it seems that they have different weights - check out the bases. These seem to be button-batteries of different types and weights. In the video, however, look at 00:52 how one of responses highlights before figurine is put down.
With multiple figurines and ipad also see bases - they seem to have different radius. Detection of direction could be a bit of a hoax, maybe figurine is pressed in a specific way to help the system.
"Coloured paper" example could be explained by different amount of surface pressed against screen. Patterns are just to confuse viewers. This one is probably a trick, while the previous ones are more practical.
Comments from people who know how Apple's touchpads work (what they are able to detect) are more than welcome.
Device with large multi-touchscreen can be used as a large multi-touchscreen! How about we just gut it and use the screen as a sensor? I know it would leave Apple out, but I'm okay with that.
I thought about this quite a bit: I don't think it will be long before a major RPG system like D&D publishes an entirely paper-free edition, where the players and the DM all have touchpads. Instead of miniatures, you have avatars of your character on the game board, and certain combat actions (like determining whether a hit landed) can be automated. Basically, the "I attack this guy with my +3 sword" command would be selected on your digital character sheet, and then you just get a prompt to roll a d20 on your touchpad. All the stuff about armor adjustments, size compensation, and whatever other rules, would be calculated automatically.
I can't wait for this obvious streamlining step. I think it will most benefit rule-heavy games like Hackmaster, which are a lot more fun than D&D but rather tedious when combat situations get big. Once we get used to this, we can actually make the hit and damage modifier system even more complicated and realistic, because using weapon speed factors, armor type modifiers, armor damage, hit location determination, etc. will actually not slow down the game at all. This stuff can return the focus of the personal interaction to where it should be, which is fantasy and role-playing. As a bonus, combat rounds can be rendered and animated once this gets good enough to help with immersion.
It won't be long before the purchase of a nice touchpad will cost you less than the purchase of a set of D&D rulebooks. Then, instead of reading about character creation, it will make much more sense to just load up the character creation wizard on the touchpad and start rolling. In more advanced versions, the software will show you the first-person view of "what you see" when you kick down a door. This really is the first step.
It is a capacitive touch screen - right? These work by having a sheet of glass with a conductive material that when your finger (also conductive) touches it is sensed. The question then becomes - how small a "spot" can it sense and how many multi-touches at the same time? If it can sense small enough and a large enough number then a braille like symbol on the bottom of the playing pieces would be plenty to identify the item along with its direction. Some of the items they are placing on the screen *aren't* conductive so they are either faking it or have something else on it. If the latter there are a number of other applications that use capacitive dots so other than applying this to game pieces not really much new.
If the screen *can not* see like that then I can't figure out how it would work. Most bases for board games are round and there is no way to "see" which way the top of the piece is facing - round items do not have direction from their bases. One may be able to have some patterns indicate direction - say each time a piece is placed you are required to move it (maybe even a really small amount) in the direction it is facing, but that isn't inherent in just dropping the piece on the surface and it knowing where it is. Further if it is just a round base and no indentifying marks can be read there is no way to know which piece is what from just setting it on the board. Again, one could make a a system where other information is used - for instance have all the pieces start in known locations. Double tap to remove a piece from the board and every time a piece is picked up it is being "moved" to the next location it touches - basically the pieces are a mouse.
Of course those are how other things "sense" what item is on the surface and direction - the hard part is the surface.
The real issue is why? How many board games can be played on a 10" surface let alone a 4" one? At least with respect to board games (would have to think a bit about other applications) I can't see playing much of anything that way. Even with something that already uses a small board (chess) you would have to have *really* small pieces and would be hard to accurately pick them up and place them. Like many products in search of an audience if their goal is to play with some neat hardware and learn something then there is no answer to why needed past that, if they are looking to make money from this good luck - I think you are going to need it (better get a a patent so you can at least hold onto the idea until screen surfaces catch up with your idea).
------- Sorry about the spelling, I suffer from two problems. Dyslexia makes it difficult to spell well, lazy makes it
I'm suspicious this is a hoax (after all, doesn't the iPad's screen detect capacitance- not visuals like patterns and shapes?) ...but even if it's real, I noticed the guy was consistently pressing down pretty hard with everything it detected. I wonder if the object's own weight isn't enough to trip whatever sensor they're using.
The ReacTable...Bjork had one of these on her most recent US tour. Lots of fun to watch in action.
The surface is camera based. It's built to look at the shape or even color of whatever is put on it. The advices use a capacitive screen. The Apis don't eem give you any information other than a distilled point where it thinks there is a touch. There really isn't the concept of an occupied area of the screen.
So this is different, but I don't understand why someone would want to do this. The real estate of the devices are small. It's not at all like a surface where placing physical objects on a table is a normal thing to do. Plus te only way I can think to do this would be to arrange little dots of conductive material underneath the objects. That would create a recognizable pattern, but it would eat up a ton of the devices finite touches. You could only put 2 things down really.
I figure you need at least 4 to distinguish different pieces and orientation. Well, you pretty much max out at 2 objects then.
it's not fake. It can only work for one object at a time though. It's pretty simpe... there are two contact points on each object that simulate the touch of a finger. the distance between the points indicates the type of object. and the angle between the points indicates rotation of course ! But they can't scale this up... from what I can tell, one object at a time is a hard limit. But this is a software limitation probably, not a hardware limitation.
You mean an RPG game like this: http://www.youtube.com/watch?v=n94E3IeBquY
And a cheap DIY Surface like this: http://www.maximumpc.com/article/features/maximum_pc_builds_a_multitouch_surface_computer?page=0,0
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
Three dots, sufficiently not in a row, are called a triangle.
Not if you do not draw lines between them. But lines do not enter into touch detection in iOS, all you ever have are points.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
On the second demo, notice that the thumb on the bottom part of the card is ALWAYS right over the home button? And notice that they don't show the app quitting? I suspect they're just tapping the home button, which the app is capturing and showing a static image in a fixed order. FAKE.
I think it could be done with no more than two. The technology behind the iPod touchscreen is capable of recognizing the shape and size of a touch point. So you should be able to determine the type of piece with a single point, and you'd just need another point for orientation. I think only oval shapes can be recognized, but if asymmetrical shapes are possible (or if you had pieces that were symmetrical and limited the forward direction to a 180 degree arc), you could do it with only one.
But if you had access to the low level touchscreen interface, it may not even be a problem. I believe the iPod filters out points below a certain size threshold as random noise. But if you know that you're looking for certain patterns, you could separate these from the noise yourself and there would theoretically be no limit to the number of recognizable shapes.
this isn't really trivial, iPad, unlike Surface is not using a camera, just the touch screen controller. It looks like they have a jail breaked iPad to get more fine grained access to the touch sensors. The neat technology is how they use flat "doped" paper stickers rather than something like a shaped rubber stamper, like other iPad/iPhone "pointers" on the market.
This is also news because it will bring things like table top gaming into the digital age, without changing the nature of the games much. Doing this with a game like DnD miniatures would be neat. You could use a phone app to keep track of the squares and do all the damage calculations, etc. allowing for more complex play. When you get more than a few pieces in play the sheer accounting takes more time than playing... watch the Warhammer 40K crowd play and the game descends in to complex damage and stat accounting very quickly and not speedy play. A technology like this would revolutionize gaming. The paper "doping" didn't look like it changed the appearance of the physical cards either. There is one card game for PS3 that tries to use the camera for reading cards and a version of Pokemon tried having embedded codes on the printed cards, but it's just too much kludge. If you could play a game like Magic:TG just by showing your cards to the device you would open up all sorts of play, from having table top referees for one-on-one matches in person to playing somebody across the internet with the PHYSICAL cards, something that has proved difficult.
So anyway, for the Slashdot crowd this is something most of us would actually use at some point.
you miss the fact that being able to recognize even simple physical objects without a camera opens up a bunch of new ideas, most readily for gaming but also for other things. Even just pulling up the stats and calculating damage for something like Warhammer 40K is a big deal. The rule and stat book is several hundred pages, they have to have a stack of several hundred cards just to represent all the unit stats "quickly". Pick up a board piece and instantly call up the rules, without having to enter search terms (which are terrible for gaming names) would be a big deal. A two item limit isn't that big of deal.
>I tried googling around to find what was behind the toucscreen technology of the iPAD but could not find enough info. ..and you probably won't. It's a trade secret.
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
Children so stupid they think America invented the Internet, computer, motor car, light bulb, telephone etc ad infinitum....
I think you picked a terrible list... I mean, other than the Motor Car all those items *were* invented in America. Sure, you could say that Swan (British) invented a good light bulb (it lasted 13 hours) - but did he invent *the* light bulb - most people would agree that a 13 hour light bulb won't fly. Or how about the Reis telephone - it was useless for talking on!! Or the computer... sure, a German computer did exist, but in terms of an electric, digital computer it's certainly the Atanasoff–Berry. And the internet... 100% American.
FAIL...
I said no... but I missed and it came out yes.
That is a good point. I hadn't thought of situations where recognizing one object would be useful. Still it seems like even that use might be better served by an RFID solution.
Why?
It's out in the open. It's only protected like a trade secret if you steal it from Apple. Reverse engineering a live example is fair game. Just that no one has posted their reverse-engineering notes. Yet.
A successful API design takes a mixture of software design and pedagogy.