OnLive One Step Closer

hysma writes "It looks like OnLive, the remote gaming system that streams HD video over the Internet, is one step closer to becoming reality, according to an article on DSL Reports in response to a lengthy video presentation by founder & CEO Steve Perlman at Columbia University. Perlman demonstrated the UI, spectating, using the service on an iPhone, and other features."

Comments

I'll believe it when I see it

[BoberFett]

Until round trips between the server and client are guaranteed to be under 50ms, the lag will feel unbearable. If someone is playing a racing game and has to deal with a second between the time they begin turning and the time they actually see it turn this service will be dead before it begins.

Re:I'll believe it when I see it

[Anonymous Coward]

Having used the beta they are giving out right now on a random computer outside of their controlled settings I have to say that their technology works. Don't know what form of magic is involved but it is playable and actually really fun. I am all for the gaming experience being opened up to people without having to keep a bleeding edge gaming rig going.

Re:Games?

[MichaelSmith]

Yeah remote desktops are the wet dream for outsourcing where I work. Imagine a system where the evil (cheap) foreigners see a video of the actual code so they can't take the revision history home on an SD card and sell it in the flea market for one tenth the real value!

Latency sensitive people

[xororand]

I also strongly doubt that any kind of game on this platform can be enjoyed by people who are sensitive to input latency. For example my old high quality PVA TFT panel used an overdrive circuit to reduce ghosting. The overdrive logic in TFT panels usually buffers about 1 or 2 full frames to analyze and optimize the pixel voltages which leads to about 20-50ms input latency. I for one already notice it when I just work to the point where it annoys me when the desktop or terminal sessions somehow always feel sluggish, let alone fast 3D games.

I can't imagine that the complete round-trip time for sending my input over the internet, waiting for a frame to be rendered and encoded remotely, sent back over the internet, decoded and displayed locally would be less than 20ms and then you'd still have the latency of your display. It might be bearable with a very fast internet connection and a CRT display which has 0ms input latency.

Maybe others aren't that sensitive to latency and can enjoy at least slower games like turn-based strategy with this service. Good for them.

Re:Latency sensitive people

[xororand]

Edit: For those who doubt that you can notice such small delays, try this:
- Connect an electronic instrument to your computer and artificially delay the audio by 30 ms.
- Try to play accurately
- ???
- No profit.

Re:Latency sensitive people

[QuantumG]

I tell ya, does OnLive have to make their case every single Slashdot article or what?

Game engines have latency in them... OnLive runs those engines at faster than realtime, so when the packet from your controller gets to the engine 300ms later than it normally would the engine has plenty of time to do its thing.

All this was explained months ago when the technical questions were asked. This article is about the business question: when are you going to ship?

Re:Latency sensitive people

[Toonol]

Game engines have latency in them... OnLive runs those engines at faster than realtime, so when the packet from your controller gets to the engine 300ms later than it normally would the engine has plenty of time to do its thing.

That's nuts. They can't run the engine out in advance of your input... unless they're rewriting core functionality of the engines, adding prediction like online FPS have. And... they aren't doing that.

If you have a ping of 100ms, you will press a key; 100ms later the onlive server will know you started turning. It generates, renders, and COMPRESSES a frame; sends it back to you. 200-250ms have elapsed. It will be like playing on a machine getting 5-10 frames a second.

Re:Latency sensitive people

[chromas]

It's okay. We knew you were already rendering your response in anticipation of what your prediction engine was going to have been expecting him to write.

*!hsoow*

Re:Latency sensitive people

[barrkel]

Assume 60 fps, synced with 60Hz monitor. That's 16.7ms per frame, and usually means a target budget of 16.7ms per iteration in the game loop - and input is probably processed exactly once in the game loop. Consider also that many decent displays already lag by a frame or two. So in a local hardware situation, you already have a built-in lag somewhere in the region of 30ms or so, pretending there isn't any lag in the actual hardware path between devices and what the OS surfaces to apps.

Now, given the above assumptions, but factoring in your posited reactive input model (i.e. no delay from game loop), you think that's good enough? The way I see it, it can only be good enough if the round trip averages to less than 16ms or so; and even then, it's not great. I've long noticed the lag in games since moving from CRT to LCD, and I can even see the lag between moving my mouse and the pointer moving across the screen - it's small but perceptible, and is either caused by the mouse / usb / driver path or by the LCD delay.

But I can't realistically see a 16ms or so round-trip being achievable outside heavily populated areas and without lots of expensive hardware very close to local loops. As it is, Google.com is 29ms away from my machine, and it's still slow to download the front page's HTML content - (yes, I know, TCP connection, several round trips, etc.) - on the order of 200ms or so.

It seems to me that round trips on the order of 50 to 100+ms are more likely, and delays of that nature are highly, highly noticeable in twitch FPSes - especially when it comes to things like changing the view direction. Pretty much all multiplayer FPSes don't wait for a server round-trip for changing the view direction. In that situation twitch FPSes will suck.

Other kinds of games may work better.

Re:Latency sensitive people

[bertok]

It seems to me that round trips on the order of 50 to 100+ms are more likely, and delays of that nature are highly, highly noticeable in twitch FPSes - especially when it comes to things like changing the view direction. Pretty much all multiplayer FPSes don't wait for a server round-trip for changing the view direction. In that situation twitch FPSes will suck.

Other kinds of games may work better.

You hit the nail on the head right there, the view direction.

Most games do all sorts of predictive wizardry to make the shooting work over internet latencies, but every game allows the view direction to occur completely locally, because even a slight lag makes the player feel like a drunkard. Many games also allow the local client to compute some of the 'game mechanics' locally, and then 'verify' the results with the server later.

Re: video as anti-copy protection

[Tei]

If ofuscated text is easily readable (anti-captcha spam bots), text with not distortion at all will be perfectly readable, so if you send video, the other side will save that video (either with hardware or software) and use OCR to get back the text.

It still wouldn't work well for rock band

[jonaskoelker]

A game like rock band could 'tune it in'. [it=500ms lag]

I really don't think it could.

Here's the reason: suppose I'm to go red-green-blue-yellow-orange-yellow-blue-green-red really fast (say, at the end of the TTFAF intro), and it's one big hammer-on-pull-off sequence which can't realistically be strummed (or the rules of the game have changed so I have to HOPO).

I miss the first green.

I only get to know that I missed the first green 500ms later. I have already HOPO'ed the rest of the sequence. There's no way I can go back in time 450ms and strum the blue I HOPO'ed, undoing the not-playing-correctly.

It's not just that you have to compensate for lag between inputs and outputs. You also have to make the lag inside a feedback loop very small. A minimal lag of 500ms is too much for rock band. ... Even if the audio and video is perfectly synchronous, and the game compensates for the output lag by virtually moving your inputs back in time. The game can never move the reaction to the output, which happens in your brain, back in time to before the output.

Re:It still wouldn't work well for rock band

[poetmatt]

I'd imagine he gets it from reality. This service not only has bandwidth requirements but serious latency requirements. We're talking considerably higher than your average hardcore counterstrike player's latency requirements. Ala 60ms pings will be required and unless onlive plans to install itself in every single state, there's no way they'll make that kind of bandwidth.

See, it's not like streaming an application, where bandwidth isn't an issue (nor resolution), and it's not like streaming video, where latency isn't an issue. You literally would need to be on a 100mb/s lan equivalent to get this to reasonably work without bandwidth and latency issues.

It just highlights that our infrastructure simply isn't there at the moment for gaming. Saying they can make 80ms within 1000 miles is a flat out lie. Someone playing in X state with the fastest server 1000 miles away is going to get at least a minimum 200ms ping.

Play music? Can't even *talk*

[Mathinker]

I remember reading in the Time/Life book on the brain that adding an appreciable delay to the auditory feedback you get makes it very difficult even to talk properly. But I'm sure that's old research.

There doesn't seem to be that much on it on the net (or maybe I'm not searching properly). The WP article on Delayed Auditory Feedback has a link to a paper with similar work but it is also from 1979.

Re:Play music? Can't even *talk*

[bitrex]

There was an exhibit I remember seeing as a kid at the Boston Museum of Science in an area dedicated to exploring the human nervous system that did this. It asked you to attempt to read a paragraph of text into a microphone while your own voice was being fed back to you via headphones, slightly delayed. I remember it being extremely difficult to read the text properly.

It's really only the mouse

[jonaskoelker]

the richer control setup of keyboard and mouse.

It's really only the mouse---you can make a perfectly fun FPS that's playable with the buttons on, say, a wiimote plus nunchuk (one stick for moving, 8 buttons in B/C/Z/d-pad/A).

What's really missing is the fine-grained relative motion of the mouse.

It needs to be fine-grained; anyone who has tried to aim with a joystick will understand why.

It needs to be relative, as anyone having played Metroid Prime 3: Corruption on the Wii will know.

Roughly speaking, you point at an absolute point on the monitor plane; your character yaws and pitches gradually to aim at that point, the speed being monotonously increasing in the distance between your current aim point and the target aim point.

What are the implications? Either you point at where you want to shoot and it takes a while to aim there, or you point way past where you want to shoot and you get to where you want to go really fast but move away again really fast.

What you really want to do is overshoot by infinity (or $BIGNUM), then aim at the target point when your character points exactly at it: then you get to your target fast and stay there. This is virtually impossible, and trying to do it is unpleasant.

That's why you need a mouse for FPSs; you can make games that only take 8 buttons, so I don't buy the "you need a keyboard". Maybe specific FPSs require keyboards, and maybe there's really no way to design around that without making the game a different game---I'll buy that. But really it's the mouse.

Is this a giant scam?

[syncrotic]

I still maintain that this simply can't work, and that it's an absolutely braindead money pit of an idea if it's not a total scam.

Idea: let's take the most latency sensitive, computationally demanding, and visually intensive thing you can do with a modern computer and try to apply the thin client model to it.

A single instance of the application in question will demand the full resources of the most powerful PC you can throw at it, but we'll just wave our hands and mutter something about virtualization to convince stupid investors that we have magic at our disposal. Because they are morons and because we put on a good show, they'll believe that you can somehow run many instances of a game on the equivalent of a single PC. We'll also be encoding 720p video in realtime at a quality / bandwidth ratio that no codec today can deliver; this will presumably happen on the same computing hardware that's already running multiple instances of cutting edge 3D games.

Finally, we'll throw in some shit about the iphone, because people can't stop fellating apple lately.

Anyone who believes this is technically feasible, much less economically viable, is fucking *retarded*.

Re:Is this a giant scam?

[setien]

I agree that on the face of it this looks like it won't work, but I can see many mitigating circumstances that means it just _might_ work.
I think there's a small chance that they might actually be able to pull it off, and if they do it really is a game-changer.

A couple of things that makes me hesitant to call everyone "retarded" if they don't dismiss this before it has even seen the light of day:
- They are aiming for The Long Tail of gaming, and I think it's easy to underestimate just how gigantic the amount of cash is in this tail
- Not ALL games are hyper timing sensitive
- Multiplexing hardware means the same computer can serve Stan in Portland and Sanjay in New Delhi at different times a day (but admittedly only if there are good pipes or the game is not super lag sensitive).
- Computer power can be spent or sold in other ways when it's not used for the OnLive gaming system (just look at how Amazon has managed to use their knowledge of scalability into a nice side business that doesn't involve books)
- For the most timing sensitive games (1st person FPS), you remove the client-to-client lag, which means the server can run a single cohesive view of the world, and pipe that to the players (so you get rid of one type of lag, which might allow for the server-to-client-video lag with no problem)
- If this gets big or they have good partner deals from the beginning, games might get engineered specifically for this network topology from the game developers side, which might take steps to minimize lag problems (I can come up with quite a few ideas just off the top of my head)
- If the video algorithm is designed for gaming (as it is), they can degrade quality in the video compression in a smart way to keep the lag to a minimum - who cares if the leaves on the trees in your peripheral vision are a bit blocky when you're in a firefight in Crysis)
- They have a few pretty strong industry profiles on their company roster

That said, I am of course also highly sceptical, but I see a sliver of a chance that they might pull it off. And if they do, I really think it will be a game changer (pun intended).

Re:Is this a giant scam?

[tbradshaw]

You obviously didn't watch the video at all. While you're being an asshole about the idea, the guy presenting during the presentation covered all of your strawmen.

1) "fill instance of the most powerful PC you can throw at it" - Uh, no. When you move from workstation class hardware to server hardware, the "ceilings" change. But, for games like Crysis, they do, indeed, use a big GPU per instance.

2) "720p video in realtime that no codec today can deliver" - Too bad you didn't watch the video. Turns out, this is the same team that brought us QuickTime before video codecs were even discussed. He also describes exactly how they pulled it off, started with scrapping the stream-based design paradigm, using a feedback loop based design paradigm, and creating a new encoder that looks great in motion encoded and decoded in real time (as one of the weaknesses, you can't pause it or it looks like shit).

3) "Presumably happen on the same computing hardware..." - Actually, no. As the presenter describes, the codec taxed even the dual quad core xeons that it was developed on. Then they fabbed custom chips that do nothing but implement the encoding algorithm. It's entirely hardware accelerated encoding, two chips per user on custom boards.

I also thought the entire process sounded like a big stupid scam, but before I declared the mighty victory of common sense, talking out of my ass, I went ahead and watched the video.

Compression?

[Rufus211]

I think the most interesting part was the (lack of) answer about how the compression works.

They claim 80ms round-trip latency from button push to image display. Running a game on a server and screen-scraping in ~20ms is fairly easy. With proper datacenter placement and peering agreements ~50ms round-trip ping times are reasonable (if somewhat optimistic). The issue is how do you compress the 720p image and send it back in 10ms with reasonable bandwidth.

They're claiming 1ms compression, 8ms decompression (125hz), and 5mbit 720p streams. The compression is using a custom ASIC, so that's completely believable. Decompressing at 120hz on any generic hardware (they specifically said no GPU help) means it has to be an extremely simple protocol. The biggest question is how do you reach "HD-quality" at only 5mbit when you are not doing group-of-pictures compression (keyframes and diffs from the keyframe). Mind you that a standard DVD is 10mbit, so they're claiming higher resolution with half the bitrate and no keyframing. Obviously H264 gets better quality/bit than DVDs, but it does so by using even more complex keyframing and diffs and is far too CPU intensive for their target platforms (it's hard to watch 30fps H264 trailers on many machines, let alone a 60fps stream). The only hint he gave was some mumbling about visual perception, and the statement that their compression only looks good in motion (if you paused the stream it would look terrible).

Any ideas as to how the compression works?

So, their business model...

[Xugumad]

Is to rent console time over the Internet, to people with enough money to have a PC that will run this stuff, and a fast Internet connection... ...or an iPhone, a platform known for its cost-effective pricing model... ...but don't want to buy their own console, because it would clearly be too expensive?

Of course, people don't want to all play computer games at the same time, so I can see they'll be balancing load throughout the day... erm... or not (and certainly, they're not going to be running connections internationally with latency that's anything less than abominable for this).

In summary: WTF?