Slashdot Mirror
Predictive Modeling To Increase Responsivity of Streamed Games
jones_supa (887896) writes Streaming game services always bump up against a hard latency limit based on the total round-trip time it takes to send user input to a remote server and receive a frame of game data from that server. To alleviate the situation, Microsoft Research has been developing a system called DeLorean (whitepaper) using predictive modeling to improve the experienced responsiveness of a game. By analyzing previous inputs in a Markov chain, DeLorean tries to predict the most likely choices for the user's next input and then generates speculative frames that fit those inputs and sends them back to the user. The caveat is that sending those extra predictive frames and information does add a bandwidth overhead of anywhere from 1.5 to 4 times that of a normal streaming game client. During testing the benefits were apparent, though. Even when the actual round-trip time between input and server response was 256 ms, double-blind testers reported both the gameplay responsiveness and graphical quality of the DeLorean system were comparable to a locally played version of the game.
Nobody can predict when I will suddenly chase a mammoth with a fork while buck naked.
Nobody.
It can't predict all three at once, I'll grant. But if you've already given the commands to strip naked and wield a pitchfork, then of course it can speculatively run the choice for pushing the joystick toward the mammoth.
Why on earth would you want to do this? Run the damned thing locally
Let me count the guesses: A publisher paranoid about prohibited copying may be willing to license its game at a lower price if the game program never leaves the server. Or it might be cheaper and faster to send a video stream than to send sufficiently powerful hardware and 50 GB of game at once. Or sufficiently powerful mobile hardware might not even exist. Or it might want to ensure that all players connected to the same server have comparable lag and the same inability to install cheat mods.
Because it's easier to extract money from people continuously if you can deny them their crack^Wgame at will.
So...branch prediction. Yea, that does seem like an interesting idea. Take it to the next step and you can render n-frames ahead based on all probable inputs and try your best to continuously pair down future paths as you get actual user input. But realistically that'd increase the bandwidth requirements 1000% or more which in most circumstances will make the situation worse. Not to mention the difficulty of actually writing games/environments to deal with these speculative paths or the generally choppiness whenever you diverge away from expectations--something which I've seen happen even on local games which were apparently profiled on set paths.
So, again, interesting. Now, if latency were closer to the 33ms range, bandwidth was plentiful, and this was trying to edge towards smooth 60fps playback...
EA better not make games streaming only as there simcity must be on line only not only was a lie the game also sucked big time.
The player would choose this because he game he wants to play is otherwise unavailable.
"I have no clue what I'm reading about but am angry about it none the less!"
- Random internet commenter
Does this mean that Microsoft will assume responsibility for creating unresponsible persons?
Time is what keeps everything from happening all at once.
A publisher paranoid about prohibited copying may be willing to license its game at a lower price if the game program never leaves the server.
*stifles laughter*
and they stop playing after they see the bandwidth bill also what about people who's only HSI is satellite?
People with download caps as low as 10GB? even 50GB is low for something light this. The lag on satellite is to high for something like this as well.
Now they want to make games essentially play themselves.
No, they don't. Read what this is about.
What happens when the player produces input the software does not expect?
If it matches none of the speculated actions then the speculative frames are discarded or it can use what they call a "misprediction compensation technique" to build the correct frame from the speculated frames. How about reading what this is about and then asking specific questions about it?
Few gamers want to play games any more, they just want instant gratification trophies. There's more focus in achievement trophies in modern games (especially AAA games) than there is on actual gameplay.
What is that based on? For years we had the promise that one day the level of immersion and the quality of the output would be such that it would be able to produce a sort of "interactive movie" experience, that is what you get from games like Call of Duty and while they are wildly popular they don't supplant more traditional games and those who enjoy one aren't excluded from enjoying the other.
A video game publisher is likely to view people stuck on satellite as a rounding error. For one thing, satellite players are already locked out of online multiplayer due to latency. For another, a publisher might be under the impression that people who can afford to live in the city are likely to buy more games and/or subscribe to a game longer. It's the same reason that many apps hit iPhone and iPad before Android: studies show that iOS users tend to spend far more online per capita than Android users.
The customer WILL buy our car because otherwise, he won't be able to drive.
Video games are less substitutable than passenger cars. Someone who wants Super Smash Bros. 4 isn't going to be satisfied with Destiny nor vice versa.
The biggest problem for streaming games is going to be the ISP.
They don't like it when people use netflix, think about something like a game, where you are sending more stuff, not to mention upstream is being used more on these.
Streaming games would kill any download limits you have on your ISP and pretty much all of them have some sort of limit in place.
But what is the good of this if your ISP cuts your account, or nutters the connection so you can't play because you went over the 200-400gb limit for the month?
Be seeing you...
Let me clarify: It's the difference between a game being available to players for rental over this sort of streaming service and a game not being able for rental at all.
Such creativeness with the language.
just a ghost in the machine.
Its even easier to pull complete support for the game when it depends on their servers!!! This way they can sell Super Mega Game X+1 to those who used to play Super Mega Game X, forcing them to pay for the new shiny versions, which is really almost identical, with no compelling new features, except maybe slightly better graphics and a +1 to the title.
I now stay 100% away from any games which demand an internet connection to play. If a game is online only, but supports local server or can be emulated via Hamachi or something, then i'm cool with it. Otherwise no.
If this means i can't play game X, then i won't die from the lack of it. Plenty of other games out there which I can play, and play how i want.
Similar in many regards, yes. Quakeworld, from what I understand and recall, focused on two things: predicting where you were going so that it could prepare those parts of the world in advance, and predicting where others were going so that it could draw them as accurately as possible on your screen even if the connection was slow (I may be incorrect about Quakeworld having this feature, where a predictive bot run locally on your machine effectively replaces the other players for a few ms at a time, but I know it's been in a number of other games as a feature to help smooth out visible lag). The difference, however, is that Quakeworld made a single guess, and wouldn't know if it guessed correctly until it heard from the input source again, so there could be some significant discrepancies between what was displayed as a result of a prediction and what the reality of the situation actually was.
In contrast, what Microsoft is doing here is making numerous guesses regarding which choice you'll make, generating the frames for each of those choices in advance, sending them all to you in advance so that they can be buffered, and then instantly displaying the correct one as soon as it gets your input to know which one it is. It's like loading frames for the next 200ms from a handful of alternate futures, and then selecting the correct 200ms once we know which potential future ended up being the actual one. Which is to say, the one it's displaying is always exactly correct, whereas in games that implement the predictive bot I mentioned above, people have been able to take advantage of the high latency and predictive modeling to invent new strategies, such as changing one's direction frequently so that the character model showing up on other's screens is rarely in the correct spot where the player is actually located.
When did they add pitchforks to Skyrim? Or is this in Elder Scrolls Online?
Trying to become famous by taking photos. Visit my homepage please.
Then I guess I'll have to deal with it the way I deal with "always online" DRM:
I'll have to abstain.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
I thought Bioshock was bad when they made a game that you literally could not lose at. It was impossible to die or fail.
It worked out quite well for Ron Gilbert and Monkey Island in 1990.
The pleasures to be found in playing a game like The Dig or Grim Fandango lie in exploring the worlds their authors create.
If you begin with something as richly imagined as Rapture or the airship city Columbia the mistake is trying to shoehorn the game into the narrow confines of a first person shooter,
Why on earth would you want to do this? Run the damned thing locally and be done with it. If it aint broke...
Ummm......online gaming?
Or do you honestly expect Microsoft, Sony et al to put their servers in your livingroom?
Please RTFA or at least RTFS before you hit the Post button....
Back in the 1990s, Microsoft developed something similar. Their idea was to render frames in layers, with the more distant or less active layers rendered less often. if the viewpoint changed, the background layers were scrolled, rotated, or transformed to match, rather than being re-rendered immediately. It never caught on, because graphics hardware became fast enough to re-render everything on for each frame.
This new thing is similar. Mispredicted frames are viewpoint-warped as a temporary measure so the user sees something. The image is wrong, but close enough to look OK until a new rendered frame is sent. It looks OK for Doom, on which it was tested, because Doom is mostly about the shooter and the opponents moving; there's not much general activity in the background. GTA IV/V would probably look much worse than normal.
The whole concept represents a desperate attempt to make something "cloud-based" that shouldn't be.
they pretty much only need to guess what happens if you pressed lmb.
of course, if they were testing with something with only digital on/off inputs, the whole thing becomes much more easier than sending 100 different frames for 100 different possible mouse moves(and calculating those frames at the server end too! it doesn't only need more bandwidth it needs the game that is being streamed do a lot more.. and have it's engine rewritten too... which makes this a lot less appealing).
it's the analog inputs that make this a fubared concept... of course they can guess that if you're moving the mouse at one frame to one direction you might be doing the same thing the next frame.
world was created 5 seconds before this post as it is.
Of course they can - you do it all the damn time!
Bioshock games are story-driven. You fight to bring a sense of immersion and interaction, but a fight on which the player can be stuck for hours would become a barrier to the story rather than a way to draw them in.
Compression quality can increase with longer processing time. If they are pre-processing frames then they can spend a few extra milliseconds on compression encoding and decoding quality.
It is double blind so we can control for people being nice.
Graphical quality is subjective, so people are going to be influenced by whether they enjoyed it more. Chances are they'd also report better sound effects, and even a more enjoyable journey home.
There are still people with bandwidth limits?
Are you sure you don't live in the past?
Download limit rather.
Assuming you can do just about anything in a game, yeah nobody can predict that.
On the other hand, in a game, with an "open world" design that actually has a corridor structure, yes they can predict a lot of movement, choices etc.
The system may backfire if you intentionally do things unrelated to game advancement but that's a different story.
A 'singular oddity' is an event that cannot be explained and only happens when you are alone.
Why on earth would you want to do this?
For example to stream windows only games from my basement server to lightweight linux-based HTPC in my living room.
Pff you dont get it.
Microsoft will be going all the way to 200 MS with this amazing new technology!
Your move Apple and Google.
Mod me down, my New Earth Global Warmingist friends!
Regular online games don't have this problem as frames are rendered locally. Only streamed games will benefit benefit, and that's what the question was referring to. His question isn't addressed in the article, so your call to read it is completely irrelevant.
Publishers treat the entire continent of Australia as a rounding error!
250ms ping to a US/EU server is about average since very few of the major titles have servers in Oz, SE Asia or Hawaii is normally is good as it gets from the arse end of the Earth. You know you have a really clean connection if it's under 200ms.
The lag from Oz to anywhere else on the planet is largely a true physical limit imposed by the speed of light, no amount of bandwidth will fix it. Anything that "tricks the users brain" into not perceiving that lag would be welcome, 4X the data for a game won't be noticed with the average data cap we have here these days. OTOH since the birth of online gaming I've been trained by experience to lead my shots, so it does mean I would have to significantly adjust my playing style to take advantage of a lower ping.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
Apparently the authors of this research have never actually driven a DeLorean. It was stainless steel, weight 3000lbs, and only has 130hp.
So basically it's a car the weight of a Buick with the engine of a Geo Metro.
This has been around for a while now, the only difference here seems that is it's being applied to streaming games and done on the server side for the player, instead of the player side.
Twinstiq, game news
Where I see this as highly beneficial is WebGL based games. You can construct most of each scene on the server, ship it to the browser & have the browser do the final rendering. But to do the scene construction, you'll need to know each player's actions. If you can predict this, then you can ship several different based on your predictive algorithms, then have the browser render the one that closest matches what was actually done. Since DeLorean includes a corrective depth & rotation matrix, you can avoid some of issues with mis-predictions.
I kinda want to see how they implement it so that I could play with it. Plus, I wonder if it could be applied to other branching activities.
I think your very last point is one of the biggest ones - the inability to cheat is a big reason why lots of games are pushing parts online. It has the downside of requiring a more powerful server (and puts full trust in the server), but it can prevent many cheats right off the bat and makes it easier to block others. Players like this because cheating sucks.
You have problems with consistent client behavior over 140 ping. This research document supposedly went up to either 200 or 400, depending on what RTT means.
Here is the real problem: What happens if you introduce random packet drop to the system? Over mobile broadband, shitty long line connections, Australias landline, and more, you get packet loss. A small amount with a random amount added.
Does this research paper even touch on the subject?
Quakeworld had limited prediction as an "improvement" over original Quake. It had a tough time with rocket jumping. More often than not the screen would freeze and a few seconds later, you'd find yourself running in a corner, the jump never having occurred, or missed badly.
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
There are still people with [download] limits?
Yes. Comcast still has the 300 GB per month limit in many markets, and cellular has a cap two orders of magnitude smaller than even that.
Are you sure you don't live in the past?
For someone born in a country whose home Internet pricing expectation is stuck in the past, such as the United States, Canada, or especially Australia or New Zealand, it can be expensive and a pile of red tape to relocate to a country in the present.
I think the point was that additionally requiring "appropriate Internet connectivity" limits the market for the product too much compared to requiring only electricity and a computer. The same is true of requiring a bleeding edge computer instead of the computer one is more likely to already own.
If you've already pushed your joystick towards the mammoth AND are known to frequent slashdot, one could speculate that you will be crazy enough to strip down and wield a pitchfork.
Wait, you said fork. Hmmm..that's a tough one.
My God can beat up your God. Just kidding...don't take offense. I know there's no God.
Selling a million game consoles will produce less benefit to fewer users than taking the equivalent hardware and putting it into a server farm.
A console in the average game console owner's home sits unpowered for the vast majority of the day. That hardware in a server farm will be used 100% of the day, meaning more hardware will be available for more people.
A server farm can be far more energy efficient than a million consoles.
Obviously there are tradeoffs, but to dismiss the concept outright is not helpful.
All my liberal friends think I'm a conservative, all my conservative friends think I'm a liberal.
This is how I always explain streaming games to people who can't immediately see the horrible problems with it:
Imagine if the Ubisoft always-on DRM were an inherent, unremoveable aspect of the game system rather than just something tacked on to a few individual games after the fact, such that Ubisoft couldn't even begrudgingly neuter it in a patch. Well, a streamed game is even worse than that would be.
All you get is streaming video/audio and all the lag you'd expect (including controller lag), which is a recipe for disaster in North America. And any interruption in the connection that lasts more than a few tenths of a second is going to be behave like the equivalent of a "freeze" or "hang" that you'd NEVER tolerate in a properly local-hosted game. Not even the most twitchy DRM existing today has that problem.
Some people consider IPS monitors unsuitable for games requiring fast reflexes (i.e. FPSes) due to their double-digit response times. Internet latency is often worse and certainly more unpredictable than LCD monitor response time, and with Onlive, etc. it applies to audio and keyboard/controller/etc input too.
Then there are the bandwidth requirements.
Let's say you're lucky enough to have a 30mb/s connection. Why would you want to use it to transfer your game's video instead of, uh, a DVI cable, which is capable of 4 Gb/s? The people who developed DVI apparently understood that that 1920 x 1200 pixels w/ 24 bits/pixels @ 60Hz results in bandwidth well over 3 Gb/s. The people who developed streamed games seem very, very confused (at best).
Those of us who know anything about bandwidth and compression and (especially) latency can see the enormous technical obstacles facing a service like this, and Onlive never did anything to explain how they intended to solve them. Instead, they ded everything they could to lock out independent reviewers with NDAs and closed demonstrations. A friend of mine described it as the gaming equivalent of the perpetual motion scam, and IMO that's spot on (except that Onlive would still have the draconian DRM issues even if it worked perfectly).
Streamed games appear designed from the ground up to benefit the game publishers and fuck the customers, exactly what you'd expect from any DRM system.
I would think one benefit would be that you can allow for a subscription service where people could try lots of different games or play portions of games without having to download hundreds of gigs of data for games they only play a few times or when they are only playing a specific mission and don't need all the maps or game modes available.
I'm not saying this is what everyone wants or that it works for all use cases, but I imagine this is one scenario.
My God can beat up your God. Just kidding...don't take offense. I know there's no God.
graphics hardware is so cheap these days. Even your $85 Amazon Fire TV can render some pretty awesome games...and it just keeps improving...so it seems like the real benefit isn't graphics per se but in AI and CPU processing, in maintaining massive worlds, and in enabling a uniform gaming experience across platforms/clients.
My God can beat up your God. Just kidding...don't take offense. I know there's no God.
even so you can download an game off peak when some ISP are cap free
If you plan to go this route, satellite is in my experience far more likely to include unmetered off-peak use than cellular.
and you can say download an game / parts of it at places with free WiFi.
If your computer happens not to be a laptop, which is likely for a gamer because laptop GPUs tend to be underpowered in both senses, watch people point and laugh at someone bringing in a desktop computer to download a game. That's the vibe I get from Not Always Right, Geekologie, and Paradoxoff.
I'm sure there's a pitchfork mod. If not now, within the next ten minutes at least.
Publishers want to rent you a game, not sell you one. Renting is lower up-front cost so it gets more people looking at the game (no one makes demos anymore), and those that get hooked on the game spend much more than the normal $50 price.
I think these are all single player games, so lag vs fairness wouldn't matter so much.
you are chasing a wooly mammoth buck-naked with a fork. we determine you need titanium toe rings with 6-carat blue diamonds. why? hell, just because. your credit card on file has been decremented ten million dollars. now your avatar jingles as you run.
perhaps you want an elephant gun now? please enter a new valid credit card number...
if this is supposed to be a new economy, how come they still want my old fashioned money?
Soon enough these streaming game services will be sending entire binaries to run on the client in order to render something quicker. First it will be the background binary, then pre-send all the textures. Soon enough the entire game will be running on the client.
Custom electronics and digital signage for your business: www.evcircuits.com
I think they're moving the mouse a "typical" amount and rounding the predicted move command to either no motion or the predicted amount of motion. Besides, in first person games, screen motion due to mouse panning is easy to predict, as John Carmack pointed out.
Let me count the guesses: A publisher paranoid about prohibited copying may be willing to license its game at a lower price...
LOL no. They will just see the increased profits as profits that were not lost to piracy.
What they fail to understand is this will not bring increased revenue or increased profits... but that is for another discussion.
"Someone needs to talk to the tree of liberty about its ghoulish drinking problem." by ohnocitizen
You probably already saw it, but Comcast is trying to weasel around it by calling them overages instead of caps.
But this is specifically about streamed games. Locally rendered games (as with WebGL) can render the scene immediately in response to the player's input, so they don't have this problem. The lag is still there, but you only have to deal with the physics side of it, rather than dealing with the physics *and* the audio and video.
The scenario you describe doesn't make any sense because the idea is that you choose the frame (or scene) based on the inputs on the local machine. So if you're rendering it anyway then you may as well just have the local client respond itself to the inputs and do away with the overhead.
Actually, the scenario I was describing is one that would benefit most multiplayer games, even locally rendered ones. With multiplayer games you aren't just rendering 1 scene & responding to the inputs from the one viewing the scene, but several players' inputs all at once, then have to make sure that the scene for all those players are correct. You can see the problems lag introduces when you do a large portion of the calculations on the server(as to prevent or deter cheating) in games like FFXIV(Google "FFXIV Titan Extreme Lag Issue" to see what I mean). Even when nobody has crippling lag issues, it's still evident when you see your entire party burst move at the last second to avoid a killing blow, when you know that they would have moved at the same time & speed you did. And this is a game where not only is the scene rendered locally, but all the primitives are available from local sources as well.
Using these predictive algorithms for player input & scene construction, would allow you to ship several scenes to each player, showing not only their immediate actions but the predicted actions of the other players. By predicting the actions of each player, the server could then more correctly account for those actions even when lag issues might have prevented the player from reacting the mob's attack quickly enough. As it stands right now, due to lag you might not even see the attack happen, even if it has a long cast/charge time, before you are hit with it.
The predictive algorithms for the scene would be deterministic, so you may as well perform them on the local machine. I still don't see how you propose this would work, so maybe I should ask this question specifically: What would be the criteria upon which the local client would select one of the pre-constructed scenes?