Slashdot Mirror
Apple's Working on a Powerful, Wireless Headset for Both AR, VR (cnet.com)
Apple CEO Tim Cook has nothing but praise for augmented reality, saying it's a technology that's potentially as important as the iPhone. It turns out he may have big plans for virtual reality too. From a report: The company is working on a headset capable of running both AR and VR technology, according to a person familiar with Apple's plans. Plans so far call for an 8K display for each eye -- higher resolution than today's best TVs -- that would be untethered from a computer or smartphone, the person said. The project, codenamed T288, is still in its early stages but is slated for release in 2020. Apple still could change or scrap its plans. It's notable that Apple is working on a headset that combines both AR and VR given its intense focus over the past year on pushing augmented reality in iPhones and iPads. Cook has said he sees bigger possibilities in AR than VR, partly because augmented reality allows you to be more present. Either way, it's vital for Apple to expand beyond its iPhones, currently its top moneymaker, and the slowing mobile market.
Comment removed based on user account deletion
If it's going to be a powerful, wireless headset, then I want it.
But I have a question. Is that power going to help me and my friend take over the world, or will the headset take over me?
Thanks,
Pinky
So it will only run at 30FPS 1080p.
I am on RR (real reality). No matter what Disney says, remaining a child my whole life is probably not the best thing for neither your personal well being nor the state of the world.
And it will run on almost all the Apple hardware, except the MacBook pro.
Good luck getting it to run on anything not Apple.
Big. Fucking. YAWN!
I object to power without constructive purpose. --Spock
Well hopefully this time Apple won't get credit for inventing it just because they added a few UI elements like they did for the smartphone.
Comment removed based on user account deletion
Powerful hardware isn't necessary when they have eye tracking and foveated rendering. The human eye only sees high resolution in a tiny spot in the center, everything else can be rendered at much lower resolutions.
There you go, champ!
#DeleteFacebook
AR doesn't completely fill the view, they only have to display the virtual objects/environment/etc.
It also depends on the level of details they're targeting. Just because they want to use 8K displays doesn't mean the end result has to look like the best games on the market today. For all we know, they could very well use flat shading with no textures, etc. The use of 8K may only be to avoid having "pixel edges".
#DeleteFacebook
Are you bullied, mentally handicapped or both?
And? Was life a contest about who lives the longest?
It's not, as long as you're not under-aged and you can smoke in moderation
Who forces you to buy a pack every day, fucking peat-gavel
Low-quality bait
Nothing to see here..
More vapor...
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
Like all your whiny victim complaints about liberal college professors holding you down lol...
Doesn't matter, it's too big. I just got a 4k TV and the thing is like 42 inches, and now they want me to strap something twice as big to each eye? No thanks.
Exactly.. sounds like someones stock is going down and they need to tell tall tales to pump it back up again. I thought apple was doubling down on leaks? I hope someone got fired for this one.
Found the guy who has no idea what vaporware means.
Maybe next they can work on a powerful escape key and usb port on the macbook pro.
Do you have ESP?
"Links or it didn't happen."
Doesn't matter, it has no headphone jack.
Latency is the problem. A wireless system will have to compress the video, transmit it with extra error correction data, and decode it for display.
It also adds more latency to the head tracking sensors.
VR is extremely sensitive to latency. Not to mention power hungry.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
"2 x 8k screens at, say, 90hz (minimum needed really) is going to require some seriously hefty hardware."
Indeed, right now, the helmet weighs around 800 pounds, so IOW, the helmet wears you, not the other way 'round.
At a minimum they can render at a lower resolution on the outer areas of the picture, then the scaling is relatively cheap. This saves on resources as peripheral details are less noticeable than what's front and center.
There's also this little feature released by nvidia a couple years ago
https://www.roadtovr.com/nvidia-explains-pascal-simultaneous-multi-projection-lens-matched-shading-for-vr/
It's geometry and projections related ; avoid rendering unneeded pixels.
Foveated rendering? That would add further savings but I doubt it's practical. You would need a ridiculously low latency, I don't believe in it. Perhaps a very broad notion of where your eyes are looking can be used to determine which broad areas are rendered at half res.. That'd be all.
If you're still reading I bet you're still not convinced at dual 8K display. Yes it sounds nuts.
It might be workable for smart-looking AR infographics, both relatively inexpensive and not covering much screen area.
Might work for some simple full graphics, but a complex games would be rendered at 3x or 2x or 4x lower resolution i.e. 16x or 9x or 4x fewer pixels, or a mix. Render your game at 2560 pixels wide = 9x fewer pixels. No LCD pixel grid artifacts at least.
How Apple would do it at all : well, see iPad Pro, iPhone 8 and X etc. They flood the foundries (TSMC, Samsung) with cash and have like a one year lead on process technology, probably get a ton of wafers made with many corrupt chips. But what works they sell at high margin.
They says it'll use a "5nm" process (thus, done with EUV lithography). Will likely make some largeish custom CPU+GPU, run at suitable low voltage where the performance per watt curve is the highest. HBM3 memory technology gets you hundreds of gigabytes per second in a small space and high capacity like 16GB.
Might be that they saw an opportunity because of their logistics : they already bought their way to being the first on new silicon processes. They also make CPUs only for themselves (they could release phones and tablets with two or three fast ARM cores while the competition was using eight slower cores. Guess what's better to run a browser etc. on a flagship device)
So they may be trying a first mover advantage of sorts on the market of AR/VR in a self-contained headset (others will do, but with lower end lower res hardware)
If they are scaling with reduced rendering then it isn't fucking 8k each eye is it. don't get me wrong their is no point in 8k per eye but I am not the one claiming that is what they are doing.
Why? Outside of valley nerds, I don't think anyone is clamoring for one. Most people don't care to strap gear on their face to accomplish normal tasks at half the efficiency. VR and AR will never be anything outside of niche gaming, just like smart watches are a niche product. Modern Apple has no vision, zero.
LCD or OLED screens etc. don't need an "Hz".
Unless you play a game and want the aim of your cross hair displayed accurately, and for that 40Hz is more than enough. The synchronization of your mouse with what you see, and via internet, were you aim at, is 100 times more important than the "refresh rate" of your screen.
Hint: the old 50Hz/60Hz CRT screens projected "half images" with that rate, and that did not really have any relation to online games or cross hairs but was the technological limitation of the CRT itself. Phosphor made to light uo when an electron beam hits it.
A modern screen does not to refresh at all, unless the video buffer changes. (Which is obviously the case when you watch a video, but a rare occasion when you read a web site)
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
A headset isn't SteveJobs launching the first iPhone category killer, industry disruptor nor bet the company innovation.
The worst part of all is: Current VR is changing fast. Pop out your GTX1080, throw it away (or ebay), and pop in a new graphics card next year. What are you going to do for this Apple crap-VR? Throw away the whole device and buy a new one.
Not only that but they're talking dual 8k screens. At 90Hz that's 138 gigabits/second of real-world throughput needed in that wireless system.
Foveated rendering? That would add further savings but I doubt it's practical. You would need a ridiculously low latency, I don't believe in it. Perhaps a very broad notion of where your eyes are looking can be used to determine which broad areas are rendered at half res.. That'd be all.
Bandwidth of present day HDMI/DP links is already something like 2000 times that of the human optic nerve and yet this is still at least 100 times short of what would actually be required to drive a VR display to the limit of human vision using current photon spamming techniques.
Regardless of challenges of Foveated rendering it WILL HAPPEN no matter what. There is no viable path forward that does not involve foveated rendering.
Foveated rendering? That would add further savings but I doubt it's practical. You would need a ridiculously low latency, I don't believe in it.
What kind of ridiculously low latency do you think you could get if you only render a 600x600 image? How long do you think it takes for a saccade to track and stabilize?
I'd imagine you'd have two rendering paths, one for a low-resolution overall image, one to track the eye and render that 600x600 image at the predicted point that the eye will be looking at in the next millisecond. Update about 1000 times per second.
Thank you. It's like I was under the impression that you'd re-render the scene every 2ms (say), even if it's just that 600x600 window. but I guess I should see it as rendering the entire scene first (geometry and all) then you "refine" it every 2ms?
I bet the GPU would have the smarts to be told, render that 600x600 portion real soon now - it's not a dumb OpenGL 1.1 or DirectX 7 device anymore. [*]
Meanwhile you'll re-draw another entire scene some 12ms later (the next frame). But.. do you need a 500Hz display? for this arbitrary 2ms I made up. Well, again, 500Hz, or 1000Hz for a small portion of the picture!, with OLED up to the job.
I guess I don't have to rile up myself either : even if this is hard or will be supported by a subset of software, you're still able to run non-foveated rendering when the software can't do it.
[*] CPU/GPU seamless low-latency collaboration (what AMD seeked with the "APU") might be a useful feature by that I mean CPU and GPU talk together at super low latency like a dual CPU system, not manually setting up transfer over "distant" PCIe. Maybe I digress, comparing PCIe latency and human latency may be ridiculous or not on-topic. But what I want to get at is, there are likely new kinds of custom rendering pipelines to invent or develop that are not available for the general public yet.
You don't even need to update the scene more often, just go from low resolution to high resolution in that one small patch. I'd think updating the scene and doing a low resolution rendering every 8 ms would be plenty fast. If the stuff being done with ray tracing works out, the high resolution patch might even be done that way.
I'm guessing you'd do this with direct updates to groups of pixels on the screen, with no reference to a scan frequency, just how fast it takes to update 600x600 pixels.
The low resolution image does need to be there, the brain needs a target to track towards, but peripheral vision is really quite low resolution, and gets lower the farther you are from the foveal area, from around 1 pixel per minute to about 1 pixel per degree at the far edges. However, the eye can track VERY fast, so the resolution would need to be high enough not to cause tracking problems as you move towards the edge.
Sadly to use it, you'll be required to enter your password on an Apple keyboard. No human eye will ever see it in action.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
First of all, they depend on PC hardware on intel and nvidia. Both not capable of features advertised. Second, Apple is a niche market for PC users. No one gonna jump on band wagon AND change platform. Third. Market is already somewhat saturated. Best they can expect is to get traction. And they probably will fail with that too. Fourth, they don't support any common tech like Direct X or Vulkan. So devs will not be catering to them, because it would mean exclusivity to Apple which already years behind and has small user base. Would love to see em try. To bleed their resources.
Comment removed based on user account deletion
Before: secret plan to make something that everyone os doing, but inovative and polished. dont tell anyone when its ready so no shortcuts because we promissed some date.
Now: tell everyone about something that everyone is doing but with a couple of higher spec features which will be standard anyways in a few years (maybe the innovation and polish is secret?) and tell everyone that the project might be dropped.
well done Apple you have finally returned to becoming another bland of predictable company like days of pre ipod.
I know it's sarcastic, and I truly understand how difficult it is to know if you are going outside of the box, and if it will be profitable. Most times not so it's the bigger risk. I wonder if ai can hep- figure that one out.
Good one...
It will likely lack ports of any kind, and only use Apple protocols :-)
Your eyes are not able to detect a difference above about 30Hz - 50Hz.
The only difference, as I pointed out, is for aiming with a cross hair using e.g a mouse, where two or three pixels difference decide if you hit or miss.
Nausea is usually caused by your balance organs in the ear. You see something but feel nothing related to it, refresh rates of screens have no influence on that.
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
Comment removed based on user account deletion
Your link supports my point.
Frame rate is only relevant if you can relate something to it, like moving the mouse.
A normal LCD or OLED has no frame rate like an CRT, a static picture is a static picture.
Frame rates come from CRTs, were you needed to refresh the screen, 50 times a second to get a picture.
Traditionally that was done with half pictures. the first picture illuminating the odd rows on the CRT, and the next one the even rows. As soon as we reached about 100Hz (actually 70Hz) the screen did no longer have that shimmering effect.
Anyway, LCDs etc. don't have that shimmering effect, hence their refresh rate is not relevant.
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
Comment removed based on user account deletion
Our heads will never feel cold again!