Apple's Lightning-to-HDMI Dongle Secretly Packed With ARM, Airplay

New submitter joelville writes "After noticing artifacts and a 1600 × 900 image in the output from Apple's new Lightning Digital AV Adapter, the Panic Blog sawed it open and found an ARM chip inside. They suspect that video bypasses the cable entirely and instead uses Airplay to stream three inches to make up for the Lightning connector's shortcomings."

Comments

Car analogy

[sinij]

Can someone please explain this with a car analogy?

Re:Car analogy

[EmagGeek]

It's like having a 300HP engine in your fancy new sportscar, but all it does is turn an electric generator that delivers 50HP to the electric drive motor.

Yet, they sell it to you as a 300HP sports car.

Re:Car analogy

[K. S. Kyosuke]

...that delivers 50HP to the electric drive motor...

...using microwaves.

Re:Car analogy

[MightyMartian]

How about a NasCar fan analogy? You think they have brains, but when you open up their skulls you find tiny Leprechauns jacking off to chrome hubcap advertisements.

Re:Car analogy

[romiz]

Don't get confused. The high-tech Intel interconnect once known as LightPeak is called Thunderbolt. Here, we are talking the proprietary, low-tech, USB-like symmetrical connector Apple uses on their recent iOS devices, whose name is on purpose confusing everyone with its better counterpart.

And from what we see here, it's markedly worse than the alternatives Apple shunned, but that were based on standards (MHL, USB3), because those would have prevented Apple from imposing drastic licensing conditions on accessory manufacturers.

Re:Car analogy

[PolygamousRanchKid]

. . . it's like opening the hood of your new car, and finding a team of miniature Steve Jobs' bike pedaling the drive train while chanting "Don’t let the noise of others’ opinions drown out your own inner voice," and blowing the smoke of hallucinogenic mushrooms out through the catalytic converter while burning their votes for the new Pope living in a Crystal palace in the sky over Apples new headquarters impounded at a dock in Amsterdam . . .

Who's been sleeping in my brain . . . ?

Re: Car analogy

[Anonymous Coward]

Airplay is a network streaming technology. The network can be wired or wireless.

Re:Car analogy

[thepainguy]

It's like a car company offering a stabilized phonograph in your car, for your ultra-high fidelity analog listening pleasure, and then not being able to make the interface between the phonograph and the stereo work and bailing and having the phonograph input through an FM band transmitter that plays through the radio.

Re:Car analogy

[Anonymous Coward]

The M5 does come from the factory with a 155mph speed limiter, actually.

Re:Car analogy

[Concerned Onlooker]

No, it's like 10,000 spoons when all you need is a knife.

Re:Car analogy

[fahrbot-bot]

No, it's like 10,000 spoons when all you need is a knife.

Don't be silly. There is no spoon.

Re:Car analogy

[ZorinLynx]

This is a silly analogy these days. There are modern automatic transmissions that are basically just automated clutch-equipped gearboxes rather than the standard torque-converter-automatic that saps power like crazy.

Those transmissions transmit no less power to the wheels than a manual transmission would. Not only that, but they can shift faster than 95% of people can shift a manual transmission, so unless you're a freaking NASCAR driver you're going to get better performance using one of these than on a standard manual tranny.

Also they often have paddle shifters or similar so if you want to shift manually you still can.

Re:Car analogy

[GrahamCox]

Thunderbolt and Lightning... very, very frightening indeed!

Re:Car analogy

[ThePeices]

How quickly you change gear makes absolutely no difference to performance. *When* you change gear is crucial, and no automatic gearbox can solve that problem.

So what you are saying, is that I can take 17 seconds to change a gear, but if I change it at just the right moment, ill lose no performance at all compared with somebody who changes gears in 1 second?

Sir, I am in awe of your logic.

Re:Car analogy

[thegarbz]

Next time you're driving behind me let me know by flashing your lights. When we take off at the traffic light going up the hill I'll put my foot on the clutch and then take a tea break while we count the number of horses powering my wheels while my foot isn't on the accelerator.

Question, do you actually believe the garbage you wrote? Automatic gearboxes can be tuned for performance to shift at the ideal spot every time. Maybe not in your shitty sedan but there's a reason why many motorsports use automatic gearboxes.

Re:Car analogy

[Minwee]

The irony is that a song called "Isn't it Ironic?" is not about irony.

Betteridge's Law of song titles clearly applies there.

Re:Car analogy

[damnbunni]

I can do that without even needing more than one make.

GTI ($26k model), Beetle Turbo or diesel version ($24.5k), Sportwagen diesel station wagon ($27k), Jetta sedan diesel or hybrid ($24k), CC ($32k), Eos $34.7k).

So that's six cars with an automated manual under $35k, and I didn't even have to leave Volkswagen.

Re:Car analogy

[mjwx]

How quickly you change gear makes absolutely no difference to performance. *When* you change gear is crucial, and no automatic gearbox can solve that problem.

So what you are saying, is that I can take 17 seconds to change a gear, but if I change it at just the right moment, ill lose no performance at all compared with somebody who changes gears in 1 second?

Sir, I am in awe of your logic.

You clearly dont drive a manual.

I can accelerate into a corner, put the clutch in, slow and gear down as I turn and use heel to toe whilst releasing the clutch as I come out of the turn for better acceleration. Admittedly, I do this at roundabouts more often than I should.

An Automatic gearbox has to wait until I start to accelerate to drop back down a gear. Automatic gearboxes are always reactionary, a good manual driver is proactive. There is no way, any current production auto can pre-empt what a driver is going to do. Maybe when we've invented enough AI but then again, if we have computers advanced enough to tell what a person will do with that regularity, a person will be sitting in the back sipping martini's whilst the car does all the work.

When I drive auto's, especially "sports" automatic transmissions, they always gear up when I put my foot down, then when they realise I've put my foot down they drop a gear and jump 2000 RPM. Not smooth at all.

What the GP should have said, is that when you drive a manual, you can be in gear before you need it, an Auto is always going to be in gear after you need it... then again if you drove a manual you'd know that (or be really, really crap at driving a manual, in either case my point stands).

Re:Car analogy

[Thumper_SVX]

Dual Clutch Transmission in a BMW 135i here. Long time manual driver prior to getting this car, and in fact my last car before it (BMW 545i) I went all the way to North Carolina to buy because it had a stick instead of an auto.

My car is really a truly clutchless manual that happens to have an automatic mode. When I start my car in the morning and shift it to "drive", the first thing I do is click the shifter into the gate to the left and then click up once. That puts the transmission in a pure manual mode that will hold a gear until you shift it. I could drive to work in first gear if I so desired and the transmission wouldn't upshift. Of course, I wouldn't be doing much good for my gas mileage either, but the point is that I could. There are paddles on the steering wheel, but I tend to use the shifter because it's natural for me to reach down there for gear changes anyway; I flip the shifter away from me to downshift, toward me to upshift. It's incredibly natural.

As for performance driving, I can anticipate the curves and downshift appropriately every time, far faster than I ever could with a stick and clutch. Sure, it's sequential in the same sense that my motorbike is; I can't go from 6th gear to 3rd, I have to click through all the intervening gears. However, the incredibly fast shifts mean that I can get from 7th (my top gear) all the way to 3rd in about the same amount of time as it would've taken me with a stick. I lose nothing.

And for those that say that shift times make no difference; I have dragged (on a track, thank you) two identical cars; one with the DCT and the other with a stick. We did two runs in our own cars and two in each others... the result was always that the DCT equipped car was a good 3/10 quicker consistently in the quarter. Part of that is final drive (the DCT has a different final drive that gets better acceleration at the cost of slightly worse gas mileage) but even calculating that in we figured the shift times were gaining 1/10 on the quarter. Not much, but still faster.

Having said all this, have you driven the most recent 8 speed autos coming out? I have driven a BMW 535i with the 8 speed and was incredibly impressed by that thing. Yes, it's a torque-converter automatic but the technology has come a long way. Modern automatics really don't lose anything to a stick unless you happen to be a professional race driver. Of course, that would have to have been a professional race driver before 1992 because almost all race cars today use sequential manuals or dual clutch transmissoins... the days of rowing your own gears on the racetrack have been over for decades.

Security?

[durrr]

So I guess it may be possible to reprogram the ARM chip to maliciously invade the users computer.
Might it even be possible to turn the adapter into a minion of evil by just connecting it to your computer assuming you have the right software running?

So borrowing someones AV adapter can now be a security risk?

Re:Security?

[Anonymous Coward]

Wouldnt it be easier to do it on the device itself?

Re:Security?

[SuricouRaven]

It does appear, from what the speculation says, that the host device sends the SoC firmware when the adapter is plugged in. Hardly unusual: Propritary firmware blobs have been the curse of linux driver developers for years. RAM is cheaper than custom-masked ROM. If that is the case, then it may be possible to simply send a modified firmware (Unless Apple have done any sort of crypto-signing). The hacked firmware would have no way to communicate back and would be lost upon reset, so you'd need to solder in a tiny battery or ultracap too. Beyond that, though, there is plenty of room in that chip to save a few frames. Hack adaptor, lend to The Boss when he goes into the super-secret HR policy review board meeting, collect it back, extract presentation, get the inside word on who is about to lose their job and who is getting a fat bonus. It's a doable exploit in theory, though the level of difficulty involved - reverse engineering the adapter and the firmware enough to edit an evil version - that anyone capable of doing so probably has no need to. The type of exploit researchers might perfect purely to prove it can be done.

Re:Security?

[AmiMoJo]

A while back someone noticed that their battery firmware updates were encrypted but the password was embedded in the updater executable in plaintext. You could replace the battery firmware, and then if you found a hole in the EFI firmware or OS gain control of the computer. I wonder if they have learned from this mistakes.

Wireless wire?

[SCPRedMage]

Doubtful. More likely that it's streaming encoded digital video via the cable itself, and the components on the connector just decode the stream.

Perhaps this is a slight step forward, as far as technology is concerned, but it's a big leap back, as far as consumers are concerned...

Re:Wireless wire?

[adolf]

But yea basically they left the parts out o the newer iCrap and then charge you for more for capabilities the older stuff had.

Rather they charge more for less capabilities: The old device supported real, uncompressed video. The new adapter has MPEG artifacts and added latency.

Re:Wireless wire?

[AmiMoJo]

After they made such a big deal of the new dock connector, turns out is is inferior to their competitors. Samsung's modified micro USB connector does uncompressed full 1080p HDMI. The cables are dirt cheap too.

Re:Wireless wire?

[SuricouRaven]

The new dock connector is superior in exactly two ways:
1. Thinner.
2. You can put it in either way up... because the device has additional electronics to detect which way around the cable is and adapt accordingly.

The second of those is a triviality: It really doesn't matter hugely if you can put the connector in first time without looking. It saves the user only a few seconds at most. The first is the only reason for lightning. Consumer demand and Apple policy are towards thinner and thinner products, with Apple leading the charge: They introduced lightning for the same reason the Macbook Pro lost ethernet. The connector became the limitation on thinnness, so it had to go.

Re:Wireless wire?

[adolf]

Perhaps you just need a different TV.

Remember, HDMI is just a superset of DVI, which works generally works fine for a myriad of desktop computers.

My own Samsung A550 from a few years back does just fine with sync, and works very well with video games. Even with layers of potential latency bolted on (playing Super Mario Bros. on an emulator on a Wii outputting component video which is then turned back into digital video at the television and then scaled), it behaves just as well as I remember it with an NES hooked to a CRT.

For that matter, both of the DVI-connected monitors on my desk also show zero noticeable latency.

As to cables, the cheaper the better, in my experience (at normal lengths): I've had expensive HDMI cables with ferrite beads on them, and had no end of problems with them. I eventually emoved the ferrites (with a sharp knife and a hammer), and they've been working perfectly for years... The cheap freebie cables that come in the box with gear or from bottom of the barrel Ebay sales seem to all work fine.

I have seen some TVs lately that had real, unforgivable latency problems, and they all happened to have been made by Sharp. These needed audio delays added in the AVR to make a movie play correctly, and were essentially unusable as a computer monitor or for video games.

Whatever the case, blaming HDMI (which really cursed piece of DRM-encumbered shit) for the wiring or latency issues is a non-starter. You're pointing your finger in the wrong direction.

Re:Good engineering?

[maccodemonkey]

Thing is, MHL sends uncompressed 1080p over a cheap, standardized cable. Apple's standard, evidently, does not. And like you said, it's worse than the old docking cable in this regard. Regression is extra silly.

Looking at most MHL cable prices from vendors, they're cheaper than Apple's adaptor, but not cheap.

And as I mentioned, MHL drives up device prices because it requires additional circuitry in the device. Standardized cable you say? Try plugging an MHL cable into a Nexus 7. Won't work? That's because the chips required for MHL were too expensive and they were left off the Nexus 7.

Shifting half the expense to the device and half the expense to the cable isn't cheaper, it's just moving costs.

Re:Good engineering?

[ninetyninebottles]

Really, needing a computerized cable is just silly.

Actually, it's a step forward and it's not the first technology to do this. The basic idea is, make the port a smart interconnect and let a smarter cable be more adaptive. That way a 4 meter cable can be tuned differently than a 2 meter cable and you can use the same port for a cheap copper cable or a long but expensive fiber cable. Regardless of how relatively expensive the cables are, replacing the computer is harder and adding new ports to mobile devices, even most laptops, simply doesn't happen. This makes a nice, future-proofed port for your laptop, phone, peripheral, etc. that will have real longevity.

Do you even know what "serial" means?

[SuperKendall]

With its limited pin count, it's not a surprise that the Lightning connector does not have the bandwidth to transfer uncompressed video.

Good grief. How many pins, exactly, would you say are needed for a serial connection?

Now look at the end of any USB cable and the end of a Lightning connector. What is the pin count between the two?

micro-USB3 would have had enough bandwidth

Also look at how many pins are in a USB 3 connector (HINT: ITS THE SAME).

This issue has nothing to do with bandwidth from Lightning.

Re:Do you even know what "serial" means?

[SuricouRaven]

"Good grief. How many pins, exactly, would you say are needed for a serial connection?"

One, if you're operating an old telegraph. Eleven, if you're doing HDMI. Four twisted pairs for differential serial, plus three that are used for control information. Monitor resolution detection, that sort of thing.

http://www.hdmi.org/installers/insidehdmicable.aspx

Some devices appear to do it with less, but they are actually using MHL, not HDMI.

Of course it has a CPU in it.

[Animats]

Of course it has a CPU in it. Something has to do the protocol conversion.

It's not clear that Apple's AirPlay protocol, which has HTTP connections in both directions, is involved. But the pictures indicate compression artifacts. The original article doesn't go into enough detail to determine whether image compression (like JPEG) or motion compression (like MPEG) is being used. An MPEG compressor would introduce visible lag between the master and slave screens.

Never apply DRM to someone else's work

[Sloppy]

Whoa. Are you saying this is applying HDCP to everything it plays?

That would be very interesting, since if I made a video of my own and played it through this device, the television would be descrambling a technological measure which limits access, without my authorization. That's circumvention. This device from Apple, would cause the manufacture and sale of all HDMI compliant TVs to become illegal.

Smoking mushrooms? Talk about drug abuse....

[Ellis D. Tripp]

What a waste of psilocybin....

But the real question is what else can it do.

[Anonymous Coward]

I think we are missing the point a little here, They released a tiny computer for 50 bucks, now we just need a port of cyanogen for it.

That is certainly one way to look at it

[nbahi15]

Fact: Apple has an ARM processor in the cable. It is fair to assume the video is processed by the chip in the cable.

The rest of the facts in this case are just speculation:
* Is design a 'limitation', or a design choice?
* Is the 1600x900 output seen by Panic a Panic problem or an Apple one? Is it a bug or a limitation of the hardware? File a bug and find out
* Is the connector providing Airplay over the 6cm cable? Pure speculation. Sounds plausible, even clever, but that is just a guess.

It seems to me that there is certainly an interesting story in this adapter, but I don't think we know what that story is yet.

Re:Good engineering?

[AmiMoJo]

10 seconds searching Amazon turned up an MHL cable for £3.50, extremely cheap. The Apple version is £37.

Standardized cable you say? Try plugging an MHL cable into a Nexus 7. Won't work? That's because the chips required for MHL were too expensive and they were left off the Nexus 7.

I'm not sure how that makes it non standard. Are you saying for something to be standardized every device must support it? That's crazy talk.

Re:Disappointing for a new connector

[green1]

I don't think you grasp the actual advantage to lightning. It has one HUGE advantage that no other cable would provide. It forces vendor lock-in while at the same time instanly obsoleting all previous Apple cables. It's a marketing dream! (nightmare for users, but since when has Apple ever cared about them?)

Re:Stop the presses!

[Anonymous Coward]

Wow, not only did you not read the article, you didn't even look at the pictures, did you?

Stop the presses! The are scaling 1024x768 content to 1600x900,

The cable is advertised as doing "up to 1080". It does not.

and there are MPEG artifacts happening as a result?!?! The deuce you say! There's never artifacts when you scale things! Never, I say!

Did you look at the picture? Those are not scaling artifacts: there is noise around edges. Those look like artifacts from MPEG or a similar compression algorithm. If it was just scaling, it would introduce aliasing patterns, which is not what they are talking about.

Next thing I know, you'll be claiming that Apple didn't replace all the already transcoded content on the Inktomi CDN with new, higher resolution content over night!

What does that have to do with this discussion?

It's almost already too scandalous that they used a CPU and software to avoid having to design and spin silicon for a Lightning-to-HDMI converter ASIC.

In fact, it looks like they did create an ARM-based ASIC, which on the face of it is bizarre to find in something sold as "an adapter cable". It's obviously doing something much more than or quite different from your standard adapter cable.

I can only echo some of the sentiments expressed in the bad ratings they received in several reviews from owners of Samsung Televisions which improperly negotiate EDID information by failing to negotiate on input sources which are not selected at the time the device comes online. One would almost think this might be an issue for Linux systems when trying to use HDMI to output to Samsung equipment, or that Dish Network DVRs might have similar problems (with the fix being to plug the device into the input channel which is selected by default when the television is powered on).

EDID? Linux? What? The article doesn't mention those topics at all. It's talking about an ARM-based chip that was unexpectedly found in a new model of a supposed "adapter cable" from Apple that is providing results that are substantially inferior to what was available on older models of Apple's similar products. As a result, if you use this cable to attach your iWhatever to a TV, you get laggy, downsampled, artifact-laden video, where Apples previous products and products from their competitors deliver sharp, un-transcoded 1080p video.

Why and how it would be done (hypothetical)

[tepples]

Consider this hypothetical: Movie studios license their works to cable and satellite networks. The studios and networks want to measure to what extent HDMI playback from iDevices competes with cable and satellite TV. (In this case, playback on the internal display of a mobile device is considered complement, not competition.) So they get Apple to add something buried in the protocol between the iDevice and the adapter to measure this. The ARM microcontroller in the adapter measures the screen size of the device on the other end of the HDMI cable and reports it to the iDevice, which sends it to Apple the next time the device connects to iCloud.

Poster/Article is way off ...

[SplatMan_DK]

The electronics involved have nothing to do with AirPlay, and this is not "news" in any way. Sorry to ruin excitement and conspiracy theories... :-)

I am willing to bet serious money that all these chips do is decode whatever proprietary protocol Apple uses for transmitting video over the Lightning port to a standard HDCP protected HDMI signal. This is needed because the Lightning port has no other way of transmitting the video - and this has been clear from the day Apple revealed the Lightning port to the world. It is really just a high-speed 8-pin serial connector. Nothing else.

In addition the chips probably try to introduce a classic vendor lock-in factor, making it hard for 3rd party vendors to provide similar cables and accessories for the Lightning port without paying royalties to Apple (read: legal tech-extortion).

Also, the scaling-problems mentioned are without a doubt due to the screen-mirror scheme involved. If they streamed an actual 1080p video file directly, the result would likely be very different.

The speculation in the article is so far from reality it almost hurts... They get points for taking it apart and all, but they could have reached the correct conclusion merely by reading up on the existing specs of the Lightning port (if they had bothered to add a bit of digital-video knowledge from Wikipedia that is).

- Jesper

I would have had first post...

[decep]

...except my Apple ethernet cable needed a firmware update.

There is no Airplay involved.

[Anonymous Coward]

Airplay is not involved in the operation of this adapter.

It is true that the kernel the adapter SoC boots is based off of XNU, but that's where the similarities between iOS and the adapter firmware end. The firmware environment doesn't even run launchd. There's no shell in the image, there's no utilities (analogous to what we used to call the "BSD Subsystem" in Mac OS X). It boots straight into a daemon designed to accept incoming data from the host device, decode that data stream, and output it through the A/V connectors. There's a set of kernel modules that handle the low level data transfer and HDMI output, but that's about it. I wish I could offer more details then this but I'm posting as AC for a damned good reason.

The reason why this adapter exists is because Lightning is simply not capable of streaming a "raw" HDMI signal across the cable. Lightning is a serial bus. There is no clever wire multiplexing involved. Contrary to the opinions presented in this thread, we didn't do this to screw the customer. We did this to specifically shift the complexity of the "adapter" bit into the adapter itself, leaving the host hardware free of any concerns in regards to what was hanging off the other end of the Lightning cable. If you wanted to produce a Lightning adapter that offered something like a GPIB port (don't laugh, I know some guys doing exactly this) on the other end, then the only support you need to implement on the iDevice is in software- not hardware. The GPIB adapter contains all the relevant Lightning -> GPIB circuitry.

It's vastly the same thing with the HDMI adapter. Lightning doesn't have anything to do with HDMI at all. Again, it's just a high speed serial interface. Airplay uses a bunch of hardware h264 encoding technology that we've already got access to, so what happens here is that we use the same hardware to encode an output stream on the fly and fire it down the Lightning cable straight into the ARM SoC the guys at Panic discovered. Airplay itself (the network protocol) is NOT involved in this process. The encoded data is transferred as packetized data across the Lightning bus, where it is decoded by the ARM SoC and pushed out over HDMI.

This system essentially allows us to output to any device on the planet, irregardless of the endpoint bus (HDMI, DisplayPort, and any future inventions) by simply producing the relevant adapter that plugs into the Lightning port. Since the iOS device doesn't care about the hardware hanging off the other end, you don't need a new iPad or iPhone when a new A/V connector hits the market.

Certain people are aware that the quality could be better and others are working on it. For the time being, the quality was deemed to be suitably acceptable. Given the dynamic nature of the system (and the fact that the firmware is stored in RAM rather then ROM), updates **will** be made available as a part of future iOS updates. When this will happen I can't say for anonymous reasons, but these concerns haven't gone unnoticed.

Re:There is no Airplay involved.

[JDG1980]

The reason why this adapter exists is because Lightning is simply not capable of streaming a "raw" HDMI signal across the cable. Lightning is a serial bus. There is no clever wire multiplexing involved.

The HD-SDI standard can transmit a full, uncompressed HD signal over a serial connection. Why wasn't that used?

Certain people are aware that the quality could be better and others are working on it. For the time being, the quality was deemed to be suitably acceptable.

Any level of compression artifacts introduced at this level is unacceptable. We understand that HD video has to be compressed to fit into a sane amount of space, but up until now all cable formats have been lossless – this is a regression.

And why does your marketing literature say 1080p output when that is clearly not true?