Ask Slashdot: Is There An Open Source Tool Measuring The Sharpness of Streaming Video?

dryriver asks: Is there an open source video analysis tool available that can take a folder full of video captures (e.g. news, sports, movies, music videos, TV shows), analyze the video frames in those captures, and put a hard number on how optically sharp, on average, the digital video provided by any given digital TV or streaming service is?

If such a tool exists, it could be of great use in shaming paid video content delivery services that promise proper "1080 HD" or "4K UHD" quality content, but deliver video that is actually Youtube quality or worse. With such a tool, people could channel-hop across their digital TV service's various offerings for an hour or so, capture the video stream to harddisk, and then have an "average optical sharpness score" for that service calculated that can be shared with others and published online, possibly shaming the content provider -- satellite TV providers in particular -- into upping their bitrate if the score turns out to be atrociously low for that service....

People in many countries -- particularly developing countries -- cough up hard cash to sign up for various satellite TV, digital TV, streaming video and similar services, only to then find that the bitrate, compression quality and optical sharpness of the video content delivered isn't too great at all. At a time when 4K UHD content is available in some countries, many satellite TV and streaming video services in many different countries do not even deliver properly sharp and well-defined 1080 HD video to their customers, even though the content quality advertised before signing up is very much "crystal clear 1080 HD High-Definition".
What's the solution? Leave your thoughts and suggestions in the comments.

And is there an open source tool measuring the sharpness of streaming video?

Comments

Copy protection?

[mosschops]

You'll probably run into copy-protection issues with many media sources. If you can get at the bits to analyze it, you can dump them out to create a copy. I'm not even sure sharpness is going to be easy to measure, if a soft/low-res source has been artificially sharpened to give the appearance of quality. Sports broadcasts seem to be the worst for it for it from my experience, with glowing halos around high-contrast edges to make them stand out even more.

Re:Copy protection?

[Jane Q. Public]

While it may be true that some TVs are not good at interlacing, interlacing in fact was designed to smooth motion, not blur it.

Theoretically, assuming your hardware is capable of doing it properly, a progressive-scan (p) video would need to have nearly twice the framerate of an interlaced (i) video, in order to achieve the same motion smoothness.

Re: What the hell?

[Anonymous Coward]

If you need a tool to tell, does it really matter?

Sharpness is an illusion

[Solandri]

It created by degrading the video in a process called unsharp masking. Basically you find the edges between light and dark objects, and exaggerate them. You make the light side lighter on that edge, and make the dark side darker. This adds no information, in fact it degrades video quality from the original unsharpened version (loses information).

But your brain has special cells which recognize transitions between light and dark, and identifies them as an edge. When the exaggerated transition from an unsharp masked edge hits those brain cells, they get more excited and signal the rest of your brain that this is a really strong edge. Thus creating the illusion that you are seeing a sharper image, when objectively it's a degraded image.

So any algorithm which detects "sharpness" as interpreted by the brain would actually rate inferior (heavily processed) video streams higher than video streams conveying the maximum amount of information possible. It's why unsharp masking is typically added by the TV or video player, rather than incorporated into the original video stream. (A slight amount of sharpening is done to counteract the blurring caused by the Bayer filter used in camera sensors, but that's another story.)

Re:Sharpness is an illusion

[Anubis IV]

Sharpening the image is an inherently lossy process. You're replacing a more detailed image containing a soft edge with something that has a crisp edge with less gradation. You can't then reverse the process to get back to the original image, since there'd be no way to tell the difference between things that actually had a sharp edge in the original image and things that had merely had their edges sharpened after the fact. They'd both look the same.

Frankly, sharpness is a really lousy measure for determining how good an image is. It doesn't make images look better, in and of itself, and it's something that the user can already introduce themselves if that's where their preferences lie, simply by bumping up the sharpness on their TV. Of course, if your goal is to have the highest fidelity image, you shouldn't set your TV's sharpness to anything over 0. Back in the day of analog CRTs it used to be the case that sharpness might improve the fidelity, but ever since the switch to digital LCDs anything more than 0 adds sharpness that wasn't there in the original image, thus decreasing the visual fidelity of the resulting image.

Re:Sharpness is an illusion

[MrL0G1C]

Because a sharpened image over-writes some parts of the image with false information, see a good example here:
https://www.youtube.com/watch?...

Video metrics

[sanf780]

There are already known metrics, like PSNR and SSIM. The thing is, these metrics are used when you encode the videos from the raw material and there is a tradeoff between bitrate and the desired metric.

The only thing you can more or less do on the broadcast video is to identify encoding artifacts like macro blocks, mosquito noise and deinterlacing side effects. MPEG2 streams are easy to identify, H264 and H265 are slightly more difficult. I do not know of a package solution to this. Be aware of news stations that use low fi mobile feeds!

Try this

[wolfheart111]

It was on G... could have done it yourself but, here u are. http://www.compression.ru/vide...

Already a solved problem

[Anonymous Coward]

There are already algorithms for doing this, like PSNR, SSIM, and VMAF. Here's a good read: https://medium.com/netflix-techblog/toward-a-practical-perceptual-video-quality-metric-653f208b9652

And there are already commercial solutions that apply these metrics as well as network analysis etc. across a range of video delivery methods, e.g.:https://www.telestream.net/iq/

Re:Already a solved problem

[DoctorBonzo]

The referenced netflix blog is probably the best answer you'll get...

What you really want...

[Graymalkin]

Is a way to tell not that the video is "sharp" but closely matched the original source in fidelity. If all you looked at was sharpness a scene that had motion blur or an out of focus effect would run afoul of the algorithm. Your end goal of name and shame is incredibly difficult to do well or even accurately.

There's ways to get video quality measures (SSIM, MSE, etc) but they require a comparison to the source (or a source you consider good enough) as they're relative measures. Interpreting the results is also not obvious.

Besides the challenge of the comparison it's also important to understand the video pipeline from raw source to what you see on screen. The video stream from the provider might be providing a high quality stream but the scaler on your TV might suck and muddies the image upon display or your LCD panel's dithering might be really shitty.

Video, especially streaming video (either Internet streaming or live delivery like cable), is really complicated. There's a lot of different dimensions where you might judge "quality" and even then it's an envelope and not a single scalar value. There's no objective "good" reference for any recorded scene, even the concept of "life-like" is not clear since the recording is entirely dependent on physical properties of the equipment.

Just use bitrate.

[sonoronos]

There's already a metric that basically defines video sharpness: bitrate.

The sharpness of H.264 and H.265 is very well known. Since commercial streaming services use commercial streaming video codecs, it's a pretty safe bet that you can almost directly correlate resolution to bitrate.

There's virtually no incentive for streaming companies to deliver lower resolutions at higher bitrates. It would be a technical challenge to deliver higher resolutions at higher bitrates.

Therefore, bitrate is most likely the simplest and most accurate measure of streaming video sharpness.

Re:Wait a second...

[guruevi]

A video is a set of images. Sharp images = sharp video.

What the poster is asking for is not a measure of "sharpness" but a measure of quality.

Re:What the hell?

[Anonymous Coward]

sounds pretty simple to me: submitter has amassed a ton of pirated media with many duplicates. now they want an automated means of choosing the highest quality files (bit rate and/or resolution alone won't necessarily do that) from those dupes so they can trash the rest.

Why sharpness?

[thegarbz]

Image quality is not determined by sharpness. This idea is the reason why those horrible TVs apply a post process sharpness increase to content (the first thing you should turn off when you buy a TV).

Quality covers a large number of metrics including artifacting, posterisation, loss of colour fidelity, and loss of contrast ratio to name a few.