Ask Slashdot: Is There An Open Source Tool Measuring The Sharpness of Streaming Video?

dryriver asks: Is there an open source video analysis tool available that can take a folder full of video captures (e.g. news, sports, movies, music videos, TV shows), analyze the video frames in those captures, and put a hard number on how optically sharp, on average, the digital video provided by any given digital TV or streaming service is?

If such a tool exists, it could be of great use in shaming paid video content delivery services that promise proper "1080 HD" or "4K UHD" quality content, but deliver video that is actually Youtube quality or worse. With such a tool, people could channel-hop across their digital TV service's various offerings for an hour or so, capture the video stream to harddisk, and then have an "average optical sharpness score" for that service calculated that can be shared with others and published online, possibly shaming the content provider -- satellite TV providers in particular -- into upping their bitrate if the score turns out to be atrociously low for that service....

People in many countries -- particularly developing countries -- cough up hard cash to sign up for various satellite TV, digital TV, streaming video and similar services, only to then find that the bitrate, compression quality and optical sharpness of the video content delivered isn't too great at all. At a time when 4K UHD content is available in some countries, many satellite TV and streaming video services in many different countries do not even deliver properly sharp and well-defined 1080 HD video to their customers, even though the content quality advertised before signing up is very much "crystal clear 1080 HD High-Definition".
What's the solution? Leave your thoughts and suggestions in the comments.

And is there an open source tool measuring the sharpness of streaming video?

Comments

Sharpness is an illusion

[Solandri]

It created by degrading the video in a process called unsharp masking. Basically you find the edges between light and dark objects, and exaggerate them. You make the light side lighter on that edge, and make the dark side darker. This adds no information, in fact it degrades video quality from the original unsharpened version (loses information).

But your brain has special cells which recognize transitions between light and dark, and identifies them as an edge. When the exaggerated transition from an unsharp masked edge hits those brain cells, they get more excited and signal the rest of your brain that this is a really strong edge. Thus creating the illusion that you are seeing a sharper image, when objectively it's a degraded image.

So any algorithm which detects "sharpness" as interpreted by the brain would actually rate inferior (heavily processed) video streams higher than video streams conveying the maximum amount of information possible. It's why unsharp masking is typically added by the TV or video player, rather than incorporated into the original video stream. (A slight amount of sharpening is done to counteract the blurring caused by the Bayer filter used in camera sensors, but that's another story.)

Video metrics

[sanf780]

There are already known metrics, like PSNR and SSIM. The thing is, these metrics are used when you encode the videos from the raw material and there is a tradeoff between bitrate and the desired metric.

The only thing you can more or less do on the broadcast video is to identify encoding artifacts like macro blocks, mosquito noise and deinterlacing side effects. MPEG2 streams are easy to identify, H264 and H265 are slightly more difficult. I do not know of a package solution to this. Be aware of news stations that use low fi mobile feeds!

Already a solved problem

[Anonymous Coward]

There are already algorithms for doing this, like PSNR, SSIM, and VMAF. Here's a good read: https://medium.com/netflix-techblog/toward-a-practical-perceptual-video-quality-metric-653f208b9652

And there are already commercial solutions that apply these metrics as well as network analysis etc. across a range of video delivery methods, e.g.:https://www.telestream.net/iq/

What you really want...

[Graymalkin]

Is a way to tell not that the video is "sharp" but closely matched the original source in fidelity. If all you looked at was sharpness a scene that had motion blur or an out of focus effect would run afoul of the algorithm. Your end goal of name and shame is incredibly difficult to do well or even accurately.

There's ways to get video quality measures (SSIM, MSE, etc) but they require a comparison to the source (or a source you consider good enough) as they're relative measures. Interpreting the results is also not obvious.

Besides the challenge of the comparison it's also important to understand the video pipeline from raw source to what you see on screen. The video stream from the provider might be providing a high quality stream but the scaler on your TV might suck and muddies the image upon display or your LCD panel's dithering might be really shitty.

Video, especially streaming video (either Internet streaming or live delivery like cable), is really complicated. There's a lot of different dimensions where you might judge "quality" and even then it's an envelope and not a single scalar value. There's no objective "good" reference for any recorded scene, even the concept of "life-like" is not clear since the recording is entirely dependent on physical properties of the equipment.

Re:Wait a second...

[guruevi]

A video is a set of images. Sharp images = sharp video.

What the poster is asking for is not a measure of "sharpness" but a measure of quality.

Why sharpness?

[thegarbz]

Image quality is not determined by sharpness. This idea is the reason why those horrible TVs apply a post process sharpness increase to content (the first thing you should turn off when you buy a TV).

Quality covers a large number of metrics including artifacting, posterisation, loss of colour fidelity, and loss of contrast ratio to name a few.