Ask Slashdot: Is There An Open Source Tool Measuring The Sharpness of Streaming Video?
dryriver asks:
Is there an open source video analysis tool available that can take a folder full of video captures (e.g. news, sports, movies, music videos, TV shows), analyze the video frames in those captures, and put a hard number on how optically sharp, on average, the digital video provided by any given digital TV or streaming service is?
If such a tool exists, it could be of great use in shaming paid video content delivery services that promise proper "1080 HD" or "4K UHD" quality content, but deliver video that is actually Youtube quality or worse. With such a tool, people could channel-hop across their digital TV service's various offerings for an hour or so, capture the video stream to harddisk, and then have an "average optical sharpness score" for that service calculated that can be shared with others and published online, possibly shaming the content provider -- satellite TV providers in particular -- into upping their bitrate if the score turns out to be atrociously low for that service....
People in many countries -- particularly developing countries -- cough up hard cash to sign up for various satellite TV, digital TV, streaming video and similar services, only to then find that the bitrate, compression quality and optical sharpness of the video content delivered isn't too great at all. At a time when 4K UHD content is available in some countries, many satellite TV and streaming video services in many different countries do not even deliver properly sharp and well-defined 1080 HD video to their customers, even though the content quality advertised before signing up is very much "crystal clear 1080 HD High-Definition".
What's the solution? Leave your thoughts and suggestions in the comments.
And is there an open source tool measuring the sharpness of streaming video?
If such a tool exists, it could be of great use in shaming paid video content delivery services that promise proper "1080 HD" or "4K UHD" quality content, but deliver video that is actually Youtube quality or worse. With such a tool, people could channel-hop across their digital TV service's various offerings for an hour or so, capture the video stream to harddisk, and then have an "average optical sharpness score" for that service calculated that can be shared with others and published online, possibly shaming the content provider -- satellite TV providers in particular -- into upping their bitrate if the score turns out to be atrociously low for that service....
People in many countries -- particularly developing countries -- cough up hard cash to sign up for various satellite TV, digital TV, streaming video and similar services, only to then find that the bitrate, compression quality and optical sharpness of the video content delivered isn't too great at all. At a time when 4K UHD content is available in some countries, many satellite TV and streaming video services in many different countries do not even deliver properly sharp and well-defined 1080 HD video to their customers, even though the content quality advertised before signing up is very much "crystal clear 1080 HD High-Definition".
What's the solution? Leave your thoughts and suggestions in the comments.
And is there an open source tool measuring the sharpness of streaming video?
You'll probably run into copy-protection issues with many media sources. If you can get at the bits to analyze it, you can dump them out to create a copy. I'm not even sure sharpness is going to be easy to measure, if a soft/low-res source has been artificially sharpened to give the appearance of quality. Sports broadcasts seem to be the worst for it for it from my experience, with glowing halos around high-contrast edges to make them stand out even more.
If you need a tool to tell, does it really matter?
It created by degrading the video in a process called unsharp masking. Basically you find the edges between light and dark objects, and exaggerate them. You make the light side lighter on that edge, and make the dark side darker. This adds no information, in fact it degrades video quality from the original unsharpened version (loses information).
But your brain has special cells which recognize transitions between light and dark, and identifies them as an edge. When the exaggerated transition from an unsharp masked edge hits those brain cells, they get more excited and signal the rest of your brain that this is a really strong edge. Thus creating the illusion that you are seeing a sharper image, when objectively it's a degraded image.
So any algorithm which detects "sharpness" as interpreted by the brain would actually rate inferior (heavily processed) video streams higher than video streams conveying the maximum amount of information possible. It's why unsharp masking is typically added by the TV or video player, rather than incorporated into the original video stream. (A slight amount of sharpening is done to counteract the blurring caused by the Bayer filter used in camera sensors, but that's another story.)
Video quality is a function of codec and bitrate. It will always be a compromise between image quality, file size and decoding power. Even if the service wanted to serve users super sharp, high preset h265 + flac audio, most users would be unable to use it - file sizes would eat their download caps and their netbooks/phones would be unable to cope with decoding. Obviously on the service side they also don't want to pay for storage and transfer of those huge files, especially if they would be hardly watched.
The only thing you can more or less do on the broadcast video is to identify encoding artifacts like macro blocks, mosquito noise and deinterlacing side effects. MPEG2 streams are easy to identify, H264 and H265 are slightly more difficult. I do not know of a package solution to this. Be aware of news stations that use low fi mobile feeds!
It was on G... could have done it yourself but, here u are. http://www.compression.ru/vide...
[($)]
There are already algorithms for doing this, like PSNR, SSIM, and VMAF. Here's a good read: https://medium.com/netflix-techblog/toward-a-practical-perceptual-video-quality-metric-653f208b9652
And there are already commercial solutions that apply these metrics as well as network analysis etc. across a range of video delivery methods, e.g.:https://www.telestream.net/iq/
You need to check with the specific streaming services terms of use to see what they say about their streaming. Most likely most will have a way out if the video is not up to par. Also, sometimes the streaming video is based on the actual speed of your internet at the time of the streaming (Amazon Prime, Netflix, etc). In addition some service providers will throttle the stream - especially if mobile.
IMO, getting a measuring tool would really be a waste of time.
Is a way to tell not that the video is "sharp" but closely matched the original source in fidelity. If all you looked at was sharpness a scene that had motion blur or an out of focus effect would run afoul of the algorithm. Your end goal of name and shame is incredibly difficult to do well or even accurately.
There's ways to get video quality measures (SSIM, MSE, etc) but they require a comparison to the source (or a source you consider good enough) as they're relative measures. Interpreting the results is also not obvious.
Besides the challenge of the comparison it's also important to understand the video pipeline from raw source to what you see on screen. The video stream from the provider might be providing a high quality stream but the scaler on your TV might suck and muddies the image upon display or your LCD panel's dithering might be really shitty.
Video, especially streaming video (either Internet streaming or live delivery like cable), is really complicated. There's a lot of different dimensions where you might judge "quality" and even then it's an envelope and not a single scalar value. There's no objective "good" reference for any recorded scene, even the concept of "life-like" is not clear since the recording is entirely dependent on physical properties of the equipment.
I'm a loner Dottie, a Rebel.
There's already a metric that basically defines video sharpness: bitrate.
The sharpness of H.264 and H.265 is very well known. Since commercial streaming services use commercial streaming video codecs, it's a pretty safe bet that you can almost directly correlate resolution to bitrate.
There's virtually no incentive for streaming companies to deliver lower resolutions at higher bitrates. It would be a technical challenge to deliver higher resolutions at higher bitrates.
Therefore, bitrate is most likely the simplest and most accurate measure of streaming video sharpness.
You are probably not aware of how video streaming works nowadays, and you should read about adaptive bitrate streaming.
TLDR, nowadays content providers encode each video at different resolutions and quality levels (and bitrates), each with a different level of "sharpness", and the client selects the one that best fits its available network bandwidth.
This means that playing the same video several times may result in different quality each time. This also implies that the quality of the video you receive may be altered by your neighbor playing netflix (in the same Wifi channel), or your girlfriend passing between the AP and your receiver (reducing the signal level; both of them cause the AP to modify the modulation and coding scheme, reduce the available network bandwidth and may reduce the bitrate (read: "sharpness") of the following piece of video.
Also, please note that "youtube quality" is a nonsense, since Youtube delivers video with different quality levels (in some cases, up to native 4K).
Fundamentally, a 2-D FFT describes the dynamics present in any image. Sure, you could compare them between a known source displayed via different streaming methods on different screens. However, these numbers are meaningless on their own without taking the context of the viewer into account.
The FFT must include physical dimensions, starting with the screen's PPI value. So a 4K phone screen is inherently "sharper" than a 4K projector on a 20 meter movie screen. But we all know the movie screen offers a far better experience than a phone, more "immersive", independent of the perceived or actual sharpness.
The viewer's perceptions are also key. Starting with their ability to perceive. For the human eye to resolve a 4K image at the pixel level, that is to detect sharpness changes at that level, a person with 20-20 vision would need to sit NO FURTHER than a SINGLE screen diagonals away from the center of the screen. That's 65 inches away from a 65" inch monitor. If you aren't sitting that close, you aren't "seeing in 4K". So you have already started to lose "sharpness" independent of what you're watching.
And if you are sitting that close, you'll want a curved monitor, since the distance from the center pixel to your eye is much closer than the distances to the corner pixels. And you'll be watching alone, since there generally isn't space for two at that ideal viewing spot. So most of us get flat monitors and sit further away. Most of us sit so far from our screens that we aren't even "seeing in 1080p"! You'll notice that most TVs smaller than 32" are only 720p, which is because you really can't see more than that if your sitting at a normal TV viewing distance.
I *DO* sit that close. As I write this I'm sitting 55" away from my curved 65" TV/monitor. For most streaming content I lean back a bit simply because it makes no difference: The media sharpness is generally sloppier than what my eyes can perceive, so sitting back doesn't reduce the quality. That does change for well-rendered 4K content, where the original material is extremely sharp (directors and editors intentionally add blur to remove distracting levels of sharpness). That's when I lean in until my eyes can take in every detail.
If you aren't close enough, the content sharpness simply doesn't matter.
Dear Salty Dude,
Your picture quality is crap because streaming services dynamically vary the quality to match the available bandwidth. If you've got less than 20Mbps or if your ISP is throttling your 20Mbps connection so that you can only get 2Mbps from Netflix, then you're not going to see 4K(UHD). https://fast.com
Even if you could programatically characterize the quality of the video, your salty rants are not going to shame the likes of NetFlix and Amazon. So don't make a spectacle of yourself or waste your time looking for pointless code.
Get a better connection or ISP. If you can't do that, don't pay for HD and UHD streaming content because you're never going to see it on your shit connection.
Love always,
AC
And that's the law!
As others here already sort of mentioned; Sharpness is the wrong metric.
Another person mentioned "macro blocks".
That's what I was going to say. Compare to competitors using the same codec and more or less bitrate = quality. It's not a PERFECT measure, but it's pretty good and dead simple.
A video is a set of images. Sharp images = sharp video.
What the poster is asking for is not a measure of "sharpness" but a measure of quality.
Custom electronics and digital signage for your business: www.evcircuits.com
First of all... sharpness is something you see and experience. There's no way to measure it. In order to measure something you need a unit and that doesn't exist. Don't tell me whether it's 1080p or 4K or 5K. Those are not sharpness units. If you want to measure sharpness, at least you're expecting certain value which you don't even know because again there's no unit.
It is very annoying to watch video on you-tube only to figure out many of them are ripped off from someone else. These thieves alter the video so YouTube doesn't kill it by changing the speed, mirroring, and other methods to get it online. It tends to make it unwatchable.
FFmpeg can do PSNR calculations. The only problem with what you call 'sharpness' (in technical terms, we call it noise or signal-to-noise ratio) is you have to have something to compare it with. Movie and video makers often use 'blur' either as a proxy to indicate motion or to put emphasis on the thing in the story that the viewer should focus on. It's also natural for things that are in the 'focus' of the camera to be sharp and everything else to be blurry (unsharp). So measuring 'sharpness' of an image is kind of pointless since 75% of an image is typically blurry.
What you need is get a comparison stream (you could use XMLTV for live channels or a Content ID system for recorded things to identify eg. a movie scene) and then run it through ffmpeg's or avisynth's many calculators.
One of the few things that do happen during cable or satellite streams and which you could minimally calculate is duplicate frames, black frames and dropped audio. Ffmpeg has calculators for all of them.
One of the few proxies you could try but which would be very hard to quantize is simply calculating how 'hard' a frame or set of frames is to compress. Blurry video is more complex and thus harder to compress, you can easily calculate these peaks if your stream is expected to be very homogenous (eg. streaming of a static image). My idea here would be to use MythTV's commercial detection algorithm and/or use OpenCV to detect and focus in on the logo. If the encoding difficulty of the logo changes significantly for periods shorter than the length of a commercial, you may just have detected a frame where the quality dropped.
Custom electronics and digital signage for your business: www.evcircuits.com
No, itâ(TM)s not. Every encoder offers configurations that trade off speed against quality. If you donâ(TM)t want to spend much on hardware, or turn around time is important, youâ(TM)ll reduce the quality irrespective of bitrate.
Then live encoding is another issue. x264 is very CPU intensive, and watch the CPU spike during (Iâ(TM)m not sure which, but they could be the same) scene detection or I-frame generation.
sounds pretty simple to me: submitter has amassed a ton of pirated media with many duplicates. now they want an automated means of choosing the highest quality files (bit rate and/or resolution alone won't necessarily do that) from those dupes so they can trash the rest.
Measuring sharpness is possible, but that would mean nothing. There are many image sharpening algorithms. Any self respecting editor can play with sharpness any way he wants, and it is even built in some TVs and video players.
What you want is a measurement of quality and it can't be done without a reference. Think about it, if you can tell how close a video is to the reference without the reference, then the same algorithm could be used to reconstruct the original from a degraded video. And guess what, compression algorithm do exactly that: reconstruct a good looking video from as little data as possible.
So the answer to your question is simple: compress the video using a good compressor (ex: x256) on constant quality mode and measure the final file size. In fact the original bitrate may be a good indicator, no need to do anything. However, note that adding noise will increase both the bitrate and sharpness, and it is a technique commonly used to make things appear more detailed. Artefacts such as poor deinterlacing will also increase the bitrate and sharpness, so don't trust these metrics too much.
Yes, because you still need an objective measure of some sort, so that prospective buyers can judge the quality of each one before they subscribe.
Have max bandwidth use by a broadcaster around the world. No attempts on adsl, low end 3rd world consumer ISP.
Work out what the original material was created in, film, HD, 4K, 8K.
Find the same show globally.
Sort for the bandwidth, compression, chip sets used.
And only selling so much bandwidth to broadcasters?
Are 2nd and 3rd world nations broadcasters trying to fit many shows over a set amount of bandwidth they can use in that part of the world?
Its not a conspiracy to make TV look bad in 2nd and 3rd world nations.
Only so much "TV" will fit in the bandwidth allowed in 2nd and 3rd world nations. To make money more "TV" is made to fit so more ads get played. So it looks different to the HD and 4K products offered in advanced normal nations. More shows, more ads, more profit.
Why? People in advanced nations "pay" at lot every month for more the amount of "bandwidth" to enjoy HD and 4K content.
Want better quality? Try DVD. Buy Blu ray. Pay for a broadcaster that "buys" the needed bandwidth for their broadcasting.
Domestic spying is now "Benign Information Gathering"
That's true, one can decide to spend less CPU encoding.
Having said that, when widely comparing services such as Dish Network vs Comcast vs Frontier, that part would be different only if they are very stupid. For a given level of quality, the higher CPU encoding gives lower bandwidth. Some of us may be used to thinking of higher quality if it were the same bandwidth, but the flip side of that is it also means lower bandwidth for whatever quality level they accept.
Since they are going to transfer the stream to millions of users, the bandwidth difference is multiplied by a million for them. It's a no-brainer to use the CPU to encode well ONCE in order to save on bandwidth a million times. I don't know if I wrote that clearly. If you're encoding once and transferring to one destination, you might not spend the CPU to make the bitrate low / quality high. Given they are encoding once and transferring a million times, it would be moronic of them to spend thousands of dollars more on bandwidth in order to save pennies on CPU.
If we assume they aren't utter morons, they'll all spend the CPU time to get the lowest bandwidth they can for the level of quality they find acceptable. That's another way of saying that they'll have the best quality they can for the bitrate try used.
Since transmission over the last mile is the expensive part, why not just check the received stream's bps? You can assume that they're not going to upsample it and pay more just for the last mile.
Your ad here. Ask me how!
That reminds me of the abhorrent sharpness setting most TVs even including 4k TVs have.
Sharpness is not a measurement of quality, quality is a measurement of quality or more specifically accurate reproduction - how close does the stream match a high quality version such as a blu-ray 4k copy etc.
If you introduce measuring sharpness as a way of measuring quality then the end result is cheating that will artificially make all streams sharper at the expense of picture quality.
So, get it right, sharpness != quality.
Here is an example of how nasty the sharpness setting can make things: https://www.youtube.com/watch?...
Waterfox - a Firefox fork with legacy extension support, security updates and better privacy by default.
Image quality is not determined by sharpness. This idea is the reason why those horrible TVs apply a post process sharpness increase to content (the first thing you should turn off when you buy a TV).
Quality covers a large number of metrics including artifacting, posterisation, loss of colour fidelity, and loss of contrast ratio to name a few.
It may seem that way but it is a real issue. Watching a football(US) game on Comcast, live, it is easy to see the lack of visual quality as the camera moves and the field just becomes a blur of green, which once again returns to detail as the image stops moving. Movies from Netflix/Amazon are often blocky (low res image) over a 1080p image choice. How can we hold providers to account and or make an informed choice as to which service provider is better? We don't watch the same instance of a game on two providers at the same time. We don't have two service providers for the same show? It's an interesting question and one which will become more of an issue as bandwidth providers race to the bottom rung of the ladder.
Unless you want to learn/acquire that ability to see the difference. I don't know a word in say spanish; but if every day I listen to a radio talk-show or a kids cartoon show in spanish for say1 hour; surely in a year or 6 months, I will hv decent ability to speak spanish (exactly the way a child learns her mother tongue). Compare that with a gibberish stream of some random spanish words.