Slashdot Mirror
Ask Slashdot: Is There An Open Source Tool Measuring The Sharpness of Streaming Video?
dryriver asks:
Is there an open source video analysis tool available that can take a folder full of video captures (e.g. news, sports, movies, music videos, TV shows), analyze the video frames in those captures, and put a hard number on how optically sharp, on average, the digital video provided by any given digital TV or streaming service is?
If such a tool exists, it could be of great use in shaming paid video content delivery services that promise proper "1080 HD" or "4K UHD" quality content, but deliver video that is actually Youtube quality or worse. With such a tool, people could channel-hop across their digital TV service's various offerings for an hour or so, capture the video stream to harddisk, and then have an "average optical sharpness score" for that service calculated that can be shared with others and published online, possibly shaming the content provider -- satellite TV providers in particular -- into upping their bitrate if the score turns out to be atrociously low for that service....
People in many countries -- particularly developing countries -- cough up hard cash to sign up for various satellite TV, digital TV, streaming video and similar services, only to then find that the bitrate, compression quality and optical sharpness of the video content delivered isn't too great at all. At a time when 4K UHD content is available in some countries, many satellite TV and streaming video services in many different countries do not even deliver properly sharp and well-defined 1080 HD video to their customers, even though the content quality advertised before signing up is very much "crystal clear 1080 HD High-Definition".
What's the solution? Leave your thoughts and suggestions in the comments.
And is there an open source tool measuring the sharpness of streaming video?
If such a tool exists, it could be of great use in shaming paid video content delivery services that promise proper "1080 HD" or "4K UHD" quality content, but deliver video that is actually Youtube quality or worse. With such a tool, people could channel-hop across their digital TV service's various offerings for an hour or so, capture the video stream to harddisk, and then have an "average optical sharpness score" for that service calculated that can be shared with others and published online, possibly shaming the content provider -- satellite TV providers in particular -- into upping their bitrate if the score turns out to be atrociously low for that service....
People in many countries -- particularly developing countries -- cough up hard cash to sign up for various satellite TV, digital TV, streaming video and similar services, only to then find that the bitrate, compression quality and optical sharpness of the video content delivered isn't too great at all. At a time when 4K UHD content is available in some countries, many satellite TV and streaming video services in many different countries do not even deliver properly sharp and well-defined 1080 HD video to their customers, even though the content quality advertised before signing up is very much "crystal clear 1080 HD High-Definition".
What's the solution? Leave your thoughts and suggestions in the comments.
And is there an open source tool measuring the sharpness of streaming video?
This whole article seems focused on one oddly specific use case and seems vaguely autistic.
You'll probably run into copy-protection issues with many media sources. If you can get at the bits to analyze it, you can dump them out to create a copy. I'm not even sure sharpness is going to be easy to measure, if a soft/low-res source has been artificially sharpened to give the appearance of quality. Sports broadcasts seem to be the worst for it for it from my experience, with glowing halos around high-contrast edges to make them stand out even more.
this class of vile racism must be resisted. we must resist. its time for anonymous comments to go and hate speech no longer protected as free speech. the time is now. this poison will end humanity.
What is "sharp"? A 1x2-pixel image where one pixel is black and the other one white is tack sharp, it can't get sharper than that. Is that what you want?
What would you measure?
More interesting is the level of compression which introduces losses and artifacts. But since the quality delivered depends on your bandwidth, make sure the bandwidth isn't the culprit.
Shut up and enjoy whatever they decide to give you, pleb.
It created by degrading the video in a process called unsharp masking. Basically you find the edges between light and dark objects, and exaggerate them. You make the light side lighter on that edge, and make the dark side darker. This adds no information, in fact it degrades video quality from the original unsharpened version (loses information).
But your brain has special cells which recognize transitions between light and dark, and identifies them as an edge. When the exaggerated transition from an unsharp masked edge hits those brain cells, they get more excited and signal the rest of your brain that this is a really strong edge. Thus creating the illusion that you are seeing a sharper image, when objectively it's a degraded image.
So any algorithm which detects "sharpness" as interpreted by the brain would actually rate inferior (heavily processed) video streams higher than video streams conveying the maximum amount of information possible. It's why unsharp masking is typically added by the TV or video player, rather than incorporated into the original video stream. (A slight amount of sharpening is done to counteract the blurring caused by the Bayer filter used in camera sensors, but that's another story.)
The tool is the human eye. At the end of the day nothing else matters.
Video quality is a function of codec and bitrate. It will always be a compromise between image quality, file size and decoding power. Even if the service wanted to serve users super sharp, high preset h265 + flac audio, most users would be unable to use it - file sizes would eat their download caps and their netbooks/phones would be unable to cope with decoding. Obviously on the service side they also don't want to pay for storage and transfer of those huge files, especially if they would be hardly watched.
Between the image and its blur. Probably want to mean center and scale to add up to 1 first (grayscale). Blurry images don't blur as fast as sharp ones.
The only thing you can more or less do on the broadcast video is to identify encoding artifacts like macro blocks, mosquito noise and deinterlacing side effects. MPEG2 streams are easy to identify, H264 and H265 are slightly more difficult. I do not know of a package solution to this. Be aware of news stations that use low fi mobile feeds!
it may be a useful illusion.
Regardless, the question asked was not about the group's opinion of the value of sharpness, but about the availability of software to evaluate it.
I do not have a direct answer to the question asked, but would suggest that a python script should be able to do something useful. Use numpy/scipy to run fft on "lines" of data across a frame. A "sharp" image will contain a higher proportion of high-frequency components (higher-order harmonics).
It was on G... could have done it yourself but, here u are. http://www.compression.ru/vide...
[($)]
There are already algorithms for doing this, like PSNR, SSIM, and VMAF. Here's a good read: https://medium.com/netflix-techblog/toward-a-practical-perceptual-video-quality-metric-653f208b9652
And there are already commercial solutions that apply these metrics as well as network analysis etc. across a range of video delivery methods, e.g.:https://www.telestream.net/iq/
You need to check with the specific streaming services terms of use to see what they say about their streaming. Most likely most will have a way out if the video is not up to par. Also, sometimes the streaming video is based on the actual speed of your internet at the time of the streaming (Amazon Prime, Netflix, etc). In addition some service providers will throttle the stream - especially if mobile.
IMO, getting a measuring tool would really be a waste of time.
Is a way to tell not that the video is "sharp" but closely matched the original source in fidelity. If all you looked at was sharpness a scene that had motion blur or an out of focus effect would run afoul of the algorithm. Your end goal of name and shame is incredibly difficult to do well or even accurately.
There's ways to get video quality measures (SSIM, MSE, etc) but they require a comparison to the source (or a source you consider good enough) as they're relative measures. Interpreting the results is also not obvious.
Besides the challenge of the comparison it's also important to understand the video pipeline from raw source to what you see on screen. The video stream from the provider might be providing a high quality stream but the scaler on your TV might suck and muddies the image upon display or your LCD panel's dithering might be really shitty.
Video, especially streaming video (either Internet streaming or live delivery like cable), is really complicated. There's a lot of different dimensions where you might judge "quality" and even then it's an envelope and not a single scalar value. There's no objective "good" reference for any recorded scene, even the concept of "life-like" is not clear since the recording is entirely dependent on physical properties of the equipment.
I'm a loner Dottie, a Rebel.
There's already a metric that basically defines video sharpness: bitrate.
The sharpness of H.264 and H.265 is very well known. Since commercial streaming services use commercial streaming video codecs, it's a pretty safe bet that you can almost directly correlate resolution to bitrate.
There's virtually no incentive for streaming companies to deliver lower resolutions at higher bitrates. It would be a technical challenge to deliver higher resolutions at higher bitrates.
Therefore, bitrate is most likely the simplest and most accurate measure of streaming video sharpness.
Most humans can't tell the difference, and now you expect a computer to? How many people I've heard say a 1080p tv isn't worth it let alone a 4k.
This is judging video, not a set of still images. Video that looks sharp and natural is a bunch of very slightly blurred stills, our brains reassemble it all to appear as such. Thereby, it would seem quite difficult to build a piece of software that could give a useful 'hard number' for this kind of evaluation.
I never thought that companies not providing a delivered product was exactly "shameful", as if they crossed some moral line. But that just seems to be product of our time.
If you really want to change things, why do you need a tool to make the comparison? Nobody is going to care that your tool said video source X was only 50% sharp, and the original source was 80% sharp. But they WILL care when you provide side-by-side comparisons of each frame.
If it's true, it's a perfect story to put on the web. If you can see the difference between the two screens use that! It's so simple that you could do it with just a few hours work. All you need to do it make screen grabs of different movies played on streaming and compare it to BluRay. You could do this in a weekend, and it's make good journalism.
So do it.
You are probably not aware of how video streaming works nowadays, and you should read about adaptive bitrate streaming.
TLDR, nowadays content providers encode each video at different resolutions and quality levels (and bitrates), each with a different level of "sharpness", and the client selects the one that best fits its available network bandwidth.
This means that playing the same video several times may result in different quality each time. This also implies that the quality of the video you receive may be altered by your neighbor playing netflix (in the same Wifi channel), or your girlfriend passing between the AP and your receiver (reducing the signal level; both of them cause the AP to modify the modulation and coding scheme, reduce the available network bandwidth and may reduce the bitrate (read: "sharpness") of the following piece of video.
Also, please note that "youtube quality" is a nonsense, since Youtube delivers video with different quality levels (in some cases, up to native 4K).
Fundamentally, a 2-D FFT describes the dynamics present in any image. Sure, you could compare them between a known source displayed via different streaming methods on different screens. However, these numbers are meaningless on their own without taking the context of the viewer into account.
The FFT must include physical dimensions, starting with the screen's PPI value. So a 4K phone screen is inherently "sharper" than a 4K projector on a 20 meter movie screen. But we all know the movie screen offers a far better experience than a phone, more "immersive", independent of the perceived or actual sharpness.
The viewer's perceptions are also key. Starting with their ability to perceive. For the human eye to resolve a 4K image at the pixel level, that is to detect sharpness changes at that level, a person with 20-20 vision would need to sit NO FURTHER than a SINGLE screen diagonals away from the center of the screen. That's 65 inches away from a 65" inch monitor. If you aren't sitting that close, you aren't "seeing in 4K". So you have already started to lose "sharpness" independent of what you're watching.
And if you are sitting that close, you'll want a curved monitor, since the distance from the center pixel to your eye is much closer than the distances to the corner pixels. And you'll be watching alone, since there generally isn't space for two at that ideal viewing spot. So most of us get flat monitors and sit further away. Most of us sit so far from our screens that we aren't even "seeing in 1080p"! You'll notice that most TVs smaller than 32" are only 720p, which is because you really can't see more than that if your sitting at a normal TV viewing distance.
I *DO* sit that close. As I write this I'm sitting 55" away from my curved 65" TV/monitor. For most streaming content I lean back a bit simply because it makes no difference: The media sharpness is generally sloppier than what my eyes can perceive, so sitting back doesn't reduce the quality. That does change for well-rendered 4K content, where the original material is extremely sharp (directors and editors intentionally add blur to remove distracting levels of sharpness). That's when I lean in until my eyes can take in every detail.
If you aren't close enough, the content sharpness simply doesn't matter.
Dear Salty Dude,
Your picture quality is crap because streaming services dynamically vary the quality to match the available bandwidth. If you've got less than 20Mbps or if your ISP is throttling your 20Mbps connection so that you can only get 2Mbps from Netflix, then you're not going to see 4K(UHD). https://fast.com
Even if you could programatically characterize the quality of the video, your salty rants are not going to shame the likes of NetFlix and Amazon. So don't make a spectacle of yourself or waste your time looking for pointless code.
Get a better connection or ISP. If you can't do that, don't pay for HD and UHD streaming content because you're never going to see it on your shit connection.
Love always,
AC
It's existed since the beginning of humanity and hasn't destroyed it yet. Your intolerance will though.
Both comments are just trolling though.
And that's the law!
First is the actual cameras that record the video
For movies >2010 this is usually pretty good,
But a lot of film conversions to digital are not so great.
Live events are all over the place, but 1080p is usually
pretty good except for a glare problem of bright lights
either directly or reflected of surfaces.
Dramatic light shows with swirling colors and so on
are hard to render without washing out colors and some
tearing and blending.
What you want is life-like skin texture on people.
But that is also dependent on the makeup they wear.
TV shows that are interviews under lights are terrible about this.
There will always be some compression in the transport
which leads to "smoothing" of subtle texture changes and
exaggeration of sharp lines, especially straight lines, and
deepening of shadows with less detail in dark areas.
The settings and capabilities of the device(s) that render the video
to final pixels on the screen are mostly about the same quality for
the same resolution, though I have found older (2010 - 2016) Vizio
units are very clear (to my eyes) compared to Sony or Panasonic or whatever.
Cable boxes HD looks worse than Blu-Ray because of the compression,
And American HD TV shows are terrible compared to UK HD.
The test is to take a streamed movie, and a HD 4K movie on Blu-ray,
and play them side-by-side on two monitors of the same brand, and if
the streamed looks bad, start reducing the res on the Blue-ray monitor output
until it looks the same. Try to match the brightness, intensity, and contrast
after every change, or put the same signal into both units and fiddle with them
until they have the best picture and both look identical.
Your mission, should you accept it, is to
take notes of everything, take pictures, and report back the results.
As others here already sort of mentioned; Sharpness is the wrong metric.
Another person mentioned "macro blocks".
That's what I was going to say. Compare to competitors using the same codec and more or less bitrate = quality. It's not a PERFECT measure, but it's pretty good and dead simple.
In the beginning, there was mpeg 1 and mpeg2.
Spatial information gets transformed into frequency domain, and then quantized.
In dumb words, sharpness is retained, but the amount and detail of sharpness becomes inaccurate.
Opposite of sharpness, the exact properties of a gradient, like a sky, also gets quantized.
Since the picture is divided into blocks, the sky starts looking blocky. Grass looks blocky. There's sharpness, and there's detail, but it's all floating around sort of disconnected in a sea of muddy blockiness. Sharp edges might get ringing around them.
Most computational and algorithmic measures say this is good. The human eye says it's bad.
So, next came h264 with its built-in deblocking filter. Not only does it try to smooth out all the sharp edges caused by the quantization, when the codec compares previous frame to next frame in order to record the differences, it uses the decoded and smoothed frame as reference. The amount of smoothing depends on how hard things were compressed.
With h264, the smoothing has temporal stability, and if the bitrate budget for the next frame is better, it can be spent on improving sharpness.
So why this long rant?
If we just go back to mpeg1 again, "Sharpness" will increase. Yay product comparison!! Goal achieved?
But it's sharpness in all the distracting places. Indeed, in a mpeg1/mpeg2 world, smoothness would be a better measure of quality!
In a h264 (and h265, which this author is too senile to be familiar with) world, the only useful measures are comparing original to encoded versions, with ssim or pant. We fon't have access to original, so we're all out of luck.
Anonymous because this was first post on Slashdot since 2001 and I forgot all login details.
First of all... sharpness is something you see and experience. There's no way to measure it. In order to measure something you need a unit and that doesn't exist. Don't tell me whether it's 1080p or 4K or 5K. Those are not sharpness units. If you want to measure sharpness, at least you're expecting certain value which you don't even know because again there's no unit.
It is very annoying to watch video on you-tube only to figure out many of them are ripped off from someone else. These thieves alter the video so YouTube doesn't kill it by changing the speed, mirroring, and other methods to get it online. It tends to make it unwatchable.
FFmpeg can do PSNR calculations. The only problem with what you call 'sharpness' (in technical terms, we call it noise or signal-to-noise ratio) is you have to have something to compare it with. Movie and video makers often use 'blur' either as a proxy to indicate motion or to put emphasis on the thing in the story that the viewer should focus on. It's also natural for things that are in the 'focus' of the camera to be sharp and everything else to be blurry (unsharp). So measuring 'sharpness' of an image is kind of pointless since 75% of an image is typically blurry.
What you need is get a comparison stream (you could use XMLTV for live channels or a Content ID system for recorded things to identify eg. a movie scene) and then run it through ffmpeg's or avisynth's many calculators.
One of the few things that do happen during cable or satellite streams and which you could minimally calculate is duplicate frames, black frames and dropped audio. Ffmpeg has calculators for all of them.
One of the few proxies you could try but which would be very hard to quantize is simply calculating how 'hard' a frame or set of frames is to compress. Blurry video is more complex and thus harder to compress, you can easily calculate these peaks if your stream is expected to be very homogenous (eg. streaming of a static image). My idea here would be to use MythTV's commercial detection algorithm and/or use OpenCV to detect and focus in on the logo. If the encoding difficulty of the logo changes significantly for periods shorter than the length of a commercial, you may just have detected a frame where the quality dropped.
Custom electronics and digital signage for your business: www.evcircuits.com
No, itâ(TM)s not. Every encoder offers configurations that trade off speed against quality. If you donâ(TM)t want to spend much on hardware, or turn around time is important, youâ(TM)ll reduce the quality irrespective of bitrate.
Then live encoding is another issue. x264 is very CPU intensive, and watch the CPU spike during (Iâ(TM)m not sure which, but they could be the same) scene detection or I-frame generation.
Measuring sharpness is possible, but that would mean nothing. There are many image sharpening algorithms. Any self respecting editor can play with sharpness any way he wants, and it is even built in some TVs and video players.
What you want is a measurement of quality and it can't be done without a reference. Think about it, if you can tell how close a video is to the reference without the reference, then the same algorithm could be used to reconstruct the original from a degraded video. And guess what, compression algorithm do exactly that: reconstruct a good looking video from as little data as possible.
So the answer to your question is simple: compress the video using a good compressor (ex: x256) on constant quality mode and measure the final file size. In fact the original bitrate may be a good indicator, no need to do anything. However, note that adding noise will increase both the bitrate and sharpness, and it is a technique commonly used to make things appear more detailed. Artefacts such as poor deinterlacing will also increase the bitrate and sharpness, so don't trust these metrics too much.
Have max bandwidth use by a broadcaster around the world. No attempts on adsl, low end 3rd world consumer ISP.
Work out what the original material was created in, film, HD, 4K, 8K.
Find the same show globally.
Sort for the bandwidth, compression, chip sets used.
And only selling so much bandwidth to broadcasters?
Are 2nd and 3rd world nations broadcasters trying to fit many shows over a set amount of bandwidth they can use in that part of the world?
Its not a conspiracy to make TV look bad in 2nd and 3rd world nations.
Only so much "TV" will fit in the bandwidth allowed in 2nd and 3rd world nations. To make money more "TV" is made to fit so more ads get played. So it looks different to the HD and 4K products offered in advanced normal nations. More shows, more ads, more profit.
Why? People in advanced nations "pay" at lot every month for more the amount of "bandwidth" to enjoy HD and 4K content.
Want better quality? Try DVD. Buy Blu ray. Pay for a broadcaster that "buys" the needed bandwidth for their broadcasting.
Domestic spying is now "Benign Information Gathering"
One element would be to Identify compression artifacts and rate the video on a scale by how much of it is present.
That's true, one can decide to spend less CPU encoding.
Having said that, when widely comparing services such as Dish Network vs Comcast vs Frontier, that part would be different only if they are very stupid. For a given level of quality, the higher CPU encoding gives lower bandwidth. Some of us may be used to thinking of higher quality if it were the same bandwidth, but the flip side of that is it also means lower bandwidth for whatever quality level they accept.
Since they are going to transfer the stream to millions of users, the bandwidth difference is multiplied by a million for them. It's a no-brainer to use the CPU to encode well ONCE in order to save on bandwidth a million times. I don't know if I wrote that clearly. If you're encoding once and transferring to one destination, you might not spend the CPU to make the bitrate low / quality high. Given they are encoding once and transferring a million times, it would be moronic of them to spend thousands of dollars more on bandwidth in order to save pennies on CPU.
If we assume they aren't utter morons, they'll all spend the CPU time to get the lowest bandwidth they can for the level of quality they find acceptable. That's another way of saying that they'll have the best quality they can for the bitrate try used.
Since transmission over the last mile is the expensive part, why not just check the received stream's bps? You can assume that they're not going to upsample it and pay more just for the last mile.
Your ad here. Ask me how!
That reminds me of the abhorrent sharpness setting most TVs even including 4k TVs have.
Sharpness is not a measurement of quality, quality is a measurement of quality or more specifically accurate reproduction - how close does the stream match a high quality version such as a blu-ray 4k copy etc.
If you introduce measuring sharpness as a way of measuring quality then the end result is cheating that will artificially make all streams sharper at the expense of picture quality.
So, get it right, sharpness != quality.
Here is an example of how nasty the sharpness setting can make things: https://www.youtube.com/watch?...
Waterfox - a Firefox fork with legacy extension support, security updates and better privacy by default.
Image quality is not determined by sharpness. This idea is the reason why those horrible TVs apply a post process sharpness increase to content (the first thing you should turn off when you buy a TV).
Quality covers a large number of metrics including artifacting, posterisation, loss of colour fidelity, and loss of contrast ratio to name a few.
One trick you can try is to export an image (or even short segment), downscale it to some resolution (say 1080p -> 720p), upscale it back to the original resolution, then optionally apply an unsharp mask filter. Compare this processed image/segment with the original and see how much difference there is. Note: use a decent scaler when performing resizes - ideally bicubic or better.
I'm not sure if there's a tool which can do something like this. I do know of some for anime (may work for non-anime content): https://iamscum.wordpress.com/_test1/_test2/resizing/
Yeah, that's working so well for Canada... Lolololol
I had a laugh when the RCMP protested those bullshit laws by shutting down lesbian bookstores with our obscenity laws, though. Probably the only time I respected them.
Many years ago my cable provider changed from analog and standard definition to digital and HD or FHD definition.
When I changed to digital myself, I set-up the TV to work with both video flows, from a spliter the analog to the TV's analog input, the other to numerical receiver and to TV's HDMI.
As long as I could receive analog signal, it was better than the digital, and at least once I had a TV's technician checking and I show him that just changing channels on TV from digital to analog for the same program, it was a big visible difference for the analog having more quality.
Now with only digital, I "enjoy" everything this technologi has to offer, macro-blocks in almost every program once per hour, and few times per week pixelation when almost a quoter of screen has 20-30 square blocks as usually FHD image.
Having also access to a streaming service, I have to say that yes, the sharpness is not always very good.
Also I saw many videos true FHD but with a low quality of image, yes, usually the sharpness.
The TOOL should accept an HDMI input (from cable's digital receiver), also being a "smart receiver" from a streaming service and pass the signal to TV.
I do have an HDMI recorder, but by re-en-coding it change the quality. Even it it is ProRes or DNxHD...
The TOOL should be able to record an image to very high quality (RAW-ish type frame) so you can analize what is at stake, the video quality and not a complex machine where the video itself is much processed.
So, in my oppinion, the TOOL should be hardware and software just to accquire proper stills from a video transport, then a suitable software (pretty complex, maybe AI driven) to be able to analyze the image and identify if a scale-up was applyied.
If you leave the still images to be acquired by each individual as he consider "available" (recorder, photo camera) it could be only this "second" part, an IQ (Image Quality) on the web you upload an image and specify the origin (from HD, FHD...) and it will give back a "mark" for quality of thar source.
Very nice ideea, indeed.
Please, you can compress the code much better than that. Did you even try?
This would be a great application for deep learning. Get 30 or so videos of each resolution and label them with their true resolutions, and train a CNN classifier on that data set to predict the resolution of a new unknown video. It's a case where deep learning is really good at learning this type of classification and would likely be highly accurate