Slashdot Mirror
HDTV via GNU Radio
NortonDC writes "High Definition TV has been successfully captured in its native data stream from an over the air broadcast by a software defined radio that is Free and open source from the GNU Software Defined Radio project."
Here is the entire collection of mailing list conversations for the entirety of this project's lifetime.
You can see how tough roadblocks were overcome by a dedicated and brilliant team of GNU coders.
Reply or e-mail; don't vaguely moderate. Ex-O'Reilly/MIT employee, now a full-time Google employee.
Seems really neat, but I found that card on a science site. ONLY 1000 pounds (Great Britan). I suspect that this isn't much cheaper in the US either ;-(
. js p?action=GetProduct&pid=91&sid=1
https://directory.adeptscience.co.uk/controller
You can get many TV shows you might have missed by using BitTorrent .
This site has a list of links to various sites which contain TV shows available through BitTorrent.
A West Wing episode is available here (The West Wing - s04e16 - California 47th [ftv].mpg.torrent) (but you need to have installed BitTorrent prior to clicking on that link).
I don't follow West Wing so I don't know whether that's from second season, and your comment is accurate -- that's the only one available from that site. One other is The.West.Wing.S04E14.Inauguration.Day.Part.I .
Enjoy!
I feel fantastic, and I'm still alive.
Think TiVo, except its easier to get at the saved programming
Manipulate the moderator system! Mod someone as "overrated" today.
At $1,299.00 for the PCI card that their driver is written for, I do not see this in my future. For that matter, I don't see that in the future of many hobbiests which makes this project rather useless to the general population at present.
See here for information on the product the GNU Radio project wrote the driver for: Measurement Computing
Maybe some day...
I have not yet got a feel for the computational power required to approach real time processing or typical performance. Does anyone else know?
All my previous sigs now look like this one, I wish they were permanetly recorded when used.
No, you need a very expensive DSP board. I'm guessing it's about 500-600$ here in the US. Look at my other post in this thread for the site.
My understanding is that it is a fight against copy protection. Open (Free) software defined radio means that the user can pull down any (unencrypted) broadcast and save it - whether or not the "broadcast flag" (no-copy bit) has been used. In a future in which hardware televisions can't save copies of anything, this will allow the user to save copies and play back later (or do anything else) on a future PC or TiVo. Good stuff.
sulli
RTFJ.
Yuck...
here's the google cache of the site: http://216.239.33.100/search?q=cache:KWJY96KyuCAJ: www.gnu.org/software/gnuradio/
also, if you're interested in HDTV samples, this site has a bunch of HDTV trailers (complete with Dolby Digital 5.1 surround sound, as per the HDTV spec):
http://www.eecs.umich.edu/~balazer/hd_ads/
It's not so much a special radio card as it is just any wide-banded data acquisition board and a little frequency-translation unit.
There is a "tuner" that multiplies the incoming radio signals by a variable frequency. When you mix two oscillating signals (by multiplication) you get harmonics. If the variable frequency is just a sine wave (i.e., not modulated with any information), then the harmonics are identical in modulation to the original, but at a difference frequency. The tuning box is used to bring various radio signals down to a frequency that can be digitized by any ordinary data acquisition board.
These data acquisition boards are designed to basically sample voltages of whatever is tied to their inputs, and to sample it very very quickly and very often. Since these boards (and computers also) are getting more advanced (i.e. faster), they are able to sample real radio frequencies (stuff in the ones of MHz ranges).
After you get the signal digitized, it's just a simple matter of writing software that mathematically performs the functions that all the circuitry in the 'old-fashioned' receivers would do with their capacitors, resistors, and inductors (and more).
That's pretty much how it works.
a little expensive for my taste.
Always going forward, 'cause we can't find reverse.
There are none. Check the Linux HTPC AVS forum for plenty of whining on this topic.
GNU Radio is a collection of software that when combined with minimal hardware, allows the construction of radios where the actual waveforms transmitted and received are defined by software. What this means is that it turns the digital modulation schemes used in today's high performance wireless devices into software problems.
Read the site! This is very important stuff and could have a huge impact on technology.Well, it looks like a DAS4020 board has 12-bit resolution and so does the BBC Micro (although it might be only 10 bits in practice). The difference comes in sample rates: 20MHz versus 100Hz! So people are not going to be recording any UHF broadcasts through the analogue port. Unless they manage some serious overclocking.
-- Ed Avis ed@membled.com
Try one of the GNU mirrors:
e s/hdtv-samples.html
s amples.html
e s/hdtv-samples.html
a ges/hdtv-samples.html
: www.gnu.org/server/list-mirrors.html+gnu.org+mirro rs&hl=en&ie=UTF-8
http://gnu.sunsite.utk.edu/software/gnuradio/imag
http://gnu.wwc.edu/software/gnuradio/images/hdtv-
http://gnu.mscnetworks.com/software/gnuradio/imag
http://www.phildowd.com:4060/software/gnuradio/im
Basically, append software/gnuradio/images/hdtv-samples.html to any of the links from here: http://216.239.57.100/search?q=cache:1KyAbWv9nRAC
Withdrawal before climax is very ineffective and those who try this are usually called "parents."
It could be used for that (assuming you have a way to rebroadcast the signal later? Without the FCC hunting your signal down?), but the purpose of the software is to provide signal-processing software targeted at radio signal processing. If that signal is an HDTV signal, than so be it, but it could just as easily be X-Ray signals from space (assuming your sampling device could capture them) or some AM Radio talk show.
If you look at the site you can see a number of other examples. Since you are no longer limited by a standard radio's hardware, you can do completely different stuff like receive two different frequencies at the same time.
If I have been able to see further than others, it is because I bought a pair of binoculars.
Well, I DO do this stuff for a living, as well as being a computer geek and a ham.
But really, $1300 for the digitizer card is a bit steep - I work with a system using a 40 MSamp/sec 12 bit flash converter and Intersil 50214. The Intersil is about $30, and I don't think the flash converter is much more. Add a $50 FPGA to do the interfacing to the PCI bus, and you could do scatter-gather busmastering capture to the main system pretty easily.
Use a $50 Intersil 50216, and you could do most of the heavy lifting with it - Final IF filtering, I/Q recovery, post-detection filtering, symbol tracking, etc. That would remove a lot of the CPU load from the system, possibly allowing for real-time aquisition and decode.
Go to one of the board fab houses, and you could probably get a board built for about $500, maybe less.
Considering that people are spending $500 for video cards, this might not be so bad.
www.eFax.com are spammers
The 20Mhz is the killer thing here. That is some Serious converting. Your Cd player converts DA 16bits at 44khz. A high end studio type- audio card does 24bits at 96khz, in either direction.
This thing is doing 12bits at 20,000khz. Holy crap. That is hauling ass, hope your damn good with a sauldering iron !
JON
Overscanning, most likely - same as what's done with NTSC or PAL to ensure that there's no weird border around your screen.
The broadcast flag is a proposal, not a regulation. But if the broadcast flag is issued, it sure looks like GNU Radio will be illegal.
This has been covered here on Slashdot before. Some of the comments in the previous post are particularily informative.
Always value the individual over the system. --Bruce Lee "I don't need a Sig - I have a custom 191" - me
"It's the DeCSS of HDTV."
Ehh... not quite. As I understand it, the standard is about copying/storing the radio transmission as-is, with no decoding of anything. It would leave any HDCP/"broadcast flags" (if present) in tact.
This is less DeCSS and more copying DVDs bit-by-bit. You'll still need a decoder.
WhenYouHaveSoManyCharactersWithoutSpacesSlashcodeB reaksItUpToAvoidPageWideningPosts(amongOtherThings I'mSure).
Not his fault.
HDTV is digital, other datastreams arent included in the video stream like they were in analog TV. The video feed is exactly that resolution, and it's the TV maker's job to make sure it shows up correctly.
Why does the image exceed 1920x1080? Isn't the highest HDTV resolution 1080p?
The highest resolution is 1080i, which runs at 1920x1080, interlaced. The GNU project threw in an extra 750 horizontal pixels for free (as in beer).
A few nitpicks:
.7 volts a 6 or a 5?), and you have to slice at the correct time (the symbols are only defined at certain times - any other time the signal isn't valid, it is a blend of the current and the (next|previous) symbol - what is called inter-symbol interference or ISI). So you have to do symbol tracking - figuring out when to sample, and at what levels to slice.
You don't get harmonics (frequencies that are related to the fundimental by an integer multiple), you get mixing products, also known as "sum and difference".
You get harmonics when you feed a single signal into a non-linear element - feed f1 in, get f1, 2*f1, 3*f1, 4*f1,... out. This is commonly used in tranmsitters to allow the use of a lower-frequency crystal to generate higher frequency carriers - you use a 10 MHz crystal, and the feed it into a non-linear element such as a squaring amp, and pick off the tenth harmonic to get 100 MHz.
Mixing involves feeding 2 signals f1 and f2 into a multiplier - you get f1, f2, f1-f2, and f1+f2 out. Mixing allows changing a frequency by a non-integer relationship. You have heard this used in the voice distorters used on TV to mask mob informants - they mix the person's voice with a low-frequency signal to change the pitch of the speaker's voice. This is also the basis of any modern superheterodyne receiver - you mix two different (heterogenous) signals together.
The idea is to take the signal from whatever frequency it is on, and move it to the frequency you have designed your circuit to work at - an "intermediate frequency", or IF. You then filter the signal, amplify it to a specified level, and repeat as necessary to get the signal where you want it. For example, a standard FM radio might go from the broadcast frequency to a 10.7 MHz first IF, then to a 455 kHz second IF, then finally to the FM detector circuit.
Eventually, in a design like GnuRadio, you sample the signal. The tricky bit is you have to sample at a frequency not less than twice the highest bandwidth in the signal (Nyquist's criterion). For a 6MHz wide TV signal, that means you need to sample at not less than 12 million samples per second.
Then, for a system like HDTV, you are dealing with a complex signal - and I mean complex as in sqrt(-1), not just as in "not simple" - you need both the real (in-phase, or I signal - the "real" part) and the quadrature (out-of-phase, or Q signal - the imaginary part). The signal is 8VSB - eight level vestigial sideband. So you have to do carrier recovery and tracking (because the carrier itself was removed - that is what makes it a sideband signal), then you have to convert the signals from the analog RF signal into one of 8 levels (slicing is the technical term). However, you have to slice accurately in 2 dimensions - you have to slice at the correct level (is
Finally, once you have a symbol stream, you then have to do all the foward error correction - you have to de-interleave the signal (think of unshuffling a deck of cards) - interleaving is done so that a transient interference (like a lightning strike) doesn't scramble adjacent bits - the errors are spread out.
Then you do your block error correction - this can undo a small number of bit errors per block (again, that's why you interleave the signal: so that block error correction needs to only correct a few bit errors per block).
Then you do some more protocol recovery, and you have an MPEG stream.
Normally, you do this sort of stuff with a big FPGA or an ASIC. The GnuRadio folks are doing it in software. The up side is that you can more easily tweak the code. The downside is that you are not going to be real-time for a few more iterations of Moore's "Law".
What gets to be REALLY fun is when, in addition to all of the above, you have to compute parametrics on the signal - not just recover the bits, but measure how far out of ideal the signal was (that's the sort of stuff I do for a living.) When you do that, you have to do all of the above, THEN once you have an error corrected bit stream you have to regenerate an ideal signal and compare the received signal against it, and measure how far away from the ideal signal the real signal is.
And THAT is when you start using multi-GHz processors, 10 million gate FPGAs, big-ass DSPs, and all sorts of other fun stuff.
www.eFax.com are spammers
Also, keep in mind that the popular CRT and projection projection TVs will purposily overscan the picture such that some of the lines are pushed outside of the viewing area. So, while 1088 lines are broadcast, a projection TV may only show 1076 of them and clip 6 lines each from top and bottom. If overscanning results in only 4 lines being clipped then you will actually see only 1080 of the 1088 lines of MPEG-2 stream.
The width of 2730 pixels appears to be intended get close to the correct aspect ratio when displayed on a computer monitor. Based on how the people's heads look on my monitor, it seems to be a little over stretched. But when I return the images to 1920x1088, they clearly look squeezed.
Actually, 20MHz isn't so bad - I work with 40 Msample/sec 12-bit flash converters, and there are 100 Msample/sec 12-bit flash converters on the market.
However, you DON'T build things like this with your brother's wood-burning kit and a old nail - These parts come in surface mount packages, and your board has to be carefully designed to maintain proper impedance matching on the RF traces, as well as having excellent grounding (RF and digital grounds meeting at one and only one point, ground planes cut as needed to prevent current loops, etc.).
Lastly, you need a proper dithering circuit to introduce noise equivelent to 1/2 of the least significant bit, in order to shape the quantization noise out of the frequencies of interest. Otherwise, you end up throwing away a couple of bits of resolution.
Those are the sorts of things you have top-notch RF designers laying out, and a top-notch fab build for you - either by having such a fab working for you, or by contracting it out.
www.eFax.com are spammers
Come up with your own set of software projects that change the world as we know it and you can name it *anything* you want.
A great site for software-defined radio:
www.nitehawk.com/sm5bsz/linuxdsp/linroot.htm
Perhaps HDTV pixels aren't square? I really don't know much about the format, but IIRC this was an issue with NTSC. (and some computers have had non-square pixels, including the Apple Lisa, I believe)
-- This and all my posts are in the public domain. I am a lawyer. I am not your lawyer, and this is not legal advice.
Are you suggesting something like this?
(I'm actually curious. At $750, this seems much more reasonable than the $1300 device listed above.)
"I assumed blithely that there were no elves out there in the darkness"
http://www.deadmilkmen.com/lyrics/stuart.html
Yes, basically what you need is a tuner, a digitizer, and enough CPU to do the job. The last one can be the killer, though.
Here's how you can receive multiple signals at once with one tuner - let's pick on ordinary FM, and let's say I want to receive 95.1 MHz and 96.3 MHz.
The first requirement is that the two signals in question have to be reasonably close in strength - if one signal is coming in at -60 dBm (one millionth of a milliwatt) and the other at -120 dBm (one millionth of one millionth of a milliwatt) it's not going to work. But let's say both signals are coming in at -100 dBm, so there isn't a problem there.
OK, so what I do is make a receiver that will amplify all signals around 95.7 Mhz (half-way between the two) and downshift that to a convenient IF, say 10.7 MHz. Ideally, I'd want to bandwidth limit the signal to about 1.5 Mhz - that way I get both the signals I want, and not much else.
So my receiver is now outputting a signal at 0 dBm (1 milliwatt). If I look at that signal on my spectrum analzyer, I see 2 peaks - one at 10.1 MHz (the signal that used to be 95.1 Mhz), and one at 11.6 Mhz (the signal that used to be at 96.3 Mhz).
OK, now I digitize the signal. I need to sample at more than 3 Msample/sec (Nyquist's criterion) - any less and the two signals will be "folded" into one, and it's game over. To keep it simple, I will digitize the signal at 40 MSample/sec.
Now, I have a bitstream that contains the information for both signals. First, I multiply the signal by a mathematically generated cosine and sine wave at 10.1 MHz. This will give me two signals - I and Q. I is the "in phase" signal - the product of the cosine signal and the digitized signal. Q is the "quadrature" signal - it is 90 degrees, or a quarter circle out of phase with "in phase" signal (hence the name quadrature).
Next, I lowpass the signal - I run I and Q through a digital filter that removes all frequency components above 75 KHz.
Now, I compute a four-quadrant arctan() on the I and Q filters - this gives me the phase angle of the signal. I then differentiate that signal - subtract each sample from the previous sample. That gives me the instantanious frequency of the 95.1 MHz carrier - and there's your audio (if you want stereo I have to go trhough a few more steps....)
Now, I do the exact same thing, except this time I use a mathematically generated 11.6 MHz cosine/sine wave. The end result is the 96.3 MHz signal.
Now, as you can guess, that's a lot of math. Hoever, there are chips that do all of that for you. The Intersil 50216 I keep mentioning has 4 independant sections, each of which will do all of those steps, on a 40 MHz signal (or faster), in real time. If you do the math, the chip works out to about 5 to 10 billion operations a second, for 10 watts.
Chips like the 50216 are used by cellular folks - the idea is they grab the entire cellular band, digitize it, and the pull the calls out of the mix. You have only 1 drify, nasty analog receiver section, and hundreds of nice, stable digital sections.
There are a few problems with this approach, however:
1) The wider the receiver, the more noise it picks up. A receiver that is listening to 1 MHz of the spectrum picks up roughtly ten times as much noise as a receiver listening to 100 kHz of spectrum. That extra noise limits what you can tune in.
2) Strong signals in your receiver's bandwidth will prevent you from hearing weak signals - this is called "desense". It's one of the things that makes CDMA and TDMA cellular harder to do than dumb old AMPS - a phone near the tower has to be told to speak softly, so that the tower can hear distant phones.
3) Faster is harder - digitizing a 455 kHz IF is dirt cheap. Digitizing a 10.7 MHz IF is harder. Digitizing a 100 MHz IF to be able to cover the whole cell band is a bitch.
www.eFax.com are spammers
degnuing your docgnumentation is easy;
sed -es"/GNU\///"g
Not being bothered by Richard Stallman is just as easy.. simply ignure him..
HDTV pixels are square (HD resolutions 143 1920x1080 and 1280x720 on 16:9 screens). I haven't a damned clue why those screenshots are screwy.
Can you imagine how long it would take to compress a 2-pass 1280x1024 2-hour movie into an xvid/ogg file?
I'd be more interested in how long it takes to downsample to a lower resolution before compressing it with XviD... Most of the DVD rips^H^H^H^H backups I do are downsampled a bit to sacrifice resolution for clarity.
HDTV at HDTV resolutions does look incredible, but I'm willing to tolerate lower resolution captures. If you consider how bad VHS is compared to a clean NTSC resolution stream (such as from a progressive scan DVD player), it's obvious the general public is willing to accept a recording solution that capures less than the broadcast resolution.
---
DRM is like antifreeze, to the MPAA/RIAA it's sweet, to the consumers it's poison.
You don't really need a $1299 A/D board if you want to start playing with GNU Radio, unless you want to decode a high bandwidth signal like HDTV.
For a couple hundred bucks, you can get an A/D board up to a couple hundred kilohertz, and then hook it up to the IF of any cheap old radio you have sitting around.
Better be prepared to buy some heavy storage equipment then too, you need 2.5 terabytes to store days worth of signal.. (Assuming the maximum sample-rate of the card, 20Msamples/second, 12 bits per sample.)
Well, in Australia you can get the DGTEC DH-2000A that supports VGA output.
Can be found here:
http://www.ettus.com/sdr/sdr_w6yx.pdf
"Consensus" in science is _always_ a political construct.