Slashdot Mirror
Tivo-like Opportunistic Recording for Linux PVRs?
fahrv asks: "I use a MythTV-based Linux PVR. I'd like to see if there are ways to implement automatic Tivo-like program recordings based on personal preferences (and Amazon-like analysis of what other people recorded based on what you've recorded.) I know that the Amazon patent on 'Other people who bought XX also bought YY' could be an issue, but then Tivo is able to record shows opportunistically as well. I don't see any technical hurdles to doing this by analyzing people's MythTV viewing habits in an opt-in kind of system; I'd be interested in building one. Might any of you have some thoughts on this subject?"
Fortuitously, this week's edition of The Economist includes, as part of its Technology Quarterly, an article devoted to this very subject. For your convenience, portions of the full text--ordinarily available only to subscribers--is reproduced below.
----
CASE HISTORY
United we find
Mar 10th 2005
From The Economist print edition
Computing: Collaborative filtering software is changing the way people choose music, books and other things, by helping them find things they like, but did not know about
EACH year, thousands of films are released and tens of thousands of books published. A big city has thousands of restaurants. How does one deal with such abundance? Reading reviews of films, books and restaurants can provide a guide, but there are more reviews than one has the time to read, and you cannot be sure that the reviewer's taste matches your own. Word-of-mouth recommendations can help in that regard; friends, after all, are often friends because they share similar tastes.
[CONTENT REDACTED]
[Image: People who liked this also liked...]
The TiVo personal video recorder, on the other hand, which can recommend programs based on your (and other users') previous viewing habits, works in a different way: the recommendations are generated by each TiVo box, not by a central server. The server generates a matrix that relates the popularity of different shows to each other, akin to the pre-calculated item lists used by Amazon to generate recommendations. But the task of making recommendations is then left to the individual TiVo boxes, which use that matrix, combined with the data they have stored locally about the viewer's preferences, to suggest shows that might be of interest. As well as unloading much of the work on to the individual boxes, this has the added virtue of preserving privacy: the central server never stores data about individual users, just aggregated data about viewing trends.
That is just one way to address what is, for privacy advocates, a major concern about collaborative filtering: that to make recommendations, it is necessary to gather information about many people in a central repository. But there are other ways too. Indeed, a scheme proposed by John Canny, of the University of California at Berkeley, shows that it is, in fact, possible for a group of individuals to pool their opinions and generate recommendations without revealing their own personal preferences to others.
Each individual encrypts their data using what is called a one-way hash--a function that is very easy to compute in one direction, but virtually impossible in the other (without a key, at least). The computations are then performed using the encrypted data. This is possible because many modern encryption schemes have the helpful property that performing calculations on encrypted data produces the same answer as manipulating the unencrypted data and then encrypting the result. The resulting matrix of recommendations is then decrypted incrementally, since each user can only decrypt a small part of it. Eventually, the whole matrix is decrypted and made available to everyone. But, says Dr Canny, "at no stage does unencrypted information about a user's preferences leave their own machine."
[CONTENT REDACTED] ::: yfnET
Here you go: link.
There was a thread about something called MythRecommend quite a while back in the mailing list. I haven't tried it, and the site looks like it's running off some personal home page, so I won't link it. Do a search and you'll find it very easily.
It looks like it has lots of room for improvement, but a neat start.
SYS 64738 NO CARRIER
There's a lot of initial data for the "people who watched X might also like Y" bit on the Internet Movie Database which covers TV programmes as well as films.
Example
They have a lot of their information available for non-commercial use on their interfaces page.
I can't spot the recommendation data on there though, but perhaps if you asked them very nicely...
I followed the link and it wasn't that hard to find anythiing that the story poster was asking about. Give it a try.
There have been multiple attempts at this. Myth Recommends, WishTV, TV Wish, Myth Suggest (?)... I can't remember the names exactly, but there are attempts out there.
I don't think any of them gained enough traction to get included in the software directly, but you may be able to kick start development on one of them.
T
---- It puts the lotion on its skin or else it gets the hose again. It does this whenever it's told.
yes... it's currently a windows program, but version SageTV/Media Center 3 is going to be also available on linux. (not OSS though commercial product -- linux media cener announcement)
I use sagetv 2.x on my windoze based PVR (currently, it's in a constant state of flux).
I'm sure there's plugins to do similiar functionality for other PC PVR software solutions (on both windows and linux)
e.
Build Your Own PVR/HTPC news, reviews, &
Tivo IS a linux PVR
Quidquid latine dictum sit, altum viditur
I've owned a TiVo for almost five years now. And for five years I've faithfully marked programs I like watching with a Thumbs Up.
All that gets me from the auto-suggest feature is those same programs I've already seen. I've seen the Suggest feature pull up a program I hadn't heard of before perhaps two or three times.
I don't know the best way to implement this for Myth, but I can tell you what finally worked at TiVo. After several years of trying to get the algorithm working well, they finally turned to collaborative filtering. Their initial algorithm was heavily genre weighted. This didn't work very well in practice. Finally, they started anonymously pulling back the thumbs up/down data from a subset of their customers (~ 50,000) and created collaborative vectors to look for program relationships. This finally did the trick and collborative rankings now outweigh genre for mainstream programs. Less watched programs and movies still use more of the original genre/actor/director calculations to make suggestions. My point is that you spend your time using community recommendations and don't waste your time trying to write a fancy algorithm which will likely produce poor results.
I don't have an immediate solution for this. Myth Recommend and WishTV have already been mentioned. Although the two of them are better than nothing at all, neither of them is what Tivo does.
However, if you are a developer and are thinking of starting such a project, I have a recommendation. I would suggest that you use Bayesian analysis of the descriptions of the user's recorded shows. Sort of like Spamassassin for TV shows. The spam detection algorithm could be adapted to identify possibly desireable shows, just as it is used to identify possible spam.
By the way. If none already exists, this post serves as prior art should anyone try to patent the idea. It is now an obvious invention.
I would like to see something similar, and even started building it myself. However as always time is a factor.
It is "usable" as far as getting recommendations from hundreds of other MythTV users. However its only "console based" at this point, but its ready for anyone to
build a native MythTV gui for it at anytime.
I would be happy to work with you on such a project if you like. I have many other ideas of things to implement, just haven't gotten around to it yet.
http://ipso.snappymail.ca/mythrecommend/
Open Source Time and Attendance, Job Costing a