"Every video on YouTube will be transcoded into WebM. They have about 1.2 million videos available today and will be working through their back catalog over time. But they have committed to supporting everything."
If true, the implications seem pretty deep. Personally, I'd be less intimidated by MPEG-LA FUD knowing that YouTube, the world's most popular source of online video, is walking the WebM walk. Add to that the fairly impressive list of supporters mentioned in the WebM announcement, and it would seem to be time to put-up (show your real legal teeth) or shut-up (no more FUD) for the MPEG-LA (and recent FUD'sters like Mr. Jobs).
On the other hand, Dark Shikari (the x264 developer) said this in his analysis: "Though I am not a lawyer, I simply cannot believe that they will be able to get away with this, especially in today’s overly litigious day and age". His comments comprise a rather more elevated class of FUD. Could this be the mother of mistakes (by Google) that will allow MPEG-LA to find themselves a mother-load of riches in such a deep pocketed infringer?
> The techology companies have paid lip service to trying to solve the problem. They offer up solutions but their heart isn't really in it.
Sure, their heart isn't really in it because they know that the "problem" is fundamentally intractable. DRM is at best a tool in an arms race, but for every DRM scheme there is always a technical counter-measure just off the horizon. Maybe it is the media companies who have the problem, that is they are the ones who only pay lip service to the idea of providing a good product at a fair price.
I think the TFA makes reasonably astute observations with respect to video's ability to use buffering to compensate for high latency. It is important to note that video is a broad term, and the importance of latency can and does vary depending on the kind of video. Video conferencing is a major challenge, because to achieve maximum quality it really does require high bandwidth and low latency simultaneously. However, there is a lot of video streaming that is only unidirectional, whether it be video on demand, or some kind of video broadcast, and in those cases, the TFA is quite right that even very large amounts of latency can be concealed with suitably designed buffering strategies. In that case, network jitter matters much less than bandwidth as far as the video is concerned (if the bandwidth just isn't there, then your video isn't going to be good).
Games are a bit different though. Most games need low latency, but compared to video, they also require a lot less bandwidth. This is important, because it opens a similar kind of opportunity to "work around" performance problems in the network. I would claim that the dominant source of delay in network games will be propagation delay (i.e. speed of light to cover physical distances traversed by the game packets). Yes, queueing delay in the network may be significant, but I doubt it will exceed the propagation delay. So my claim would imply that preferential treatment (away from the game traffic) will hurt games most when it causes some of the game packets to be dropped, possibly causing the game to cope with missing data, or longer latency (if the game reponds by retransmitting data). One simple approach is to send the packets redundantly (pro-active retransmission, FEC, etc.). This redundancy means that even if some of the packets get dropped, others may still get through. This will increase the overall bandwidth of the game traffic, but compared to other traffic (i.e. video) the amount will still be relatively low. Just like video, there may be certain types of game where this may not be feasible... if a game for some reason requires very high bandwidth, then its requirements will look a lot like video conferencing---it will need both high bandwidth and low latency. Almost none of the current games that I know of actually fall into that category.
You may be interested in a paper we wrote a few years back [1]. We also started with the premise that some applications require both minimal latency and maximal bandwidth. In our case the application was our own media streaming system. We came up with our own patch to TCP (in Linux). The patch provided a new socket option, we call TCP_MINBUF. The idea is that you need a certain minimum amount of buffer allow TCP's congestion window to function, but no more. Indeed, in the paper we show that the delay due to socket buffer beyond the congestion window is often by far the dominant source of latency--not retransmissions, or delayed acks, or all the other more commonly cited things. So basically what TCP_MINBUF does, is dynamically size the socket buffer to follow the congestion window size. It had a huge impact on latency.
[1] "Supporting Low Latency TCP-Based Media Streams", Ashvin Goel, Charles Krasic, Kang Li, and Jonathan Walpole. Tenth International Workshop on Quality of Service (IWQoS), May 2002.
The GPL is like capitalism
on
Why I Love The GPL
·
· Score: 2, Insightful
There's the quote we all know about capitalism that says "it is not perfect, but it is the least flawed system we know". For open source software, this is exactly what a lot of people (myself included) think of the GPL. [ My apologies for paraphrasing and the lack of proper attribution ]
At the same time, a lot of people dislike or even hate the GPL. In my view, their opinions usually come down to the fact that there really is no general example of how to make a business of open source software development. To people who's primary concern is the business of software development, the wealth of GPL software is a false wealth, it's analogous to teasing a baby with candy that it can not have. This seems especially true for entrepreneurs, for whom the bootstrapping potential of open source systems is always a huge temptation. The reality is that when an software entrepreneur deals with the "money people", the ideals of free software go out the window pretty quick, because they have to justify their existense with some kind of asset, usually of the IP (intellectual property) variety. This pretty much eliminates the possibilty of using GPL software, because it prevents them from claiming the degree of ownership of the code they produce that investers demand. Paradoxicallly, the situation that some of the biggest current backers of open source software (especially GPL) are giants like IBM. They have loads of assets, and lawyers, and all the rest. Where it makes sense to them, and right now it does a lot, they are willing and able to play by the open soure rules.
I consider myself an open source advocate, but I can sympathize with the business issues, especially entrepreneurial ones. After all, the GPL is espcially good a protecting the rights of the little guy, which in the business world are the entrepreneurs. I think the world desparately requires a working, repeatable business model for (open source) software development--one that spans the whole corporate life cycle. I just don't see one today, and I don't see how to make the GPL business friendly without breaking it.
On the other hand, I only have limited sympathy for the software profession, and the defacto standard model for the software business. The truth is that the majority of what we produce is utter crap. In light of this, I think it is paramount that as a profession, we do everything possible to ensure that software gets better, and good software is as unhindered as possible in getting better and making it to end users. I know of no technology that I believe can achieve this better or even close to the open source process. The GPL is the center of mass for that process. So yes, the GPL isn't perfect, but for now, I say, tough. It is the best we have.
My advice, if you want to try DVI to your TV, that's fine. But if you don't get satisfactory results initially, don't waste any more time. After huge amounts of research, I found out that my TV (a 30" Widescreen CRT by Toshiba) is basically impossible to get working (beyond 640x480) with DVI from a computer. I think the same is true of many TVs. The path of least resistance lies with VGA to component adaptors like the Audio Authority VGA to Component (Y-Pb-Pr) Transcoder (Model 9A60). The 9A60 is a bit pricy, but it was well worth it for me. My theory is that the component input (Y-Pb-Pr) circuitry in TVs is often much more flexible than their DVI interfaces.
As for quality. The VGA route gives you perfectly clean text and menus, pretty much like you are used to on a proper monitor. Video quality is another matter. My set is HDTV capbable, but I don't feel the cost of HDTV where I live is acceptable. So my video quality benchmark is DVD quality. On my TV, watching a DVD via MythTV (actually xine) on my computer is indistinguishable from watching the same DVD via my progressive scan player (also made by Toshiba).
I've researched this a lot, and the only other option I'm still curious about is using an Xbox for the front end. The Xbox can do Y-Pr-Pb directly, and the linux Xbox claim to have it working. My main concern is that the CPU may not be powerful enough in some cases. For example, some of my cable channels are of truly horrible quality (Shaw cable in BC). I use the MythTV feature that transcodes video to compact MPEG-4. This space savings substantial, but the quality on the poor channels is what I'd call VHS level (occassionally much worse). There are playback postprocessing filters that can help this, but they seem to need a lot of CPU.
I do have my computer connected to my HDTV. I don't use 1080i, I find 720p works better for me. It took me a lot of effort to make that work. I learned about the above card in that process.
The card has two strengths:
native component out. this is the way to
go with HDTV. DVI is a nightmare, at least
with my HDTV (a Toshiba).
nvidia. I'm a linux user. Lots of linux
users take issue with nvidia because of binary drivers, but for linux nothing works better
Re:Transport latency and TCP
on
Replacing TCP?
·
· Score: 1
The size of the socket buffer (sndbuf) and the congestion window (cwnd) are separate things. sndbuf is amount of space in the kernel to hold application data. cwnd limits the amount of data TCP will put "in flight" in any given round trip time. generally, sndbuf > cwnd. Usually, sndbuf is a fixed value (64k, 128k), whereas cwnd is adjusted by TCP continuously (as in the sawtooth pattern of TCP sending rate).
It turns out that the (sndbuf - cwnd) can and does contribute a huge amount to the transport latency, often much more than packet retransmissions. Transport latency is the time to send a single byte through a TCP connection from sender to reciever application.
The MINBUF mechanism in our paper essentially makes sndbuf == cwnd by having sndbuf follow cwnd.
-- Buck
Re:Transport latency and TCP
on
Replacing TCP?
·
· Score: 1
There's a whole body of research that concerns improving TCP's congestion control to deal better with long fat pipes.
I think we're talking about different definitions of latency. By transport latency, I mean the time to send a given byte (or small number of bytes, like a single video frame) from the sender to the receiver. Adjustments to TCP's window size happen on a coarser timescale than this, so they are not strictly relevent.
Another type of latency might be the time to transfer a whole file using TCP. I would tend to use the term throughput in that case though.
Re-using window-sizes has been proposed, but is generally regarded as unsafe, and certainly violates current TCP standards. Linux does re-use other values, like the TCP slow-start threshold, I believe.
As for RED, that is what I meant about AQM, of which RED is just one example. An ideal AQM + ECN combination would completely eliminate lost packets for TCP.
-- Buck
Transport latency and TCP
on
Replacing TCP?
·
· Score: 5, Informative
This work seems to be about two things (which I am not sure I see a strong connection between): lowering transport latency, and using available bandwidth better. The latter has been the subject of many papers in the last few years. There are now several serious proposals of how to fix TCP with respect to long fat pipes. They don't seem to support the idea that retransmissions are harmful. So I'm going to talk about the first issue, transport latency.
The idea of using error-correcting codes (ECC) to eliminate the need for retransmissions is an interesting one. The main benefit is to reduce transport latency (the total time it takes to send data from application A to B). Here is another paper proposes has a similar idea, applied at a different level of the network architecture.
The root problem here is that network loss leads to increases in the transport latency experienced by applications. In TCP, the latency increases because TCP will resend data that is lost. That means at least one extra round-trip-time per retransmission. This "Rateless TCP" approach uses ECC so that the lost data can be recovered from
other packets that were not dropped. In this way, the time to retransmit packets may not be needed. I say may, because there will be a loss rate threshold which will exceed the capability of the ECC, and retransmission will become necessary to ensure reliability. But, as long as the loss rate is below the threshold, then retransmissions will not be necessary. Note that the more "resilient" you make the ECC (meaning supporting a higher loss threshold), the more work will be needed at the ends. So you are not eliminating latency due to packet loss, you are simply moving it away from packet retransmission into the process of ECC. However, if you've got good ECC, the total latency will go down.
The ECC approach may be a nice middle ground. But, it the ultimate solution to minimize latency is probably through a combination of active queue management (AQM) and early congestion notification (ECN). Unlike ECC, this approach really would aim to eliminate packet loss in the network due to congestion, and therefore completely eliminate the associated latency. Either ECC or regular TCP would benefit. In a controlled testbed using AQM and ECN, I've completely saturated a network with gigabits of traffic, consisting of thousands of flows, and had virtually no packet loss.
It should also be noted that retransmission is NOT the dominant source of transport latency in of TCP. I am a co-author on a paper that shows another way (other than eliminating retransmission) to greatly reduce the transport latency of TCP.
The basic idea is that the send-side socket buffer turns out to be the dominant source of latency (data sits in the kernel socket buffer waiting for transmission). In the above paper, we show how a dynamic socket buffer (one that tracks the congestion window) can dramatically reduce the transport latency of TCP. We allow applications to select this behaviour through a TCP_MINBUF socket option.
I still get an uneasy feeling from parts of this essay. The link between community governance and control of the commit authority is played up a little too much for my comfort. Open source has a fallback mechanism for users/customers who are unhappy: the code fork. This is one way in which the analogy with newspapers is a bit week. A newspaper is ephemeral, the stories change every day. A "fork" doesn't make sense. Sure you can make your own by going to base news sources, but you can't re-use the mechanical bits that make up the NYTimes layout or the website. If you tried to my a MYTimes that re-cycled the NYTimes content directly, you would certainly be violating their terms of use and copyright.
This article gives me the impression that Sun is still clinging to control of the commit mechanism as a way to exercise ultimate authority over the community. In contrast, if you read interviews with Linus Torvalds, he is usually very careful to express how limited his control is, downplaying the fact that he holds the ulimate "commit" keys, and emphasizing that his true power comes from the amount of respect he has earned (and is able to sustain) from his fellow kernel developers.
The article mentions near the beginning that he tried Fedora, but says nothing more as far as I can tell. He says scarcely more about Redhat. In the Suse rant section, he does complain about Redhat's pricing model being out of reach, but since he's a Ph.D student, I must assume that he is unaware of Redhat's recently announced educational pricing (which is pretty cheap).
I've been reading about the AMD 64 bit processors with great interest. I really like many of the things AMD has done in the x86-64 designs. But the one thing that blows me away is that many of the "desktop" mobos for AMD 64 still only allow a maximum of 2 or 4GB of phyisical RAM. What the hell is the point of a 64bit architecture if you can't use more of the address space than with IA32 processors? Surely not 64bit math?
I would think that machines with 2-16GB of RAM would be the natural zone where AMD64 starts to really do things that are a pain in the ass on IA32. As far as I can tell, few of the current AMD 64 motherboards fall into that space. Bah.
Now there's some overstatement. 802.11b for example: 11Mbs? Yea, we all know that the user only sees about half that.
Yes, I know where the 11Mb number comes from, but these Wi-Fi folks have really pushed the envelope on deceptive marketing with that one.
But is GPL compatible DRM possible?
on
Linus on DRM
·
· Score: 1
Linus' post seems all well and good, except that if falls well short of decribing how a full DRM scheme would actuallly work. Perhaps Linus is just saying "let them try", as long as they follow certaing ground rules. However, the inconsistency in his statement is that he seems to imply that thinks that effective DRM is actually possible with his ground rules. I do not. As most readers here, I'm skeptical that effective DRM is possible, period. But DRM with open source, I seriously doubt it.
I subscribed to Netflix for about a year and began to have the same experience. I suppose Netflix could argue that I was the one who lost the DVDs, not them or the post office. They didn't actually say this. Instead I just noticed one day that my account was "on hold" or something when I tried to select more rentals. I could have called Netflix and argued about it, but I was already fed up with how much the service had deteriorated, so I just cancelled.
Artagel wrote:
"Having read the article, I thought - finally, they came up with a justification that can be sold to consumers for DRM - privacy protection."
The two, privacy and DRM, are *not* the same thing. No amount of slick Microsoft marketing can change that.
Privacy is about communication among a small number of trusted parties. When, I send e-mail to mom, I don't care about preventing mom from broadcasting to the world. I do care that "the man" doesn't know what I said to mom, and that "the man" can't manipulate or tamper with my communications to mom. Public key cryptography can work to solve these problems.
DRM is about controlling communication between a small number of producers and large numbers of "untrusted" customers, for the purpose of maximizing profit. DRM is now, and always will be pure snake oil. If I can see it and hear it, there will be a way I can make an "unauthorized" copy of it. That is what computers *DO*. There is no way that DRM can replace the social trust relationship that works among small numbers of individuals, like mom and myself, with a technology solution enforced between a vast corporate entity and the untrusted hordes, like between Microsoft and everybody else.
I can sustain super smooth 1.5Mbps with UDP, very low jitter, distributed evenly. Or I can burst at 4Mbps and see a complete deaad zone for over 500ms. That is shaper, not TDMA. If it was TDMA as you surmise, the dead zone would occur regardless of whether I did a burst.
I don't claim cable modems don't do TDMA. I know they do. The DOSCIS protocol is basically IP over syncrhonous MPEG-2 TS, but the TDMA period is much more fine grained than 1s.
I live in the Portland, Oregon area. As others have noted, the transition had some hiccups with DNS in the first few hours, but since then it seems OK.
I decided to do a little bit throughput checking with the new setup.
Prior to the transition, I was able to to consistently achieve 3.5-4Mbps downstream speed (using a traffic generator from the place that I work). For about a year now, I have had a 128kbps upstream cap.
Post transition, I see that there is a 1.5Mbps downstream shaper in place. My measurements indicate that this shaper is configured to allocate the 1.5Mbps in 1 second chunks. It is possible to burst to as high as 3.5-4Mbps (I surmise this is the link limit speed imposed on the coax side of cable modem), but the rate drops off a cliff to zero as soon as the 1.5Mbps quota is reached. The overall rate curve for continuous traffic looks like roughly like a square wave.
This has suboptimal interactions with long-lived TCP connections, like file download via ftp or http. The most they will achieve, by my measurements, is about 1.2-1.3 Mbps. This suboptimality is due to the interaction between TCP's congestion control and the shaper (the cliff causes TCP to experience multiple timeouts).
I discussed this setup with a colleague at school, specifically why they would choose such a coarse grained shaper (1s period). We concluded that it favors short bursty traffic, i.e. normal web surfing. Since the average http request is less than 1.5Mb (~180KB), the speed should "feel" more like the 3-4Mbps link rate, even though the maximum average is the 1.5Mbps limit. So, surfing should feel upto 200% faster than the shaper limit, while long-lived TCP connections take about a 20-35% hit.
In summary, their setup favors web surfing over downloads and streaming.
This issues seems to recur frequently, but it still leaves me confused. If it is proprietary hardware technology that these companies fear revealing, then why can't they secure patent protection?
When the patent is issued, then the technology ceases to be a secret. The intellectual property is now protected by legal power of the patents. So releasing an open-source driver should do little to alter the value of the IP to the inventor.
Perhaps they'd rather retain secrecy rather than have to enforce the patent. Not likely, as is there is so little extra secrecy. It's just that it's slightly easier for competitors to look at source than find the relevent patents. And perhaps the patent doesn't reveal or explain some details that driver source might.
Maybe its the uncertainty of the patent process. It takes several years to actually have the patent issued. If the patent application fails, then the cat is out of the bag with no return.
Software technology is obviously different. If hardware guys want to protect software IP, then I have no sympathy. That attitude places them squarely on the slippery slope to proprietary OS's, each aiming to differentiate mediocre hardware. We've been there, and it sucks royally for the consumer.
Slashdot Mirror
I chose it mainly for security. As a former Google engineer, I feel that Google's security expertise is top notch.
According to this link:
http://hacks.mozilla.org/2010/05/firefox-youtube-and-webm/
"Every video on YouTube will be transcoded into WebM. They have about 1.2 million videos available today and will be working through their back catalog over time. But they have committed to supporting everything."
If true, the implications seem pretty deep. Personally, I'd be less intimidated by MPEG-LA FUD knowing that YouTube, the world's most popular source of online video, is walking the WebM walk. Add to that the fairly impressive list of supporters mentioned in the WebM announcement, and it would seem to be time to put-up (show your real legal teeth) or shut-up (no more FUD) for the MPEG-LA (and recent FUD'sters like Mr. Jobs).
On the other hand, Dark Shikari (the x264 developer) said this in his analysis: "Though I am not a lawyer, I simply cannot believe that they will be able to get away with this, especially in today’s overly litigious day and age". His comments comprise a rather more elevated class of FUD. Could this be the mother of mistakes (by Google) that will allow MPEG-LA to find themselves a mother-load of riches in such a deep pocketed infringer?
Interesting times ahead!
> Parent writes:
> The techology companies have paid lip service to trying to solve the problem. They offer up solutions but their heart isn't really in it.
Sure, their heart isn't really in it because they know that the "problem" is fundamentally intractable. DRM is at best a tool in an arms race, but for every DRM scheme there is always a technical counter-measure just off the horizon. Maybe it is the media companies who have the problem, that is they are the ones who only pay lip service to the idea of providing a good product at a fair price.
How about this: http://conal.net/Vertigo/
I saw Conal give a talk on this, and I was very impressed. Pity it didn't seem to lead to anything since Conal has left Microsoft(?).
-- Buck
I think the TFA makes reasonably astute observations with respect to video's ability to use buffering to compensate for high latency. It is important to note that video is a broad term, and the importance of latency can and does vary depending on the kind of video. Video conferencing is a major challenge, because to achieve maximum quality it really does require high bandwidth and low latency simultaneously. However, there is a lot of video streaming that is only unidirectional, whether it be video on demand, or some kind of video broadcast, and in those cases, the TFA is quite right that even very large amounts of latency can be concealed with suitably designed buffering strategies. In that case, network jitter matters much less than bandwidth as far as the video is concerned (if the bandwidth just isn't there, then your video isn't going to be good).
Games are a bit different though. Most games need low latency, but compared to video, they also require a lot less bandwidth. This is important, because it opens a similar kind of opportunity to "work around" performance problems in the network. I would claim that the dominant source of delay in network games will be propagation delay (i.e. speed of light to cover physical distances traversed by the game packets). Yes, queueing delay in the network may be significant, but I doubt it will exceed the propagation delay. So my claim would imply that preferential treatment (away from the game traffic) will hurt games most when it causes some of the game packets to be dropped, possibly causing the game to cope with missing data, or longer latency (if the game reponds by retransmitting data). One simple approach is to send the packets redundantly (pro-active retransmission, FEC, etc.). This redundancy means that even if some of the packets get dropped, others may still get through. This will increase the overall bandwidth of the game traffic, but compared to other traffic (i.e. video) the amount will still be relatively low. Just like video, there may be certain types of game where this may not be feasible... if a game for some reason requires very high bandwidth, then its requirements will look a lot like video conferencing---it will need both high bandwidth and low latency. Almost none of the current games that I know of actually fall into that category.
You may be interested in a paper we wrote a few years back [1]. We also started with the premise that some applications require both minimal latency and maximal bandwidth. In our case the application was our own media streaming system. We came up with our own patch to TCP (in Linux). The patch provided a new socket option, we call TCP_MINBUF. The idea is that you need a certain minimum amount of buffer allow TCP's congestion window to function, but no more. Indeed, in the paper we show that the delay due to socket buffer beyond the congestion window is often by far the dominant source of latency--not retransmissions, or delayed acks, or all the other more commonly cited things. So basically what TCP_MINBUF does, is dynamically size the socket buffer to follow the congestion window size. It had a huge impact on latency.
i wqos2002.pdf
[1] "Supporting Low Latency TCP-Based Media Streams", Ashvin Goel, Charles Krasic, Kang Li, and Jonathan Walpole. Tenth International Workshop on Quality of Service (IWQoS), May 2002.
http://www.eecg.toronto.edu/~ashvin/publications/
There's the quote we all know about capitalism that says "it is not perfect, but it is the least flawed system we know". For open source software, this is exactly what a lot of people (myself included) think of the GPL. [ My apologies for paraphrasing and the lack of proper attribution ]
At the same time, a lot of people dislike or even hate the GPL. In my view, their opinions usually come down to the fact that there really is no general example of how to make a business of open source software development. To people who's primary concern is the business of software development, the wealth of GPL software is a false wealth, it's analogous to teasing a baby with candy that it can not have. This seems especially true for entrepreneurs, for whom the bootstrapping potential of open source systems is always a huge temptation. The reality is that when an software entrepreneur deals with the "money people", the ideals of free software go out the window pretty quick, because they have to justify their existense with some kind of asset, usually of the IP (intellectual property) variety. This pretty much eliminates the possibilty of using GPL software, because it prevents them from claiming the degree of ownership of the code they produce that investers demand. Paradoxicallly, the situation that some of the biggest current backers of open source software (especially GPL) are giants like IBM. They have loads of assets, and lawyers, and all the rest. Where it makes sense to them, and right now it does a lot, they are willing and able to play by the open soure rules.
I consider myself an open source advocate, but I can sympathize with the business issues, especially entrepreneurial ones. After all, the GPL is espcially good a protecting the rights of the little guy, which in the business world are the entrepreneurs. I think the world desparately requires a working, repeatable business model for (open source) software development--one that spans the whole corporate life cycle. I just don't see one today, and I don't see how to make the GPL business friendly without breaking it.
On the other hand, I only have limited sympathy for the software profession, and the defacto standard model for the software business. The truth is that the majority of what we produce is utter crap. In light of this, I think it is paramount that as a profession, we do everything possible to ensure that software gets better, and good software is as unhindered as possible in getting better and making it to end users. I know of no technology that I believe can achieve this better or even close to the open source process. The GPL is the center of mass for that process. So yes, the GPL isn't perfect, but for now, I say, tough. It is the best we have.
My advice, if you want to try DVI to your TV, that's fine. But if you don't get satisfactory results initially, don't waste any more time. After huge amounts of research, I found out that my TV (a 30" Widescreen CRT by Toshiba) is basically impossible to get working (beyond 640x480) with DVI from a computer.
I think the same is true of many TVs. The path of least resistance lies with VGA to component adaptors like the Audio Authority VGA to Component (Y-Pb-Pr) Transcoder (Model 9A60). The 9A60 is a bit pricy, but it was well worth it for me. My theory is that the component input (Y-Pb-Pr) circuitry in TVs is often much more flexible than their DVI interfaces.
As for quality. The VGA route gives you perfectly clean text and menus, pretty much like you are used to on a proper monitor. Video quality is another matter. My set is HDTV capbable, but I don't feel the cost of HDTV where I live is acceptable. So my video quality benchmark is DVD quality. On my TV, watching a DVD via MythTV (actually xine) on my computer is indistinguishable from watching the same DVD via my progressive scan player (also made by Toshiba).
I've researched this a lot, and the only other option I'm still curious about is using an Xbox for the front end. The Xbox can do Y-Pr-Pb directly, and the linux Xbox claim to have it working. My main concern is that the CPU may not be powerful enough in some cases. For example, some of my cable channels are of truly horrible quality (Shaw cable in BC). I use the MythTV feature that transcodes video to compact MPEG-4. This space savings substantial, but the quality on the poor channels is what I'd call VHS level (occassionally much worse). There are playback postprocessing filters that can help this, but they seem to need a lot of CPU.
The size of the socket buffer (sndbuf) and the congestion window (cwnd) are separate things. sndbuf is amount of space in the kernel to hold application data. cwnd limits the amount of data TCP will put "in flight" in any given round trip time. generally, sndbuf > cwnd. Usually, sndbuf is a fixed value (64k, 128k), whereas cwnd is adjusted by TCP continuously (as in the sawtooth pattern of TCP sending rate).
It turns out that the (sndbuf - cwnd) can and does contribute a huge amount to the transport latency, often much more than packet retransmissions. Transport latency is the time to send a single byte through a TCP connection from sender to reciever application.
The MINBUF mechanism in our paper essentially makes sndbuf == cwnd by having sndbuf follow cwnd.
-- Buck
There's a whole body of research that concerns improving TCP's congestion control to deal better with long fat pipes.
I think we're talking about different definitions of latency. By transport latency, I mean the time to send a given byte (or small number of bytes, like a single video frame) from the sender to the receiver.
Adjustments to TCP's window size happen on a coarser timescale than this, so they are not strictly relevent.
Another type of latency might be the time to transfer a whole file using TCP. I would tend to use the term throughput in that case though.
Re-using window-sizes has been proposed, but is generally regarded as unsafe, and certainly violates current TCP standards. Linux does re-use other values, like the TCP slow-start threshold, I believe.
As for RED, that is what I meant about AQM, of which RED is just one example. An ideal AQM + ECN combination would completely eliminate lost packets for TCP.
-- Buck
This work seems to be about two things (which I am not sure I see a strong connection between): lowering transport latency, and using available bandwidth better. The latter has been the subject of many papers in the last few years. There are now several serious proposals of how to fix TCP with respect to long fat pipes. They don't seem to support the idea that retransmissions are harmful. So I'm going to talk about the first issue, transport latency.
The idea of using error-correcting codes (ECC) to eliminate the need for retransmissions is an interesting one. The main benefit is to reduce transport latency (the total time it takes to send data from application A to B). Here is another paper proposes has a similar idea, applied at a different level of the network architecture.
The root problem here is that network loss leads to increases in the transport latency experienced by applications. In TCP, the latency increases because TCP will resend data that is lost. That means at least one extra round-trip-time per retransmission. This "Rateless TCP" approach uses ECC so that the lost data can be recovered from other packets that were not dropped. In this way, the time to retransmit packets may not be needed. I say may, because there will be a loss rate threshold which will exceed the capability of the ECC, and retransmission will become necessary to ensure reliability. But, as long as the loss rate is below the threshold, then retransmissions will not be necessary. Note that the more "resilient" you make the ECC (meaning supporting a higher loss threshold), the more work will be needed at the ends. So you are not eliminating latency due to packet loss, you are simply moving it away from packet retransmission into the process of ECC. However, if you've got good ECC, the total latency will go down.
The ECC approach may be a nice middle ground. But, it the ultimate solution to minimize latency is probably through a combination of active queue management (AQM) and early congestion notification (ECN). Unlike ECC, this approach really would aim to eliminate packet loss in the network due to congestion, and therefore completely eliminate the associated latency. Either ECC or regular TCP would benefit. In a controlled testbed using AQM and ECN, I've completely saturated a network with gigabits of traffic, consisting of thousands of flows, and had virtually no packet loss.
It should also be noted that retransmission is NOT the dominant source of transport latency in of TCP. I am a co-author on a paper that shows another way (other than eliminating retransmission) to greatly reduce the transport latency of TCP. The basic idea is that the send-side socket buffer turns out to be the dominant source of latency (data sits in the kernel socket buffer waiting for transmission). In the above paper, we show how a dynamic socket buffer (one that tracks the congestion window) can dramatically reduce the transport latency of TCP. We allow applications to select this behaviour through a TCP_MINBUF socket option.
-- Buck
I still get an uneasy feeling from parts of this essay. The link between community governance and control of the commit authority is played up a little too much for my comfort. Open source has a fallback mechanism for users/customers who are unhappy: the code fork. This is one way in which the analogy with newspapers is a bit week. A newspaper is ephemeral, the stories change every day. A "fork" doesn't make sense. Sure you can make your own by going to base news sources, but you can't re-use the mechanical bits that make up the NYTimes layout or the website. If you tried to my a MYTimes that re-cycled the NYTimes content directly, you would certainly be violating their terms of use and copyright.
This article gives me the impression that Sun is still clinging to control of the commit mechanism as a way to exercise ultimate authority over the community. In contrast, if you read interviews with Linus Torvalds, he is usually very careful to express how limited his control is, downplaying the fact that he holds the ulimate "commit" keys, and emphasizing that his true power comes from the amount of respect he has earned (and is able to sustain) from his fellow kernel developers.
The article mentions near the beginning that he tried Fedora, but says nothing more as far as I can tell. He says scarcely more about Redhat. In the Suse rant section, he does complain about Redhat's pricing model being out of reach, but since he's a Ph.D student, I must assume that he is unaware of Redhat's recently announced educational pricing (which is pretty cheap).
I've been reading about the AMD 64 bit processors with great interest. I really like many of the things AMD has done in the x86-64 designs. But the one thing that blows me away is that many of the "desktop" mobos for AMD 64 still only allow a maximum of 2 or 4GB of phyisical RAM. What the hell is the point of a 64bit architecture if you can't use more of the address space than with IA32 processors? Surely not 64bit math?
I would think that machines with 2-16GB of RAM would be the natural zone where AMD64 starts to really do things that are a pain in the ass on IA32. As far as I can tell, few of the current AMD 64 motherboards fall into that space. Bah.
Now there's some overstatement. 802.11b for example: 11Mbs? Yea, we all know that the user only sees about half that.
Yes, I know where the 11Mb number comes from, but these Wi-Fi folks have really pushed the envelope on deceptive marketing with that one.
Linus' post seems all well and good, except that if falls well short of decribing how a full DRM scheme would actuallly work. Perhaps Linus is just saying "let them try", as long as they follow certaing ground rules. However, the inconsistency in his statement is that he seems to imply that thinks that effective DRM is actually possible with his ground rules. I do not. As most readers here, I'm skeptical that effective DRM is possible, period. But DRM with open source, I seriously doubt it.
I subscribed to Netflix for about a year and began to have the same experience. I suppose Netflix could argue that I was the one who lost the DVDs, not them or the post office. They didn't actually say this. Instead I just noticed one day that my account was "on hold" or something when I tried to select more rentals. I could have called Netflix and argued about it, but I was already fed up with how much the service had deteriorated, so I just cancelled.
Artagel wrote:
"Having read the article, I thought - finally, they came up with a justification that can be sold to consumers for DRM - privacy protection."
The two, privacy and DRM, are *not* the same thing. No amount of slick Microsoft marketing can change that.
Privacy is about communication among a small number of trusted parties. When, I send e-mail to mom, I don't care about preventing mom from broadcasting to the world. I do care that "the man" doesn't know what I said to mom, and that "the man" can't manipulate or tamper with my communications to mom. Public key cryptography can work to solve these problems.
DRM is about controlling communication between a small number of producers and large numbers of "untrusted" customers, for the purpose of maximizing profit. DRM is now, and always will be pure snake oil. If I can see it and hear it, there will be a way I can make an "unauthorized" copy of it. That is what computers *DO*. There is no way that DRM can replace the social trust relationship that works among small numbers of individuals, like mom and myself, with a technology solution enforced between a vast corporate entity and the untrusted hordes, like between Microsoft and everybody else.
Ooops. That should be 1Gbs. :-}
Guess that's what preview is for.
Bullshit. See my other post. The E1000 XT rocks.
The driver seems to be updated regularly, but not because it is flakey. They have loads of advanced features: VLANs, teaming, failover, etc.
They're not GPL, but the source is fully available.
We've just setup up a gigabit cluster with new
u pe rServer6012P-6.htm
:-} I havn't been able to test how much more CPU headroom they give me.
1u servers from supermicro:
http://www.supermicro.com/PRODUCT/SUPERServer/S
I've tested it using a CISCO catalyst 3500XL switch. Our new 4000 switch goes online this week.
I can get a fully saturated link using a program I wrote:
http://mxtraf.sf.net
I'm using a 2.4.18 kernel with the latest drivers from Intel.
BTW, anybody know how to enable jumbo frames in IOS?
-- Buck
I think you are wrong.
I can sustain super smooth 1.5Mbps with UDP, very low jitter, distributed evenly. Or I can burst at 4Mbps and see a complete deaad zone for over 500ms. That is shaper, not TDMA. If it was TDMA as you surmise, the dead zone would occur regardless of whether I did a burst.
I don't claim cable modems don't do TDMA. I know they do. The DOSCIS protocol is basically IP over syncrhonous MPEG-2 TS, but the TDMA period is much more fine grained than 1s.
I live in the Portland, Oregon area. As others have noted, the transition had some hiccups with DNS in the first few hours, but since then it seems OK.
I decided to do a little bit throughput checking with the new setup.
Prior to the transition, I was able to to consistently achieve 3.5-4Mbps downstream speed (using a traffic generator from the place that I work). For about a year now, I have had a 128kbps upstream cap.
Post transition, I see that there is a 1.5Mbps downstream shaper in place. My measurements indicate that this shaper is configured to allocate the 1.5Mbps in 1 second chunks. It is possible to burst to as high as 3.5-4Mbps (I surmise this is the link limit speed imposed on the coax side of cable modem), but the rate drops off a cliff to zero as soon as the 1.5Mbps quota is reached. The overall rate curve for continuous traffic looks like roughly like a square wave.
This has suboptimal interactions with long-lived TCP connections, like file download via ftp or http. The most they will achieve, by my measurements, is about 1.2-1.3 Mbps. This suboptimality is due to the interaction between TCP's congestion control and the shaper (the cliff causes TCP to experience multiple timeouts).
I discussed this setup with a colleague at school, specifically why they would choose such a coarse grained shaper (1s period). We concluded that it favors short bursty traffic, i.e. normal web surfing. Since the average http request is less than 1.5Mb (~180KB), the speed should "feel" more like the 3-4Mbps link rate, even though the maximum average is the 1.5Mbps limit. So, surfing should feel upto 200% faster than the shaper limit, while long-lived TCP connections take about a 20-35% hit.
In summary, their setup favors web surfing over downloads and streaming.
This issues seems to recur frequently, but it still leaves me confused. If it is proprietary hardware technology that these companies fear revealing, then why can't they secure patent protection?
When the patent is issued, then the technology ceases to be a secret. The intellectual property is now protected by legal power of the patents. So releasing an open-source driver should do little to alter the value of the IP to the inventor.
Perhaps they'd rather retain secrecy rather than have to enforce the patent. Not likely, as is there is so little extra secrecy. It's just that it's slightly easier for competitors to look at source than find the relevent patents. And perhaps the patent doesn't reveal or explain some details that driver source might.
Maybe its the uncertainty of the patent process. It takes several years to actually have the patent issued. If the patent application fails, then the cat is out of the bag with no return.
Software technology is obviously different. If hardware guys want to protect software IP, then I have no sympathy. That attitude places them squarely on the slippery slope to proprietary OS's, each aiming to differentiate mediocre hardware. We've been there, and it sucks royally for the consumer.