Slashdot Mirror
A New Way to Look at Networking
Van Jacobson gave a Google Tech Talk on some of his ideas of how a modern, global network could work more effectively, and with more trust in the data which changes many hands on its journey to its final destination.
Watch the talk on Google's site
The man is very smart and his ideas are fascinating. He has the experience and knowledge to see the big picture and what can be done to solve some of the new problems we have. He starts with the beginning of the phone networks and then goes on to briefly explain the origins of the ARPAnet and its evolution into the Internet we use today.
He explains the problems that were faced while using the phone networks for data, and how they were solved by realizing that a new problem had risen and needed a new, different solution. He then goes to explain how the Internet has changed significantly from the time it started off in research centres, schools, and government offices into what it is today (lots of identical bytes being redundantly pushed to many consumers, where broadcast would be more appropriate and efficient).
"He then goes to explain how the Internet has changed significantly from the time it started off in research centres, schools, and government offices into what it is today (lots of identical bytes being redundantly pushed to many consumers, where broadcast would be more appropriate and efficient)."
Multicast.
The talk was held on Aug 30, 2006.
Lars T.
To the guy who modded me down from perfect to terrible Karma - Apple haters still suck
"...where broadcast would be more appropriate and efficient"
If this means airwaves, same as TV, sure. Why not, since the whole thing is one big info-mercial swamp already. Otherwise, it also means guaranteed next -packet delivery, without any pauses, resends, spinning cursors. And the internet is not going to deliver that, sorry.
How is this anything new? Everything in that summary was already covered by Tanenbaum in his excellent book on networks - of course it's easier to hear it from someone if you're too lazy to read :p
I have spoken'eth.
How is the 3rd comment redundant?
Idiot.
There is no reason you can't multicast across a large segmented network, i.e. the internet, and get good delivery. Radio, television, audio, phone, movies are all latency sensitive but not particularly bit sensitive so you can drop some packets here and there. That also means that some things would need QoS (VoIP) while others would need intelligent caching and buffering (movies, etc.).
[RIAA] says its concern is artists. That's true, in just the sense that a cattle rancher is concerned about its cattle.
So stop posting videos! I wish we still had those 14kbps modems...
This was something we watched in the networking class last semester (fall 2006). Very interesting though.
"(lots of identical bytes being redundantly pushed to many consumers, where broadcast would be more appropriate and efficient)"
The first part is true, but does not necessarily lead to the conclusion in the second. There is a huge, very important IF that belongs between them. Specifically, "if the recipients are all prepared to receive those bytes at the same time". The problem with the conclusion is that the evaluation of the "if" part is nearly always "they're not". This is yet another case of "if the internet were like television, it'd be more efficient". Yes, but it would then no longer be the internet people like. The great promise of the internet is information on demand. All this bullcrap about broadcast, push, and the like, it's all the efforts of 20th century throwbacks trying to fit the internet into their outdated worldview of "producers" and "consumers". They need to quit it. Broadcast is a square peg and the internet is a round hole. Every time anyone suggests putting the two together, they simply look like a bloody idiot.
If a job's not worth doing, it's not worth doing right.
I 'browsed' some of this video and book marked it for later: Van Jacobson's background is awesome.
A bit off topic, but there are two things that I want to see happen: a complete upgrade to IPv6 and the creation of an alternative 'public Internet' based on emerging long distance wifi and software that lets people volunteer to be part of this new open grid, and optionally share some bandwidth bridging the 'real' Internet.
It may seem pointless to want both higher performance (multi-casting UDP, essentially infinite IP address space) and low performance and ad-hoc systems, but please consider: the UK and USA seem to be going down the wrong path of surveillance and citizen control, the Internet may someday be viewed as something that the public just should not have because it is too free a source of information. I hope that I am wrong about this, but this unpleasant possible repressive future is a possibility.
Of all things, how the hell is this post redundant?
I think bittorrent is the internet answer to the broadcast problem. Bittorrent is intrinsically adapted to the way the internet works. Data which is most sought by people will be found on more nodes around the net, less popular data can be downloaded directly from the primary servers.
This is something that's been on my mind for a long time. In fact, I thought that's how streaming was done because I couldn't understand why the load would increase so much as more people watched. This should be especially true for internet radio/TV, and for ads that really suck up the bandwidth, which why I block them (the ads that is). I didn't know that each stream was being fed to only one viewer/listener. It always seemed kind of odd to do it that way.
What?
Based on all the measurements I'm aware of, Linux has the fastest & most complete stack of any OS (source)
The talk is 1 h 21 m long. It's actually quite fascinating, Van Jacobson is teh bomb.
...
...
But a few things caught my OCD in the closed caption stream
awk => oc
combinatorics => common torch
The whole time I'm watching this thing, I'm thinking to myself, why doesn't that girl at the front of the audience just shut up and listen? She's gesturing wildly to her neighbor and not paying attention at all! Then the revelation hits me, she's signing for the hearing impaired. Pass the "insensitive clod" hat, I'm out
It was to be the next big thing in the 90's. I think it was just before push technology that everyone was talking about multicast turning our PC's into TV's with "high quality" video through dial-up speeds.
So, sending identical packets to everyone is somehow more bandwidth efficient than sending packets to only those who want them? Doesn't that seem backwards to anyone else? Furthermore, couldn't you define broadcasting as precisely the act of sending identical bytes to many consumers?! I'm teh confused.
TLF
I do not respond to cowards. Especially anonymous ones.
Well yes, he does.
Also at 7:01 he says "I'm an old fart" but the subtitle gives this as "I'm an old part".
Who cares?
Paid Q&A/Research
Is there a transcript of the video available (e.g. just the subtitles pulled out)? It's a bit tedious watching this when reading it will take 1/10th of the time of the video...
haven't we solved that with proxies?
FTA: and with more trust in the data which changes many hands on its journey to its final destination
I hope it doesn't change my hands!
Everything he says applies to server>client. Producers>consumers. And he proposes a change to the current model of conversation to a model of multicast the same data to many consumers. And support this by findings that 99% of data is structured that way. I guess that's wishful thinking though, because he works at google, the massive meta-producer of data we all consume.
What about bittorrent though? Uses the same TCP/IP protocol, trusts the data, not the source (like he says we must do), answers to the contemporary question "Who has the time" (ditto) AND DELIVERS.
They are also other decentralization options to bittorrent, like trackerless torrents (DHT) and torrentless torrents (Magnet links)
Ditto there are implementations for p2p phones, p2p tv, etc.
I think google and other big companies are just pushing a way to change the Internet to suit their business models and nothing more.
The summary was posted on May 06, 2007 at 08:20.
I just read Slashdot for the articles.
- Only people actively downloading or seeding content are available to redistribute it, so there's a built-in time-dependence which makes in unsuitable for small pieces of data.
- It has no knowledge of topology, so it can't take advantage of topology to cache data closer to endpoints.
I wish someone had asked about freenet, since that seems much closer to what he describes."If you look 'round the table and can't tell who the sucker is, it's you." -- Quiz Show
The architecture he describes is reminds me surprisingly closely to freenet.... other than the ability to do really short term routing, and any sort of 'fairness' built into the network (he talks about nodes that offer more connectivity or storage should somehow be given priority so that there's a feedback loop that encourages people to share more resources)
...does anyone have an answer?
I wonder if he has an answer for the latency involved in the freenet implementation...
Wow, some great subtitling here. The guy programs in oc and apparently speaks about item potent data packets...
I enjoyed this talk very much. It was more than just a statement of Van Jacobson's thoughts on data dissemination. It showed his analysis of the relationship between infrastructure and application across two generations of networking, and it pointed out very nicely why it's time now for phase 3: we've moved our usage goalposts compared to when the IP network was designed. Great stuff, and I agree completely.
:-)
:-)
The article submitter didn't seem to "get" what Van Jacobson was saying though, as the talk had almost nothing to do with broadcasting or multicasting. Indeed, Van Jacobson actually pointed out why multicasting and broadcasting were inappropriate in most situations in this new world (they carry implicit time sync), so only use them as accelerators on LANs or in other special cases. The slightly wrong article description may have misdirected some of the posts here since not everybody reads TFA, and even fewer sit through an extended talk. It wasn't about broadcast or multicast at all, except in passing.
Maybe it'll help to summarize his thrust briefly.
What he said was that the network underneath doesn't actually matter, and that the wires and fibre underneath don't actually matter either -- TCP/IP has abstracted away from them. However, the client-server model on which TCP/IP is based is no longer strictly relevant either, because it is founded on a somewhat obsolete concept, the "conversation". The vast bulk of our Internet traffic is no longer "conversations", but "data dissemination" (the migration of identified data objects from place to place), and actual conversations are just a special case of that.
Data dissemination is utterly different to conversation as a communications paradigm, and that's what he's getting at. Fully identified, self-validating items of data as discrete entities are really where our focus needs to be, and how they get to us is rather immaterial, or abstracted away. *Where* they come from (ie. the actual server to which we connect) is quite immaterial too --- getting it from a passing plane would be as good as from a known server, when you can rely on data identity. Furthermore, if the data items were fully self-descriptive then many of the current problems like spam would go away as well. What's more, the nodes of the network would be able to work more intelligently too (and hence efficiently), if they were aware of data identity rather than just treat everything as a conversation.
That's a very brief summary and can't hope to do the talk justice. Go listen! He's dead right.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
You give a pretty good short summary of a long and interesting talk.
One thing I pulled most out of it was the analogy to 60s and 70s networking and how it is only after technology has been adopted that we see what its used for.
When the telephone was invented Bell didn't know what it would be used for, its a strange concept but he really didn't know what a "phone call" was. He just knew he could transmit voice. Not only that but you had to have wires to connect people, so there was this very expensive business of putting wires everywhere. What happened was that people used those wires to make conversations. To establish a conversation you had to have a path between two nodes. This encouraged a monopoly because the best known way to make paths was to have control of all the wires.
When the idea of what TCP/IP was to become was introduced people thought it was lunacy. What they were proposing was adding all this crap onto your data to explicitly name your destination so that it could travel any path to get to its conversation partner. All the networking researchers didn't get it because they already had implicit addresses by way of making the path. Turns out that the supposed innefficiency solved several problems simply by construction. Being able to take any path meant not caring about the underlying topology.
What Van Jacobsen is proposing is another abstraction. Essentially adding another layer of "crap" that will allow us to ignore the underlying network. He mentions how several technologies are working towards these ends to some degree like bittorrent and akami CDN, but I think he is advocating for something like a new protocol. This new protocol would then end up solving some of our current problems simply by construction. Broadcast and one-to-one will become the same thing. Whether you are sending a secure email (pgp signed and named) or downloading the front page of the nytimes you could rely on the nature of the new protocol to deliver you authentic data, no matter where it comes from.
Personally I think its genius, I'd like to follow the progress of such a protocol if it exists. I just got done watching the talk so I'll be googling around for a little I suppose.
"how can they call it a MINE if everything here is THEIRS?!?!" -Straight Jacket
...but the more he talked, the more it reminded me of some halfbreed between akamai and freenet.
Basicly, he's speaking of named resources, that an URL would be key like KSKs in Freenet
Content would self-verify, that's basicly CHKs in Freenet
Then you need add security into it which pretty much amount to SSKs
Only in his case, it wasn't talk about making the end nodes treat information this way but rather the core of internet, and it didn't involve anonymity. But the general idea was the same, to grab content from a persistant swarm of hosts who doesn't need a connection to the original source. Unfortunately, most of the examples he gives are simply false, like the NY Times front page. If I want up-to-the-minute news everybody need to pull fresh copies off the original source all the time, reducing it down to a caching proxy. Any sort of user-specifc content, or interactive content won't work. For example take slashdot. I've got my reading preferences set up, which means my content isn't the same as yours. Also my front page contains a link to my home page, which is not the same as yours. Getting a posting form and making a comment wouldn't be possible. Making any kind of feedback like digg, youtube, article feedback etc. isn't possible. Counters wouldn't be possible. The only thing where it'd work is reasonably static fire-and-forget content, and even then there's the problem of knowing what junk to keep. Notice that when asked about BT he said that only worked for big files, so the idea is that everyone will have some datastore where they keep small files until someone needs them. The only good example is the Olympic broadcast, which is exactly the same content at exactly the same time. Oh wait, that's classic broadcast. Classic broadcast works best in a broadcast model? Who'd think that.
Live today, because you never know what tomorrow brings
I see one problem with his idea of ignoring where data comes from.
Corporations make money by restricting access to information.
It doesn't seem that it will be possible for them to continue to do that with this model, so I don't think any of this will come to pass any time in the near future.
1) Optical only routing at the backbone, and traffic monitoring to ensure Tier1's don't
let major choke points choke due to not wanting to spend money on equipment,
share the costs like a Co-op if necessary amongst Tier1's to keep the backbones
and choke points scaled up. People who run trace routes see tier1 choke points now.
2) In the Tier1 core make sure it is DWDM folding many layers of Sonet/ATM into
a multi-channel frequency spectrum, at some point plan on phasing out asynchronous
communications and go with on time in order delivery of packets, aka Sonet and its like protocols.
*** VoIP/streams works best with synchronous***
3) Each Metro Area Network has its own Squid box cluster for reducing repetitive traffic,
filling all identical requests for data vs. pulling it long haul a 2nd time,
alot of ISPs have their own squid like system now, but unify and optimize it.
4) Create a spam registry, like ACLs for routers, don't blacklist by single IP,
blacklist by ISP, then Inclusively activate by white list single IP's within that range.
Offenders would essential end up shutoff from most of the world.
5) Create a modified Torrent protocol that can multicast via VPN like hamachi.cc,
so that a single peer can send the same data at 50kb to a million ppl,
make it like tuning into net radio audio stream in progress picking up the
data in progress, and getting the first to middle bits on the 2nd "play" of the data stream,
set a threshold for the modified torrent protocol to switch to this multicast mode.
for ppl with privacy concerns make it virtual ip's on a VPM network like hamachi.cc
6) All monetary transaction sites require end to end encryption and authentication,
and some means of client IDS that verifies the client isn't compromised,
a cursory one would be patch level and malware/virii client acknowledgement.
CERN could maintain a central footprint repository or another agency could
do it, but have open peer review and input.
Verifying legitimacy of all in memory processes would get rid of the obvious
intruders, but the ones that can 'checksum' mimic known registered processes
would be more difficult to detect. Verifying client integrity particularly for
Windows OS would require some brilliance beyond what we have now.
7) Zombie / IDS Net Spiders that roams the internet looking for compromised systems
and notifying the owner, and the ISP, and reports it to all other ISP's and
is added to a blacklist at ISP of origination. Systems routing to known
zombie network reporting IP's or security threat sites would be placed on watch lists
and client warnings of possibly being compromised.
8) Global and Nation based NOC's that monitor for network issues,
and traffic control system that allows truly functioning
reroutes via higher layer routing protocols for alternate routes.
Just a few ideas I had, some it may not be workable with what we have now.
google "32 trillion offshore needs IRS attention"
The data contains all the information, so why not an authentication system. DRM for data. If you can validate if the data is from Company X then the data should be able to validate that you are allowed to see it. You still can have a copy of the data on your server, just can't use it.
What power has law where only money rules.
I think the idea of having a fully transparent networking paradigm is what is of paramount importance to both data and software that manages that data. Adding application layer logic to routers that will effectively cause data I want (or need) to be available everywhere would (in my mind) restrict the things that we can do with the net. I'm a bottom up kind of guy.
We need not only the data but the intelligence to manage that data spread around. I can't think of any news or information site that doesn't require direct interaction with the user of that data. I mean, take Slashdot for example, it tracks your usage of the site and gives you moderator points when you meet certain criteria. How is that going to work if somewhere along the line you don't have some kind of interaction with Slashdot's software? How does that get distributed as well? Are we going to allow server algorithms to run everywhere as well? I guess if we solve the secuirty problem, that would actually be quite nice.
I agree there needs to be a better mechanism for delivering content everyone wants more efficiently. I'm reminded of this as well when I go to watch a live event and the response is very slow (not to mention 30 to 45 seconds behind), or I try to download a large file. Also with VOIP routing headers taking up 30% of wireless bandwidth alone, something has to be done with our protocols so they are lighter weight.
Just as mechanical switches had to be programmed and were part of the problem, I think TCP/IP is part of the problem here as well. There are still (as he mentioned) all kinds of barriers that have to be known about and planned for. As he said, it is a very static system requiring a day or at least minutes to pass before you can be seen by someone else on the net. I use a networking system that is tightly integrated into an operating system (QNX) that I like very much (still has its weaknesses) but it goes a long way to making the entire computing platform very transparent with the ability to add security.
Content can be very static or it can be extremely dynamic (sensor data in the dynamic example). I think his ideas seem focus around managing very static data or data that can be "late" and making it more readily available. When it comes to dynamic content (such as live video or VOIP) he didn't seem to offer much in the way of substance, or I may have just missed his point. I guess I see the need for direct "conversation" like connections as well as data "dissemination" type connections.
In fact, I'm not sure things like the NY Times daily is a good example, as dynamic advertising is very much a part of what they sell, and they need to have their individual subscribers connect to their servers so they can analyze what advertising to focus on the poor unsuspecting reader.
I personally think that the dynamics of the internet are moving towards very dynamic content and this requires direct connections to the servers which manage that content. Unless this content management (software) can be distributed to other devices along with the data, I'm not sure its going to fly. Microsoft already tried that and now most interactive content is blocked (unless specifically enabled) because of security and other concerns.
However, these are all application layer problems, do they really belong at the bit level of data being bounced around? I don't know and it is an interesting question.
We can add layers that do all kinds of fancy things with data to get it to places that were harder for it to get to previously (mirrors, caching, etc). However, in order to make work what he is indicating makes the internet look like nothing more than a big informational resource (file versions, news, video, etc). Like a big file system or database, like a big static storage system. However, the internet is much more dynamic than that and I don't have the feeling that he addressed that aspect well enough.
What we need is something that will enable us to make "conversation" connections
I got it, I just posted this late at night with a poor description. In any case people are watching and talking about it, which was my goal.
this space intentionally left blank
That was one of very few useful talks I've *ever* seen on shortcomings in the Internet.
Akamai Technologies is really very much in the business of solving the main problem Jacobson describes. Yes, lots of people want the same information. Jacobson is a very bright man, and he got pretty much everything right except: "You can't Akamize dynamic content." Yes, you can -- unless live feeds of sporting events (NCAA March Madness) aren't considered dynamic enough.
That said, there probably is room for a truly open platform in this case. If it can do half of what the Akamai network can, I wholeheartedly congratulate the creators their amazing feat of engineering. To my knowledge, the closest thing these days is the coral cache, which actually is restricted to static content.
-Amalcon
Corporations make money by restricting access to information.
:-)
Judging by the many jabs he took at Microsoft and at the PTTs and the RIAA etc, I'd say that Van Jacobson is very much in touch with companies and their shenanigans. He almost certainly expects them to have learned nothing from their experience so far, and to stick DRM into the data objects distributed by his "3rd phase" network as well.
In any case, his data dissemination approach would work regardless of what's actually in each data object, and whether it's DRM'd or not. He seems to be very agnostic on content as befits a networking scientist, but on the issue of control then he's strongly in favor of empowering the user. Spammers won't like that, and nor of course will control freaks like media corporations. Tough.
At one point in his talk, Van Jacobsen talks about segmenting. He mentions that segmenting solves a problem that is metaphorically like having trains and cars on the same city streets; one doesn't want to wait around in a car for a train to clear the intersection. Then he says something that I've heard before, but never made the connection:
"It would be nice here to not have big trucks, just little cars."
And suddenly I realized that Senator Stevens had gotten a lecture that he completely misunderstood.
Granted, the man shouldn't have tried to give a speech on something about which he was completely clueless, but now I think I know where the heck that truck he mentioned came from.
I've got my reading preferences set up, which means my content isn't the same as yours.
It's partially the same as mine. I have a different set of preferences to you but there is some overlap between us --- that's extremely likely to be the norm.
Consequently any data dissemination router on the path between Slashdot and our two machines needs to fetch those data items that we both want from upstream only once, and then we both get it sent to us out of its item store. As he said, it's somewhat like a cache, but differs in not being tied to any specific upstream source.
Making any kind of feedback like digg, youtube, article feedback etc. isn't possible.
Of course it is. All feedback does is create a new data item. If the result is private then it's no different to now, and if it's shared then this merely adds your data item to the set of current items on the server. Anyone who is interested in items of your type will then see it the next time that they request the list of new or updated objects.
It's worth noting that even if this list of updates comes from the central server, the data items themselves needn't --- they may already be stored in intermediate nodes on the path, as a result of someone else's slightly earlier query. In effect, this is a data-centric distributed cache with the role of the servers reduced to state updates and data injection only.
When I'm logged into slashdot and browsing news, there's only a very small part of the page that is customized specifically to me. Everything else is the exact same content that everyone else is seeing. Currently the web browser does separate queries to pull down the images in the page, which are mostly the same for everyone. Perhaps under VJ's scheme the text parts that are specific to me would have to come pretty much directly from slashdot's site, but it could contain references to all the common content on the site which would be fully cacheable by his dissemination model.
Instead of slashdot's content authors writing templates that are processed on their servers before giving the web page to my browser, something similar to their template would be shipped to my browser, which would then ask for all the common (cacheable) content to fill out the page. Image-heavy pages would already work well with his scheme if the browser just requested the images using the new semantic.
I don't think he plans for the "request/respond" protocols to be used everywhere in a network. The issue is when many users are requesting the same data. Take for example when a site gets "slashdotted," it'd be more efficient if there weren't several thousand requests for the same exact index.html from some host being individually transmitted through the network.
In your example of tracking users on Slashdot, parts of that could fall back on the current conversational protocols. It should probably be possible for clients to decide on whether they want to request data by means of the broadcast system or the conversational system. Then I guess the issue is how does the client decide when it should use one or when it should use the other?
Or, maybe custom homepages and whatnot would have to start being initiated by the client sending a request indicating it needs a specific set of data, instead of the server side realizing that a specific client is requesting data and that it should send the modified data. So in other words, you'd still have to know you want to receive a certain set of data.
There's definitely a lot of finer details to this...
This is the water system...translated for data...that's the only way to make it work...All the information (non-specific-user-defined) would have to be in the system at all times...
If you want a drink you don't specify where the water has to come from...you open the tap and out it comes...but when you want to make lemonade or distilled water lets say...you have to have software on your end (pot and fire) to get out of it what you want...
But that presents the problem of too much bandwidth usage at old usage methods...unless we go to completely over the air data transfer which takes the phone system out of the loop and allows a more radio style internet...where the data is already being disseminated just waiting for you to tune to that station...you're not asking for the information you're allowing your device to consume it...
hrmmm
"Helping to keep you two steps ahead of the Thought Police!"
Some of these issues seem to be addressed (or are being attempted, at least in the early stages) by metalink which was discussed at http://slashdot.org/article.pl?sid=07/02/25/144209 a few months ago, but I don't think people really understood what it was.
In his talk, he alludes to it solving "the spam problem", as did a couple of commenters here.
It doesn't.
Nor does it solve the "phishing problem".
Why?
First, spam.
Spam is unwanted email that is being delivered to you. Spam isn't resitricted to email, we also get snail mail spam that is also known as "junk mail" - advertising pamphlets from stores, your credit card company, etc. To eliminate spam requires the computer to undertsand what we think we want and don't want. In some cases this is easy - anyone that we've sent email to we probably want to receive email from. Easy. The problem is how do I get email from strangers that I want to read? And the problem for spam software is how to distinguish that from the email coming from strangers that I don't want to read? Changing the model of the internet to require data to always be authenticated by someone or something doesn't solve the problem of "do I want it?"
Next is phishing. Phishing scams take advantage of naive people by presenting them with something that looks like something else and tricking them into doing something they shouldn't. This is equally achievable using surface mail except that it may require the offender to leave more soid footprints. If I give you an image that says "nytimes.com" on it, but I present it as being from x.com and it validates as being from x.com, then no red flags are raised - it is left to the user to be somehow discover that what they're looking at comes from x.com and not "nytimes.com". Software won't be designed to do that too prominently because it will interfere with the delivery or real content where people don't want the user's screen cluttered with "this came from blah" everywhere.
But if you can solve the spam problem then you're on track to also solve phishing because phishing is spam.
And as has been seen so far with SPF for SMTP, spammers are quite willing to do whatever is necessary technologically to overcome simple hurdles.
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> /* <![CDATA[ */
/* ]]> */
<html>
<head>
<style type="text/css">
<!--
@import "/branding/css/tigris.css";
@import "/inst.css";
-->