UDP - Packet Loss in Real Life?

PacketStorm asks: "There's always an argument between TCP and UDP users. TCP users complain that UDP is non-determinstic and lossy, while UDP users complain that TCP is slower and nobody needs all the features anyway. So the question is - has anyone actually seen/experienced UDP loss in high-traffic environments? Is the degeneration of UDP any better or worse than TCP in a congested environment? TCP also craps out in times of congestion, but at least you know - or do you? Experiences?"

Comments

UDP packet loss

[fraxas]

I don't know if it means anything, but I have experienced UDP packet loss on a regular basis -- playing online games. Unit positions are most often tracked over UDP because it doesn't really matter if things warp around a little when the client receives an update that puts a unit somewhere other than where the client extrapolated it was. There was a big fiasco at launch-time for Anarchy Online -- they used TCP and not UDP for about 8 hours, during which time the game was unplayable because the servers couldn't maintain all the TCP connections. Interpret this as you will.

Re:UDP packet loss

[Polo]

Hmmm... it's LOTS easier to write an application to handle UDP than to handle TCP.

With TCP, you have a listening socket and then a socket for each client. Each socket has a file descriptor, and you have to select() on all of them to check for activity. There is a lot of housekeeping to do - though it is a solved problem - all webservers solve it. Timeouts are harder to determine from the application's perspective because there can be retransmits and stuff going on that you have no idea about.

However, with UDP, it's laughably easy. You just do one recvfrom() and you get a packet and it also fills in a data structure to tell you where it came from. No filling in fd_set structures and no running out of file descriptors. When you got the packet, you got it.

Most online games and stuff would really like to know when packets were sent, when they were received and if packets are being dropped. With UDP you can usually find out these values directly. With TCP I think it would be more like an educated guess.

One drawback is that UDP is quite easy to spoof. I can send a packet and it is up to the application to figure out it has been spoofed.

If I were downloading a patch to a game I think TCP would be the better choice. It already has the smarts to pace the connection, transmit the data reliably and prevent spoofing.

Re:UDP packet loss

[Jeremi]

However, with UDP, it's laughably easy. You just do one recvfrom() and you get a packet and it also fills in a data structure to tell you where it came from.

... or maybe you don't get the packet, because a router was loaded down and had to drop it. Now you gotta implement a timeout and retransmit protocol. Not to mention that packets may arrive out-of-order from the way they were sent, so you if ordering is important (and it usually is) you have to implement some sort of sequencing tag systems too. A few dozen hours later, you find out you've implemented something that looks suspiciously like a primitive version of TCP... :^)

Re:UDP packet loss

[Polo]

Well, in relation to the post about anarchy online I replied to, it seems like TCP didn't work.

I would think SOME game data would need to be reliable, but most isn't. The problem with TCP for an online game is that you can never THROW AWAY data that's too old or unnecessary. It will be transmitted and retransmitted further delaying current data. At some point, things will become unusable.

I don't think you could play a multi-user game for over an hour and not run into this problem. I think on lower-bandwidth connections, you might never be able to catch up once you fall behind.

Of course, you could do what microsoft does with DirectPlay: open multiple connections, sometimes on multiple protocols, some of them asymmetrical, while ignoring silly details like firewalls and NAT.

I think the initial handshake for dungeon siege was something like:

first packet: udp my_ip:6073 to server_ip:6073
reply packet: udp server_ip:6073 to my_ip:2302 (what!?!?)
next packet: udp my_ip:2302 to server:6073

or some garbage like that. I think the dungeon siege guys just changed it.

Re:UDP packet loss

[crisco]

Many games are designed to not care about the dropped packets. Sure, they can be unplayable if packetloss gets too high but the occasional dropped packet doesn't matter.

Some tasty articles from gamasutra (might require a login, you might also find these in Google's cache):
"TCP is evil. Don't use TCP for a game. You would rather spend the rest of your life watching Titanic over and over in a theater full of 13 year old girls."
article on WON's servers for Half-Life.
Dead Reckoning Latency Hiding for Multiplayer Games.

Other software might need the benefits of TCP, but game development is one familiar illustration of where UDP often wins out.

Re:UDP packet loss

[Jeremi]

The problem with TCP for an online game is that you can never THROW AWAY data that's too old or unnecessary

Well, that's not entirely true -- while it is the case that once you have called send() on the data, it can't be thrown away, it's also true that if you are maintaining an outbound message queue (to avoid blocking on a filled TCP send buffer), then you can remove or replace data that is in the outbound message queue, that hasn't been sent yet. I have used this technique with reasonable success.

I do agree that UDP has its place, however.

both are usefull

[f0rtytw0]

UDP for streaminig video and games and other sorts of things where it doesn't matter if you miss a couple of packets and TCP where you can't miss packets such as file transfers. There everyone happy go home now.

Parts of the Internet currently collapsing?

[Anonymous Coward]

Check the Internet Traffic Report

Why UDP in games, TCP for bulk tranfer

[MonMotha]

UDP is commonly used in games and other time sensitive environments precisely because it lacks reliabilty. With time sensitive data (such as streaming video or unit positions in a game), if the data gets dropped it's not worth it to retransmit, because it woudl be out of date. Therefore, the program just transmits the next update and the user sees a small skip. This is better than getting "out of sync".

TCP is designed to make unreliable networks (like the internet, which only gives "best effort delivery") reliable by ensuring that a stream can be reassembled, in order, with no missing pieces. Read the RFC for more info here. This reliability makes it good for things that need zero corruption (file transfers for example), and aren't time criticial.

Hope this helps.

--MonMotha

Re:Why UDP in games, TCP for bulk tranfer

[Electrum]

This reliability makes it good for things that need zero corruption (file transfers for example), and aren't time criticial.

TCP and corruption is an interesting subject. TCP's error detection is only designed to solve the problem of data not being delivered. It is not designed to stop data corruption. If the transport layer corrupts data, then eventually, TCP could deliver incorrect data.

What happens if data is corrupted so that a corrupt checksum matches corrupt data? Eventually, from a probability standpoint, there will be undetected errors.

Re:Why UDP in games, TCP for bulk tranfer

[stilwebm]

It is *very* unlikely that a chucksum being corrupted will match the checksum of corrupted data. Possible yes, but you're looking at very small fraction there.

UDP Experience

[mchappee]

I'm firmly in the UDP camp. About 4 years ago we replaced our timeclocks at work (manufacturing facility) with hardened, wall-mounted PCs. I wrote a GTK app that started at boot up that takes the users card swipe, grabs their name from the database (for display only), and sends the clock number via UDP to the timeclock server.

During my initial proposal I mentioned to the PHBs that I would use the UDP protocol. One of my colleagues, wanting to sound important, said that UDP can be lossy. He went on to explain packet loss to the befuddled crowd. Well, the PHBs latched onto the term "packet loss". Packet loss this, packet loss that. They had no freakin' clue what it was, but it sounds pretty cool.

Anyway, I had to set up a test in which I had all of the timeclocks start a program at the same time. This program went into a tight loop of sending UDP packets (clock numbers) across the network to the server. Each one sent 1000 clock numbers, and every single one made it across. Obviously our 100mb network and proper use of subnets helped, but we haven't experienced any packet loss in the four years that these things have been running. So there. :-)

Matthew

Re:UDP Experience

[mgblst]

What is a packet became corrupted, something that will happen. YOu need to build reliability into the application layer, whereas with TCP it is there already.

Definitely

[austad]

I run a cluster of high volume mail servers that use syslog-ng to log to a remote syslog server. This reduces disk IO for each message and actually gives a very noticable performance increase. However, I noticed under periods of very high volume, I was losing nearly 50% of my log entries. Syslog-ng has the ability to use TCP for logging, so I switched, and I haven't lost any entries yet.

UDP is great for things you that aren't useful if retransmitted later, like real time apps (Quake, video streams, audio streams, etc.) But to ensure no loss at all, you need to use TCP.

What argument?

[jkujawa]

Jesus. Both have their uses. You use TCP if you need the reliability, and a stateful connection. You use UDP if it doesn't matter that a packet gets dropped here or there. Things like games and streaming media are good examples. This is rather like comparing Pepsi to milk. They both have their place. It's the job of a good engineer to determine which is most apropriate.

Re:What argument?

[tunah]

This is rather like comparing Pepsi to milk. They both have their place.

Except for pepsi.

Re:What argument?

[4of12]

Seems like UDP would work OK on an unclogged network and have the benefit of adding less congestion compared to TCP with all of its handshaking.

For an low traffic, more or less isolated network, UDP might be a simple solution. OTOH, with PCs with 100 Mb cards so cheap these days, people could probably afford the extra buzz of TCP without noticing any appreciable degradation of performance or incurring great cost.

The day the network starts to become the least bit congested and any UDP packets get dropped, then everyone shifts to TCP and things start to really get congested.

A binary QoS mechanism exhibiting an inherent problem.

Re:What argument?

[Muad'Dave]

Along with some <FatB*stard> Baby - the other, other white meat!</FatB*stard>

I wouldn't know.

[Wakko Warner]

The only protocol I ever use is ICMP.

- A.P.

Re:UDP Experience QWZX

[Anonymous Coward]

/me bangs his head on his desk over and over at the sheer stupidity of this.

Dude, I hate sound flame-y, but do you have any understanding of what you implemented? That you got lucky and it "works" is totally irrelevent to the fact that it's completely unreliable. All it takes is one flakey piece of Ethernet dropping packets to SCREW UP FREAKING TIMESHEETS.

This reminds me of the morons who use MySQL for financial transactions (i.e., no transactions, no foreign keys, etc) who justify their ignorance with "well, it works so far!!"

They point isn't whether it works or not, the point is that when it fails, it fails spectacularly. Like your system. Just because you can keep spinning the chamber and the russian roulette gun never goes off doesn't mean it never will.

Seriously, you screwed up bad. These are the kinds of stories that really make me think that programmers should have some sort of licensing.

non-determinstic ?

[heliocentric]

Could you please elaborate on that for me? I always though NTM's (non-determinstic Turing Machines) were so powerful since they always made the "right" decision and didn't need luck or a specific rule set, they just non-determinstically chose the right path.

If you have a network that is just sending packets correctly with out decisions then all the better and we should take it on tour. Maybe, however you're refering to the complete broadcast mentality that packets aren't sent to specific hosts as required but to all hosts since this is how non-determinism is typically simulated, but simulated non-determinism and true non-determinism are a tad different.

Maybe it's just that I don't really know a lot of networking theory, but that term just sort of jumped out at me. I've often equated non-determinism with magic...

Multicast

[stere0]

I used to work with a company that sent data (mostly video) to all customers via satellite, in multicast. The uplink went through the terrestrial network. IIRC, the whole traffic was sent over UDP.

So, what you're saying is...

[Wonko42]

I'm a skydiver. I never use a parachute. Instead, I just jump out of planes and hope that I land on a large air mattress. It's worked perfectly so far. Why should I switch? All you silly fools using parachutes make me laugh.

Multicasting MPEG

[Martin Spamer]

One of the points for using UDP over TCP is where 'guaranteed' data data delivery is not a 'hard' required. i.e. It does not cause a catastrophic failure.

We use Multicasting (IGMP is built on UDP) to 'broadcast' MPEG Video over an IP network for an Interactive DTV (www.kitv.co.uk) project.

This functions largely without problems, because maintaining an MPEG steam is highly time sensitive but it is not catastrophically sensitive to lost or dropped packets. These lost/dropped packets lead to video artifacts and not total loss of data, because the video stream can continue from the next received packet.

The quality issue is governed by *minimising* the lost packets not 'guaranteeing' them

Wireless

[epsalon]

Also, there is an issue of IP over wireless, where packet loss is a given, at quite high rates. Thus, if you need the reliability either use TCP or add a reliability layer over UDP.

In most cases using UDP is a loss

[WolfWithoutAClause]

If you have very special requirements, or very non special requirements, or you simply can't use TCP for some reason, then UDP can give you better performance. But it usually won't.

About the only reason for using UDP is if you deliberately want to circumvent the congestion avoidance protocols that are built into TCP. So, if you are playing a game, and you need the packets to get through at all costs, but you aren't sending many packets- by using UDP you can agressively defend the small amount of bandwidth you need- any TCP connections around will tend to back off and get out of your way; and that's reasonable if you code it carefully. But writing the protocol to do that is hard, you have to understand not only UDP but also TCP, as well as your game requirements.

And that's the real problem. In most cases people think that waving UDP at the problem will solve their problems- in fact it makes them worse; and TCP has solutions to problems only PhDs have even thought of, and the solutions are built in.

As an example, somebody I know implemented a tftp protocol using UDP. The guy is off the chart in his software abilities (trust me the guy is amazing, he's in the top 2 percent of software engineers according to the tests). Anyway in a back-back comparison against a standard ftp protocol- the tftp protocol loses by a factor of 10 or more (on a network with some congestion, I expect a quiescent network would have been much more level). Of course tftp isn't supposed to handle congestion. But that's the problem- UDP can't handle network congestion out of the box... indeed if anything it tends to create network congestion.

The main algorithms in tcp include 'slow start' and 'exponential backoff'. Both of these are missing in UDP, and both improve the network performance enormously. If your application doesn't affect network performance and doesn't worry about packet loss much, then UDP may be the way to go, otherwise stay away from UDP.

Re:In most cases using UDP is a loss

[sudog]

You don't know what you're talking about.

You started out okay and then sank into total bullshit land as soon as you mentioned your uber-friend's tftp implementation. Newsflash: tftp is UDP from the get-go. That's the whole point. And since tftp is a request-response type protocol, of *course* it's going to see less performance out of your networks than ftp. On the other hand, try these guys. They built a faster-than-tcp implementation of a streaming protocol using UDP and it outperforms TCP on high-path networks and does *at least as well as* TCP on the LAN.

There is no way to "aggressively defend" your bandwidth if (for example) you're playing a game. If someone else comes along with their own UDP application that doesn't back off when detected packet loss gets extremely high, you lose out just as much as they do. There is no defence in this case, there is only *higher packet loss.*

And claiming your friend is "off the chart" like that--what is that supposed to do--lend credence to your exaggerated and false claims about UDP? Bzzzt. Nobody cares about test scores--especially for someone who re-implemented a trivial file transfer program using the very protocol it was designed for to begin with. Tell you what--you show us something impressive he did and we'll be impressed. Don't beak off about "trust me" and "he's in the top 2 percent". That's just crap--you didn't even tell us what the tests were or who administered them!

Finally, UDP wasn't designed to handle congestion. But that doesn't mean it can't. Counter-example: build congestion avoidance into your application.

Readers, don't listen to the parent of this note, he doesn't know what he's talking about. Anything TCP can do, UDP can do--the problem lies in how much work you have to do to implement it. TCP is convenience because all the work for a streaming, in-order, semi-reliable, congestion avoiding protocol has been done for you. Unfortunately you can't turn the major features you don't want, off.

Yeesh. Someone mod parent down, it's really not worth a 4.

Re:In most cases using UDP is a loss

[WolfWithoutAClause]

Newsflash: tftp is UDP from the get-go. That's the whole point. And since tftp is a request-response type protocol, of *course* it's going to see less performance out of your networks than ftp.

Precisely! It's supposed to be a lightweight protocol, which is one of the reasons people choose UDP, but it turns out to be slower in this case. And it's because of what's been left out of the UDP protocol stack; UDP isn't inherently bad because of this, but usually you would want what's been left out. Right now I would not want to deploy tftp ever, it lacks passwords, and it's performance is poor on a congested network.

Finally, UDP wasn't designed to handle congestion. But that doesn't mean it can't. Counter-example: build congestion avoidance into your application.

Absolutely, but then you have to implement it yourself! And it is very much not simple.

There is no way to "aggressively defend" your bandwidth if (for example) you're playing a game. If someone else comes along with their own UDP application that doesn't back off when detected packet loss gets extremely high, you lose out just as much as they do. There is no defence in this case, there is only *higher packet loss.*

Being aggressive does not guarantee more bandwidth. But in most situations most other people are using ftp/http etc. they normally will back off and you will get more bandwidth if your protocol is aggressive. Of course if everyone is aggressive, especially inappropriately- you and everyone else ends up losing worse than if you'd have used TCP; it's rather like real life. It probably only makes sense in cases where you only need a small amount of bandwidth; but you NEED that bandwidth. If you start to lose packets due to congestion, you are supposed to send more slowly. However, if you increase the rate you send, you end up with a bigger slice of the pie; if the slice is then enough for what you need, you can control your bandwidth to some extent. However, ultimately you can't send more than your pipe will take, so really heavy congestion will still kill you.

Anything TCP can do, UDP can do--the problem lies in how much work you have to do to implement it.

Yes. Exactly. It's often a lot more work, and unless you really, really know what you are doing UDP is likely to be the wrong choice.

Interestin paper on Checksums, CRCs

[d-rock]

This is a very interesting paper on why things don't always work like we think they should...

Derek

Re:real life experience

[iangoldby]

On a simple network, you won't get out-of-sequence packets with UDP.

Out-of-sequence delivery happens when there is more than one possible route for the packet, such as occurs over the internet. On the internet, routing is done on a case-by-case best effort basis. It's possible then for the first packet to be sent via a longer route than the second and therefore for the second packet to arrive first.

So remember - UDP across a LAN is quite a different beast to UDP across the internet.

Lemme get this straight.

[FreeLinux]

NTP was too complicated(??) so, you designed built and tested your own homegrown version of NTP??????

To quote Dr. Evil, "Riiiight...".

Re:Lemme get this straight.

[schon]

All I can remember that in those days we looked at NTP (or maybe it was NNTP??)... All we needed was a simple way to keep system clocks in line, with an accuracy of a few tenths of a second.

Ahem.. you were going to use usenet posts to synchronize your system time?!?!?!

Bwahahahahah...

(wipes tear from eye)
*sigh*, that was a good one - To quote FreeLinux, 'To quote Dr. Evil, "Riiiight...".'

Coding philosophy

[kevin42]

I've spent the past several years writing networking applications, and this is my philosophy: Anytime there is *any* chance of a packet being lost or arriving out of order, you must write code that assumes every packet has a high probability of being lost. I've seen people make assumptions that since it's running on an ethernet lan they will never lose UDP packets, then their app becomes very unstable. Even when sending UDP packets to localhost they can be lost. When designing UDP software it shouldn't be a matter of how often packets are lost, but how well your code deals with lost packets.
Now if only someone would standardize a reliable datagram protocol implementation. :)

Re:UDP and TCP flow control

[kevin42]

UDP does have flow control, as does all IP traffic. Unfortunately the sockets API don't consistently support it. It's called an ICMP source quench message, and it tells the sending application that the receiving side couldn't keep up (or a router in the middle couldn't keep up). It's not UDP's fault that the APIs don't support flow control well, it's the fault of the OS and the sockets layers that it's not easy to use.

Packet loss == upgrade network

[bluGill]

OKay, a very small amount of packet loss can be normal and should be ignored. However if you have anything other than a tiny amount of packet loss your network is in trouble, and in serious need of upgrades. Remember packet loss starts to casscade, because the droped packets have to traverse the network two or more times, and each time it crosses the network it uses some bandwidth. One droped packet, but when it is re-transmitted some other packet is droped and re-transmitted, and as a result your network gets really slow. In theory tcp will just slow down, but users will re-start the slow jobs trying to get a fast connection.

Sure, TCP will get through even when you have 85% packet loss, (I had a customer who had 85% packet loss once, a babybell I won't name) but your applications will often start timing out in other areas. In theory things should still operate, but they just get slow, but many programs have their own timeout outside of tcp so they can detect when the connection when down.

Don't use tcp where you don't need it though. I once had to debug a heartbeat for a failover system, where the system only provided tcp packets. When there was a failure in the network we could switch the network easially enough, but then we had a lot of code to try to figgure out if the heartbeat that just arrived after the network switch was old (and contained invalid information about the failed node), or correct. It always seemed to work, but I didn't sleep well many nights knowing that a customer could lose a critical computer because of code that I was supposed to make work.

So in theory you can say TCP is better when there is expected to be packet loss, and UDP is better when lost packets should be ignored. In parctice though, if you have significant packet loss you need to upgrade the network.

Re:packet loss expierenced using tcp

[Jeremi]

Just to give you an idea how bad it was if I downloaded 40 - 15 meg files, at least half would have CRC errors [...] I thought tcp wasnt supposed to let that happen, then again, maybee i should just blame the router. Has any one else have simular problems with cheap nat routers or is this post on topic?

Alas, I've seen this too. I wrote a file sharing program that uses TCP for file transfers, and has a download-resume feature that does an md5 checksum of the user's local file portion, and only starts the resume if the local file portion matches the corresponding portion of the uploader's file.

Every few days, I would get complaints from users that the auto-resume wasn't working. It was refusing to resume because the two file fragments' hashcodes didn't match. After going over all my checksum code checking for bugs, I finally got paranoid and added an application-computed checksum every so often to my TCP data.... and sure enough, in some cases the TCP data I sent would arrive at the downloader's machine with a checksum mismatch. Unfortunately, I haven't been able to finger a particular piece of software or hardware that causes this, but I can confirm that it does sometimes happen. :^(

tcp vs udp..

[josepha48]

you may or may not know when one craps out. That all depends on the programmer and what they check for. In the case of tcp / ip if a programmer assumes that all is okay they can keep sending data to a connection that is failing. If they do not check for certain statuses then it could fail and you wont know it. Same is true I'm sure for udp.

The best method is really a send / reply setup in your code. You send data and the other side replys with I got it (or something). Like what is done in RFC 821 - SMTP.

What is UDP good for?

[Nyarly]

In what really ought to be called the Slashdot Effect, I've just read 50 or so comments where the most useful 5% all said the same thing: if you're writing a game or streaming video, use UDP, otherwise stick to TCP. But, what bugs me is that this is a very narrow view of the differences between the two protocols.

Frankly, the single most important use of UDP is for sending singleton datagrams. Not to be pissy but, um... duh. Consider how DNS works: queries are sent as UDP, and if the reponse is small enough it goes back in a UDP packet. If it's too big, it's sent as a TCP stream. Prime example of what UDP is good for.

Frankly, as far as UDP streams are concerned, I've never found a use for them that didn't involve a realtime response on the receiving end. Network gaming and video streaming is one idea, but certain kinds of telecontrol are another.

The thing that bugs me most is that, by trade I am not a network programmer. I have done network stuff in the past, but it upsets me that the sum of information in all the commentary on Slashdot is more and more frequently less than I already knew about the topics.

Maybe Bruce Perens is right.

UDP can be made to go as fast or faster than TCP

[Great_Jehovah]

I just finished writing a UDP file transfer protocol that goes just as fast as TCP on a clean network and handily outperforms it on noisy ones.

Exponential backoff is braindead when you need good thoughput on a fat but noisy pipe. The reason TFTP gets lousy throughput is not because it uses UDP but because it waits for every ACK before it sends the next packet rather than having a receiver window feature like TCP does.

Apparently you have to be smarter than mister Top 2 Percent to make this work.

Here's some links for you:

• http://www-iepm.slac.stanford.edu/monitoring/bulk/ summary1.html
• http://www.babinszki.com/Networking/Max-Ethernet-a nd-TCP-Throughput.html
• http://www.lariat.org/Berkeley/node8.html#SECTION0 0040000000000000000

Re:UDP can be made to go as fast or faster than TC

[WolfWithoutAClause]

Apparently you have to be smarter than mister Top 2 Percent to make this work.

Actually he's very capable of doing this kind of stuff right. More or less he had to follow the tftp spec, so what's he gonna do?

But my real point is still that most people don't know enough to do it right.

You certainly sound like you might have your head around what TCP does, so probably your protocol works very well.

Exponential backoff is braindead when you need good thoughput on a fat but noisy pipe.

Yes, you are optimising your protocol to deal with cases where packet loss is caused by noise. The internet is built mostly on the assumption that packet loss is caused by congestion, so of course TCP will perform more poorly, and a more tailored protocol is a win, and UDP allows you to do that.

I do wonder if your protocol might trigger congestion collapse, but I have no doubt you've designed against that and tested for it too.

pr0n suckage causing crc errors?

[LWolenczak]

I have a fairly high traffic enviroment.... (my boss pulling pr0n from news servers on our internet connections constant......) Most days.... I see about 97% usage of the connections... Scarry huh? I haven't noticed many UDP dropped packets.... Infact... I think I'm seeing more mangled UDP packets that are failing the crc test then packets being dropped. I do have QoS enabled, which allows my web browsing... or software downloads to take his bandwidth right out from under him.

Re:UDP Experience QWZX

[RobinH]

Dude, I hate sound flame-y, but do you have any understanding of what you implemented? That you got lucky and it "works" is totally irrelevent to the fact that it's completely unreliable. All it takes is one flakey piece of Ethernet dropping packets to SCREW UP FREAKING TIMESHEETS.

Ok, calm down...

Now, I agree that UDP is built to be a fast, not-so-reliable protocol. One would initially presume that this is not the best thing to use on a punchclock system.

However, you have to look at the whole system.

First of all, if you put all the punchclocks, and the servers on the same subnet, then you've eliminated dropped packets due to routing.

Secondly, if you send an acknowledgement back to the clock, then it can display to the user "OK, you're signed in" or "ERROR, please retry". If you lose either the "sign-in" packet, or the acknowledgement packet, then all the user has to do it swipe again to retry.

Thirdly, these kind of systems always (should) have a manual backup, so if, for some reason, the system records Buddy punching in at 7 am, but never punching out, then a supervisor can go in and manually update the database to fix the problem.

Just remember that programs like this don't exist in a vaccuum; they are always part of a larger picture, and they need to function in that framework.

Re:packet loss expierenced using tcp

[Jeremi]

It's called WINDOWS.

Doubtful, in this case.... this program was only available for BeOS.

Re:UDP Experience QWZX

[bugg]

Secondly, if you send an acknowledgement back to the clock, then it can display to the user "OK, you're signed in" or "ERROR, please retry". If you lose either the "sign-in" packet, or the acknowledgement packet, then all the user has to do it swipe again to retry.

As a rule, people who build acknoweldgements into their UDP based protocol should have used TCP.

Re:UDP Experience QWZX

[pthisis]

As a rule, people who build acknoweldgements into their UDP based protocol should have used TCP.

Not necessarily. A better rule is that people who build sequencing and acknowledgements in should have used TCP. But if you can do without one or the other then sometimes UDP is worth it, but not often.

Sumner

NFS uses UDP.

[Jason Pollock]

Everyone's arguing back and forth about which is better, UDP, TCP, some are even presenting other protocols such as SCTP. None of it matters, because the protocol you use depends on what the problem is.

From what I understand, UDP will lose packets in a congested environment when passing across a router. Your stack will guarantee that the packet gets onto the wire, if it gets onto the wire, it will make it onto the other machine if it is on the same wire (subnet) (barring sunspots). As soon as it hits a router, then it can be dropped.

TCP has some nasty timers in it that make it entirely unsuitable for real-time traffic. It assumes that a packet was lost because of congestion, and backs off on the retry, rather than retransmitting immediately. In modern corporate/telco IP networks congestion simply isn't the case. What really is bad is having to wait up to 4+minutes to find out the connection is dead.

UDP allows you to do what you want, and avoid anything you don't need. However, if you need things like in-order, reliable transmission you are probably better off with TCP. If you are simply providing a response, then you should be fine with UDP.

If you are after high-traffic, high connection, high throughput, UDP seems to be the way to go. If you are after easy programming and guaranteed in-order delivery, TCP is your tool.

As someone pointed out, there is a new player in town, and that's SCTP. It was invented specifically because TCP is bad for low-latency transmissions (such as Telephony!). It is used in the SS7 over IP protocols, such as M3UA, SUA, etc.

That is why you will see a mix of streams in various protocols. H.323 uses TCP for control, and UDP for speech/video. SIP allows you the choice of UDP/TCP for call control.

Jason Pollock

Re:UDP Experience QWZX

[fwc]

So exactly WHAT is the advantage of using UDP rather than TCP in this instance?

I of course don't know about the original poster's actual implementation.

I'm making the assumption timeclocks themselves are very lightweight (or thin) devices, CPU and memory wise.

The problem with TCP for a truly embedded application is that you have to play a lot of keep-state programming to implement even a minimal TCP stack.

A minimal UDP stack is much smaller. Everything you need to do with UDP is rather stateless.

Ignoring ARP and ICMP for a second (which is the same for TCP and UDP), sending a single message via UDP consists of encapsulating the packet and sending it. With TCP you have to send a SYN, then wait for a SYN-ACK, then send an ACK back (possibly containing data), and then you have to deal with closing the connection.

Maybe a better way to describe this is that UDP is a DATAGRAM protocol. I.E. you format a datagram and send it. By contrast TCP is designed for a mostly-reliable almost-serial-like stream, with congestion control, retransmission, etc. etc.

I was going to try to make a case for TCP in the timeclock example where a larger machine is available. I just can't see a good reason for the TCP overhead in this app. I mean, what exactly are they going to be sending... Something like "Employee 308 clock in", which the server is going to say "Got it, employee 308"?

If you haven't written low-level embedded networking code you really don't realize how much overhead is in TCP. Heck, just to get a TCP session set up and torn down you need 7 packets:

--> SYN
<-- SYN-ACK
--> ACK

(Session Established)

<-- FIN
--> ACK
--> FIN
<-- ACK

With UDP you only need 2 (1 each way) to send your message and get it acknowdged.

Don't get me wrong, I'm a fan of TCP in the right places. Any message over the ~1400-1500 MTU size of a UDP packet should probably be sent via TCP.

Re:UDP Experience QWZX

[Wonko42]

This reminds me of the morons who use MySQL for financial transactions (i.e., no transactions, no foreign keys, etc) who justify their ignorance with "well, it works so far!!"

When was the last time you used MySQL? MySQL now supports several table types other than the default MyISAM type, including types with full transaction support.

Re:UDP Experience QWZX

[RobinH]

Not necessarily. A better rule is that people who build sequencing and acknowledgements in should have used TCP. But if you can do without one or the other then sometimes UDP is worth it, but not often.

I agree. In this case, you don't need sequencing, because there's only ever one transaction out there from each client at any given time. The transaction is as simple as "12345"->"OK". The UDP protocol is so much simpler to program (and you don't have to worry about managing connections) that for this particular application, it certainly seems worthwhile.

Re:UDP Experience QWZX

[RobinH]

I was going to try to make a case for TCP in the timeclock example where a larger machine is available. I just can't see a good reason for the TCP overhead in this app. I mean, what exactly are they going to be sending... Something like "Employee 308 clock in", which the server is going to say "Got it, employee 308"?

Actually, the timeclocks he describes are fully fledged PCs, I believe, and they do support a full TCP stack already. However, considering the differences in complexity of the server, I still would favour a UDP solution.

As an example, for the server to implement TCP, you would need to listen on a port, deal with an incoming connection (maybe fork off another process), deal with connection timeouts, etc.. With UDP, you can just have one single procedure that's called when a packet arrives. The procedure parses out the employee number from "Employee 308 signing in", updates the employee database, and sends a UDP message back, i.e. "Employee 308 signed in". This tolerates dropped packets (the employee just swipes again), and most importantly, the implementation is practically stateless, and that's a much simpler architecture. Never underestimate the KISS principle.

Rate Policy vs Priority Policy

[macdaddy]

I ran into this very problem a while back and solved it last week. Have have a Packeteer PacketShaper 4545. We use it at my Unv to slow down P2P and speed up interactive traffic such as SSH. I also use it to limit the amount of bandwidth an application or set of applications can consume. One of these sets is Games. I limited them to a slice of bandwidth during the day and raised that after hours (neither limit has ever been reached). After hours I also garuntee a small slice to help kick start apps. I also use dynamic partitions to garuntee each flow a certain amount of BW within the garunteed slice. I've made various changes to the Games class over time to try and make it better. It's been a bit of a guinea pig to test settings on for other types of traffic. One of the things I've done is raise the classes Rate Policy after hours to something slightly higher than most other traffic. None of the things I tried helped. Users still complained of exceptionally high ping times (ie, 9999ms) even during after hour times. Most of these corresponded to bursts of other traffic, not usually with a higher rate policy. Finally I called tech support. One of their techs had me switch the Rate Policy (default) to a Priority Policy. He also explained the difference between the two. Using a rate policy to slow TCP traffic means that the ACKs are delayed to slow responses down. This isn't possible with UDP though. Using a rate policy on UDP resulted in either dropped packets or an entire datgram being delayed. There is no backoff implementation in UDP so the queue would fill with these UDP datagrams. Using a simple FIFO, the 1st delayed datagram would be delivered considerably late. The another and another. If the queue was full and a datagram was received, it was dropped. Bad news. The client would experience really crappy performance. Video and audio would be extremely choppy. It would just suck. A Priority policy works differently though. Let's say a UDP Quake packet and a TCP HTTP packet arrive at the same time. The Quake packet has a higher priority. The Quake packet goes 1st. No queuing, no delay. There's other technical stuff that goes along with this but I won't bore you with it. All in all, because this traffic was UDP, the policy method I was using (again, the default) caused horrible service. Does UDP have it's uses? I'm sure. Does UDP have benefits over TCP in some cases? Sure. Would I consider switching everything to UDP? Hell no.