Slashdot Mirror
Usenet Encoding: yEnc
Motor writes "Anyone remotely interested in usenet binary newsgroups must have noticed the spread of yEnc. yEnc is an encoding scheme for usenet binaries which avoids the enormous (30-40%) bloat associated with the schemes currently in use - which all have to produce 7-bit data to stop ancient newsservers from choking. A good thing, surely? Well, not according to some people. The guy has some good points about yEnc and standards, but I can't help thinking that "standards" people have endlessly discussed better encoding schemes, and nothing has come out of it. yEnc may not be perfect, but it works and it's here - hence the rapid adoption. What do you think?"
In fact, Yenc will help pay-per-gigabyte Usenet users achieve a greater bang for their buck. Anything that saves money is a good thing!!
www.lonseidman.com
There are some proposals in thee XML and Web Services arena for dealing with some of the problems tha yEnc is skirting.
One, called DIME, is a MIME-like system that handles binary content, chunks, etc.
http://www.oasis-open.org/cover/dime.html
Forte released Agent 1.91 2 days ago with yEnc support. it looks like Mr. Nixon is fighting a losing battle.
Breaking MIME is not something I would (do) lose sleep over. People in the MIME community screamed at us when we had the temerity to introduce the text/html content type, rather than use application/binary. They were completely obstructionist when it came to insisting on 8-bit clean transport for HTTP. In the end we treated them as damage and routed around them. HTTP uses several headers that the MIME people villified.
The functional issues raised are significant and it would be good to see them addressed. In particular using the subject line is pretty lame. Either you want the encoding format to be completely independent of MIME or you don't. I think that MIME independence would be the better route since then it would be easier to move to a more modern protocol such as BEEP. But using magic numbers and MD5 inside the encoding does not seem like a bad move.
The more interesting 'meta-point' however is that tweaking the encoding format is only scratching the surface when it comes to fixing UseNet. The main problem with USEnet is that it still has to route every single article to every single node whether it is going to be read or not. While the flood fill routing was a good scheme when NNTP was developed and the number of nodes was small it is needlessly wasteful now that we have hundreds of thousands of NNTP servers, it is just not necessary to have that level of redundancy to route arround censorship.
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/
Well then. When I put that page up, I honestly didn't expect many people to read it outside news.software.nntp and a few curious folks in alt.binaries.news-server-comparison. I certainly wan't expecting to get Slashdotted. Well, that's fine, except that the uproar might have waited a little bit.
In my essay, I state that what Usenet needs is "a better way to post Binaries". The next piece of the puzzle, of course, is to answer the question, "What IS a better way to post binaries?" I was thinking about finishing that page up tonight, but I am writing code at the moment instead.
So, when reading my comments, just keep in mind that, yes, I DO have some answers to that question, too. It's just that it's a bit of a more time-consuming question, so that page isn't done yet.
This time around, though, I will make sure to include a prominent warning to NOT run off and implement the ideas as quickly as possible, and to please not use all of Usenet as beta-testers. The idea that whatever gets done fastest is best just doesn't work for me. There were good reasons I didn't go and get people to implement my smaller encoding ideas when I first wrote the code. If only the yEnc implementor had continued where I left off rather than going down his rather misguided path...
All the comments are welcome. I've been getting some interesting email, too, of course. Many programmers of Usenet client software absolutely despise the thing and are quite annoyed at the amount of their time it is wasting. I guess it's just more of that never-ending divide between the users and the techies. So it goes.
yEnc is here, that's for sure. Now we just have to try to deal with it.
Jeremy
See my other post in this thread. It's not all about dl speed or even drive space. Most serious usenet large binary downloaders use a premium news service, and they ALL have download/upload quotas and/or pay by the byte schemes. yEnc gets you 30% more for your buck on those servers.
... the implementation sucked.
I leech regularily from alt.binaries.anime and the related newsgroups. When the yEnc posts started coming in, I simply upgraded my newsreader to the newest version. But a LOT of people out there use Agent, and it was absolute pain to combine/decode all the yEnc posts that started popping up all over the place. The worst of it is that the yEnc posters were basically saying, "Start living in the present and upgrade". Nevermind that at the time that only yEnc-capable newsreaders were for Windows...
I mean, I don't know, but this sounds a lot like the OS wars that have been going on for quite some time. Some people simply don't want to have to switch newsreaders. Some people don't want to have to switch OSes. And that's fine, because it's a free world out there. I like the idea of yEnc (I get more out of my Easynews account), but I really don't think it should have been introduced so quickly.
~ Firecaster ~"Whatever kind of "confusion" this guy is referring to is nonexistent. yEnc downloading and decoding with the right software is as transparent as uudecoding. Promise. People don't have to relearn anything."
Well people do now have to think of yENC's longer lines. Where before my ISP newserver let me post 5000 lines of uue material now I only get 2280 with yENC. Longer lines under yENC. That's why Agent 1.91 has such a small line length as the default. Yes the payload is the same but you have to think a little different or just rediscover your servers limits.