Usenet Encoding: yEnc

Motor writes "Anyone remotely interested in usenet binary newsgroups must have noticed the spread of yEnc. yEnc is an encoding scheme for usenet binaries which avoids the enormous (30-40%) bloat associated with the schemes currently in use - which all have to produce 7-bit data to stop ancient newsservers from choking. A good thing, surely? Well, not according to some people. The guy has some good points about yEnc and standards, but I can't help thinking that "standards" people have endlessly discussed better encoding schemes, and nothing has come out of it. yEnc may not be perfect, but it works and it's here - hence the rapid adoption. What do you think?"

Comments

Intertia vs. Good Ideas

[ksw2]

The article points out some interesting points why yEnc shouldn't be adopted... none of which will probably keep the community from adopting it, however. If it's here, and being used, that is a whole lot more intertia than common sense can usually gain. Er, betamax, anybody?

Re:Intertia vs. Good Ideas

[Zeinfeld]

I have some sympathy with the article author, but not when it comes to the MIME issues. I have written plenty of IETF and other standards, I know the value of going through a standards process, however the IETF is not a place to do research, it is a place to standardise and improve existing protocols. The idea is that you start from code.

Breaking MIME is not something I would (do) lose sleep over. People in the MIME community screamed at us when we had the temerity to introduce the text/html content type, rather than use application/binary. They were completely obstructionist when it came to insisting on 8-bit clean transport for HTTP. In the end we treated them as damage and routed around them. HTTP uses several headers that the MIME people villified.

The functional issues raised are significant and it would be good to see them addressed. In particular using the subject line is pretty lame. Either you want the encoding format to be completely independent of MIME or you don't. I think that MIME independence would be the better route since then it would be easier to move to a more modern protocol such as BEEP. But using magic numbers and MD5 inside the encoding does not seem like a bad move.

The more interesting 'meta-point' however is that tweaking the encoding format is only scratching the surface when it comes to fixing UseNet. The main problem with USEnet is that it still has to route every single article to every single node whether it is going to be read or not. While the flood fill routing was a good scheme when NNTP was developed and the number of nodes was small it is needlessly wasteful now that we have hundreds of thousands of NNTP servers, it is just not necessary to have that level of redundancy to route arround censorship.

Re:Intertia vs. Good Ideas

[Spy Hunter]

I would argue that the adoption of a standard is a much better indication of its "goodness" than its technical features. yEnc has been adopted by lots of people because it solves problems that they have, therefore it is proven to be good. If someone fixes the flaws that this author talks about and makes a new scheme that works better, then it might get adopted. If it does, it will be because it solves real problems people have with yEnc. If it doesn't, that means that it is too much of a pain for people to switch and that the problems yEnc has are not that much of a problem for real users. I think this is probably the case. So you can't use filenames with double quotes. Big deal! Change them to single quotes or something! So one out of a thousand posts will be corrupted because of mis-recognized magic strings or something. Its not any worse than it was before, and the downloads are smaller! If the problems really are THAT bad, a solution will come and people will use it.

Re:Intertia vs. Good Ideas

[Tofuhead]

Yes. A shipping product beats theoretical vapor every time, or, as previous generations would have said it, "A bird in the hand is better than two in the bush." But, that doesn't mean the product couldn't have been done right the first time (particularly with the apparently large number of folks who had made recommendations that would have improved the spec, which the yENC author is only now hacking into it). I believe this is the article author's point.

< tofuhead >

Re:Intertia vs. Good Ideas

[xmda]

Er, betamax, anybody?

And I say, er, Microsoft, anybody?...

Re:Intertia vs. Good Ideas

[JordanH]

• Er, betamax, anybody?

Not really comparable to the betamax vs. VHS debate. I've not seen anyone arguing that the alternatives, uuencode or base64, are better than yEnc, just that yEnc has serious deficiencies.

Perhaps Mr. Nixon is arguing that yEnc is worse than some wholly theoretical alternative.

Some of Mr. Nixon's points do seem interesting, but if he is convinced that there is a better alternative to be put forward, he should get the code out there. Anything else is just sniping.

Re:Intertia vs. Good Ideas

[Rui del-Negro]

Betamax failed because it was a proprietary format. If Sony had allowed other companies to make it (like JVC did with VHS) it would still be around (20 years ago it was better than VHS is today). Betacam still dominates the broadcast world.

Now, yEnc looks like it was created by Microsoft. It's not a standard, it's a hack. The only way it can become a standard is by pushing it down people's throats and then using "public pressure" to force applications to support it.

To use a videotape analogy, it's like releasing magnetic tape reels after people had been using cassettes for years, just because the reels use slighlty lighter tape.

I hope yEnc in its current form is *not* supported by the industry. I think a company such as Forté could create a real standard using an encoding method similar to yEnc (it wasn't "invented" by yEnc's author, anyway). I think Agent's programmers, of all people, should know how hard it is to deal with these (non)standards, and could save themselves a lof of work in the future by making sure it's done right.

As it stands, yEnc is the same as UUEncode, only in smaller portions (actually it's worse, because you can sort of wrap UUE in MIME; you can't with yEnc).

Re:Intertia vs. Good Ideas

[Anonymous Coward]

I just wish USENet was still about news and discussions.

I'm worried that it's changed so much that it's beyond hope. It's just a big pipeline of porn and 'entertainment' content now. A picture is NOT worth a thousand words if I'd rather discuss NetBSD or Linux and the bandwidth is flooded with bad pictures of vulvas and chopped up movies and music.

Re:Intertia vs. Good Ideas

[Chasing Amy]

> it is needlessly wasteful now that we have hundreds of thousands of NNTP
> servers, it is just not necessary to have that level of redundancy to route
> arround censorship.

I disagree entirely. Never underestimate the government's ability to stretch censorship to new levels.

Unless the very way NNTP servers operate is to gulp down and pass on each article for each newsgroup, the government would easily target those servers that spcifically carried groups or posts it doesn't like.

Pressure for news providers to drop certain groups began several years ago when the Vacco busts of people trading in child pornography led a news service to be criminally charged for the content of some groups and led other news servers in that state and elsewhere to drop gcertain groups thanks to their content. The charged news service took a plea even though they clearly would have won at trial or on appeal by claiming common carrier status, but hey, nobody wants to be the expensive test case.

Some may not see the problem with news servers being coerced by the government to drop those particular groups thanks to their contents, but the principle it sets is horrid. Certain "content owners" have of late been threatening to use the DMCA as a club to get news servers to drop groups which share TV shows and other such copyrighted material. If groups were more "localized" to a set of specific servers, or articles were localized to their originating servers, that would make it exceptionally easy for the DMCA to be used to require the "closure" of groups or removal of articles from USENET.

Furthermore, in this time of anti-terrrorist hysteria, the government has gotten away with the USA/PATRIOT mess already and is continually making some questionable choices. If it finds a newsgroup dedicated to dissent, or more spcifically dedicated to anti-globalism, for example, it cannot easily dstroy such a group because of the nature of USENET--the damage would be routed around by servers in other countries, even if every U.S. server could be forced to remove a group or article (not that they could be).

However, if the architecture of USENET were redesigned to localize groups or articles to subsets of servers--the likelihood of a government censoring USENET speech is magnified considerably.

It is the redundant architecture of USENET which will keep it free of censorship long after the WWW has been tamed--as it will be. Just look at the broiling mess within ICANN over officials trying to hand control of the WWW over to government-appointed reps. Eventually something like that will happen, and governments will cooperate with each other to make censorship in their mutual interests easier. Thanks to the architcture and nature of USENET, it will remain free and uncensored long after the WWW has fallen to censorship.

Just my 2 pence, though...

Re:Intertia vs. Good Ideas

[Chris Siegler]

The main problem with USEnet is that it still has to route every single article to every single node whether it is going to be read or not. While the flood fill routing was a good scheme when NNTP was developed and the number of nodes was small it is needlessly wasteful now that we have hundreds of thousands of NNTP servers, it is just not necessary to have that level of redundancy to route arround censorship.

Just as a guess, I'd say most news servers only get a partial feed. Some avoid alt.binaries.* altogether certainly. And even when an ISP gets the whole shebang, they often sell access to smaller ISPs, so that the "tree" isn't nearly as nearly as big as it might be. So it's not really that bad.

Re:Intertia vs. Good Ideas

[3247]

Betamax failed because it was a proprietary format.

Well, Betamax was better but more expensive. The same reason IDE is winning against SCSI.

Re:Intertia vs. Good Ideas

[3247]

While the flood fill routing was a good scheme when NNTP was developed and the number of nodes was small it is needlessly wasteful now that we have hundreds of thousands of NNTP servers, it is just not necessary to have that level of redundancy to route arround censorship.

I believe this is no longer true for binary Usenet. There are a few hosts that carry binary newsgroups in a usable way (having some 100MB is not useful). Most servers can't handle the traffic induced by binary newsgroups.

Re:Intertia vs. Good Ideas

[InsaneGeek]

>While the flood fill routing was a good scheme when NNTP was developed and the number of
>nodes was small it is needlessly wasteful now that we have hundreds of thousands of NNTP
>servers, it is just not necessary to have that level of redundancy to route arround censorship.

I don't think anybody is seriously thinking of having hundreds of thousands of NNTP servers for the purpose of getting around censorship. It's more of a way to minimize bandwidth, and works pretty much off the same principal as a cache server. It's main purpose has not changed much over the years bring the content closer to the end user who will want it. With the advent of broadband users pushing >1mb downloads from the a NNTP server, you want it as close as possible. Get 10-20 users downloading from an external usenet server simultaneously you tend to have big issues since instead of having on constant 1-3mb connection you might have a full T3's worth.

Re:Intertia vs. Good Ideas

[Anonymous Coward]

ICANN has no more control over the "WWW" than it has over USENET.

ICANN's scope is the DNS system, upon which WWW, MAIL, NEWS, and everything else on the net currently relies.

Re:Intertia vs. Good Ideas

[Chasing Amy]

> ICANN's scope is the DNS system, upon which WWW, MAIL, NEWS, and everything else on the net currently relies.

Umm, that's the point.

He who controls DNS exercises immense influence on the Net, particularly the WWW. Giving control of the DNS system to delegates directly appointed by governments is a recipe for fostering censorship in the interests of those respective governments.

I.e., if DNS were in the hands of representatives appointed by governments, and some given websites were to be deemed undesirable--delete their DNS entries, and they go away. Poof. The instant DNS is controlled by governments, is the instant they begin implementing a system to instantly pull the DNS entries of websites which are "dangerous" and "patently offensive." Governments hardly ever agree on anything, but you can bet they'd agree to some deal-brokering--you support the "delisting" of our seditious sites, I'll support the delisting of yours.

USENET isn't as vulnerable because of its architecture. You can't just "delist" one newsgroup the same way you can delist one website.

If the ICANN higher-ups have their way and hand over the reigns to government-appointed reps, I guarantee you governments will take the opportunity to at least consider using their control of DNS to enable censorship.

Re:Intertia vs. Good Ideas

[Rui del-Negro]

In fact that's what he did (yEnc is based on that code). But the point is, you can't just "get the code out there" and pretend that creates a standard. When you create a new kind of aircraft, you need to make sure it can land and operate in existing airports; you shouldn't just force all airports to change to accomodate your new design, even if it is 20% or 25% faster.

Besides, size isn't the only problem with the current method for posting binaries in Usenet. If we are going to change, might as well spend a couple of months discussing the issues and come up with a real standard that solves as many problems as possible. Instead of a hack that creates more problems than it solves. And note that standard does not mean the same as common. It's like the difference between science and common sense.

Would you rebuild your entire system and re-install all your software just for a 20% increase in CPU speed? I wouldn't.

Re:Intertia vs. Good Ideas

[Zeinfeld]

I disagree entirely. Never underestimate the government's ability to stretch censorship to new levels.

I think it is possible to reduce the bandwidth consumption of USEnet by at least one order of magnitude and introduce much better censorship resistance.

It is actually quite easy to block posts in the current flood fill algorithm, just introduce your own post with the same article ID.

Re:Intertia vs. Good Ideas

[JordanH]

• In fact that's what he did (yEnc is based on that code).

He did get the code out there, but he didn't push the concept for years. He's to be applauded for not putting any sort of restrictive copyright on the original code, but he actively discouraged people from using his code.

• And note that standard does not mean the same as common. It's like the difference between science and common sense.

I think you mischaracterize the Internet standards process here. Internet standards, in the RFC process, have always been about competing implementations that get thrown out there for people to test extensively. Eventually, after widespread acceptance, they get elevated to the level of Standard. I seriously doubt that yEnc is there yet, but perhaps it will light the way toward a better Standard eventually.

• Would you rebuild your entire system and re-install all your software just for a 20% increase in CPU speed? I wouldn't.

This is a poor analogy. If you don't like yEnc, don't use it. Nobody's asking you to switch to a new standard overnight. Uuencode and base64 aren't going anywhere. Now, true, you might lose out on not being able to download from those who do like yEnc, but that's like saying "nobody should use Java because I don't like it and I won't benefit from Java code on the net".

Re:Intertia vs. Good Ideas

[Salamander]

The main problem with USEnet is that it still has to route every single article to every single node whether it is going to be read or not.

First, don't say "Usenet" (not an acronym BTW) when you mean "NNTP" and vice versa. While there are strong and obvious relationships between the two, they are different things.

Second, censorship-resistance was not and is not the main reason for NNTP working the way it does. I know somebody else already pointed that out, but it bears repeating.

Third, NNTP doesn't do flooding ("flood fill routing" is not a phrase with any kind of common meaning and is thus equivalent to a typo). NNTP is a point-to-point protocol, and is this incapable of flooding. Usenet doesn't use flooding either. For one thing, flooding is generally understood to mean "pushing" data everywhere it can go when it becomes available, whereas Usenet and NNTP have always been based on sending information where and when it is requested. For another, Usenet connectivity patterns are mostly tree-like and flooding - which refers to sending the same data along all possible paths to a single destination - is generally considered an inoperative term when only one path existed to begin with.

Re:Intertia vs. Good Ideas

[DavidTC]

ICANN's scope is the DNS system, upon which WWW, MAIL, NEWS, and everything else on the net currently relies.

Mail and www, yes. ICANN can instantly stop any email at all, as email requires DNS. WWW doesn't technically require it, but it sure would screw up all current links. If ICANN started playing with that for political reason, I suspect people would just start linking to 'non-approved' content using IPs.

However, ICANN can't hurt Usenet at all. Unlike email and WWW, you just need one computer's address to use all of Usenet. And, just as much to the point, you already have some sort of relationship, usually contractual, with your provider(s). You just have to call them up, or contact them in some way, and they'll be glad to tell you their IP, as you're usually giving them money, and it's in their best interests to not breach their contract by being unreachable. If they aren't helpful, you'll just give your money to someone who is. (Assuming you can find them, with this hypothetical censored DNS.;) )

If DNS went, though some unimaginable error, completely and totally offline tomorrow, it would get rebuilt by people using Usenet, as that's about the only thing that would still work. (Not that there wouldn't be an interuption, but only as long as it took people at various central providers to call up other central providers and get their IPs. And, yes, they do know these people's phone numbers.)

Re:Intertia vs. Good Ideas

[WNight]

Would I go through a huge hassle to make my web browser and MP3 player work 20% faster, or even Quake3? No. But I don't wait on those.

Would I go through the same hassle for 20% more bandwidth? Hell yes. I often end up waiting for downloads because I download 700MB images. Would I pay much for 20% faster CD Burning? Perhaps, if it still took 70 minutes like on my old single speed. But I wouldn't pay for the extra minute saved now that I can burn a disk in five minutes.

So perhaps you wouldn't care much about big savings in downloading from Usenet. But do you download much from Usenet? If not, does your opinion matter? Why not ask the people who use it? I think they've strongly indicated their views by starting to use yENC.

This isn't the same as a company forcing people to use something. This is the people choosing for themselves to find a tool to fill an important niche. It may not be terribly standard, but it's open. And as long as its consistent, that'll enable it to become a standard.

Re:Intertia vs. Good Ideas

[Anonymous Coward]

it would be a temporary censorship at best, but
plausible that the war planners would consider
their leverage of the DNS in some future washington-meets-and-
is-in-love-with-hollywood scenario for 'necessary'
nuclear war to thank those bio-terrorists that are busy at
this very moment training and developing hate for
the other. hate (for sin, bad, obviously) taught at
the church, mosque, synogogue.

hard to shut down the whole internet, criminalizing
usenet, well, even cheney must know that it's hard
to do and would be a public relations disaster.

people still spend more time looking at TV, and
newspapers, etc., so on the whole right-thinking
people go along with these specialist-spin-doctored
myths, the amazing hypnotic capacity of swirling
music words and imagery.

there are many more effective ways of intimidation,
guvmint is not addicted to this information medium

Usenet

[Anonymous Coward]

Heck, my provider doesnt even offer a Usenet NNTP
server anymore. I know about Newzbot, but most
"public" NNTP servers are read-only, and there
are a few that are post-only.

Are there any newsreaders that allow you to read
from one server, and send posts to another? Does PAN do that?

Yenc is great!

[Mean_Nishka]

I for one am happy about Yenc's rapid adoption. My newsreader ( Xnews )software supports it, and I have noticed no difference in using Yenc over traditional binary encoding.

In fact, Yenc will help pay-per-gigabyte Usenet users achieve a greater bang for their buck. Anything that saves money is a good thing!!

Re:Yenc is great!

[Wesley Felter]

...I have noticed no difference in using Yenc over traditional binary encoding.

Did you read the article? One of its major points was that traditional binary encoding sucks, and instead of yEnc, people should come up with something better.

Re:Yenc is great!

[atam]

The question is if the author of the article knew of the issues to implement a better binary encoding method, why didn't he do so? He even claimed that he had tried something like yEnc years ago, so he should be technical capable to come up with a better way. So I urge him: Stop whinning, create a better competing format instead. Even if he doesn't have enought time to commit on such project, he could just find some able bodies through internet to do actual work and act as an advisor or technical assistant to this project. yEnc is still in its infant stage, so it is still not too late to do something right.

Re:Yenc is great!

[BlueJay465]

Dammit! They should do something better! What exactly, I dunno. But they gotta do something.

This is what I interpreted from the article, A rant about the use of a new type of encoding because it is a new take on the old. So what if it is kludgy. It seems to be working. I don't see any new type of encoding coming down the pipe that would be considered a next-generation-USENET-type-of-thingy.

Hell, maybe USENET has lived beyond it's usefulness and does need an overhaul, but I seem to remember a time when people were bitching "Stop posting in base64! The rest of us can't read these files!" or even "Stop posting in JPG format! It takes too long for my poor 286 to decode JPGs instead of GIFs"

All this appears to be is more of the same. The haves versus the have-nots.

Re:Yenc is great!

[Anonymous Coward]

Instead of saying yenc sucks, the author should
come up with a solution long before yenc was
released.

Code talks, bullshit walk.

Re:Yenc is great!

[Apache]

He say in the article that he did try to help on it, but the author ignored him.

Re:Yenc is great!

[Dwonis]

That's pure bullshit. Criticism should be welcome in all technical circles. If you don't like criticism, then go do something else.

Re:Yenc is great!

[tricorn]

Wouldn't compressing the newsfeed transmission help a lot more than using a mostly-8-bit encoding method? UUencode compresses fairly well; if you're already compressing the transmission and try to send compressed data down the line, you basically get no additional compression, so 8-bit or 6-bit encoding schemes are almost identical. It'll save a bit of disk space if you're storing the newsfeed uncompressed, but disk space is cheap, right? Note that for people connected via modem, virtually all of them are already using compression, so pre-compression doesn't really help much, if any.

As an example:

bash is 316848 bytes on my machine; compressed with gzip (-6) it is 142711 bytes long; compressing again with gzip saves less than 100 bytes.

compressing with gzip, then uuencoding, then compressing with gzip again yields 148962 bytes, or about 4% more than sending the compressed bash through gzip.

if you use LZW for the transmission (e.g. as with compress), it depends on what you send; sending bash through 8-bit vs. uuencoded does give an increase of about 28%; however, if you gzip bash first, then the uuencoded version is actually smaller after compressing with compress (187423 vs. 177937).

At the same time, you're also compressing all those text messages, with all those lovely redundant headers to help increase the compression ratio. I've always assumed people pulling down large newsfeeds (as opposed to retrieving individual articles) were using compression (or connecting using compression from the modem).

the point of the article

[dunkelfalke]

instead of breaking the standard, code it right, wait till it matures, use it then.

What are standards?

[ndnet]

I really don't believe this to be a big issue. So yEnc isn't an official standard; big deal. Most 'standards' didn't start out as such, instead being what was adopted and then standardized. Look at basic hardware designs for great examples of this principal.

What is inportant is that it works, it is open format, and that it has a large enough user base now to become a standard. It can be changed to fix any problems it has.

I'm suprised that Slashdot posted this. It isn't a groundbreaking issue. Odd...

Re:What are standards?

[Anonymous Coward]

Of course, by doing this, yEnc is now a de facto standard, just like MS Word doc format . . . And what a good standard that is.

Re:What are standards?

[Anonymous Coward]

"Of course, by doing this, yEnc is now a de facto standard, just like MS Word doc format . . . And what a good standard that is."

Yet yEnc is completely open for all to see, use and implement. Apples to oranges.

Re:What are standards?

[ndnet]

Well, let's see.

Is MS Word .doc:
1) Functional? Yup. It can do what it advertises.
2) Open or reverse-engineered? Yup, the latter. Almost every office suite (open-source ones included) have sufficient reading and writing capabilities.
3) Support? Only under Office, but then it's covered by MS and by forums and such.

Am I arguing the MS .doc of even yEnc are perfect? no. All I say is that somethimes a de facto standard must be established. This was one of those times.

DIME?

[leighklotz]

There are some proposals in thee XML and Web Services arena for dealing with some of the problems tha yEnc is skirting.

One, called DIME, is a MIME-like system that handles binary content, chunks, etc.

http://www.oasis-open.org/cover/dime.html

This is why it is bad

[oPless]

This is why it is so bad:
(quote)
The problem is that doing [using) yEnc within MIME will break the current MIME specification, and very likely cause problems with existing software.You cannot add yEnc to MIME without first having two small changes made in the MIME specs. Bypassing the standards process would basically sabotage MIME by making it so that coding to the spec will produce software that doesn't work in the real world
(end quote)

Anyhow do we really need yet another damn standard for posting stuff. We're still having trouble with people posting html messages on usenet (Thanks outlook) - not to mention email...

Usenet really isnt the place for posting pics of you shagging your favorite goat near the railway lines, or the latest mpegs of enterprise etc.

In fact, lets kill off mime altogether, and get this multimedia crap out of our inboxes. Email/Usenet is for plain text, period.

Bah humbug. :-)

okay so I /do/ use outlook occasionally, and sometimes email attachments but only as a last resort.

Re:This is why it is bad

[dwsauder]

You've obviously not spent much time in the "binaries" newsgroups. These groups are very popular. I have personally downloaded over 1000 MP3 files from the alt.binaries.sounds.mp3.* newsgroups. I hate to break the news to you, but Usenet is not just for text discussions any more. When Napster and all of its clones are taken down by lawsuits, we all be getting our music from Usenet.

Re:This is why it is bad

[Anonymous Coward]

You're absolutely right! Let's kill all the MIMEs. I can't stand when those bastards follow you down the street while they mimic your every move.

When they keep running into those invisible walls, I just want to punch a hole through that wall and hit them in the face.

I declare open season on MIMEs!

...

Oh wait, I guess you didn't really mean MIMEs did you?

Re:This is why it is bad

[dvdeug]

In fact, lets kill off mime altogether, and get this multimedia crap out of our inboxes. Email/Usenet is for plain text, period.

Yes, because everyone who doesn't speak English should just shut up. You did realize that MIME is the only standard on how to send non-ASCII plain text across email, didn't you.

Re:This is why it is bad

[wheany]

Just as long as you keep your binaries in binary groups, I won't mind...

Re:This is why it is bad

[Anonymous Coward]

TEXT ONLY!!! Are you mad!! The value of email and newsgroups would be severely diminished without the ability to attach. Sure, some short run problems (like virii) would be eliminated, but you would be taking us back to the 80's!

How the hell would you send a spreadsheet to someone without mime? What are you going to do, dump the delimited text in the message body? Are you stupid?

This is the dumbest post I have ever read here. Not using attachments? Also, without an effective way to squeeze multimedia files into nntp posts, how the hell I am going to see any movies or listen to any music?

I am not buying any more content from a group of criminals how have taken arms against me. (Read: if encryption is a weapon, as claimed some years ago to prevent 128 bit patches "falling into the wrong hands" and the dmca specifically forbids even attempting to remove the encryption - then the movie companies have put a gun in every DVD they sell and have pointed it at us!)

Re:This is why it is bad

[ncc74656]

I hate to break the news to you, but Usenet is not just for text discussions any more.

Has it ever been? I was downloading software posted to comp.binaries.apple (what's now called comp.binaries.apple2) as far back as 1989. If Usenet was ever text-only, that would've been before my time. :-)

yEnc is terrible.

[Anonymous Coward]

I had scripts that automatically got some a.b* newsgroups, but after the invention of this bastardized yEnc piece of crap, all my scripts are broken, and I'm 2 months behind on data for our clients.

BTW, I work for a pr0n site :) Only e-biz that's thriving still.

Re:yEnc is terrible.

[Anonymous Coward]

Ok, so you were stealing free porn from the newsgroups for your employer to sell, and you arent/werent able to modify your scheme to support yenc..

Why is it that we should feel sorry for you?

Agent

[Account 10]

Forte released Agent 1.91 2 days ago with yEnc support. it looks like Mr. Nixon is fighting a losing battle.

Re:Agent

[Anonymous Coward]

Use XNEWS... it's free and easier to use than AGENT

Re:Agent

[Anonymous Coward]

NewsBin

Re:Agent

[Typingsux]

I wish I didn't blow my mod points or you'd get one from me.
I have played with Xnews, grabber, newsbin. Agent for some reason I like best. Now with yenc support it's over for me. Back to agent.

Re:Agent

[Anonymous Coward]

The funny part is the Mr Jurgen doesn't even like it's current inception. Check the Follwing Link: link

Re:Agent

[evilviper]

Well, just about every operating system on earth has PLIP available. By your logic, that must mean PPP will be dead soon.

Re:Agent

[IHateEverybody]

The funny part is the Mr Jurgen doesn't even like it's current inception. Check the Follwing link

That message is a week old and was sent in response to a one week delay in the release of Agent 1.91 (the first version with yEnc support). And in any case, the link you provided shows Mr. Juergen being very complementary to Forte Inc. and the Agent development team.

Since then Agent 1.91 has been released and Mr. Juergen posted this message to the Agent newsgroup thanking the Agent team.

Re:Agent

[hkmwbz]

Actually, I find Agent to be easier to use than Xnews. But that's probably because I use Agent regularly and haven't tried Xnews much. Perhaps this is what you meant, only that you have used Xnews more?

Just use the software which best suits your needs :)

I've been considering having a longer "test drive" of Xnews, though, also to test the yEnc support. It's just so hard to get used to new software!

Regarding the yEnc issue, from what I've read so far, I agree with Jeremy Nixon's criticism of yEnc. It should have been planned better and made a proper standard, and only then be used, rather than the "shotgun tactics" used to force the spreading of yEnc perhaps without considering the potential consequences. Then again, I am somewhat of a pessimist. Most people probably won't have any problems. We'll see what the future brings.

Is usenet dead?

[Warped-Reality]

NOTE: This is actually a question, not a troll

Does anybody really use usenet anymore? everytime i've poked around on my ISP's NNTP server, it seems to be filled with 90% spam, and non-spam posts seem to always be grossly offtopic. And no, i'm not just talking about the alt.binaries groups either.

Re:Is usenet dead?

[mmusn]

Brings to mind Yogi Berra: "The trouble with that place is: it's so crowded that nobody goes there anymore."

Re:Is usenet dead?

[Anonymous Coward]

usenet is the place for

free pr0n vidz ..
alt.binaraies.multimedia.erotica
free tv ..
alt.binaries.multimedia
free mp3s
alt.binaries.mp3.*

you name it .. you got it on alt.binaries.*

dont forget *nix help as well !!! job postings .. *.forsale newsgroups .. fan discussion ..

NEWSGROUPS WILL NEVER DIE .. *i hope*

Re:Is usenet dead?

[dattaway]

Usenet is very much alive and a great way to bring together communities who like to share original material, even in binary newsgroups. My favorite is alt.binaries.pictures.motorcycles.sportbike where I can see many insane pictures posted on an impulse not found on websites.

The binaries groups I have seen have been pretty much noise free. If the users are vigilant and like to use physical means to squash spammers, the forum is void of abuse.

Re:Is usenet dead?

[mmontour]

Does anybody really use usenet anymore?

I do. Some groups are mostly noise, but others are still pretty good.

As for the spam problem, maybe you can suggest to your ISP that they install Cleanfeed or similar. (Yes, that's the same site as the anti-yEnc page, which BTW I agree with.)

Re:Is usenet dead?

[Graymalkin]

I use Usenet all the time (pun intended). While there is a good deal of spam in some groups others are pretty light on the spam. For me Usenet is the place to turn when I run into a problem in Linux because I've found there is a lower proliferation of idiots using it unlike IRC which is filled with 15 year olds blabbering about RTFM. It also has some pretty good discussions on several topics if you hunt around a bit.

Re:Is usenet dead?

[maggard]

Does anybody really use usenet anymore?

Yes.

That's why folks care about the yEnc issue. If it were a fight over, say, a Gopher implementation do you imagine there'd be much discussion?

Usenet is in trouble, it may be mortally wounded, but it'll be around for awhile yet and in that time lots of folks use it.

Re:Is usenet dead?

[Anonymous Coward]

I use Usenet all the time (pun intended).

Unfortunately there isn't a pun in your sentence.

Re:Is usenet dead?

[CrazyDuke]

Usenet is very alive. The problem is finding a decent newsserver that doesn't have To get this I've had to subscribe to a commercial nntp server and paying for the membership. But, it is worth it.

If you are just interested in text posts, you can search google (click the groups tag) for posts up to way back when the internet was mainly populated by college students, researchers, and the gov.

Spam is a problem, but one learns to ID it from just a glance at the subject line. Some are more devious and annoying. But, thats spam for ya.

I don't like yenc personally, but my big worry about usenet is that the corps are starting to sue the crap out of nntpd hosts that have groups that the corps find their copyrighted material in. Oh, and watch your step, some of the porn is a little...um...extreme for some people.

(props to a.s.s, a.b.a, a.b.g.a, a.b.m.e, and a.b.p.e.a!) ;P

Re:Is usenet dead?

[Sabalon]

Yeah...but once you get outside the porn groups the signal-to-noise ratio gets better :)

As someone else said, there are plenty of idiots posting the same questions over and over (groups.google.com anyone?), and topics tend to degrade, but for the most part, the groups that are actually dedicated to something are pretty decent. The Tolkien groups, the linux groups, some of rec.arts groups.

Re:Is usenet dead?

[MillionthMonkey]

I never post to it. (And after finding my ten year old posts to alt.drugs sitting in plain sight on Google, I doubt I ever will again!) But I do searches through newsgroups all the time when I'm looking for answers to obscure technical questions, or when I want to know if anyone's come across the same bug that I'm having trouble with at the moment. USENET might have more crap, but sometimes crap is what you want! You don't always want your search results to get choked up with corporate stuff. If you're doing Java programming for example, and you want to find out why class X doesn't work, a normal web search is difficult to control because all you get is bullcrap from Sun, and a zillion identical javadocs for class X that people leave around on their HTTP servers. I want to find out what people are complaining about, or whether anyone actually USES a certain API I'm considering. For getting a feel for what's going on in the field, without getting snowed under by marketing materials from a vendor, USENET is great. It isn't as corporatized.

Of course, the existence of alternate web based bulletin board systems like this one decreases its relevance for search purposes. And it's suffocating under the weight of all that spam. But USENET is still the biggest forum out there, and it's still the one that's the most easily searched.

Re:Is usenet dead?

[thogard]

I run about 700 groups on my own server and the level of junk is very, very low. Most of the groups I read on a regualr basis have almost no spam.

There are about 20 groups I read almost daily and about 100 I drop into from time to time. About 400 of the groups are associated with ones I read (I read sci.geo.sat-nav so I also get sci.geo.*). I also have regional groups for three areas.

I've got about 7 news peers. These are all smallish servers or smallish ISP's and their sysadmins have their pet groups as well so I carry them (they help me, I help them). That adds up to about 700 groups. Total in the spool is about 600 meg for 15 to 30 article storage.

My server is called news.abnormal.com and is open for public reading.

Re:Is usenet dead?

[Graymalkin]

Thats why it is fucking funny. Get back on the fucking short bus.

Re:Is usenet dead?

[Anonymous Coward]

What is the alternative? Web discussion groups/forums. Those fucking suck and have fascist moderators. USENET rules -- WWW forums drool.

Re:Is usenet dead?

[RFC959]

Oh, hell no. comp.os.linux.*, comp.os.solaris.*, comp.sys.sgi.*, comp.unix.*...all are pretty active and relatively spam-free. Plus, your ISP may have local groups: depending on the quality of the ISP and its users, these can be very good and are entirely spam-free (unless one of the local users decides to spam them, which would be a spectacularly stupid move).

Re:Is usenet dead?

[Warped-Reality]

Thanks, i'll give some of these groups a try

Re:Is usenet dead?

[Anonymous Coward]

the "X-No-Archive" header should prevent google.groups from archiving it (in theory anyways :)

Re:Is usenet dead?

[ncc74656]

Does anybody really use usenet anymore? everytime i've poked around on my ISP's NNTP server, it seems to be filled with 90% spam, and non-spam posts seem to always be grossly offtopic.

Maybe it depends on what newsgroups you frequent. The worst that I usually see is flamage over abandonware in comp.sys.apple2, which is jihad fodder for some. Aside from a few easily-filtered kooks, several of the comp.sys.ibm.pc.hardware.* groups offer decent S/N. Last time I checked, comp.os.linux.* wasn't too bad either.

Re:Is usenet dead?

[DavidTC]

If you want google to delete your post, just ask them.

Re:Is usenet dead?

[DavidTC]

'Usenet is dead' is just one of those things that everyone repeats but isn't actually true. You just need to get a server with Cleanfeed, ignore the 5% of the groups that have been taken over by idiots, and be ready to killfile about 1% of the people in the groups you frequent, and it works just fine.

I'm really baffled by others saying it is dead. I'm not exaggerating, I truly am confused by this idea. I'm simply not seeing any thing that could be making people think that. These people must have had a fundamentally different experience there than I have. Maybe they stumbled into a group under a flood or something.

no no no

[SETY]

The author of the article seems to keep saying there isn't a problem with USENET encoding, but then goes onto complain that yenc shouldn't be used. He points out flaws in how this encoding scheme is implemented. fine fine fine.

There was a market for this thing, it spread like wild fire. It's too bad that no one made a better spec and program (the author aludes that there was planty of time to do this). yenc meets the "GOOD ENOUGH" criteria, thus it will be used, shitty, non-robust standard or not.

Re:no no no

[Plutor]

It's this kind of thinking that has made MSIE's broken HTML and Outlook's poor security the industry standards. Just because people can stand shitty software doesn't mean we need to give it to them.

Re:no no no

[0x0d0a]

I have to agree. Yes, it sucks that there's no longer any carrot to convince users to upgrade to any new standard -- the speed issue is gone. However, if someone was even in the progress of making a standard, I think that the argument against yEnc would be a lot stronger. There was nothing going on...so yEnc will be the new standard.

yENC

[arsaspe]

I'm all for standardisation... but sometimes it takes _forever_ to get something standardized. If someone writes a better product, they generally don't want to wait for it to be declared a standard, especially with something like uuencoding which has been around as long as usenet, and isn't going to be replaced in a hurry unless someone comes out and waves a product around yelling "hey try this. it works better". Ogg Vorbis isn't a standard by any means. Hell, it is still on RC3. _but_ a lot of people are using it because it has far better sound compression than mp3. You don't hear people complaining that Vorbis has jumped the standardisation process do you?

Personally I can't see why we can't just send the data as 8-bit binary. uuencode and similar encoding formats should have died out with UUCP years ago, since there is no physical reason why 8bits can't be sent over the wire anymore.

Re:yENC

[MSG]

Ogg Vorbis isn't a standard by any means

Ogg Vorbis has a specification for its file format that doesn't break any previously specified standard. Therefore, Ogg Vorbis can be called a standard.

yEnc, on the other hand, breaks the NNTP standards, and will likely break the MIME standard. That's the difference. yEnc must fit within the standards already specified for the transmition methods it uses, and does not.

Re:yENC

[dwsauder]

Personally I can't see why we can't just send the data as 8-bit binary. uuencode and similar encoding formats should have died out with UUCP years ago, since there is no physical reason why 8bits can't be sent over the wire anymore.

Because many of the protocols we use, like SMTP, IMAP4, POP3, and (yes!) NNTP, use "lines" (of text) as the "framing" method. As you probably know, TCP doesn't provide any kind of framing, so framing has to be handled at a higher layer in the protocol stack. If you use "lines" to define framing, then you search through the content to find CR LF, which marks the end of a line. The alternative is to provide a length at the beginning of segment. There is an extension to SMTP to allow that, but there is no similar extension to NNTP. But, as long as framing is done by breaking content into lines, you won't be able to transfer unencoded binary data using NNTP or similar protocols.

Re:yENC

[Anonymous Coward]

Hey -- I'm sure there's still a couple ESR-like characters out there getting their porn feeds over UUCP over a 9600 kbit serial link strung between their shacks/university offices/whatever.

Re:yENC

[arsaspe]

Yes, but my point was that there isn't a physical reason why data cannot be sent in 8bit format, only a software one. Rather than trying to accomodate ancient protocols that do not make use of modern technology, we should be updating them. (And yes I am aware that getting everyone to update to a new version of NNTP that isn't fully backwards compatible would be a big job... but it needs to be done. The best example of this is IPv6- we need to upgrade, but no one wants to go through the teething problems that will result)

Re:yENC

[Anonymous Coward]

*giggles* Actually that makes one wonder.... how _did_ people download their porn with the slow ass modems of yesterday? I'd imagine your average jpeg would take about as long to download on them as a small mpeg on a 56k

Re:yENC

[thogard]

So your saying I should upgrade my server (which has been running for nearly a decade) so that people can use it to pirate binary stuff?

The next time I do an update, I'll be adding a cute patch that kills all binary posts at the nntp level, not after the fact like now.

Re:yENC

[arsaspe]

Thats not what I'm saying.... I'm saying that a lot of protocols, NNTP included, are outdated.

Re:yENC

[Anonymous Coward]

I seem to recall porn was pretty popular on the BBSes in the 1200 bps days. Mostly small GIFs though.

Before there was good reader support, Usenet binaries (porn) was a real bitch -- all those stupid UUENCODEd segements had to be manually reconstructed and decoded. My old roommate was into it, and it would take him about 15 minutes to get a single JPEG viewable (SunOS shell account, tin? and some csh scripts, 14.4 modem, ZMODEM, and an old Mac LC running Zterm and JPEGView). He was real excited when he found this tool called "www" which let him get actual binaries.

Re:yENC

[Rui del-Negro]

A "standard" is not the same as "a common thing", or even "something that most programs can read". For example, most programs can't read YCrCb TIFF files (Photoshop included), but these files do follow the TIFF specification, hence they are standard.

When you create an encoding method that is going to be used to transfer data across a network, you need to ensure that this method is compatible with everything along the way. When you send an e-mail with an Ogg file attached, this file is encoded in a way that enables the servers and the client at the other end to identify it, check its integrity, reconstruct it, process it, delete it independently from the rest of the message, etc. It doesn't matter what the file itself is (Ogg, MP3, TIFF, Doc, XYZ, etc.); these methods work with any file, regarldess of its type or contents.

8 bits can be sent over the wire, and in fact are sent over the wire. But to make sure the servers (and the clients) can tell where one file (or part of the file) ends and the next one begins, you need to "wrap" the data in a package that programs can understand. That's what MIME does. It says "this part is the message text in HTML", "this part is the message text in plain text", "that part is an image", "that part is an executable file".

Instead of using this "universal" packaging system, yEnc forces programs to look for specific strings and try to guess where things begin and end. And it has no mechanism for identifying individual parts in a multi-part post (again, programs must look at the text or message subject and try to guess).

Doing something right is usually not much harder than doing something wrong. And when people get used to something that's broken, they won't want it fixed.

Re:yENC

[Anonymous Coward]

"Instead of using this "universal" packaging system, yEnc forces programs to look for specific strings and try to guess where things begin and end. And it has no mechanism for identifying individual parts in a multi-part post (again, programs must look at the text or message subject and try to guess)."

Your so called '"universal" packaging system' can't even package yEnc according to the article link. If it's so universal why are you worried about a little programmer doing something? I'd say it's far from universal. MIME may be the standard on paper but UUencoding is the defacto standard in practice.

"Doing something right is usually not much harder than doing something wrong. And when people get used to something that's broken, they won't want it fixed."

That pretty much sums up my feeling on the MIME standard, and why I feel it was nice that someone cut through the BS and made something evolutionary called yEnc to replace UUE. The MIME body now has a higher bar to raise itself to and has healthy competition for mindshare. These are both good things and things not present before.

Re:yENC

[JamieF]

It's interesting that it took this long for an ancient problem to be sorta kinda attacked, if not solved. Maybe this is because the Usenet infrastructure has changed to the point where 7-bit is silly, but why is this something that's just being addressed now? Don't tell me this is a brand new problem that the best minds of the news admin community couldn't have figured out by now.

One approach to short-circuit standards is to take the rogue approach that Netscape did, which is very different from the one folks keep accusing Microsoft of. That approach is to take an almost-suitable standard and extend it, in the same spirit, with the intent that your well-thought-out extension will be adopted later. Of course they did a less than perfect job, but the idea is sound: don't wait for the spec, lead it, while making it possible for it to remain open. That's different from the Microsoft approach of "Embrace, Extend [and Exterminate]" wherein you add undocumented proprietary features that lock you into a single vendor's solution. A hell of a lot of what's in the HTML 4 standard was released as a non-standard but documented extension by Netscape in 1.1 and 2.0. Some of that was bad (blink, and arguably frames), but when you judge Netscape, try to separate the new HTML tags from the bugs, because it's the bugs (and having to code around them) that make the web such tower of Babel.

So basically what I'm saying is, if MIME is almost there, CAREFULLY THINK THROUGH and implement the extensions you want, and implement it already. Or, pick something other than MIME. But, don't start from scratch and make mistakes that have already been learned from and solved. Build on the good decisions of the past. The real crime is solving the same problem, badly, over and over; getting ahead of a standards body while keeping intentions pure and decisions wise is not nearly as bad.

Re:yENC

[Anonymous Coward]

I'd just like to point out that if Microsoft did this, you'd all be up in arms, hypocrites.

Re:yENC

[Anonymous Coward]

> I'd say it's far from universal. MIME may be > the standard on paper but UUencoding is > the defacto standard in practice. MIME and UUE are two different things. It's like comparing an alphabet with a language. There's no problem in using MIME to structure UUEncoded data, and in fact most modern programs do that.

Re:yENC

[drinkypoo]

Personally I can't see why we can't just send the data as 8-bit binary. uuencode and similar encoding formats should have died out with UUCP years ago, since there is no physical reason why 8bits can't be sent over the wire anymore.

First: UUCP is not dead. It is still in use, and still useful in places where there is no (afforable) broadband and the quality of POTS lines is trash.

Second: UUCP is capable of sending binary data. uuencode/uudecode is just for embedding 8 bit data in something which doesn't support it, such as mail, or USENET news. This is why we have base64 encoding in MIME mail.

News send via UUCP was batched, which translates to being tarred and compressed, though it was possible in later versions to plug in other transfer protocols to replace the "g" protocol (Zmodem was a popular replacement, for obvious reasons) and to use other batching methods, including zip.

Why this is bad

[Anonymous Coward]

Yes, it's supported in Forte, and it would probably be supported in any windows implementation.

But, for standard loving Unix people, slrn and the likes, this would break everthing, and this would eventually shut us off most of the binary usenets. When yEnc goes to e-mail, shudder...

This is very much like the way Microsoft does stuff, yEnc is a standardbreaking, non conforming tool that when loaded on the windows hoards would eventually bring about a catasrophy as mentioned above.

What can we do about it?! Flame any one who posts with yEnc.

Do you want to be shut out of the Usenet too? Do you want non-standard shit to finally close you down? BTW, yEnc is very compatibile with microsoft products and their schemes.

Re:Why this is bad

[oolon]

Pan supports yenc to name one, lots of *nix tools available to decode, encode and post. Don't want to change software? You can still use tin to tag all the parts save them to disk and use an extenal decoder. This was the only way before with many *nix news readers. So not much change really just a different filter to use! Yenc like another much loved binary tool "par files". Seems to have been available on *nix first not second.

Is it good is it bad? To late its here.

James

Re:Why this is bad

[domc]

Do you know of a PAR tool for unix?

domc

Re:Why this is bad

[bighousehx]

Try parchive.sourceforge.net.

Also, SmartPAR works (for me, anyway) under WINE.

Re:Why this is bad

[oolon]

Yes I did know it was available and it use it, and this unix was my point, I obviously didn't make it clear enough. People were saying yenc was a windoz thing. I was trying to say it is not, as par is not. Windos users were the first to get clicky clients, not working impletementations.

Re:Why this is bad

[domc]

I realized what your point was. I genuinely need to find a PAR tool for linux.

domc

Re:Why this is bad

[domc]

Thanks a bunch! Booting into windows just to run a parity check was becoming quite tiresome.

domc

Change we must

[Anonymous Coward]

Dude, I remember years ago people on usenet whining about this new "JPEG" format. "STOP POSTING IN JPEG! POST ONLY PIX IN GIF!" they cried. They WHINED and WHINED about jpeg.

You tell me how that turned out.

Re:Change we must

[Anonymous Coward]

Exactly. And all the 'GIF ought to be free' whiners should just cough up the dough that the patent owners have a Constitutional right to collect.

Fuckin' no software patent fucks... I guess not being able to make money coding is fine, since they all live together on a commune anyway.

Re:Change we must

[Wakko Warner]

The key theme here is that people on usenet whine. A Universal Truth, as it were.

- A.P.

Re:Change we must

[Anonymous Coward]

I think they should patent the dictionary. That way you wouldn't be able to use any of the words in it to spread your bullshit.

Re:Change we must

[Anonymous Coward]

Whatever, cockwad. Go back to your Linux commune and suck RMS's natty-pubic-haired nuts.

there is no real bloat associated with uuencode

[mmusn]

Uuencoded text will compress down to nearly the same size as its corresponding binary (or less, if the binary can be compressed). That kind of compression is now a standard part of modems, Internet protocols, and many file systems. Even the CPU overhead of compressing and decompressing that kind of data is negligible. If yEnc doesn't end up using less space on disk and doesn't end up using any less bandwidth than uuencode, indeed, "why encode" in yEnc and break a lot of software that expects USENET posts to be text-only?

Re:there is no real bloat associated with uuencode

[A Commentor]

Uuencoded text will compress down to nearly the same size as its corresponding binary (or less, if the binary can be compressed). That kind of compression is now a standard part of modems, Internet protocols, and many file systems.

I have yet to see any DSL or cable modem with compression. So for most of the heavy binary users, uuencode data will not be compressed. And on regular modems it won't be smaller than the the yEnc, since if, as you say, the binary can be compressed, then the yEnc will be compressed..

Even the CPU overhead of compressing and decompressing that kind of data is negligible.

Do you run a heavily used news servers to provide proof that the CPU overhead is 'negligible'.

Re:there is no real bloat associated with uuencode

[mmusn]

I have yet to see any DSL or cable modem with compression. So for most of the heavy binary users, uuencode data will not be compressed.

Oh, come on, think. Even if your modem doesn't compress, you can get compression at the level of a tunnel or at the level of your news transfer protocol.

Do you run a heavily used news servers to provide proof that the CPU overhead is 'negligible'.

Not anymore. But I do use software compression with most network connections, and it doesn't seem to load the CPU much.

If it really worries someone enough, they could come up with a compression envelope that works for all USENET postings and is used both during transmission and storage by servers that know about it. But it's apparently not enough of a problem for anybody to seriously worry about.

Re:there is no real bloat associated with uuencode

[A Commentor]

Oh, come on, think. Even if your modem doesn't compress, you can get compression at the level of a tunnel or at the level of your news transfer protocol.

For 99% of the people, getting the tunneling software on your system will be easy, getting your news server to install software on their software will be alot harder...

Re:there is no real bloat associated with uuencode

[mmusn]

Huh? A compresing tunnel is transparent. It takes one command to set up (two if you don't want to change the NNTP host in your news software).

Oh, wait, you may be using Windows, and Windows users perhaps just can't figure this out? They can't even figure out what software to buy to do this for them? So, because Windows users can't figure it out, instead of just adopting a simple compressed NNTP protocol, everybody is supposed to change their software to accomodate binary postings, and everybody is supposed to post in binary?

Sorry, but that strikes me as enormously stupid. If you want efficient news transmission, do it right: use a compressing news transfer protocol, either at the level of news transfer or at the level of your networking protocols. Imposing a new, incompatible format on everybody else because you can't figure out how to compress your I/O makes no sense.

Re:there is no real bloat associated with uuencode

[Anonymous Coward]

Several years ago I worked my butt off to get a faster UUdecode algorithm working in x86 assembler. Assuming 60 characters and the '\n' (unadorned) per line, and even including the per-line looping overhead, I managed to get the 45 binary bytes out of there through the execution of very few 32-bit instructions. (Some byte-wise swapping "magically" alleviates lots of bit-wise shifting.) The thing about Usenet is that you _can_ assume no crlf problem most of the time, thereby eliminating a lot of so-called "robustness" overhead. _During_ my work on the algorithm, it became clear to me that my work was obsolete. Moore's Law clobbered the value of the optimization _back then_. Furthermore, even though I was proud of my achievement, I realized that equivalents of similar performance were _too easy_ to code for me to place any hope in cashing in. In short, it ain't broke. Why fix it?

The Internet is not a places for this... yet

[Anonymous Coward]

Sorry, but I think the internet is not a place to be testing standards in such a highly used area as Usenet. It's too massive and restricts access to binaries to too few people as the new system (y-enc) is continually updated, changed modified, etc. It affects nntp style servers like news.uu.net and affects binary image news servers like www.WebBinaries.com meaning over 15,000 servers need to be code synchronized, and countless millions of browsers on numerous platforms. That's ridiculous.

Wait till it's done, then propose it as a standard like almost every other piece of the Internet. (or at least wait till it's close enough)...

Re:The Internet is not a places for this... yet

[Anonymous Coward]

Yeah, but if its alt.binaries.windows, does it matter if UNIX users can get to it? Seriously, think about it.

Adoption of yEnc

[jonbrewer]

I am seeing smaller binaries as a result of yEnc. This is fine. The problem is, my favorite binaries grabber has no idea what to do with the files, and won't even download them. I figured out how to make Agent download them, but A. I hate Agent (and don't understand why anyone likes it!) and B. the binaries don't always decode.

As I'm a lifetime lurker (well eight years, but it seems a lifetime!) I can only choose not to download yEnc encoded binaries. And no one will know! (my news server doesn't log downloads) It's all up to the posters to adopt or not.

Re:Adoption of yEnc

[Backov]

Check out Newspro, it's better than most binary grabbers.. I wouldn't use it for reading text newgroups, but XNews is good for that.. Newspro lets you combine headers from multiple servers. I download from my paid usenet server plus the @home servers and regularly saturate my connection at 500k/sec.. A buddy across town gets 700k/sec.

Newspro - http://www.usenetopia.com

XNews - http://xnews.3dnews.net/

Newspro is shareware, XNews is freeware.. They're both good, for different things.

Cheers,
Backov

Re:Adoption of yEnc

[McChump]

I recommend Binary News Reaper, aka BNR2

http://www.co.jyu.fi/~ap/bnr.html

It's still in beta, but it's pretty great, it's freeware and supports yENC.

Re:Adoption of yEnc

[Anonymous Coward]

some one said the same in abpua ;)

That's a frightening segment of society. If we can call it society.

Dunno why they marked you offtopic though. They should have modded you up as, +3 sick and informative.

Yeah, and off the web too!

[eddy]

In fact, lets kill off mime altogether, and get this multimedia crap out of our inboxes. Email/Usenet is for plain text, period.

I second this. Also, let's make it so that non-validating and just generally malformed XHTML documents are rejected by browsers, and can be filtered out of google. plain HTML =

I'd just love to turn on a "[ ] Reject non-validating pages" option in google and see the world wide web with new eyes :-)

Re:Yeah, and off the web too!

[eddy]

Oh crap, missed entitiy'ing the less-than. The irony, eh? The part missing is:

[...] plain HTML =< 4.01 is a lost cause, but already I've seen sites claiming XHTML in the DTD, just to go on and PISS all over the spec (see forums.bioware.com for instance, though I guess the problem is with that bulletin-board package).

Re:Yeah, and off the web too!

[Anonymous Coward]

If you wanted to reject invalid HTML, you wouldn't be able to read slashdot.

Didn't you know that slashdot HTML shits all over the spec?

See this post for more information.

Re:Yeah, and off the web too!

[Anonymous Coward]

Get a fucking webpage.

fine with me

[jest3r]

Anything that can get rid of 30% of legacy bloat is ok in my books ..

I am surprised something like this wasn't adopted years ago when we were all on dialup .. It would seem thats when we needed smaller file sizes the most.

Warez and pr0n

[Doom Ihl' Varia]

Armed with such resources, Usenet shall never die.

Re:Warez and pr0n

[Anonymous Coward]

Usenet is dead. The body is being used by smugglers to get porn and warez across 'the border.'

It's sad, really. It's the way things are.

Re:Warez and pr0n

[Da Masta]

Amen brotha!

Usenet? Please go away!

[fm6]

What do you think?

I think anybody with any common sense should do everything possible to avoid using Usenet. Especially for file sharing. Usenet was designed for an environment where all the network connections -- user-server, server-server, eveything -- were modem speed. Not 56K modems, mind you, but 1200, or even 300 bits per second. So replicating every message, every file made sense. But how many people does this describe now? Hello! Slashdot! FTP! HTTP! Napster!

I suppose you can justify using Usenet as a free-form discussion channel -- mainly because nobody seems to be working on web oriented replacements that do a better job of matching up people with common interests. (I swear, 60% of all the content on my company's newsgroups consist of threads that start with "Hello, I'm new at this, how do I...", followed by an explanation that's already in the FAQs and/or a reminder to post to the correct group.) But what's the possible justification for continuing to use Usenet to share files? Aside from inertia.

Re:Usenet? Please go away!

[Anonymous Coward]

"Especially for file sharing"

No no no. Usenet is good for everything because a web site can be stopped with a single letter from an aggrieved party.

Try it sometime. Try setting up a website posting some copyrighted scientology stuff. It will be up there about 24 hours.

Now, upload the same stuff to Usenet. Its out there, and NOBODY can stop it.

Do you understand why that makes usenet infinitely more powerful than the web?

Re:Usenet? Please go away!

[Sabalon]

Lets see...Napster is dead. Morpheus is dead. Morpheus part 2/gnutella is a zombie (perhaps it's just me - I have yet to have it actually pull down a file for me. It keeps telling me the other end is busy or something.) Even when they were alive, you get half way through and someone cuts you off...or you find out that 80M download that took a whole day was actually mislabled.

On Usenet...sure - you don't get to search and finding a file involves posting a request and hoping someone fulfills, but you get bandwidth - assuming you want to pay for it...you get files that are there (assuming you have decent retention.) and not dependant upon someone being online. And unless you have a crappy server, you don't get halfway through a download and someone decides to kick you off. And 99% of the time, what something is posted as is what it is.

As for the replication - well, there is no one point of failure. As well, you don't have one site getting the shit hammered out of it either. I pay $9/mo for usenet...I get three fast servers to choose from and some high GB limit.

If I had to go to the same server as everyone else, you'd have the same problem that moviefone had when star wars tickets went onsale online - DOA - all with nice corporate control of content.

Re:Usenet? Please go away!

[Anonymous Coward]

Or better yet, set up a web site and offer free DivX porn movies. Then count the minutes until you exceed your bandwidth threshold and the provider shuts you down or wants more money.

Usenet is kind of a hacky solution, but it does corretly push the costs of bandwidth back to the consumer. And that's why some larger ISPs like it -- because they can keep the transfers within their own network.

Re:Usenet? Please go away!

[Shelled]

"But what's the possible justification for continuing to use Usenet to share files? Aside from inertia."

Discovery, that's why. Napster (Napster?, talk about dead!) and the like are great for finding a name you already know. By design, that's all it can do. Do a search for "early '80's Boston punk" for example and see what you get. This is where Usenet excels. Enthusiasts congregate into groups and upload files they enjoy, meaning a far better chance that I'll enjoy them too. I'm continually discovering great bands through Usenet that would remain unknown had I relied on name-search based file sharing.

Re:Usenet? Please go away!

[Anonymous Coward]

Yes, but the point is, there could be a text discussion site where people talk about "early '80s Boston punk", where people even talk about and cite FTP or WWW sites where the files are available, without it involving them sticking the attachments right in the middle of the discussion.

Re:Usenet? Please go away!

[Anonymous Coward]

I meant to type 'text discussion group' and not 'text discussion site' which badly obscures my point. sorry.

Re:Usenet? Please go away!

[Anonymous Coward]

I completely agree. The groupings in UseNet is what makes it so good. When you subscribe to a group a majority of postings will be what you are interested in and you can discover new groups or information. UseNet is far superior in this respect. I have been using UseNet for several years now and it is far superior to any other way of finding people with similar interests (even better than Slashdot I dare say).

yEnc = XMODEM part deux

[phr2]

In the old CP/M days there was no standard way to transfer files over serial connections, except maybe Kermit. Kermit was slow because of its ping-pong protocol (no packet window--that was added later) and because it encoded binaries as printing characters. Ward Christensen invented XMODEM, which basically dumped the file through the wire as 8-bit characters, with very crude error checking and file headers. yEnc does something pretty similar for Usenet articles. It's a crude method for posting binary files as 8-bit characters instead of 6-bit characters. That of course cuts down transmission time considerably.

Despite its problems, XMODEM took off because it filled a need, just as yEnc does. Nixon's complaint that shrinking files by 35% won't make Usenet any smaller because people will just post more files is besides the point; it's like saying getting a 35% salary increase won't help your finances because you'll just buy more stuff with the extra money. Most people want that extra 35%, and Jürgen stepped up to the plate and delivered it.

Thankfully, as far as I know, nobody railed against Ward Christiansen the way Nixon does against Helbing. XMODEM's problems became obvious and the solution was to introduce YMODEM and then ZMODEM. XMODEM is still around, but its successors (and of course serial IP) have pretty much supplanted it. Ward's initial efforts are still deeply appreciated.

Yes there's the problem of legacy software, but a protocol that's only been around for a few weeks or months can't have that much of a legacy. The only programs that currently support yEnc are the ones whose maintainers react pretty fast to new developments, and those maintainers are likely to also quickly pick up any revisions/fixes to yEnc.

So the solution Nixon should be calling for is not a years-long bureaucratic standardization process that will get yEnc 1.3 entrenched while the standardization is happening. The solution is to fix yEnc's problems and release a new version as fast as possible, before the old version gets spread around too widely.

Re:yEnc = XMODEM part deux

[dbarclay10]

Yes there's the problem of legacy software, but a protocol that's only been around for a few weeks or months can't have that much of a legacy. The only programs that currently support yEnc are the ones whose maintainers react pretty fast to new developments, and those maintainers are likely to also quickly pick up any revisions/fixes to yEnc.

The incompatibility problem isn't with clients. The problem is with the NNTP(newsgroup) servers. Some ancient servers will choke if there are eight-bit characters in the message.

I still feel that moving to an eight-bit encoding system is fine, but let's be clear about what the issues are, okay? :)

Re:yEnc = XMODEM part deux

[Anonymous Coward]

xmodem existed before kermit.

Re:yEnc = XMODEM part deux

[0x0d0a]

Hmm...IIRC, wasn't kermit designed to handle situations like over IP where you could have wacky latency?

Re:yEnc = XMODEM part deux

[Anonymous Coward]

I remember using 'modem7' and wondering what that new fangled xmodem was about.

Of course, at the time I was using 'modem7' as a terminal program on CP/M, not as a transfer protocol.

The old confusing days of not having a fucking clue but getting by.

Re:yEnc = XMODEM part deux

[Stuart Park]

Xmodem may have been something that filled a need, but Kermit was by far the more widespread protocol (and to a certain extent still is) - it handled errors well, and has been written for just about EVERY operating system that ever existed.

Considering that Usenet is also used on many different systems, then we need a "kermit" not an "xmodem".

Re:yEnc = XMODEM part deux

[Fatal]

Yes, but the problem is that Jurgen started at yENC. That doesn't leave much room for improvement! There's only zENC left.

If he started at aENC, he'd have a whole alphabet of versions to improve his software.

Re:yEnc = XMODEM part deux

[tomstdenis]

The problem is only a minority of usenet users want to send a majority of usenet traffic.

Nothing against new programs and all but seriously. If you think new encoding schemes on usenet is a good idea than you should have your head cleaned out with a nice CZ75 .40

Tom

Re:yEnc = XMODEM part deux

[slashdot_commentator]

Yes there's the problem of legacy software, but a protocol that's only been around for a few weeks or months can't have that much of a legacy.

But what if it takes a year before an "improved" standard comes about? Everyone starts jumping around, forcing the new standard down people's throats. Then these poor non-commercial coders have to drop what their doing and implement the new protocol (including interpretation, and debugging time). (The commercial programmers have to go through it too, but I have no sympathy for people who are being remunerated for it.) If they don't react, they risk losing their userbase because they are perceived as being slow with "critical" improvements.

Nixon's complaint has to do with the fact that its not a thoroughly thought out implementation of a standard. Its a kludge. Nixon feels is that it is so apparently flawed, it will need to be reimplemented with corrections. This will cause a lot of angst for developers and users to deal with flawed software, and frequently upgrading it. And for what? The benefits may not even be that spectacular.

Frequent changes to a program is no big deal. (What didn't work right, now does.) Frequent changes to a protocol is a disaster. The programs' problem is not limited to correctly implement the new standard; it needs to interpret and then "properly" deal with the previous version(s). After all, how is program supposed to figure out if its a yEnc v0.9a and a yEnc v.093x? yEnc doesn't embed that kind of information in its format. And will the Forte reader be able to properly decode yEnc encodings by Pan? That can only be certain when there is an unambiguous protocol definition. ...and then everyone needs to do the upgrade tango! This is the mess that Nixon (and others) would like to avoid.

The only programs that currently support yEnc are the ones whose maintainers react pretty fast to new developments, and those maintainers are likely to also quickly pick up any revisions/fixes to yEnc.

Interesting, but all this change is actually being driven by the posters, not the newsreader developers. I've seen this firsthand in the alt.binaries.multimedia.* groups I frequent. They replace the uuencoding phase with the yEnc, treating it like old-style uuencoded posts, and then send the nntp articles. If people want their movies or tv episodes (before they expire in 2 days), their only choice is a yEnc capable newsreader. I'm sure the newsreader maintainers would have been more than happy to wait for a better standard.

So the solution Nixon should be calling for is not a years-long bureaucratic standardization process that will get yEnc 1.3 entrenched while the standardization is happening. The solution is to fix yEnc's problems and release a new version as fast as possible, before the old version gets spread around too widely.

I agree; waiting for the MIME committee to releases a standard before implementing yEnc would be like waiting for the Federal Gov't to issue TCO standards before creating the first personal computer. The solution was for the media posters to pass on the current yEnc implementation. If the flaws were obvious enough to require reworking yEnc, then the public should wait for the better implementation. I can understand why Nixon would have reservations about implementing a MIME-like standard without the MIME committee's approval. But then yEnc should not try to implement a MIME compatible kludge, and Nixon should ditch the complaints of yEnc not being a MIME compatible standard.

Re:yEnc = XMODEM part deux

[phr2]

But what if it takes a year before an "improved" standard comes about?

That's the point--right now, the only yEnc implementations are in programs maintained by people who jumped on things quickly. So by fixing yEnc right now instead of waiting a year, you avoid trapping users of the slower moving programs. The slower moving programs simply aren't using yEnc yet.

I agree yEnc should have been more carefully thought out before being released. But its problems are not obscure. They are obvious to people who have dealt with these issues before. I don't think years of study and a lot of iterations are needed to figure out how to fix yEnc's problems. It should be possible to fix them quickly and get the fixed version out there before too many people are using the old version.

Re:yEnc = XMODEM part deux

[slashdot_commentator]

NO, you miss the point! yEnc flaws cannot be corrected and implemented in a week!!! It cannot be fixed right now! And 30 different individuals trying to implement what they think is wrong with yEnc gives you 30 incompatible news readers!

I am not advocating waiting a year for a better standard. I am not in favor of waiting for the MIME committee to define a new standard. I am against "forcing" users to use yEnc as is, right now. (And this is being "forced" by the media newsgroup posters. The yEnc author cannot force yEnc's adoption on USENET. But he is actively encouraging it, even though it will need to be trashed in a couple of months.) It will result in a lot of grief when the yEnc author puts together a revised protocol in a couple of months. You do not grasp the problems that will be created by adopting the flawed yEnc now.

Re:yEnc = XMODEM part deux

[kesuki]

You're forgetting after z come aa ab ac ad... all the way to zz.

concretely: this is why you don't need it

[mmusn]

Here is a typical binary file (a Mach executable, if you must know), sizes in bytes. Feel free to repeat with your own favorite binary data.

42244 file
17136 file.gz
23636 file.gz.uu
17894 file.gz.uu.gz

file.gz is what a raw binary encoding transfers, file.gz.uu.gz is what a communications protocol with a good compressor does when it sees a uuencoded file. Assuming everything works out, you save (drumroll) 4%. yEnc probably has more overhead than that.

The new modem standards apparently even have special compression modes for HTML, and if uuencode and base64 matter, modems and other compressors could recognize them as well. This used to be limited by the available computing power of, say, a PDP-11, but in the days of GHz signal processors and handhelds, textual encodings are OK. Really.

Re:concretely: this is why you don't need it

[willy_me]

I'm glad I'm not the only one who saw this. Seriously, the arguement that yEnc encoded files will consume less space on servers seems very flawed. Any good server should see a uuEncoded message and store it in compressed form on the server.

When you know it's a uuEncoded file, it should be very easy to write a speedy compression algorithm. Even just use a standard Huffman algorithm - it was originally designed to be implemented by hardware for storage devices (think tape drives) and wouldn't impose any real overhead on a modern server. I just can't see why we have to implement a new standard before it's ready.

Willy

Re:concretely: this is why you don't need it

[Reality Master 101]

The new modem standards apparently even have special compression modes for HTML, and if uuencode and base64 matter, modems and other compressors could recognize them as well.

That's great ... if you use a dial-up modem. Personally, I think any more incentive we can give to stamp out dial-up modems, the better.

Re:concretely: this is why you don't need it

[jridley]

See my other post in this thread. It's not all about dl speed or even drive space. Most serious usenet large binary downloaders use a premium news service, and they ALL have download/upload quotas and/or pay by the byte schemes. yEnc gets you 30% more for your buck on those servers.

Re:concretely: this is why you don't need it

[mmusn]

It's no different over other TCP connections: people for whom this is a concern will transfer USENET postings in compressed format.

Besides, your attitude seems pretty selfish. Many people don't have a choice but to use analog modems, and that's not going to change for a long time. Unless, of course, you have a few trillion dollars to donate to the cause of stamping out analog.

Re:concretely: this is why you don't need it

[mmusn]

Oh, come on, think about it. If pricing isn't completely arbitrary, it reflects costs. In that case, if you compress the content more, the prices are going to be raised.

Re:concretely: this is why you don't need it

[Anonymous Coward]

How retarded is that?

No service changes their bandwidth charges based on
whether you are downloading compressed data vs.
uncompressed data. That's like saying you would have
to pay the same to download a .bmp file and the same
file encoded as a .gif. Same data, different size,
different price?

Ya, that happens.

Re:concretely: this is why you don't need it

[Anonymous Coward]

That's great ... if you use a dial-up modem. Personally, I think any more incentive we can give to stamp out dial-up modems, the better.

Goddamn. I was half agreeing with some of what you were saying. Now it's clear that you're just fucking trolling to rile up anybody whose belly doesn't rest on the front of his chair as he sits on a DSL wire.

I may have to actually get a Slashdot account (again!) to plonk you. (Rob's 'neo-plonk', actually)

Re:concretely: this is why you don't need it

[mmusn]

You didn't think it through quite right. Your point is: you get charged by volume of news. My point is: the communications costs are the same whether you compress or not since the protocols already compress. If you reduce the volume you get charged for by compressing it yourself, but the company still incurs the same costs, you do indeed save--until the company adjusts their costs to account for this. In short, if you make the data you transmit harder to compress, prices go up. I'm sure if you think it through, you'll eventually get it...

Re:concretely: this is why you don't need it

[Anne Thwacks]

If there is any reason why you dont need it, its becauwe People using 7-bit for pass-though traffic should be excommunicated.

7-bit died BEFORE THE PC- That's over 20 years ago. I seriously doubt there are any PDP11s still on the internet, and AFAIK nothing else felt the need to pack three 6-bit chaaracters into a 3 character file extension. Apart from that, it was a 16 bit machine and did 8-bit chars. Almost all other 7-bit hacks were dead before the PDP-11 was even launched. (except POCSAG pagers - now there is a seriously SAD protocol!)

The best answer is dotn compress for transmission - let the modem/imodem/NIC/why do it IF THERE IS A SUITABLE STANDARD, and if not, let the appropriate standards committee fix it, cos they have the tools for negotiation a compatible standard at run time.

Re:concretely: this is why you don't need it

[Reality Master 101]

Now it's clear that you're just fucking trolling to rile up anybody whose belly doesn't rest on the front of his chair as he sits on a DSL wire.

Sheesh, dude, it was a joke -- relax. I realize I didn't put a smiley on it, but...

Re:concretely: this is why you don't need it

[rhaig]

Any good server should see a uuEncoded message and store it in compressed form on the server.

wow...

you've never run a news server have you?

I just can't see why we have to implement a new standard before it's ready.
Do you work in Redmond?

Re:Screw luddites

[npietraniec]

I post HTML to Usenet. Intentionally. And I will continue to do so.

That's your right... Don't be suprised if a certain percentage of users don't read your post because you unnecessarily post with red text in an ugly font.

Re:Screw luddites

[Anonymous Coward]

You're a fucking moron. You know that, right?

Save /. Money...

[smoondog]

Hey, if this guy adopts it, he could save slashdot some money...

-Sean

Re:Screw luddites

[Anonymous Coward]

As an example, I post HTML to Usenet. Intentionally. And I will continue to do so. The green-screen luddites need to stop whining and upgrade their newsreaders. Yes, ASCII newsreaders can strip out (or render to some degree) the HTML. And no, I don't care about the extra bandwidth. There's a reason that newspapers and magazines use italics, boldface and bullet lists.

Unfortunately, not all of us are from the USA with unlimited flat-fee usenet access. Some of us still have to pay for each byte we download. I hope you will be more considerate in the future.

He can't be actually using binaries groups

[Talla]

What he's whining about is that it didn't fix every other problem in addition to overhead, and if anyone should actually bother making some huge new mime standard, now they won't have that carrot.

Obviously, if the rest of the problems were as big as he's trying to claim, yEnc would only be a minor setback for a new and more comprehensive standard, but the fact is that the 35-40% overhead of current standards is by far what's most annoying to usenet users. After we got PAR (parchive.sourceforge.net), reposts have been reduced drastically (except for pr0n and partly warez groups, where the dumb people with shitty servers rules).

Also, he's trying to say that because the increase in volume will outgrow the savings, there really is no savings. What kind of logic is that? Let's stop making processors faster, we'll just find bigger problems for them to be too slow for anyway, so what's the point?

After the introduction of PAR and yEnc, as a long time binaries downloader, I'll say the actual content of multimedia groups has more than doubled, and probably tripled, the last 6-9 months. That's progress to me.

Re:He can't be actually using binaries groups

[Anonymous Coward]

From the user point of view:

Ones that pay for news server - yEnc doesn't matter as a.b groups are always 100%. It might take a little less time to download.

One that doesn't pay for news server - YEnc doesn't help as he/she would see 1-2% complete binary files.

Standards ARE important... here's why.

[gambit3]

In one sentence, standards ARE important because they allow for the most people to get the most benefit.

I work in an industry that relies heavily on standards, and my job deals specifically with standards. Making sure that WE follow standards, and making sure that other vendors follow standards.

Sure, they're slow to develop. But they're the best for interoperability, and that's crucial. In my line of work (for a major Mobile Phone System NSS provider), I have to deal with other providers that have to follow the same standars we do. That allows both of our products to communicate. This gives the end consumer (i.e., Cingular, Sprint, etc.,) the option to buy from different vendors. This forces us to make better products. This forces us to be more efficient. This forces our competitors to do the same thing. In the end, everybody wins.

The other alternative is what I see as the Micro$oft approach: Standards be dammed, I'm going to do it this way, and f*ck everybody else. It's the same approach that gives you security holes in your browser, because, well, who needs the standards?

I can't believe I'm reading comments like "well, it's here and it works so what's the problem?"
The problem is the future.
The problem is the inability to send an SMS from a CDMA service like Sprint to a GSM one like Voicestream. That's what happens when you blow off standards.
The problem is the inability to read an M$ Word doc that was sent to a Linux user.
Ignoring standards and going off on your own (especially, going off BADLY on your own) just divides us.
Good standards help us all. They give us better products. The lower costs.
CD-Rs. FireWire. PCI. countless others.

Besides, as the article begins by asking: Just what problem were they trying to solve?

Re:Standards ARE important... here's why.

[Anonymous Coward]

"Besides, as the article begins by asking: Just what problem were they trying to solve?"

Faster, better and cheaper posting downloading of files.

"Standards be dammed, I'm going to do it this way, and f*ck everybody else."

That's what the standards guys said to the yEnc guy so he released his specs and source and said have at it. Now instead of waiting for a solution, we have one. Can it be improved? Sure but now the bar is raised for those who want to do it right

Re:Standards ARE important... here's why.

[Anonymous Coward]

I can send SMS from a TDMA service (AT&T) to GSM (Voicestream) and CDMA (Verizon).

I'm not sure if this relates to your argument or not.

I'm just saying... I can do it (and often do).

Re:Standards ARE important... here's why.

[Lars T.]

Faster, better and cheaper posting downloading of files.

Of pr0n and warez, stuff that could just as well be propagated through the web and FTP.

Old standards die hard

[oodl]

Speaking of legacy "standards", how about ditching gzip and zip compression in favor of bzip2? bzip2 compresses files significantly better.

http://sources.redhat.com/bzip2/

Re:Old standards die hard

[Anonymous Coward]

Better yet http://www.rarsoft.com/

You can thank me later after un-raring all those cool games and pr0n!

Re:Old standards die hard

[goldid]

Ah, but bzip2 doesn't always compress things better. That's the whole thing with encoding for compression: some work better than others. (Think why some images are GIF, some are JPEG. There is cause for use of gzip still, I've had it work better than bzip2 in the last week.

Re:Old standards die hard

[oodl]

bzip doesn't have to compress better all of time. It compresses significantly better *most* of the time, and nearly as well for the few times it doesn't compress as well as zip/gzip.

BTW, GIF is lossless, JPEG is lossy. That is comparing apples and oranges.

Re:Old standards die hard

[goldid]

OK, so even if we assume that my GIF to JPEG analogy was bad, there are still reasons to keep multiple standards.

Is there a good reason why we *shouldn't* have them both. (to save disk space on the binaries? :) )

For instance, you might use a Huffman-based encoding for files from a word processor, but an Arithmetic-based encoding scheme for images. (Both loss-less). I just think that it's silly to insist everyone use bzip2 all the time when it's not always the best for the job.

Choice is good.

:)

Re:Old standards die hard

[0x0d0a]

bzip is noticeably slower than gzip. I copy files using nc piped through gz frequently on my LAN. Using bzip would make things much slower, since the CPU is the bottleneck.

Re:Old standards die hard

[Anonymous Coward]

gzip and bzip do not compress multiple files a la zip. Thus the need for tar before [gb]ziping.

In Agreement, but powerless to do better.

[Anonymous Coward]

Excellent comments, I agree totally, but I'm already doing the best that I can. Sadly, this is the pinnacle of my life. My youth fades away, I'm powerless to do better.

--A typical slashdot reader/poster.

P.S. Hurry up download those skipjack ISO's. Redhat lost over $100 million dollars last year
The open source experiment has proven that it's not possible to make money by giving away a
substandard operating system. Likely skipjack will be the last release before they shut their
doors.

Humor - "standards"

[Seth Finkelstein]

I see nobody has said it yet:

Standards are crucial. And the best thing about standards is: there are so many to choose from!

Sig: What Happened To The Censorware Project (censorware.org)

Re:Humor - "standards"

[Anonymous Coward]

Look, I've enjoyed your sig as much as anyone else, but let me state that there's nothing so profound or humorous that it must be stated in any discussion about a subject. In short: thanks, heard it.

Re:Humor - "standards"

[Anonymous Coward]

You could at least credit Andy Tanenbaum if not quote him correctly.

Re:Humor - "standards"

[Seth Finkelstein]

You could at least credit Andy Tanenbaum if not quote him correctly.

Thanks for the correction. I got the version I used from the "fortune" files, and it didn't have an author attributed.

Sig: What Happened To The Censorware Project (censorware.org)

Re:Humor - "standards"

[Anonymous Coward]

Hey, aren't you the freak stalking our beloved Michael?

yENC isn't even important

[Anonymous Coward]

The big savings on binaries is coming from .PAR files.

If you don't know what they are, then you haven't been on usenet for a while.

But essentially, it allows you to stripe you sets with parity so that you can lose up to "n" posts and the PAR programs can rebuild the missing pieces.

I believe this has helped the backbone tremendousl.

Re:yENC isn't even important

[Anonymous Coward]

And if you yEnc encode and post parity files when reposts are required then the savings are even bigger.

Re:yENC isn't even important

[Sadfsdaf]

Incidentally, many people in alt.binaries.news-server-comparison (one of the places Jeremy hangs out) are very opposed to PAR files and say they cause an increase in bandwidth. I don't know if Jeremy's one of them.

An excellent idea would be to make a new format where yEnc and PARs are combined. Imagine this: get rid of RARs, PARs and SFVs altogether. Post a file as one big file and have 1% parity. Tbe parity will be for individual posts (not enire RAR archives like before). 1% is more than enough since a 500 message post will have 5 parity messages. Most incomplete files only have about 3 or 4 missed messages and the 5 will fix those INDIVIDUAL POSTS. If that isn't enough, a few more parities are generated.

I find this idea MUCH more important than silly nitpicks on implementation. Of course, this can eliminate yEnc from the face of the earth if this was implemented, simply because it makes downloading binaries much much easier and simpler.

I've emailed Jurgen about this, but his reason was that it'd make adding in this way too complicated for news client maintainers.

Those who want to push a MIME format should implement this and create a new standard, everyone will jump from the yEnc bandwagon onto this like they jumped from the UUE bandwagon to yEnc.

Re:Screw luddites

[Reality Master 101]

I hope you will be more considerate in the future.

Not a chance. It's not my fault you have a bad Internet connection, and I don't think it's reasonable to hold back progress for some proportion of people.

If this needs a solution, then the solution is to implement a "strip" protocol at the Usenet server level. But the solution is NOT to penalize everyone else who doesn't want to live in a world of monospace fonts.

Pan

[Alien Being]

And Pan's got it too. Tastes great, less filling!

I wonder if it's in CPAN yet...
Module Convert::yEnc (P/PN/PNE/Convert-yEnc-0.03.tar.gz)

Yep. Works for me!

Re:Screw luddites

[Anonymous Coward]

Why not use Word document files instead?

Counterpoints to all of Jeremy Nixon's main points

[Harumuka]

Uuencoding relies on searching for "magic strings" in the message body of a Usenet post. This is unreliable, error-prone, and has already led to problems with certain client software. It is absolutely the wrong way to go about tagging message content, because what you really want is something reliably machine-readable and precisely specified. However, yEnc also relies upon magic strings in the body.

There is no reason to despise magic strings. They work, and cannot ever occur in the user data. All yEnc magic strings start with =y, = being the escape character. Ctrl-Y does not need to be encoded, so yEnc is free to use =y for it's own purposes (e.g. =ybegin, =yend). Jeremy Nixon continues his misled rant...

With a uuencoded multi-part post, client software typically uses the Subject line of the post to attempt to determine the filename, and to tell where the segment falls in the sequence. This is obviously a terrible way to do it.

No, using the subject line is not obviously a terrible way to determine filenames, segments, and anything else. I find it very convienent to know exactly what my yEnc files will be saved as, how big they are, and how many parts they are in inside the subject line. Nixon says "Sure, it works out most of the time, but it is imprecise and error prone (especially when spaces are used in filenames)" This is blatently false nonsense. Quotes reliabily allow clients to discern the filename. It's not "imprecise and error prone" by any stretch of imagination.

When non-ascii characters are used in message headers, software currently just has to guess what they mean. Jürgen's filename specification cannot even be used to reliably reproduce his own name.

I give them that. Non-USASCII data in headers is a pain, and a large powerful organizational bodies needs to agree on a character encoding standard. Oh wait, they already did - Unicode!

but gives no method to specify a filename which happens to contain quotes, which is not uncommon

False again. I've never had a filename containing quotes on my Windows box. If we expect newsgroups standards to reach everyone, we must use the lowest common denominator. Similar to how ISO9660 used 8.3 filenames, but on a higher level.

And the bandwidth savings? That's an illusion. A smaller encoding scheme gives us exactly one benefit: faster downloads and uploads for the users

Which is exactly what the creators of yEnc intended.

Meanwhile, the transition creates confusion for the users

They mean "AOL users" of course. Usenet hasn't had a new encoding format in 6 years, it's about time. Adopting this format should be as easy as switching from Napster to OpenNap to Morpheus to Grokster to Blubster and so on.

When Jürgen found that going through an actual standardization process within MIME would take time, he chose to ignore MIME in favor of getting something out there right away.

I don't blame him. Jurgen is a coder, not a politician. I would have done the same thing.

In short, yes I agree yEnc needs to be more polished. But the point is it works right now, and it's working great. It filled a gap in Usenet, itched a stratch to borrow an ESRism. Once yEnc is standardized as Y.32049 Annex D or whatever those standard organizations call it, we will use it. Until then, yEnc forever!

Re:Screw luddites

[Wolfier]

While I quite agree to your "screw the luddites" idea, and I think yEnc is a progress, I'm not sure if HTML postings in usenet can be classified as a "progress", especially when this trivial idea does not need any imagination to accomplish. "Yay, we have HTML, let's slap it onto everything!!"

Not only isn't HTML postings an important "technical advancement", not everyone thinks HTML postings look better, either - surprise.

I, for one, deliberately killfile HTML postings and filter HTML emails because I don't need the funny colours that distract contents, and the security concerns associated with scripts, images, "runnable" postings and cookies.

Plus, every single piece of HTML posting and email I've seen are spam anyway. Why bother?

If you want to post HTML contents, there is a place it should be done and it is called "the web". I don't complain HTML postings in web pages because it is what the HTTP was designed for - and there are already many decent methods built in or around an application called "browser" to protect yourself from security hazards and spams. I cannot say so for newsreaders, at least yet.

On a related note...

[dimator]

I haven't been able to connect to my damn ISP's news server for over a week now. They just ignore my complaints. I can't ping news.gte.net from anywhere that I've tried.

I'm dying here, a man can only live off the same porno for so long...

Re:On a related note...

[Account 10]

You didn't get the mail from verizon? You need to use news.verizon.net now

Dear Valued Verizon News Customer,

Verizon Online is improving the quality and performance of our
Newsgroup service. As a result, we will be conducting an important
upgrade to our servers on February 19, 2002 between 4:00 a.m. and
5:45 a.m. EST. You will be unable to access your Newsgroups during
this time.

Before February 19:
If you wish to save any previously downloaded articles, please do so
before the upgrade on February 19. If you don't save them, they will
be permanently deleted from your Newsgroup file.

On or after February 19:
To ensure uninterrupted access to News after the upgrade, please
follow these steps:

Step 1: Check that your News server name is "news.verizon.net".
Step 2. Re-subscribe to your Newsgroups to ensure you receive all new
articles. Although you may download the entire archive of articles,
we
recommend you set your Newsreader to download only the last four
days
to minimize download time.

If you require assistance or more detailed instructions, please visit
http://help.verizon.net/webcgi.exe?new,kb=d sl,dtre e=3946 or call
1-877-222-2375 and select the Technical Support option. We apologize
for any inconvenience but are confident you will be pleased with the
improved performance of our Newsgroup service.

Sincerely,
Verizon Online

*Please note: This e-mail was sent from a notification-only address
that cannot accept incoming e-mail. Please do not reply to this
message.

Re:On a related note...

[dimator]

Thanks for the tip, but I've tried that too (in fact, they resolve to the same IP). The last line of the traceroute is:

16 so-1-3-0-0.SVCS-RTR1.RES.verizon-gni.net (141.156.255.46) 97.166 ms !A 97.214 ms !A 97.336 ms !A



Where the !A means access to the host is prohibited. (I'm going to have one hell of a time trying to convey this to a tech support moron.)

Re:On a related note...

[Hyped01]

You are probably being blocked from it... I could access it, though since I am not in the correct IP pool, the 200 (OK) message states I need a login (authinfo stuff)

200 This connection requires username/password authentication (Twister v2.0.5.1)

You post something they dont like or that someone complained about?

- Robert

Re:On a related note...

[Hyped01]

• Where the !A means access to the host is prohibited. (I'm going to have one hell of a time trying to convey this to a tech support moron.)

That means you are being blocked on a firewall level someplace... either on the machine itself or before it.

Re:On a related note...

[dimator]

You post something they dont like or that someone complained about?

Nope, just downloaded pr0n... not that much either, and not everyday. I get similar traceroutes from a couple shells I have as well, so I don't think they're blocking me specifically.

Re:On a related note...

[Hyped01]

They probably havent set up their filters or IP allow list properly yet then. We had to be added to our ISP's list (not really theirs) as they license out someone else's news server... if they've changed over to another server, highly likely they need to update their files to allow all the gte customers (or the subsets they have forgotten) back on. Possibly, they had been blocked in the past from access to the verizon servers due to issues, but now that verizon owns it, needs to be fixed... best guess I have... and of course with verizon, any guess is probably the best you'll get, no matter how off the wall since it is almost always more than you will get out of them... :-(

Best of luck to you...

Robert

www.WebBinaries.com

Re:Screw luddites

[squiggleslash]

As an example, I post HTML to Usenet. Intentionally. And I will continue to do so.

Please continue to do so. Killfiling by Content-type: text/html is relatively easy to do, and it's certainly easy for the rest of us to avoid downloading form-over-content crap posted by clueless bandwidth hogs.

Death to the luddites!

Whatever.

Personally I don't think someone who wants to be able to ssh into their shell account from work/etc and read USENET, rather than rely on (easily monitored) Google, or who doesn't want to run up a large phonebill downloading approximately 5-6x as much data as necessary, or who finds that there's no way of GUIfying the newsreader they've found works best for them, is a "luddite." I'd say they're using their common sense. And I'd say someone who describes them as a "luddite" and "living in the past" is someone whose respect for others is demonstrably lower than ought to be necessary for posting on Usenet. If there was a Usenet "licence", I'd argue that the name callers and bandwidth hogs ought to have their's revoked.

But YMMV.

Being on the Internet is about accepting standards. Some of those standards may not be perfect for you, but it's certainly not impossible to stick by them, and you can make your point easily enough and still follow them. Most people do. Get over it.

Not all programs require yEnc in the subject line.

[bryanp]

He mentions this -

It relies upon "yEnc" being in the Subject to determine that the message contains a yEnc binary

Not necessarily. My personal favorite Usenet binaries file grabber, Binary Boy not only handles yEnc nicely but doesn't rely on the subject header to determine whether it's really a yEnc encode or not. Instead it scans the file itself. At least it does as of the current build. (yeah, it's a Windows program, it's closed source and you actually have to pay him money - get over it)

Re:Screw luddites

[Reality Master 101]

Not only isn't HTML postings an important "technical advancement", not everyone thinks HTML postings look better, either - surprise.

So should the web return to the days of pure ASCII? Should Slashdot disallow any HTML postings? HTML is useful. Look at how it used on Slashdot -- italics, boldface, links, etc. Not to mention that proportional fonts are infinitely easier to read.

I don't complain HTML postings in web pages because it is what the HTTP was designed for...

How can you possible be in favor of HTML on the web, yet not in favor of HTML in Usenet? Put it this way: If Usenet were invented today and they included HTML, could you honestly say it would occur to you to say, "you know what would make this better? ASCII only!"

I cannot say so for newsreaders, at least yet.

Depends on your newsreader. If you're using a Windows newsreader, it typically uses the IE COM component to display the HTML.

bad article

[hidden]

This guy has some points, maybe. I'm not sure.

I don't know that much about yenc (beyond using it)

however I DO know that yenc binaries are recognised by the
=ybegin at the beginning of the binary. Not, as this guy complains, by the subject line.
he says:
"yEnc continues to use the Subject line for this. It relies upon "yEnc" being in the Subject to determine that the message contains a yEnc binary ..."

if the article can't even get that basic a fact right, I wonder how valid the rest of his complaints are?

Re:bad article

[Ruddygore]

Read the yEnc spec before you say idiotic things like that. How do you suppose the client is meant to know there's a yEnc binary in the post BEFORE it downloads it?

The filename and other information is also in the Subject line. According to the spec. Which you obviously have not read.

Jeremy

Re:Screw luddites

[Reality Master 101]

So if straight ASCII is so great, why do you use HTML in your Slashdot posts?

Grumble mutter bah humbug

Your sig is particularly appropriate for your post.

There's a lot of irony

[dwsauder]

There may be problems with yEnc. But there are so many more problems with uuencode. For crying out loud, uudecode is hopelessly broken! There is no "uuencode" standard. There are many variants of uuencode. One would have to wonder how uuencode could become the de facto standard for Usenet binary newsgroups. To put it another way, MIME is infinitely better than uuencode, which was a pure hack from the beginning. MIME has been around for 10 years, but it has not been adopted by Usenet.

I can't believe that Nixon is advocating that we stick with uuencode.

Here are the advantages of yEnc:

• Contains a CRC, so you can detect corruption.
• It is a more efficient encoding.
• Clearly indicates the parts of a file

Even if you don't think the first advantage is a big deal, the second and third ones are.

Nixon's out of his mind if he thinks we should stick with uuencode. He's also out of his mind if he thinks we will ever adopt something better that is based on MIME. MIME has been around for 10 years, and still has not been widely adopted in Usenet, despite the fact that it's widely deployed for email. I don't think it ever will catch on in Usenet. You can sing MIME's praises all day long, but you can't force users to adopt it. That's why I think if there is momentum in favor of something that is significantly better than uuencode, let's go for it.

This whole situation reminds me of the difference between TCP/IP and the OSI standards. On the one hand (OSI) you had bureaucratic committees designing extremely advanced (read: complex) protocols that are designed first, then implemented second. On the other hand, you had a chaotic situation (TCP/IP at the IETF) where the guys who implemented a protocol started talking to get it standardized. My point is, that maybe MIME is just to complex for Usenet, which likes things simple, although maybe a bit chaotic.

BTW, considering that uuencode is almost never used in email, and that MIME is almost never used in Usenet (specifically, the binary newsgroups), I think there is no chance yEnc will cross over into email.

Re:There's a lot of irony

[J'raxis]

How is MIME complex? Heres a simplistic MIME message I just wrote out by hand in about 5 minutes:

From: jraxis ("J'raxis 270145")
To: dwsauder
Subject: Foo
Content-Type: multipart/mixed; boundary="foo12345"

Here's your foo.jpg.

--foo12345
Content-Type: image/jpeg
Content-Disposition: attachment; filename="foo.jpg"
Content-Transfer-Encoding: base64

[ base64 data ...]

--foo12345--

You want checksumming? MIME provides a Content-MD5 (which is more reliable than CRC). yEnc could have just been registered as a new form of transfer-encoding (along with base64, quoted-printable, etc.) and happily merged into MIME.

Re:There's a lot of irony

[Anonymous Coward]

The way it's taken off, it sounds like he wouldn't even had had to register it with mime, he could have just gone ahead and used mime and eventually it would end up in every meaningful mime spec anyway.

Just seems like a way of using mime that's more consistant with what just happened :)

Technical arguments against yENC, blah

[Tofuhead]

It should be pointed out that this site, linked from yENC's own website, goes into more technical detail regarding the technical flaws of yENC. The fact that it's linked from yENC's own site is proof that the author is at least familiar with the concerns that people have with his implementation.

I personally still find it difficult to argue against the article author's point that THERE WAS NO RUSH to force yENC out the door in such an unpolished form. After so many years of waiting for something better, why ignore the recommendations of those you are trying to help?

< tofuhead >

Re:Technical arguments against yENC, blah

[Anonymous Coward]

THERE WAS NO RUSH

Bull-fucking-shit. How long have standards-bleating wankers delayed while binaries are massively bloated?

Give me a fucking break... the fact that yEnc has taken off so fast is a perfect demonstration that IT WAS NEEDED BADLY. There are times when I despise Microsoft for it's attitude to standards, and there are time when I can well understand their frustation with standards bodies.

How many times do we have to listen to some guy whining about 8-bit data and holding everything up because he doesn't want to update his 15 year old newsserver software (that doesn't get binary groups anyway). yEnc ain't perfect, but it's a big kick in the balls and a jab in the ass to those fucking losers who would delay forever if you let them.

Re:Screw luddites

[squiggleslash]

Erm...

1. Slashdot is only accessable from web browsers.
2. Slashdot is an HTML forum.
3. HTML is fine. Posting it to Usenet isn't. See comments about cost of downloading, newsreaders, standards, etc.

Stop setting up straw men.

A few words from the original author

[Ruddygore]

Well then. When I put that page up, I honestly didn't expect many people to read it outside news.software.nntp and a few curious folks in alt.binaries.news-server-comparison. I certainly wan't expecting to get Slashdotted. Well, that's fine, except that the uproar might have waited a little bit.

In my essay, I state that what Usenet needs is "a better way to post Binaries". The next piece of the puzzle, of course, is to answer the question, "What IS a better way to post binaries?" I was thinking about finishing that page up tonight, but I am writing code at the moment instead.

So, when reading my comments, just keep in mind that, yes, I DO have some answers to that question, too. It's just that it's a bit of a more time-consuming question, so that page isn't done yet.

This time around, though, I will make sure to include a prominent warning to NOT run off and implement the ideas as quickly as possible, and to please not use all of Usenet as beta-testers. The idea that whatever gets done fastest is best just doesn't work for me. There were good reasons I didn't go and get people to implement my smaller encoding ideas when I first wrote the code. If only the yEnc implementor had continued where I left off rather than going down his rather misguided path...

All the comments are welcome. I've been getting some interesting email, too, of course. Many programmers of Usenet client software absolutely despise the thing and are quite annoyed at the amount of their time it is wasting. I guess it's just more of that never-ending divide between the users and the techies. So it goes.

yEnc is here, that's for sure. Now we just have to try to deal with it.

Jeremy

Re:A few words from the original author

[Wesley Felter]

Why use Usenet for binaries at all? Why not some kind of more efficient P2P system?

Re:A few words from the original author

[Bishop]

Because Usenet works, and an "efficient P2P system" does not exist.

Re:A few words from the original author

[0x0d0a]

Mmm...I disagree.

USENET is wildly inefficient for binary distribution unless someone downloads every binary file from every server. I admit that most P2P systems don't allow offloading network load for files that you are publishing -- with Gnutella, you have to wait until someone else decides to share the file you're sharing if you want any load balancing.

But Freenet exists, though it's a bit rough at the edges and there is no non-Java implementation (blech). And it is much better for this sort of thing and much more efficient than USENET.

Re:A few words from the original author

[Anonymous Coward]

You mean: 'because Usenet allows a reasonable measure of anoymnity and P2P systems make it hard for someone to not get hammered for the porn and wares we all want.'

It's the old kiddie-porn thing again.

The preee-verts have taken over the discussion forum, and they'll be the ones who excuse it being destroyed by those 'in charge.' It's sad, really. Like somebody showing up at 'Town Hall' meetings handing out printed pornograhy, and the political powers-that-be using it as an excuse to ban all Town Hall meetings.

It hasn't happened yet, but it's coming.

Re:A few words from the original author

[Apache]

IMHO, Usenet is good for binaries because the person doing the transmission only has to upload a file once for lots of people to download it (like a web server) but it is hard for an ISP to squelch it because of the automatic propagation to other ISPs.

And the idea that it's only efficient if someone on every server that gets the message downloads it is mostly false. For it to be more efficient it only has to be downloaded by more users that servers the message was propagated to (assuming that some of those users are not using the same news server as the poster).

Re:A few words from the original author

[Wesley Felter]

IMHO, Usenet is good for binaries because the person doing the transmission only has to upload a file once for lots of people to download it (like a web server) but it is hard for an ISP to squelch it because of the automatic propagation to other ISPs.

I agree that these are good properties, but they aren't unique to Usenet. Both Freenet and MNet only require a file to be uploaded once and they both automatically propagate popular files. But they don't automatically propagate unpopular stuff like Usenet does.

Re:A few words from the original author

[sparrowjk]

Your essay is the best summary I've seen so far of the reasons not to use yEnc. You have done a service to those of us who have been annoyed with yEnc -- now we don't have to explain it to anyone, we can just point them to your essay.

So, be it resolved that yEnc leaves much to be desired.

However, if yEnc is the impetus which actually gets the community moving toward implementing a good, solid standard, then it will have served its purpose. Perhaps if we had had yEnc 5 years ago, we would have a standard already. But we didn't, and now we must pay the piper.

Since people aren't going to give up the advantages of yEnc without a substitute, the priority going forward is clear: to develop a better standard. If it truly is better (and not simply another hack) then ensuring its wide adoption shouldn't be too much of a problem. If, however, people can't be persuaded to switch, so much the worse for Usenet -- but no point in dwelling on doomsday scenarios. As you say, the cat is out of the bag, and all we can do is damage control.

Re:A few words from the original author

[Crass Spektakel]

Usenet is not Internet.

Usenet works without IP.

Usenet can be used with a C64, a 1981 IBM-PC, a Amiga500 and everything else.

Usenet needs its own way of distributing data.

Seems silly to be complaining....

[mollusk]

about a completely open encoding method. yEnc was released into the public domain. Jeremy Nixon has the spec and the freedom to fix it. If he feels that yEnc is flawed in some way, there is nothing to prevent him from changing the spec to address its perceived shortcomings. Isn't that the one of the supposed benefits of open source software? Asking the developer to change his project when he has already provided you with the means to do it yourslef seems rather selfish. If Mr. Nixon feels so strongly about the problems with yEnc, he should create his own version. Only then should he engage into a debate about with approach is better. It's easier to ask people to choose between 2 working implementations, than it is between something that works and nothing.

Re:Seems silly to be complaining....

[khuber]

This is wrong thinking. NNTP is an internet protocol and yEnc should play by the rules of internet protocols. It does not and that's why it is bad. You can't break building codes to build your house and justify it because you saved a few bucks. "Hey...it seems to not collapse" "it hasn't caught on fire yet!"

In particular, there is no yenc RFC and yenc does not use MIME which is the agreed upon standard for encoding binary attachments. Yes, uuencode is a gross grandfathered format, but it is still 7 bit clean.

Releasing problematic improperly specified encodings that break internet protocols is not being a good citizen. "it works" is a poor justification. it does not work, and breaks compliant software.

-Kevin

alt.slack and sputum

[Anonymous Coward]

S.P.U.T.U.M. came to mind when I read your post about "physical means".

Don't bother downloading now

[DiveX]

Either it is /.ed to hell or the server is having major problems (not that they are exclusive). A lot of people are having the download stop at 10-15%. All this for a simple 2 meg file. If someone already has it, throw it to a binary group and provide a link.

Re:Don't bother downloading now

[Account 10]

It is posted to alt.binaries.warez.ibm-pc

Subject: As Requested a UUencode version of Agent 1.91 with yEnc [1/2] - a32-191.exe
Message-ID: <q7Pm8.81615$rr6.1172226@news.webusenet.com>

Re:Don't bother downloading now

[tomstdenis]

Why would you put it in a .warez group? Are you agreeing that the only use for this is to distribute warez like the commy loving swine you are?

Re:Don't bother downloading now

[Mik3]

If someone already has it, throw it to a binary group and provide a link.

Should they use yEnc on it first?

Re:Don't bother downloading now

[spudnic]

Because the next message posted was the crack for it.

.

Everyone has an angle...

[lanalyst]

From this page, Mr. Nixon admits his bias: I am currently employed by one of the below providers, so I do not give recommendations as I would be biased. Fair enough but it seems the whole idea behind yEnc is to maximize bandwidth - which would cut into his employer's profits - more encoded content at the same price. Spread that over thousands of users, the number becomes significant.

So Jürgen Helbing creates a public domain solution which is gaining popularity and Mr. Nixon is being ignored - and why? Implementation nitpicks? Please.

A more useful excercise may be to guess his employer from his list of 'reviews'

Re:Everyone has an angle...

[rudy_wayne]

"the whole idea behind yEnc is to maximize bandwidth - which would cut into his employer's profits - more encoded content at the same price. Spread that over thousands of users, the number becomes significant."

This is a bogus argument. As he says in his article:

"And the bandwidth savings? That's an illusion.... Do you really think people aren't going to post more, if they can do it faster? Of course they are. They're always going to post more, with or without yEnc. And, with yEnc, they are even more likely to post more, because posting the same amount of material will take a shorter time ..."

And he's exactly right. yEnc will have no effect on the revenue of news-servers that charge by the gigabyte. Although he only mentions uploading, the same thing applies to downloading as well. People will still download their 10GB a month, they'll just get an extra porn video or two for the same money.

Re:Everyone has an angle...

[lanalyst]

If the overhead of uuencoding is indeed 33% vs yEnc's 2%, this gives metered customers another what 3GB a month on a 10GB account? Not insignificant. Personally, I use a service that's metered per day - with no bandwidth cap and unlimited uploads (teranews) so my savings is even more signicant.

Your quote states that people will of course post more (and download more). This equates to greater bandwidth requirements, larger/faster servers and more disk for the NNTP provider. And of course, this means more value to the customer.

Mr. Nixon's interests are indeed those of his employer. If he has/had a solution, why all the grousing? I think it's more a matter his employer won't allow him. This isn't bogus: Mr. Nixon should put up or shut up.

Re:Everyone has an angle...

[hkmwbz]

Basically, what you are saying here is that this affects the news server provider both positively and negatively. If users use less space and bandwidth, the provider can cut down on hardware/bandwidth costs. If they use more, it will be more expensive for the company.

I fail to see how this would affect Nixon's view in any way based on where he is employed. It would only do that if it was only negative for his employer. However, that is not the case.

Consider the source

[JohnA]

I'm not sure if it is still true, but I know that Jeremy Nixon (the author of the article) worked at Supernews (now ReMarq) as one of their chief engineers. Not to be jaded, but it stands to reason that he would be against a technology that will decrease the data transferred by customers who pay by the gigabyte.

Re:Consider the source

[Anonymous Coward]

Uh, it's considerably more likely that he dislikes it because it makes it considerably more difficult for him to maintain Cleanfeed, which by many news admins to filter out spam and "stealth binaries" (binaries posted to non-binary groups). Binary detection is made tricker when people start throwing them around in some new half-baked encoding format.

Re:Consider the source

[Burdell]

Actually, Supernews is now part of Critical Path (the RemarQ name went away). And yes, Jeremy Nixon does work for Supernews. He also wrote Cleanfeed, the anti-spam program that Usenet admins everywhere to make news bearable to read.

The "he's against it because it saves bandwidth" argument makes no sense. If it saves users a little bandwidth, it saves Supernews many many times that much bandwidth, lowering their costs (which means they don't have to charge users as much to provide the same service). It also saves disk space, meaning Supernews doesn't have to buy new disks quite as soon. And a good bit of Supernews' business is in the corporate (outsourced ISP) service, which they don't charge by the gigabyte (they have speed caps, not monthly download quotas).

The problem is that any savings are just an illusion; this is just a momentary blip in the growth of Usenet. Since yEnc doesn't have the 100% market penetration that uuencode and MIME have, people are more likely to post binaries in multiple formats, causing storage and bandwidth needs to increase, not decrease.

Re:Consider the source

[Ruddygore]

If you think this is going to decrease the amount of data transferred by customers, you need to take off those rose-colored glasses. I suspect it won't affect that at all.

Smaller encoding helps us (the providers), it doesn't hurt us. (Besides, one look at the Supernews price list versus the competition will reveal that metered individual accounts are hardly the main focus.) With the customers the actual money comes from, less downloads would increase the margins, not decrease them. In fact, to a lesser extent, it would do so with the individual accounts as well -- the pricing on individual Usenet accounts (at almost all of the top providers) is such that the margins are lower at higher download levels. At the highest levels, it's nearly break-even. So less downloading would even be somewhat better there.

If you really think that I'm against a smaller binary encoding scheme, then you completely missed the point of my essay -- and you also must have missed the part about how it was my first experimental code, implementing smaller encoding, from which yEnc was hatched. If I truly wanted to avoid smaller encoding, why would I have implemented it in the first place? Why would I have done it in public and then sent the code to several people, explicitely releasing it into the public domain? You think I would have done that to prevent the spread of smaller encoding?

Had Jürgen picked up where I left off and did the thing right, I would be singing an entirely different tune.

Jeremy

Re:Consider the source

[bighousehx]

It *WIILL* decrease the amount of data downloaded on a per message basis. Of course, this means the customer will simply download more messages... good for the customer, bad for the service provider.

Jeremy, where is YOUR answer to yEnc? I admit I'm not up on the history of your involvement in Usenet encoding, save for the info on your web page, but I'm curious as to why you didn't follow through and persue the new format.

Re:Consider the source

[Ruddygore]

More, smaller messages -- what difference do you think that would possibly make?

My answer to yEnc is brewing, and frankly, I'm not arrogant enough to push anything which is "my" answer to yEnc. There are a lot of people out there who know what they are talking about and it would be stupid not to listen to them. So, don't look for "my" answer to yEnc... look for "the" answer to yEnc, developed not by one hacker but by a group of people who know what they're doing.

Jeremy

Re:Consider the source

[bighousehx]

>More, smaller messages -- what difference do you think that would possibly make?

Correct me if I am wrong, but I believe you are speaking in regards to the NNTP server. I was speaking in regards the end user.

An end user will download FEWER posts of a given size, if those posts are yEnc encoded, as compared to posts that are uuencoded for the same binary.

If you are referring to the fact that users will simply post more files, well, that will happen in any event, regardless of encoding scheme. If you or any "group of people who know what they're doing" can devise an encoding scheme that will reduce both post size and number of messages, for a given binary, I shall switch to it immediately. Provided, of course, that it is implemented in my lifetime...

As I said in my previous post, yEnc certainly has lit a fire under the binary posting issue.

Re:Consider the source

[0x0d0a]

How is this bad for the service provider?

They pay for upstream bandwidth, which is now less. They pay for downstream bandwidth, which is equal if the customer is ineed transferring the same number of files.

Heck, if the service is more attractive, they *get more users* and make more money.

I may not agree completely with Jeremy, but I have to say that this attack on him is completely bogus.

Re:Consider the source

[hkmwbz]

Had Jürgen picked up where I left off and did the thing right, I would be singing an entirely different tune.

I read your text with interest, and you have many valid and important points. However, I think you are being too harsh towards Helbing. At least this is my impression. It sounds too much like Helbing did this on purpose to try to spread yEnc "by force". I don't think it should be necessary to mention Helbing at all, as it is the yEnc issue which is important. According to Helbing himself, the release of yEnc was too early, and a mistake. Indeed, Helbing might not be the one to blame at all. I don't want to start the "pointing" game and find someone to blame, but Helbing can hardly be criticized for others releasing yEnc implementations too early.

My impression is that Helbing is aware of these issues, and that he really wants to do something about it. Perhaps it is still time to do this "the proper way" and create a well thought-out standard? Currently, updates to yEnc will cause problems for developers, but there is nothing saying that yEnc will continue to change forever.

What we are seeing right now is basically a massive public beta test, and various newsreader are adding support for the current version of yEnc. Perhaps this beta test will end one day, and we will have a mature version of yEnc which will have addressed all or most of the issues you point out?

Communicate with Helbing and continue to point out flaws in yEnc. I am sure he is more than willing to work on addressing these issues.

You can't stop yEnc now, but we can all help it mature into something which doesn't cause all these problems.

Don't you agree?

Re:Consider the source

[Talla]

The problem is that any savings are just an illusion

As another person stated, I guess you don't need that 30% salary raise, then, because you'll just spend all the money you earn anyway.

people are more likely to post binaries in multiple formats

You mean you *think* they are. I've followed all the big binaries groups to check for just this, and on average there are way less than 20% reposts in uue/base64 from original yEnc posts, which means a net gain.

Re:Screw luddites

[Reality Master 101]

Stop setting up straw men.

No, it's not a straw man. Look past your "it's always been done that way" attitude, and think about it. Why is a Slashdot HTML forum "good" and a Usenet HTML forum "bad"? History is not a reason. Standards are meant to be enhanced.

Sheesh, some geeks are worse than PHBs on this issue. It reminds me of that shipping company commercial where the boss drones on about the fact that "we've always shipped this way. We will always ship this way" while the young employee stands there mocking him.

Why do I have a feeling that all the old farts are going to have to die off before this issue goes away?

Encoding efficiency isn't a BIG deal

[Sloppy]

If, by any chance, you're transferring things over a modem (v42bis' lzw) or ssh vpn (zlib's deflate) or possibly other types of links, then you're probably not going to notice a difference anyway. The systematic encoding inefficiency that goes with base64 and uuencoding, results in a substantial lack of entropy that will be picked up on and exploited by good compression algorithms. Then end result won't be quite as good as having efficient encoding to begin with, of course, but it will be in the same ballpark. There's no way it'll be anywhere near a 33% difference.

This sounds like something that would have been useful 15 years ago before compression was widely used, and when people were still writing newsreaders. Now it looks like a waste of time and an excuse to get people to "upgrade" their software.

Re:Encoding efficiency isn't a BIG deal

[Anonymous Coward]

If, by any chance, you're transferring things over a modem (v42bis' lzw) or ssh vpn (zlib's deflate) or possibly other types of links

That excludes the average alt.binaries customer who would be a Windows user on broadband. (Unless I'm my ass and DSL/Cable uses v42bis.) Also, I can't remember anyone arguing zip and gz because some people have compression on the transport level.

Re:Encoding efficiency isn't a BIG deal

[marx]

This sounds like something that would have been useful 15 years ago before compression was widely used, and when people were still writing newsreaders.

Maybe you should try to actually use Usenet before speaking about it. There's no compression between a Usenet server and client.

Re:Encoding efficiency isn't a BIG deal

[Sloppy]

I wrote:

If, by any chance, you're transferring things over a modem (v42bis' lzw) or ssh vpn (zlib's deflate) or possibly other types of links, then you're probably not going to notice a difference anyway.

You wrote:

Maybe you should try to actually use Usenet before speaking about it. There's no compression between a Usenet server and client.

Maybe you should try reading things before replying to them. Granted, my ssh-based vpn example is probably pretty uncommon for nntp, but modems sure aren't.

Re:Encoding efficiency isn't a BIG deal

[marx]

Granted, my ssh-based vpn example is probably pretty uncommon for nntp, but modems sure aren't.

The problem is that you've never actually tried what you're proposing.

I don't know if broadband modems support compression, mine certainly doesn't, or the compression sucks tremendously.

I made a simple test. I created a file with random data and uuencoded it. I then transfered these two files through my ADSL modem and measured the transfer times. The uuencoded file is 38% bigger, and surprise, surprise, takes 38% longer to transfer.

Maybe it would be a good idea to actually try using the things you talk about in the future.

Re:Screw luddites

[Anonymous Coward]

I have no problem with text-only, it's the manual linebreaks that piss me off.

I have a 21" monitor, but because some fuckwits still read news on a DEC VT52 connected to a PDP-11 that doesn't have the CPU to autowrap, we're stuck with hard-to-read, mandatory-for-everyone 76-character lines. Ain't that a communist solution for ya!

So I support you in your quest to post HTML to Usenet. If only because my reader groks it, and it wraps properly.

Re:Screw luddites

[Anonymous Coward]

Dude, you're a fucking moron. I bet you post HTML 'cause you don't know how to turn it off in outlook expre$$

Problems, Solutions, and Change

[_Sprocket_]

This post has me thinking a bit. At first, I'm all with it. Yea! Progress. Let's go.

In the case of yEnc, someone found a problem and (Freely, as in public domain even) offered a solution if people wish to adopt it. People are adopting it. Progress is happening.

And then I hit:

As an example, I post HTML to Usenet. Intentionally . And I will continue to do so. The green-screen luddites need to stop whining and upgrade their newsreaders. Yes, ASCII newsreaders can strip out (or render to some degree) the HTML. And no, I don't care about the extra bandwidth. There's a reason that newspapers and magazines use italics, boldface and bullet lists.

And I can't help but think "wow - what a jerk." Why? Not because I don't have newsreaders and email clients capable of handing HTML based messages. Not because I dislike HTML itself (open standard - yay). But because HTML seems to be completely unneccisary in these environments.

Now... sure. There are some forums that might bennifit from it. Times where typefaces and bulletlists, etc. really add value to information. Maybe even times where some forms of information NEED this technology or it becomes very difficult to portray. But I rarely see it.

Instead, HTML based messages in Email and Usenet tend to increase the overhead with no real added value to the content. In some cases, they are used to attempt various shennanigans such as web-bugs (not to mention worms, etc). Little wonder tech-heads dislike it.

Change must happen. But when you run around trying to force change for change's sake rather than to solve real problems, then you simply become a problem yourself.

Re:Screw luddites

[Anonymous Coward]

You're probably not wrong in this regard; they will all have to die or become such a small percentage of the net population that they can be ignored as ancient bass-ackward pretentious folk, like prudes and such are now.

Thank goodness the person you're responding to isn't responsible for any meaningful developments in the IT industry.

Re:Is usenet dead? Usenet = MP3 galore

[Anonymous Coward]

I have a 130GB mp3 collection thanks to Usenet.

Transfer speed isn't the only reason for it.

[jridley]

Because people with premium news services (AKA, ANYONE that's serious about large binary downloads, the people at which yEnc is aimed):
- have megabyte quotas, both upload and download
- pay by the megabyte for their downloads

Also, this saves space on the server hard drive. NO WAY are usenet servers compressing data on their hard drives. It's one of the most challenging situations for a hard drive, they're not going to wreck their performance by using compression. Having less data means more retention on the server.

I personally have a Newsguy extra account, grandfathered at 1GB per day. I EXHAUST MY QUOTA by about 10AM, most days. I still do, but by then I've gotten 30% more data.

It's not all about transfer speed. yEnc means people with download quotas get 30% more stuff per day.

Re:Transfer speed isn't the only reason for it.

[GigsVT]

What do you do with a whole GB of porn each day? Who goes through it and deletes the crap and dupes?

Re:Transfer speed isn't the only reason for it.

[mmusn]

That's a bogus argument. Dowload quotas are set taking into account that the data is compressible. If you make the data incompressible, then your download quotas will end up being set lower, or your prices will be raised.

As for compression and all that, if you believe that the constant compression/decompression during transmission and storage is too costly, then the right thing to do is to come up with an extension for NNTP that lets you transfer and store data in compressed binary form. Coming up with a new user-visible change that only a fraction of postings adopt and that breaks a lot of software is just plain stupid.

I, however, think that there simply isn't a problem. If you can't be bothered to pay for the bandwidth you want, why should the rest of the world put up with another uselss protocol?

Re:Transfer speed isn't the only reason for it.

[jridley]

No, they're NOT counting the data compressed. At least, none of the servers I've subscribed to.

Well, it's a moot point anyway because pretty much everyone's already using it.

I can tell you that my upload and download times are way down for posting now that I'm using yEnc.

Re:Transfer speed isn't the only reason for it.

[jridley]

Oops, sorry, I misread your comment.

This doesn't make any sense. The cost to the provider is primarily the bandwidth to get the data to me. In the agreements we have at work with our IP peers, I don't see anything that says "we're billing you for compressed data amounts, not actual data amounts." I assume the news providers have the same deal.

I don't really know, so I'll shut up about it until I can find out.

BTW the "add to the NNTP spec" argument is not terribly practical. First, most people don't have the knowledge to define a spec that would be acceptable to server operators (I'm sure there are a lot of things to be considered). And even if it was defined, you'd have to wait for the server software companies to implement it, and then wait for the news providers to roll it in. I bet it couldn't be done in less than 2 years.

By defining the yEnc spec as they did, it's totally in user-space and up to the user and user's programs to implement.

Re:Screw luddites

[squiggleslash]

Why is a Slashdot HTML forum "good" and a Usenet HTML forum "bad"?

I've already given you the reasons why posting HTML in Usenet is *BAD*. Posting HTML in Slashdot is OK because Slashdot is designed to be read by webbrowsers. Oh wait, I've already told you that too! Additionally, just as going out of your way to deliberately format stuff as HTML in Usenet is likely to make things harder for normal users, going out of your way to make things appear as "plain text" (which Slashdot, unlike most graphical newsreaders, forces to be shown as fixed font text), will make it more difficult for users to read your comment.

Standards are meant to be enhanced.

Standards are not meant to be "enhanced" unilaterally, otherwise they're not standards.

Why do I have a feeling that all the old farts are going to have to die off before this issue goes away

So we're back on the insults again. Come on, I've already explained to you exactly why posting HTML is a bad thing in Usenet. You so far have set up a straw man hypocracy argument about posting HTML in an HTML forum, and now you're back to name calling and claiming anyone who disagrees with you lives in the past. What you haven't done is address the actual reasons why people want HTML out of Usenet.

And until you do, and perhaps even come up with practical solutions to these problems, I don't plan to take you seriously. I doubt anyone else does either.

YOU are the luddite

[mmusn]

But I get annoyed with people who live in the past

Why don't you get annoyed with yourself, then? Binary encodings are a holdover from the 1960's. For most content, we don't need them anymore because our infrastructure now handles all that automatically.

As an example, I post HTML to Usenet. Intentionally. And I will continue to do so.

You are confusing novelty with innovation, and generally seem to understand little of what has come before. The fact that you are obnoxious about it only makes you more annoying.

Re:Screw luddites

[Glytch]

I was going to post a long technical piece on why what you're doing is wrong, but, essentially, it all boils down to the fact you're just a prick.

Re:Screw luddites

[mikeage]

Let's have some moderation (not the select a category, but some moderate views) here. I agree that <i> and <b> can enhance a posting. What the other poster was commenting on, and which I agree (since I often connect via modem, often 28.8, sometimes even 14.4)is that excessive HTML is useless. It doesn't add to the content, it just gives it a little extra pizzaz, which is not always good.

Re:Screw luddites

[Anonymous Coward]

How do you reconcile putting in this assertion:

So should the web return to the days of pure ASCII? Should Slashdot disallow any HTML postings? HTML is useful . Look at how it used on Slashdot -- italics, boldface, links, etc. Not to mention that proportional fonts are infinitely easier to read

when the comment you're replying to contains (and you even read it, as you quoted it):

I don't complain HTML postings in web pages because it is what the HTTP was designed for..

One of the surest signs that someone is losing an argument is when they pretend their opponent is arguing something that in actual fact they're arguing the opposite...

(Posted as AC by Squiggleslash (who's also responding to you in a seperate subthread) because this is intended to be a heads up that you're coming across as a kook rather than on-topic discussion)

"The specification seems to rely..."

[Anonymous Coward]

"The specification seems to rely on the principle of "it'll probably just work most of the time."

Does that sound familiar??? Slashcode, for example??

making a standard (or) hack for a hack?

[fred911]

Complaing over what should be a standard for transfering binary data over NNTP? Doesn't anyone get it? I mean come on. Binary transportation over usenet is a hack in itself (not that it's bad) but the network just wasn't designed and isn't intended for binary distribution. There *are* better models for distributing binarys.

That said, If joe blow wants to encode his binary in X method the recipient should know how to parse the data. It isn't all about point and click. The standard will be what everyone uses.

Re:Screw luddites

[dvdeug]

Sheesh, some geeks are worse than PHBs on this issue.

You propose a method that many people can't read,
and while it's supposed to look better if you can read it, it frequently just looks like over-elaborate crap.

Geeks believe in content over packaging, at least more than your average person. HTML offers very little in an email or newsgroup environment, where messages are quickly written, but often come with huge overdone packaging. Geeks also like to use their computers via text links, and HTML is not friendly to that. Somehow it should not be surprising that they aren't happy with HTML in those enviroments.

Re:Screw luddites

[pyramid termite]

So should the web return to the days of pure ASCII?

And when was that? You do know that HTML is an intregal part of the WWW, don't you?

How can you possible be in favor of HTML on the web, yet not in favor of HTML in Usenet?

Because the newsreader I use doesn't know what to do with it and the way a lot of the people who use HTML on Usenet do it, it doubles the length of the post.

Put it this way: If Usenet were invented today and they included HTML, could you honestly say it would occur to you to say, "you know what would make this better? ASCII only!"

For a medium that outside of the binaries newsgroups was designed to post text messages? Why wouldn't ASCII only be better for plain text? I actually think a lot of web pages have gone overboard and have worsened the bandwidth/content ratio considerably. There's nothing wrong with lean and mean.

If you're using a Windows newsreader, it typically uses the IE COM component to display the HTML.

While allowing other possibly malicious components to wreak havoc with your system - a problem that never happens with plain text messages, does it?

Go ahead. Post HTML on Usenet. But you should know if I read it, it's going to look like a hard to read text message with a lot of bracketed garbage and I'll be more likely to skip over your posts in the future.

Re:Screw luddites

[Anonymous Coward]

There's nothing to stop you from posting how-ever many characters across even in text/plain. There is a MIME standard to make it easier to do without being obnoxious to other users, I forget the type (usually recognisable when it's posted that way because people with non-MIME-compliant readers see equals signs on the end of every continued line)

However, I must admit that I fail to see the value in having 16" long lines of (presumably, for if the font size was bigger, you wouldn't notice) 10pt characters. Imagine sweeping back that distance with your eye every time you finish reading a line of text, you'd never find the start of the next line.

So: Long story short: No need for HTML in this case, and you might want to be careful what you wish for.

What newsgroups do you read, anyway??

[Russ Nelson]

I've heard this "is usenet dead" crap one too many times. Comp.lang.python isn't dead. Soc.religion.quaker isn't dead. comp.protocols.tcp-ip.ibmpc isn't dead (okay, so it doesn't have a lot of traffic, but the principals read it, which is what matters).
-russ

yEnc: Anyone remember GIF vs. JPG?

[bighousehx]

The whole yEnc brouhaha reminds me of the GIF vs JPG Usenet debates of the late80s/early 90s. Each side claimed their format was "obviously" better and produced "data" that supported their arguments.

Time will tell, of course, whether yEnc emerges as the predominant standard or not; just as it did with GIF and JPG.

FWLIW, I support yEnc. No, its not a formal standard and its not perfect. It does, however, force the issue of encoding for Usenet. If left to the powers that be, Usenet encoding would never change as there is no incentive to do so.

If you don't like yEnc: create a better format, formalize it, issue an RFC, wait, make changes, discuss the issue some more, wait, hope that end users accept your format, wait. etc., ad nausium.

In the meantime, I (and many, many others) will happily make use of yEnc. Whether you like it or not, you have to deal with it, as a user or as an admin.

...just like those ancient GIF proponents had to deal with JPG.

Standards for Usenet encoding???

[roystgnr]

Can *anyone* look at the uuencoded, mime encoded, and other similarly mangled into 6bit, 70 character-per-line standards, and honestly tell me that Usenet was designed with binary file transmission in mind?

There are no Usenet binary transmission standards, just a few different hacks to make it work. If this guy's new hack makes it work better, good for him.

Look ma, no mouse!

[dattaway]

Oh no! slashdot is beautifully rendered in a textual browser such as lynx and lightning quick over a 9600 baud cellular modem. If you never tried viewing this site under lynx, you may be surprised at the artistic detail of the formating.

Over to the left where the links take the shape of an ascii candle flame followed by more links presented in an intuitable format. Rob was generous leaving this site accessable to the more mature historical browsers.

What a miserable bugger

[darien]

I lost sympathy about here:

A smaller encoding scheme gives us exactly one benefit: faster downloads and uploads for the users. It is not going to make Usenet smaller. It is not going to allow servers to increase retention. Do you really think people aren't going to post more, if they can do it faster? Of course they are. They're always going to post more, with or without yEnc [...] big deal.

So effectively, what he's saying is, in effect: "this system changes nothing, and is of no benefit, except that it makes more data available on the Usenet and gives users faster uploads and downloads. So it's worthless."

This guy obviously hasn't had to use a metered dial-up account for a while. A 33% saving on transfer times is an enormous benefit. I feel quite insulted by the way he seems to think it's of no importance, as if my time and money aren't worth anything. "What's the rush" indeed! I'd happily tear up MIME and MD5 tomorrow if it would speed up my transfers by a third.

If yEnc is so widespread, it can only be because there's a demand for it. And if there's a demand for it, why the hell shouldn't programmers support it? Last time I checked, RFC's weren't enforced by law. The Net has seen a million non-standard hacks, and has, for the most part, assimilated the good ones and outlived the bad. yEnc is by no means the worst, and it brings real benefits to tens of thousands of people every day. I say leave it alone - or if you have to oppose it, at least oppose it constructively, for Christ's sake!

Re:What a miserable bugger

[Ruddygore]

You just gotta love the speculation that runs rampant around here.

FYI, I am using a metered dialup account (actually ISDN, which isn't much faster, just more reliable). I cannot get unlimited, high-speed access (yet, I keep hoping). I pay well over $100 per month for Internet access at a fraction of the speed of peoples' cable modems and DSL lines, and it would be quite a lot more if I left it nailed up 24x7.

You, like several others here, seem to have somehow gotten the impression that I am opposed to smaller encoding. I can only conclude that someone is spoofing my web host's DNS and some of you are reading a different page than the one I wrote.

Jeremy

Re:What a miserable bugger

[Anonymous Coward]

Sounds like you're in for an expensive month, what with the slashdotting of your article and site, and whatnot.

Re:yEnc = XMODEM part deux [off topic]

[cymen]

Speaking of Xmodem - remember Ymodem, Zmodem, and all those? What was the one that allowed you to do bidirectional transfers? I can't remember what it's called and I'm bugging out. I remember using it on OS/2 and it had a $15 license fee (shareware).

Ah! Google has this site:
http://www.simtel.net/pub/msdos/commprog/

Unfortunately I don't think it was HydraCom.

That was ages ago. I remember it rocking because it made it much easier to keep up with the upload/download ratio.

Re:Screw luddites

[Reality Master 101]

How do you reconcile putting in this assertion: [etc]

Because it's absurd to argue about features in an application (i.e., Slashdot, Usenet) based on what protocol it uses. Either a feature is useful, or it is not. You can't have it both ways -- either HTML is good for discussion groups (Slashdot OR Usenet) or it is not.

Screw Elegance

[Deflatamouse!]

If everyone is as anal as the poster of this story and cares about elegance and "pretty code" and whatnot, we all would not be using a x86 processor because its architecture is just "sooo damn ugly!!". Get over it. If it works, it works, does it really matter how anything is implemented if it does its job just perfectly fine?

Personally, I will adopt anything that will make my life (not the developer's life) easier. yenc allows me to download 30-40% less, so I use it. Why? because if I don't use it, I won't be able to obtain the mp3s, the games, the warez, the pr0n, etc, etc, etc that I wanted :) Real life is not pretty, deal with it... stop living in fantasy world.

Re:Why yEnv is good for the software companies

[foonf]

Most progressive newsreaders group multi-part binary posts together pretty well. That really isn't a problem at all. And the reason for the multi-partness doesn't have to do with a particular encoding scheme, but with the line number limit many servers impose on posts.

Re:Screw luddites

[Anonymous Coward]

Usenet is not an application, it's a protocol. Additionally, the WWW is something that requires more than plain text - for example, to include links.

Usenet doesn't.

Either way, you pretended that the person you were arguing with opposed HTML. He didn't. He thought it had its place, but that place wasn't Usenet. Phenomincally easy to understand, you pretended - you lied, if you like - that he was arguing that HTML was an evil by itself.

So my point still stands. Why did you put that argument in?

Good idea, but...

[Firecaster]

... the implementation sucked.

I leech regularily from alt.binaries.anime and the related newsgroups. When the yEnc posts started coming in, I simply upgraded my newsreader to the newest version. But a LOT of people out there use Agent, and it was absolute pain to combine/decode all the yEnc posts that started popping up all over the place. The worst of it is that the yEnc posters were basically saying, "Start living in the present and upgrade". Nevermind that at the time that only yEnc-capable newsreaders were for Windows...

I mean, I don't know, but this sounds a lot like the OS wars that have been going on for quite some time. Some people simply don't want to have to switch newsreaders. Some people don't want to have to switch OSes. And that's fine, because it's a free world out there. I like the idea of yEnc (I get more out of my Easynews account), but I really don't think it should have been introduced so quickly.

~ Firecaster ~

Two Problems:

[NeuroManson]

One: yENC, when it was unveiled, did not really allow most conventional newsreaders any opportunity to adapt, til after the fact. This is akin to perhaps releasing zip files long before any archival software was actually available to open them... So do most of the folks using usenet for binaries get the opportunity to at least *choose* the way they do their downloads? Nope, they also are forced to adapt, or lose out...

Two: Loss in transmission... I've been downloading yENC attachments for the last month, and out of them, found over 50% loss/corruption in posting... Not due to retention/propagation either... Just files missing large chunks... Now this *could* be due to some problems on the senders' end, but it seems just a little *too* coincidental that almost all of the losses have occured with yENC uploads...

Re:Two Problems:

[weave]

Yeah, I've seen the same thing. Xnews started supporting it almost right away and hence I started to use it (or basically, not care about it since Xnews handled it transparently).

I too have noticed a large number of corrupted files in downloads from giganews, something I never saw before yEnc. (giganews has large retention and excellent completion rates. It wasn't lack of completion that caused it...)

At this rate, I'm paying more (by the gig) because I am throwing out so many corrupted downloads.. :-(

Re:Two Problems:

[sircrown]

I think there's something wrong with Xnews' yenc decoder. I tested with both Xnews and Agent using an external decoder and the Xnews files came up bad whereas the latter method was fine.

Re:Two Problems:

[atholbrose]

Um, when zip files were introduced, there was one program to deal with them: Phil Katz' pkunzip for DOS. How is this any different?

Re:Two Problems:

[NeuroManson]

The difference is pretty simple when you consider the following: The average number of computer users numbered in the low millions, if not the high hundreds of thousands... AND the zip format was almost immediately known (practically open source when you think about it)... Those who sent out files with .zip extensions would let people know where they could find the software to decompress said files...

Making matters worse, you have millions of people looking at these files asking "WTF?!? How do I open these?!? (especially with Outlook Express or other generic off the shelf news clients)

Look at the yENC format, and you'll immediately note the difference... Nobody posting it tells you WHERE to find the nessesary software to open the attachments, at best you will find a description of why yENC is being used, nothing more...

Re:Screw luddites

[dvdeug]

either HTML is good for discussion groups (Slashdot OR Usenet) or it is not.

One huge difference between Slashdot and Usenet is that Slashdot only permits a subset of HTML tags in posts. If Usenet HTML would limit itself to a similar subset, then it would be much more platable.

Also, Slashdot has already limited itself to those who can handle HTML, so why not permit HTML in posts? Usenet is and always has been available by plain text, so HTML gets much more scrutiny.

pan does yenc

[accessdeniednsp]

PAN has yenc support in the latest dev code, however it's pretty rough. 0.11.3 is gonna have it fixed tho i think. if you're looking for a great gtk news reader, this is it! 100% gnksa approved too. been using it for a while now.

Works for me.

[Typingsux]

Free pictures. pRon. Software. Pyr8ted. Simply. yenc makes it easier and faster to get. Good for me. The author seems bitter since he's a developer that made a similar software and didn't get it implemented on a wide scale. It's out there, and this webpage created out of jealousy will not do anything to stop it. Who made it the standard of RARing the files when they are large? Or HJsplit? (RAR appearing to be the preferred method.) Definitely newer: Who made PAR files? (Love them BTW.) Much less begging for fills, if ever. Moving forward. That's how I see it. So take your whiny commentary down, Mr Jeremy Nixon.

Reminds me of Napster data format

[topham]

This just reminds me of the napster data format. Anybody ever read the reverse engineered specs? It's scary. It looks like it was designed by a monkey. And not a smart one.

yEnc sounds like a good idea, and a horribly bad implementation.

Re:Reminds me of Napster data format

[MmmmJoel]

If it was designed by a monkey, then we'd all be in good hands.

Eight bit characters and old servers

[phr2]

What happens to those servers now, on the normal text newsgroups, where lots of messages appear with accented characters?

And how many "ancient" servers are really handling the (huge-traffic) binary newsgroups where yEnc encoding is appearing? I would have thought old servers were made for much lower news volume and couldn't handle the load.

I'm not a news guru though. I just read the stuff sometimes.

Re:yEnc = XMODEM part deux [off topic]

[Anonymous Coward]

Maybe you are thinking of szmodem? Or was that just good because of the built in tetris game? :)

Re:Is usenet dead?you are joking right?

[Anonymous Coward]

my poor pc dumbfounded friends are still broken hearted over napster (or as they refer to it, "napSTAR")

now they can't figure out why they can't make morpheus work

they are getting their asses kicked by Outlook viruses, fucking web popup's galore...

me?

1. There is hardly a Redhat question I can think of that has not been already asked and answered. Thanks http://groups.google.com/ ...I'm well on my way to becoming windows independent.

2. Ditto that on Cisco...I'm just about to take my CCNA exam.

3. I can try-before-I-buy just about any software package ever released. Thanks newsgroups.

4. I can try-before-I-buy just about any music...all sorted by category. Celtic, 70's rock, classical or techno anyone? Thanks newgroups.

5. Great starting place to get an opinion and further links on any topic, from Scientologists to Dodge trucks, to import beer.

6. Last but not least, all time best deals when i need new gear, i just check out the austin.forsale news groups...and there's always tons of goodies forsale ...CHEAP. (sometimes even dfw.forsale ...when i visit)

lock down the web...whatever...just don't take my newsgroups away!

one of the best "secrets" ever...and some day i'm sure will get squashed, if it ever gets too easy to use or too popular.

Re: Also, GrabIT

[Self-Important]

offers completely transparent decoding of usenet binaries.

Re:Screw luddites

[Anonymous Coward]

"There's nothing to stop you from posting how-ever many characters across even in text/plain."

You must have missed all of the flaming retroguard assholes who populate usenet. Besides, it's not my posts I'm worrying about -- It's everyone else's.

"However, I must admit that I fail to see the value in having 16" long lines of"

Talk about strawman city. FYI, right now in my browser, your post is soft-wrapped to about 100 proporational characters wide, and it's perfectly readable. The point is that should be under the reader's control. (And if you know of a reader that will rewrap text, please let me know.)

I think it's a bad idea...

[seebs]

Because binary newsgroups themselves are a fundamentally stupid and bad idea, which do nothing but damage the discussion medium. Usenet isn't designed for binary distribution, and it's not good at it. A real solution would involve getting away from usenet entirely.

Re:I think it's a bad idea...

[Meowing]

Well not entirely getting away, but otherwise I Agree With This Post[tm].

All of Mr. Nixon's points are easily refuted

[Self-Important]

I'm a pretty active Usenet poster and binary downloader, and let me assure any of you who are riding the fence on this that yEnc *is* much, much faster. It is light years ahead of uuencoded binaries, and the coolest thing since .PAR to hit Usenet.

Whatever kind of "transition period" Mr. Nixon is referring to has already passed. Let me just cry a freaking river for the poor programmers who have to update their code to accommodate the changing spec--it seems that the most popular usenet progs have *already* done this. If "Joe's Special Shareware Usenet Posting and Spyware Program" needs to be updated so that its 7 users can stay happy, then so be it.

Whatever kind of "confusion" this guy is referring to is nonexistent. yEnc downloading and decoding with the right software is as transparent as uudecoding. Promise. People don't have to relearn anything.

I read Mr. Nixon's "I invented better binary posting, but didn't rush to claim credit" anecdote and can't even force myself to care. Sounds like sour grapes to me.

-----
Nothing to see here, go back to downloading pr0n now.

Re:All of Mr. Nixon's points are easily refuted

[Anonymous Coward]

"Whatever kind of "confusion" this guy is referring to is nonexistent. yEnc downloading and decoding with the right software is as transparent as uudecoding. Promise. People don't have to relearn anything."

Well people do now have to think of yENC's longer lines. Where before my ISP newserver let me post 5000 lines of uue material now I only get 2280 with yENC. Longer lines under yENC. That's why Agent 1.91 has such a small line length as the default. Yes the payload is the same but you have to think a little different or just rediscover your servers limits.

Re:All of Mr. Nixon's points are easily refuted

[Ruddygore]

As a "pretty active Usenet poster and binary downloader," I bet you know your stuff pretty well. You could switch to anything in a short time, I bet. That's great, so could I. You're not looking at the average user, though.

Since I put my page up, my mailbox is overflowing with messages from people who are agreeing with me, but for the wrong reason -- they are saying, yeah, right on, this yEnc thing sucks, I can't figure the thing out and I can't get half the binaries anymore, how do we get everyone to go back to the old way?

If you just look at the people who know what they're doing and know it well, of course you won't see much confusion. But a popular binaries group easily has several thousand times as many people downloading as it has posting, so I think you are grossly underestimating the level of confusion this is causing among the regular users.

I'm not worried about confusing them; it's inevitable. What I'm worried about is confusing them too many times in a row.

Jeremy

Re:Screw luddites

[Reality Master 101]

He didn't. He thought it had its place, but that place wasn't Usenet. Phenomincally easy to understand, you pretended - you lied, if you like - that he was arguing that HTML was an evil by itself.

*sigh* Of course, it has be a problem with me ("lied"??) -- it couldn't be that you're not understanding the point I'm making.

The point is that he recognizes the advantages of HTML on the web and on Slashdot, yet is unwilling to consider those advantages for Usenet. I turned the argument around -- if ASCII is so great for Usenet, then why not all ASCII for the web?

Both the web and Usenet are publication protocols. They just go about them in very different ways. I find it very useful to be able to italicize, boldface, embed links, have lists, etc in my publications, not to mention the advantages of proportional fonts.

Jeremy's right, but it's too late now.

[Charles Kerr]

I'm one of the authors of the Pan newsreader and agree with Jeremy's analysis of yEnc. yEnc repeats many of uu's mistakes, so news clients have to search text/plain messages for =ybegin and =yend blocks instead of looking in the headers.

But yEnc's bandwidth savings are real, which is a huge win for alt.binaries users. yEnc has been the most-requested feature for Pan over the last month. (0.11.2.90 supports it.) IMO yEnc is the format to use for multiparts right now.

Hopefully yEnc will motivate others to come up with a mime-friendly alternative encoding for Usenet. yEnc Considered Harmful is another yEnc opposition page that suggests mzip compression, but I haven't seen any public discussion of it yet.

If/when such a replacment comes along, Pan will support it too and add an are-you-sure dialog for yEnc postings.

Re:Jeremy's right, but it's too late now.

[who what why]

Good Job! I've been using Pan for a couple of months and I love it. The whole yEnc issue is a moot point, since binary downloads are so transparent and effective in Pan that it's hard to realise exactly what I *don't* have to do... I just look at the "please help because Outlook is making me grab several parts to each binary and click on them and combine them and decode them" posts and shuddder. Software exists that takes yEnc, and even uuencode into it's stride. Get over it! and thanks again for Pan.

Re:Jeremy's right, but it's too late now.

[slashdot_commentator]

BTW, kudos on the 0.11.2.90 release.

My response to yEnc was basically to sit on my thumbs watching all the movies & tv shows go into a yEnc-encoded oblivion until you guys put out the yEnc compatible Pan. (...Like I was going to switch to a windoze newsreader... bah!) Only lost two weeks worth of TV and movies...

Re:Jeremy's right, but it's too late now.

[Anonymous Coward]

Seriously, Pan fucking rocks.

Re:Jeremy's right, but it's too late now.

[Anonymous Coward]

..authors of the Pan newsreader ...

This expands as the "pimp-ass newsreader newsreader". Have you already forgotten your roots?!

Re:Jeremy's right, but it's too late now.

[Scurrilous Knave]

Another big RIGHT-ON to you and Matt and Cristophe and the other Pan contributors for making an excellent piece of software. You guys rule.

And a special thanks to you guys and Gabi for adding yEnc support so quickly.

Try www.WebBinaries.com or Re:On a related note...

[Hyped01]

or www.Pictureview.com (I'd suggest WebBinaries...)

Re:Screw luddites

[Anonymous Coward]

>The green-screen luddites need to stop whining and upgrade their newsreaders. Yes, ASCII newsreaders can strip out (or render to some degree) the HTML.

Go ahead. I just ignore you anyways, unless I'm feeling nasty. In that case I'll just include some nasty popup goatse.cx javascript in my post. You might have that filtered out, but other users aren't that smart and will just turn off their HTML support, thereby reducing your readership even further. Slowly you are censoring yourself off the internet as you both make it into killfiles and more and more users choose not to read your crap.

BTW: I'm running windows XP with Forte FreeAgent, which can't read your shitty postings. And no, my monitor isn't a green screen ASCII terminal (it would be interesting to see what windows would make of that, though).

Re:Screw luddites

[shepd]

>The point is that should be under the reader's control

Then take the problem into your own hands instead of depending on the world to do it for you. Add a module to your newsreader to strip all CR/LFs from postings unless they are followed by another set of CR/LRs. This isn't even high-school programming level work -- anyone with a "programming for dummies" book could do it.

Your problem is fixed, and you can post the solution to benefit everyone tomorrow.

Re:yEnc = XMODEM part deux [off topic]

[bovinewasteproduct]

I think the one your looking for is called BiModem.

BWP

Techie YENC Bad Design Info ! (Please mod up)

[Anonymous Coward]

yEnc considered harmful yEnc considered harmful

yEnc is yet another encoding method for binary files. Unlike Base64 or UUENCODE, it uses nearly all octet values, reducing the overhead from 33% to about 2%. Problems with yEnc No MIME or Back to the UUENCODE mess

A huge mistake is not to use MIME for yEnc. Let me explain why: In the pre-MIME era there was UUENCODE, which had several problems:

• There are no clear delimiters. UU-encoded files can start anywhere in a text message. As a result, some newsreaders incorrectly see attachments where there aren't any. This has been partly addressed by yEnc which uses =ybegin...=yend instead of begin...end. But there's still a chance that the markers appear in normal text.
• No labelling of file types (Content-Type in MIME).
• Charset recoding problems. Note that this is a much larger issue with yEnc than with UUENCODE. UUENCODE is only corrupted on gatways that can't handle some ASCII characters (esp. EBCDIC systems, which should now be extinct). For yEnc, every 8bit charset translation is fatal.
• No standard labelling of split messages. (This is addressed by yEnc, but only within the body.)

MIME provides a proven solution to these problems:

• It clearly labels all data out-of-bound, using Content-* headers. It clearly seperates text from binary data.
• Base64 uses an alphabet that most likely survives all charset translations.
• message/partial provides a standard way to split large messages. It even allows MTAs to split messages on the transport level (granted, that should NEVER happen on Usenet).

yEnc-over-Quoted-Printable and Charset Fun

Back then, when UUENCODE was state of the art you could just cut and paste a UU-encoded file into your text message. With modern, MIME-aware newsreaders, this is no longer the case:

• Newsreaders might be tempted to apply Quoted-Printable (or Base64) encoding on the text. Yes, I've seen Quoted-Printable encoded UUENCODE attachments.
• Even if they can be convinced to use 8Bit (or Binary), they might suddenly do some charset recoding:
  • ISO-8859-15 and UTF-8 become more and more popular, especially due to the Euro Currency Sign. This means the charset has to be recoded from the systems native encoding (e.g. Windows-1252).
  • Even with plain old ISO-8859-1, DOS and OS/2 newsreaders have to recode from the DOS charset and Mac newsreaders have to recode from the Mac charset.
  Further, there's no way to include yEnc within UTF-8-encoded text, which is slowly becoming the worldwide standard: If you include the data as-is you'll have invalid UTF-8 sequences. (This also applies to other Multibyte Charsets, which are quite common in Far East countries.)
  (Note that this is no problem with UUENCODE or Base64, which only use an unproblematic ASCII subset.)

Small problems that should be addressed

There are also some smaller problems which should be addressed:

• Using CRC32 as a checksum. There are much better hash algorithms like SHA-1 or MD5 available.
• Bad Extensibility and less features than MIME Content-*: Could be solved by integrating yEnc into MIME or vice-versa (e.g., Content-* headers could be allowed directly after =ybegin and before an empty line after which the binary starts.)

Solutions for yEnc yEnc as a MIME Content-Transfer-Encoding

One of the solution, of course, is to embed yEnc into MIME. The first idea is to do that as a Content-Transfer-Encoding.
There have been some arguments against MIME, however, which should be addressed here.

The creation of new Content-Transfer-Encoding values is STRONGLY discouraged. (Quote from RFC 2045)

This is true and there's a very good reason for it: A lot of software needs to be updated to support it.
On the other hand, the situation is no better when MIME is not used: Users would still need new software. If the news client does not support the format, users can just export the message (nearly every newsreader can export a message in source format) and process it with external tools.
You have to ask whether a new, news-only encoding is a good thing. If yes, then it does not make a difference wheter it's embeded in MIME or not.

message/partial MUST not have binary content.

This is because it couldn't be recoded as neccessary on gateways. But with yEnc, recoding would only happen where the message would have been recoded anyway.
It only raises the question whether a news-only encoding is a good thing again. (With yEnc, recoding would only happen where the message would have been corrupted anyway.)

There's no per-part integrity checking for message/partial.

There is no reason why Content-MD5 could not be used on message/partial: RFC 1864 only disallows Content-MD5 on multipart/* and message/rfc822, not message/partial. (The reason for this is that these type can contain data that could be recoded at gateways. This would not happen with message/partial or, if it happens due to yEnc, the message would have been corrupted anyway.)

There's no easy way to find all parts from a single part (i.e. find the message ids of all parts) with message/partial

Neither is there with yEnc. All proposed solutions would both work with a non-MIME-yEnc and message/partial.

You can't use your standard newsreader by pasting the encoded data.

You haven't been scared enough by the yEnc-over-Quoted-Printable and Charset Fun chapter, have you?

Conclusion: yEnc as a MIME CTE is much better than yEnc without MIME. yEnc as a MIME Content-Type

The other solution, of course, would be to introduce yEnc as a MIME Content-Type. It then would have to use the binary 8bit CTE.

This would be a similar approach as used for application/mac-binhex40, which is also defined as a Content-Type. Also note that many compression and archive formats are a Content-Type.

As yEnc contains additional information (such as file name, markers, etc.) which a CTE usually does not, this seems to be the better solution.

Conclusion: yEnc should be a MIME Content-Type. Alternatives

Of course, one should ask what alternatives are there to yEnc (or any other super-Base64 encoding). Not using Usenet

Usenet is, even without the Base64 overhead, a horribly inefficient method to transfer large files. Because of the flood-fill mechanism, the articles are sent to all news-servers carrying the specified newsgroup, even if there's no user that wants the article there.

Peer-to-peer networks, such as Gnutella or Freenet are much more efficient and can transfer binary data as-is.

Conclusion: Binaries should not be sent over Usenet. This is not expected to happen any time soon, however. Use all MIME features

MIME already has most of the features necessary to send binary data over Usenet:

• An encoding: Base64.
• A standard to split messages: message/partial.
• An integrety check: Content-MD5 (MD5 is much better than CRC32.)
• Out-of-band labelling of data types.
• No mixing between text with charsets and binary data.

Of course, you would have to use all features provided by MIME to provide everything that yEnc provides (today):

• Use Content-MD5 on each embedded file.
• Use Content-MD5 on each message/partial
• Use Content-Disposition to transport file names and attributes.
• Use the number and total parameters on every part.
• Use the Message-ID convention described below.

Some features proposed for yEnc, such as assembling a file (message) from different sets of partial messages, could also be integrated into MIME - in a backward compatible way!

There is only one real argument agains MIME: efficiency. The Base64 encoding produces about 33% overhead.

Conclusion: MIME provides a proven solution to send binary data over Usenet. The only problem is efficiency. Link-level Compression

The 33% overhead can be more than nullified by using an compression method over bandwith-sensitive links.

There's a proposal for a MIME-aware compression scheme Assembly of partial messages

See my document about a Message-ID convention for partial messages. Conclusion

MIME already provides a solution to most problems that yEnc is supposed to solve. There's no reason to reinvent the wheel for yEnc, which causes some problems. Only the more efficient encoding remains. This problem can be addressed by introducing yEnc as a MIME compression scheme or by introducing a link-level compression/filtering.

Revisions
2002-03-19yEnc as MIME type would only require 8bit, not binary.

Some minor fixes. 2002-03-04Initial version.
Claus Frber <claus@faerber.muc.de>

Re:Screw luddites

[Anonymous Coward]

If you're using a Windows newsreader, it typically uses the IE COM component to display the HTML.

While allowing other possibly malicious components to wreak havoc with your system - a problem that never happens with plain text messages, does it?

I have specifically turned off the 'Use Microsoft HTML viewer' in Eudora, my Mail client of choice.

That way, I get the formatted text from people who send me HTML mail (the worst is people who send me a Web page as a paste-in), but it filters out any of the evilness of the Microsoft tags, etc.

It's disappointing that Eudora by default uses the Microsoft viewer, rendering it at least half as bad and security dubious as using Outlook products.

More like it used to be

[medcalf]

It used to be that someone did something useful, then the community, through use choices, adopted it as standard. Then, if there were flaws, these would be ironed out with an updated standard, usually all or mostly backwards-compatible with the original implementation. It's gotten to where new standards are useless, either because companies (like, say, RealNetworks or MS) refuse to submit their protocols/formats for public use/review, or because the standards committees (say, for Java (before it was pulled) or the W3C) argue for years without actually doing anything.

I, for one, am happy to see a useful format publically available.

Re:Screw luddites

[Anonymous Coward]

So, really, what you're saying is that as long as there's one dood out there using a VT-100 terminal to read Usenet, we have to accomodate him.

And actually, screw that. VT-100 has escape sequences. We need to accomodate anybody who uses a printing teletype (i.e. a ASR-33 or a new-fangled DecWriter that supports Upper/lower case) to read Usenet.

Yep.

Re:Screw luddites

[shepd]

>So should the web return to the days of pure ASCII?

For someone who calls themselves "reality master", one might hope you would realize that HTTP/HTML have supported extended ("foreign") character sets since inception, thereby violating ASCII standards all along (for the client).

You can access them by typing an ampersand, followed by the mnemonic of the character you are accessing, followed by a semicolon.

This process is described in my 1995 HTML sourcebook.

So, in other words, there were no such "ASCII only HTML" days you speak of.

>Should Slashdot disallow any HTML postings? HTML is useful.

Not when people abuse it to make page widening, huge blinking text and scrolling marquees. Not to mention linking IMG tags to child-porn and other illegal items.

>Look at how it used on Slashdot -- italics, boldface, links, etc.

Look at how that's been done since ASCII was used for conversation.

Bold: *bold text*
Italics: ~italicized text~
Underline (betcha that would confuse you if I did it with HTML): _underlined text_
Link: You can go to http://slashdot.org/ for more info. [Most text-only readers will auto-parse anything with the http:// infront of it as a web-link and treat it as such].

>Not to mention that proportional fonts are infinitely easier to read.

A: Not for ascii art.
B: Invalid argument. Proportional fonts came out of my 1985 Star NX-1000 Dot-Matrix printer, well before HTML.

>How can you possible be in favor of HTML on the web, yet not in favor of HTML in Usenet?

I would suppose because they are different?

Next thing you know people will want to implement FTP over usenet...

>Put it this way: If Usenet were invented today and they included HTML, could you honestly say it would occur to you to say, "you know what would make this better? ASCII only!"

Hell yes.

>Depends on your newsreader. If you're using a Windows newsreader, it typically uses the IE COM component to display the HTML.

*Some* newsreaders do. Hardly even close to typically (netscape news, freeagent come to mind).

Whine!

[Anonymous Coward]

Sounds like a great big whine to me. If he doesn't like it, then he should put his money where his mouth is and DESIGN AND PUBLISH something better.

Fact of the matter is, YENC caught on because there was nothing else that solved the problem. If the author of that article had got off his butt and written something better as he claimed he planned to, chances are that we'd be using that now.

So don't give us this "it sucks because it's not as good as it could be" attitude, and if you're going to improve on it, DO IT, don't just bellyache about it here.

Re:Whine!

[Anonymous Coward]

While JPEG was being developed some people rushed out file formats based on the same kind of compression. The files were as small as JPEG, but the file structure was crap. Does anyone remember those formats today? My guess is in 1 year no-one will be using yEnc, and most people won't even remember it existed. The code that yEnc is based on (which was not written by yEnc's author!) will be used to create a proper format, which will do more than simply make files smaller. Wait and see.

Re:Whine!

[Anonymous Coward]

Yeah, before the JPEG people published the JFIF file format standard, there were bunches of proprietary formats put out by various shareware types.

Hopefully the same thing will happen to DivX files (MPEG-4 in AVI) someday...

Use the braincell, Luke...!

[Rui del-Negro]

> It *WIILL* decrease the amount of data downloaded
> on a per message basis. Of course, this means the
> customer will simply download more messages...
> good for the customer, bad for the service provider.

Why is it bad for the service provider...? Do you think they care how many messages you download? The only thing that matters to the provider is the volume. Doesn't matter if it's 10 big messages or 15 slightly smaller ones.

It's bad enough that the big companies ignore the standards and keep showering us with "brilliant innovations" that last one year and then need to be replaced (after everyone wasted time adding support for them). Now we have to deal with public domain hacks, too?

I guess this shows how Microsft got to where it is. Given the choice between a good thing and a bad thing, people will pick whichever one they can get their hands on sooner.

yEnc reminds me of the Morse code. It worked, it was crap, and some people are still having to put up with it today.

Re:Use the braincell, Luke...!

[bighousehx]

> The only thing that matters to the provider is the volume. Doesn't matter if it's 10 big messages or 15 slightly smaller ones.

yEnc will allow me to download a given binary in 10 small messages, using your analogy. This is a reduction in volume, and a good thing for those of us with metered connections.

yEnc is not perfect, but it *IS* an improvement (even yEnc detractors admit this) over uuencode. I find it hard to dismiss yEnc simply because it is an incremental, rather than complete, solution.

Re:Screw luddites

[pyramid termite]

The point is that he recognizes the advantages of HTML on the web and on Slashdot, yet is unwilling to consider those advantages for Usenet.

What's the major advantage to HTML on the web? You can link to other pages on the web, knowing they're there.

Now how would one use HTML to link to other posts on Usenet and be sure they're there? Could you write a post linking to several later posts you were planning?

No. The only thing you can really do with HTML on Usenet, seeing as binaries such as images are prohibited on text groups, is to present text. (Unless you're running malicious scripts.) And guess what? You don't need HTML to do that - you don't even need it to post web links, as one can just copy and paste them into a browser.

I turned the argument around -- if ASCII is so great for Usenet, then why not all ASCII for the web?

Because, dumbass, it couldn't be a web, then, could it? Maybe, before telling us how Usenet should work, you ought to learn how the web works first.

Well

[Trepidity]

I'm glad you agree that Microsoft Word is proven to be one of the best programs in existence. =]

the author responds...

[flip-flop]

As some of you might already have noticed, the author of yEnc responds to several of the points raised in his FAQ (scroll down to near the end of the page). His replies (to questions such as "Some people say that yEnc is badly designed and was rushed without proper discussion. Is this right?") seem quite realistic and sensible to me, and I get the feeling some of the accusations levelled at Jürgen in the article weren't entirely fair.

Re:the author responds...

[Anonymous Coward]

yeah, like unix, on the surface it looks like a hopeless collection of horrible kludges, but if you look deeper there's reason behind most of the decisions.

Re:yEnc = XMODEM part deux [off topic]

[cbensinger]

Could've been BiModem; but more likely HS/Link.

It worked very well for my BBS under OS/2 (assuming you had the SIO drivers running). It's been a long while, but it seems like it had a two way chat feature, some sort of built in game (Tetris clone?), etc. Nothing like the good old days.

Nice Logic

[PhilosopherKing]

You do realize that you both put forth a theorem and then disprove it all in the same post. It is proper form, but not what you were going for, I think.

A shame

[wdr1]

A shame his article isn't better written. It seems Nixon has some good arguments as to how to make yEnc better, but is having trouble making them.

Nixon seems intelligent, but he doesn't want to understand that his problems, as an admin, are different from those of his users. To quote:

And the bandwidth savings? That's an illusion. A smaller encoding scheme gives us exactly one benefit: faster downloads and uploads for the users.

... and ...

The problem here isn't that we need a smaller encoding scheme, the problem is that we need a better way to post binaries on Usenet.

Well not really. What Nixon wants is a better way to distributed binaries on Usenet. Based upon the popularity, what the users want are faster downloads. That "exactly one benifit" seems to be pretty signifcant to them. As a admin, I'd guess that Nixon has pretty much high-speed access almost all the time. In that type of an enviroment, it's easy to forget the pain of life with a modem.

That disconnect is further demostrated by the iron-clad assertation that Usenet, in it's current form, is near perfect:

What was so broken? Nothing. Usenet was working just fine, and people were posting and downloading binaries just fine.

To me that conveyed a very unwelcoming aditude to anything that might rock the boat. Nixon feels everything is okay right, but it seems a lot of users seem to feel there is room for improvement.

More progress is likely to be made if the admins stop and understand why users are eger to adapt yenc instead of trying to dismiss as ignorant why they do so.

-Bill

Re:Is usenet dead? Usenet = MP3 galore

[TheSolution]

Amateur.

Microsoft Word IS the best program...

[Megs]

...for opening and writing Word documents =)

In an alternate universe, that might be a weird fringe activity, like speaking in Klingon or playing chess by mail. In this one, of course, it can be quite important or at least rather convenient.

Meghan

Re:Microsoft Word IS the best program...

[Anonymous Coward]

Not to my experience. Not even that. You know why? 90% of time when i get a word doc and i try to open it on word 95, it just refuses to. However, staroffice eats it with no problem.

Re:yEnc = XMODEM part deux [off topic]

[cymen]

Bingo! I forgot about the two way chat feature. I think I used that all of one time. Definately used the SIO drivers. HS/Link... Those certainly were the good old days. You ever search groups.google.com to see if any of your handles show up on there from the BBS days? I found some of mine the other day (apparently some of the discussion groups were broadcast to usenet).

Thanks for the post. I hate little things that keep churning away in the back of my mind :).

Re:Is usenet dead? Wow, the web, too!

[Anonymous Coward]

"Does anybody really use usenet anymore? ...it seems to be filled with 90% spam..."

You know, every time I poke around the web, I notice the same thing.

--Richard

Where are Amazon and BT?

[Snowfox]

Shouldn't nVidia, Bezos or British Telecom be stepping in about now to claim a patent on encoding 8-bit data as 8-bit data?

Whats your point?

[Crass Spektakel]

In the very early 1990 I was using subnet, which was assimilated by usenet meanwhile, with uucp and wazoo.

Even way back then there was vital interest in more efficient filetransfer. uudecode simply sucked. Everybody agreed to this.

But what did happen in the last 12 years?

Nothing.

There are now more powerfull uudecode-implementations, but I havent really seen anything practical invention in the basics.

I totally agree that yEnc is a quickshot with none thinking at all and its weird and so on. But the usenet-gods havent done anything wort mentioning in the last 12 years and they will not do in the next five years. They lost it and are now crying for not getting asked. Sad fate, but graveyards are half full of indispencable people and half full of people who dispenced with them..

Re:Whats your point?

[Narsindal]

I agree. Binary posting in newsgroups hasn't changed at all compared to advances in other areas over the last ten years. First PARs and now this. It may not be well written but at least it's something.

Usenet needs a good kick in the pants if they want to stick around.

7plus...

[Anonymous Coward]

This guy apparently complains that the new encoding does not fix the problems he has with binary encodings, yet he has done nothing to specify an encoding that does.
Basically he just not wants others to solve part of the entire problem.
In fact, this problem has been solved a decade ago in the amateur radio packet network. The 7plus encoding has all the things he likes: specification of parts, checking of integrity, it even has a limited form of FEC.
However, the usenet community never saw or used this encoding. It just went on using uuencode and later MIME.

Re:7plus...

[Meowing]

The encoding is not the real problem. Also, the send-it-everywhere approach to distributing articles isn't a big problem for binaries. It's efficient enough in real life, though it does limit what can be done in the more interactive [text] parts of Usenet. Binary groups are't really intereactive (well, fnarr I guess, but...).

The use of a protocol and article format designed for text messages rto distribute stuff that isn't text is the problem.

There are ready-made solutions, even standardized ones, to all of this. It's a (not necessarily simple, but you might be surprised) matter of reaching into the Obvious Bag, pulling out the stuff that's already made to do the job, and coding it.

That's Bullshit

[Anonymous Coward]

98% of binaries traffic on Usenet is pirated software or illegal pornography (by volume), and 93% by number of posts

I'd say there's at least as many mainstream movies on Usenet than there are porn/software; also, the pornography found on Usenet is legal in most jurisdictions. Your numbers are bogus.

Kiss my....... Ass!!!!!

[ainsoph]

you may be surprised at the artistic detail of the formating.

LOL...man...

History

[stox]

Speaking as previous long time user of CBBS ( the birthplace of Xmodem ), I will point out that Xmodem enjoyed a great acceptance in the BBS community prior to adoption of support for Kermit. Early versions of Kermit were extremely inefficient compared to Xmodem, but it was the only game in town if you were interested in tranfers to and from MVS mainframes. Kermit quickly ruled the domain of inter-architecture tranfers, while XModem, and it's derivatives quickly dominated the PC domain.

And years later, there was the legendary KA9Q code. :->

Not so new

[nrosier]

I don't know who was first and I'm sure this is not the only other tool/compressor but here in Belgium, we've been using a tool called Bommanews (b-news.sourceforge.net) for a while now because of rediculous upload-limits by the local cable-ISP. We're not allowed to run servers (ports &lt 1024 are blocked), have un upload-speed-limit of 128kbit/s and only 15% of our total traffic can be upload so some guy came up with this to allow easy distribution of files through the news-server. Because it's so popular, our ISP finally had to setup a 2nd news-server just for binary postings as the other one was overloaded (serves them well).

Am I the only one who....

[tomstdenis]

...thinks distributing binaries through usenet is a huge waste of bandwidth? I mean if you really got to get your kiddy porn out or a hack of XP why not just post links to websites with it?

What people don't realize is that when you post a 250KB attachment that 80,000 servers around the world will download that ... thats 19.3GB of traffic or so. Just for one measily attachment.

Any ISP that bans attachements in usenet is fair game in my books. They are doing the rest of us a service.

Tom

Re:Why yEnv is good for the software companies

[IHateEverybody]

As a Deja.com shareholder, I followed the talks between Deja and Google last January very closely. I learned many interesting things about company strategy, usage patterns, and targeted markets. But one thing that I'll never forget is this one chilling statistic: 98% of binaries traffic on Usenet is pirated software or illegal pornography (by volume), and 93% by number of posts.

Ummm, you are familiar with the three kinds of lies right? Besides, pirated software and porn (whether legal or illegal) posts can take up hundreds of megabytes and are blown up by about 33% by less efficient encoding schemes like MIME and uuencode. This is why their "volume" is so high.

For the first time, I truly felt appreciative to Deja for all of the efforts they expended in keeping this illicit content off of their fine service.

Actually, it was quite easy for them. They just didn't save any binary content on their fine service so it was easy to keep the illicit stuff off.

And just for the record, whoever wrote up that 93% pr0n and warez posts statistic wasn't focusing just on alt.binaries.pictures.aviation or alt.binaries.pictures.astro. I have never once encountered anything like what you'd see on goats.cx (something I alas can't say about /.) in a binaries group that was dedicated to something wholesome like aviation or astronomy. Those groups tend to be filled with pictures of airplanes and stars.

And that brings me to my point: yEnc sucks. yEnc is a horrible standard. Just like uuencode, it forces you to track down all 56 posts that constitute the 1337 warez release or all 7 posts of your kiddie porn movie.

And just like uuencode, most serious Usenet users can use yEnc savvy newsreader like XNews or Agent to track down, join, and decode all 56 posts of your illegal copy of Bloatware 2002 automagically.

yEnc - is now

[Anonymous Coward]

Look... I read the article, and the responses... and while many have hinted at it... here's the biggest point I think that needs to be made:

Some time ago, a discussion in the newsgroup alt.binaries.news-server-comparison led me to the idea of a low-overhead binary encoding system. I wrote some code and made some test posts using an encoding system very much like the one used in yEnc. It worked. I didn't go out and get people to start using it right away, though, for reasons which will become clear shortly.

Alright... yes - you started it - but you never finished it. If you're not part of the solution, you're part of the problem... He asks - what's the rush? It's been a "problem" for 20+ years... if people had gotten off their asses and written it the right way (including the author of this article) - they he wouldn't have to be spending the time complaining about it.

The yEnc author finally wrote the damn thing - and sure it's not perfect... but it's now... it exists. Maybe _this_ will get people off their asses to write a better one.... if they don't want yEnc to be the real solution

It's like the whole "full disclosure" issue on software bugs. arn

Re:GO OUTSIDE.

[netdemonboberb]

Balance is important, but stuff like this is useful. The point you have is not to spend your whole life in front of the computer. There is nothing wrong, though, with spending a lot of time at the computer. You also have to realize that a lot of people do this for a LIVING. Knowledge about what's going on in the tech world is part of staying alive in this business. Therefore, this is how THEY make their money instead of working at McDonalds.
Balance is key, and you have to balance time on the computer with excersize, sleep, and time with friends / fun other than computers, possibley school and job, and possibly spirituality and/or religion. If you are letting one of those slip, then you are missing out on part of life.
We live life for the experience of it. That is the key to life, getting the most experiences from it. In the world we are in now of fast-paced technology evolution (which I often wish would slow down a little, or become more orderly in growth), one experience that can be gained is being a part of it.
The meaning of life is to gain experience and knowledge to help you on the other side. Therefore, everything that exists on this planet is something that can be experienced to gain knowledge and insight, and even though you might not agree with certain experiences or want to try them doesn't mean that there is anything wrong in doing so unless it causes pain or hurt for others as you are also hurting yourself in the process as we are all one. Of course, what is harmful for others can be left up for debate and argument.
Try to be open minded before judging others.

j00 lu53r

[Anonymous Coward]

YHBT. YHL. HAND.

Re:Not all programs require yEnc in the subject li

[nagora]

Instead it scans the file itself.

What happens when the posting is about yEnc and includes a description of how the encoding starts and ends? Does it know that that's not really an attachment?

TWW

Re:Why yEnv is good for the software companies

[Anonymous Coward]

"we can show the software companies that we are responsible enough to make it a pain in the ass to pirate software"

Don't be a fucking moron.

You act as if software companies are a monolithic "thing" looking for the proper "behavior" from us.

They could give a crap what you or I think. They only care about profits. Trying to "demonstrate" something is the mentality of a sheep and a loser.

Re:Counterpoints to all of Jeremy Nixon's main poi

[nagora]

There is no reason to despise magic strings.

You don't have much programming experience, do you? Magic strings are a dire way to do anything. MIME has been around for quite a long time now and was invented as a solution to all the problems of magic strings in messages, the obvious one being how difficult it is to handle a message about the magic strings (Outlook still can't handle these). MIME makes sure that the strings are unique to the message, not the class of attachment.

Data about the message should all be in the header. This is so obious that it is hard to imagine what you think the header of a message is for.

No, using the subject line is not obviously a terrible way to determine filenames, segments, and anything else.

The subject line is for the subject. What the fuck are you smoking? Why not the From line or the X-Envelope line? Because that's not what they're for!

find it very convienent to know exactly what my yEnc files will be saved as, how big they are, and how many parts they are

Your client should tell you that from properly formatted headers; the Subject line is for the subject.

If we expect newsgroups standards to reach everyone, we must use the lowest common denominator.

So, no yEnc then. You are arguing that we can't change anything but that we should embrace any half-arsed new encoding that comes along

Which is exactly what the creators of yEnc intended.

I think you are confused about who the creators of yEnc was; one aim of any new encoding should be reliability.

I don't blame him. Jurgen is a coder, not a politician. I would have done the same thing.

Class-level magic strings belong in the museum along with line numbers in source code; Jurgen isn't much of a coder if he doesn't understand this.

But the point is it works right now, and it's working great.

It sort of works sometimes right now, if the spec had been put in as a MIME type it could have worked well all the time in a few months. As you say, Usenet has waited 6 years so why not a few months extra and get it right?

TWW

Deja Vu all over again....

[cmkrnl]

I've just had this strange usenet flashback to the 1991-93 timeframe where there was a very very vocal element on alt.binaries.* bitching something rotten about posters using JPEG instead of GIF.

Strange how spurious BS arguments get recycled again and again in defence of the status quo.

If I was bold enough to make a prediction, I would suggest that within 18-24 months, posters into alt.binaries.* using anything other than yEnc will be in a very tiny minority.

Like all great software that scratches an itch for just about everyone , a de facto standard some becomes a de jure one. The RFCs will be along later.

As a real life example. How many european govts were mandating OSI and X400 to the forcible exclusion of everything else back in the late 80s/early 90s ? How many are doing so now ?

Anyone want to take bets on how long it will take MS to give it the ultimate seal of approval by supporting it in OE ? I reckon its sooner rather than later.

Curmudgeon

Re:Counterpoints to all of Jeremy Nixon's main poi

[3247]

There is no reason to despise magic strings. They work, and cannot ever occur in the user data. All yEnc magic strings start with =y, = being the escape character. Ctrl-Y does not need to be encoded, so yEnc is free to use =y for it's own purposes (e.g. =ybegin, =yend). Jeremy Nixon continues his misled rant...

Nowhere does the yEnc spec talk about "Ctrl-Y" (octet value 25). It does talk about "=y", being the equal sign (ASCII 61) and the Latin small letter y (ASCII 121).
This character sequence, of course, can appear in any text message.

Inform yourself before propagating such nonsense!

Not quite...

[Da Masta]

What exactly, I dunno.

That's something resembling a flame. In the article, he mentions quite a few ways to improve yEnc, including the use of MIME, and some other stuff I can't remember. I'm just wondering why there's no code if all the technology needed exists.

News Readers yEnc and pr0n

[ellem]

Forte Agent _finally_ has yEnc support in the product (version 1.91.)

My guess is Outlook Express will follow.

People will not have their pr0n and warez messed up to the point that they cannot download it.

As for the "war" these two seem to be having let me boil it down in schoolyard terms:

Nixon: It's mine give it back!
Jurgen: I already gave it away!

Re:Screw luddites

[Reality Master 101]

What's the major advantage to HTML on the web? You can link to other pages on the web, knowing they're there.

Wha'choo talkin' 'bout, Willis? There is NOTHING about the web that guarantees a page "is there". In fact, the whole "linked" nature of the web is totally an illusion. All you're doing is marking a piece of text as something that should be inserted into your browser's address bar if you click on it. There's nothing magic about a link.

Now how would one use HTML to link to other posts on Usenet and be sure they're there? Could you write a post linking to several later posts you were planning?

Who cares about linking to Usenet posts? Just being able to link to web sites would be an advantage.

You don't need HTML to do that - you don't even need it to post web links, as one can just copy and paste them into a browser.

As you can on the web. Who needs links when you can just cut/paste into the browser? After all, that's all a link does anyway. The point is that it's a convenience, which is just as convenient in a Usenet post.

Maybe, before telling us how Usenet should work, you ought to learn how the web works first.

Perhaps you should take your own advice.

Redhat closing up?

[Anonymous Coward]

Too many companies _need_ RedHat, and would
prop it up or buy it to avoid it's disappearance.

you meant:
"...last release before IBM buys RedHat"

Number of servers.

[mindstrm]

Also.. are there hundredsof thousands of news servers nowadays? I think we are dealing with smaller numbers of larger servers now.. with news services like newsfeeds.com and such becoming increasingly popular hubs for usenet.

yEnc will be in Mozilla at some point, BTW

[slashbrent]

Just thought some of you Mozilla fans might be interested to know that there is already an open bug in bugzilla 119964 to support yEnc in Usenet postings.

Recent posting indicate that some of fellow Moz contributors may be taking this issue and fixing it ahead of the planned "future" date already assigned by the developer - if any of you can do this fairly quickly, i'm sure the rest of us MailNews users would appreciate your efforts!

Re: Microsoft Word .doc files?

[Self-Important]

"The problem is the inability to read an M$ Word doc that was sent to a Linux user."

I know you're throwing out examples of perceived "problems" that stem from a lack of standards, but this just reads like a pandering attempt to garner the sympathy of Linux users.

Face it, bub, .doc files *are* a standard. It's a documented (yes, the spec *is* publically available), ubiquitous file format. If you run Linux and *still* haven't availed yourself of the many M$ .doc file solutions, then I don't know what to tell you. Boo hoo?

Let's think about this for a second: If there isn't even standardization with ASCII text files between DOS/Windows-based systems and Unix-based ones, then why would anything else be completely, unequivocally standardized? For documents, .docs are what *most* people use, and I really don't see that changing any time soon just to accommodate a few rogue Linux users who can't be troubled by OpenOffice or StarOffice.

Not the same thing

[Rui del-Negro]

They could have made it cheaper to the consumer, it would have made no difference. The problem was Sony charged a license to whoever wanted to make Betamax equipment, so other companies simply decided to support VHS instead. In the professional world, quality is more important, so Sony managed to get Betacam (which is actually very similar to Betamax) accepted by the broadcast industry. Since this industry has a lot of money, other companies were able to license the format from Sony and still make a profit.

IDE and SCSI are two different markets. Most (all) companies making SCSI drives also make IDE drives. IDE is perfect for single users. It's not so good when you need to read and write 20 things to the same disk simultaneously. And it only supports drives (no scanners, etc.). End-user pricing rarely "kills" a standard; if sales are low, they will simply lower the price (unless it really isn't cost-effective). But (lack of) industry support will kill a standard, and that's what happened with Betamax.

standards compliance

[MenTaLguY]

That could be because yEnc breaks the NNTP and MIME standards. Some (many?) NNTP and MIME-compliant servers won't pass yENC correctly.

Why yEnc rules.

[Anonymous Coward]

Because now you can download a lot more child-pr0n for the same amount of money. Is that great or what?? Kiddypr0n groups have been using it for ages (isn't that where it started?). Only I heard it has a backdoor that identifies the poster ID to the FBI, and thats how they caught all these guys last week...

Agent 1.91

[alexo]

I think a company such as Forté could create a real standard using an encoding method similar to yEnc (it wasn't "invented" by yEnc's author, anyway). I think Agent's programmers, of all people, should know how hard it is to deal with these (non)standards, and could save themselves a lof of work in the future by making sure it's done right.

Too late...

Re:Agent 1.91

[Rui del-Negro]

Yes, that's precisely why I mentioned it. When a real standard comes out, Agent will have to add support for it and keep supporting the current (poor) implementation. All it means is more work for them. But I can understand it (after all, they depend on sales, and having extra "features" increases sales, no matter how poorly designed they are - again, just look at Microsoft).

What I definitely can't understand is why there's talk of including support for yEnc in Mozilla. Some people are using the argument "we could pull the carpet from under MS Outlook's feet if we implement it quickly enough". What for? Mozilla is free. It's not about how many people use it, it's about how well it works. Who cares if some stupid people prefer slow, crappy software? Now we need to make a program worse so it can compete with other bad programs? I think some of the people creating alternatives to Microsoft worry too much about Microsoft to actually go anywhere...

BTW, yEnc saves about 21-23% of space compared to traditional encoding (UUE, etc.), not 30% or more like some people are claiming.

Re:Agent 1.91

[spudnic]

You're right, the main goal for Mozilla isn't selling more units to be profitable. However, in something like browsers, market share is indeed important. The more people who adopt Mozilla as their primary browser the better. Once companies start seeing that the number of IE users is decreasing as Moz users increase, they'll be more receptive to not using IE only features in their web design. This would be a win for everyone.

.

Re:Agent 1.91

[Rui del-Negro]

So to fight Microsoft's "hacked" design, Mozilla should... add hacked features...? I really don't judge paintings by their authors. Poor design is poor design, I don't care if it comes from Microsoft or from Mozilla.

And anyway, I use Opera. :^)

Re:Agent 1.91

[Anonymous Coward]

You miss the fact that 99% of Mozilla development is funded by a large corporation. This corporation might lose interest in continuing to work on this project if it doesn't successfully attract users.

(Which it isn't - Netscape 6.2/Mozilla have been "ready" for a number of months now, but haven't made a dent in even the tiny Netscape 4.x userbase.)

And of course you have the code, but considering the amount of external involvement right now, who is going to continue to work on it?

Re:Agent 1.91

[Anonymous Coward]

> You miss the fact that 99% of Mozilla
> development is funded by a large corporation.

No it's not. 99% of Mozilla development is done for free.

And yEnc doesn't even have an RFC, for heaven's sake. The author was clearly more concerned with getting famous than with actually making a decent piece of software.

I don't know if I trust it...

[Kymermosst]

After all, the website has a decoder in "Pearl", but not "Perl."

Re:Is usenet dead?you are joking right?

[wheany]

3. I can try-before-I-buy just about any software package ever released. Thanks newsgroups.

4. I can try-before-I-buy just about any music...all sorted by category. Celtic, 70's rock, classical or techno anyone? Thanks newgroups.

You mean you can "try"-and-never-buy just about any software or music.

Oops.

[hkmwbz]

My last post sounded a bit more harsh towards Juergen Helbing and what I called "shotgun tactics" than it was supposed to. Helbing has said that he never intended yEnc to be spread this way. A mail has been published on Forte's web site, where Helbing explains that this was not his intention:

The first yEnc draft was never meant to be the base of a fundamental change to Usenet (this was a discussion proposal). And all I can do today is to apologize for the mess I did create with it.

Thanks to the AC who posted this link in another comment.

Nevertheless, the way yEnc has been spread is not beneficial. But it can't really be "blamed" on any single person as such.

Let's take it one step further

[Stavr0]

We just saved about 30% bandwith with yEnc. Let's implement PARchive redundancy (reed-solomon) right in the protocol. Sure, we're back to the original size, but all that'd be needed is 70% of any parts of the usenet post to successfully reconstruct the original file, so the 30% is not wasted.

My ISP (cable) has a quite short retention period in the binary newsgroups (it's like they do it on purpose!) and many files are broken from missing multiparts. Integrating yEnc and PAR would solve that problem once and for all.

Re:Let's take it one step further

[Meowing]

A decent upgrade to news would eliminate the multipart posting siliness. You can resume FTP and HTTP transfers, why not make news be able to do the same?

Re:Let's take it one step further

[rhaig]

that's a client issue. If you want to tke the code for your favorite client and have it watch subject headers for multi-part posts and have it "resume" then go for it.

If you're thinking that you should be able to post your 56MB file in a single post to alt.binaries.Imadumbass, then you don't get why they are multiparted in the first place. news servers are typically configured to limit article size. Default on INN is about 100K. depending on the news admin, sometimes it's more, sometimes less.

Re:Let's take it one step further

[Meowing]

You're seeing the problem backwards.

It's not a client issue but a protocol issue. NNTP doesn't have a good mechanism for indicating the size of a message (there's a size field in XOVER, but that's not useful for POST, IHAVE or TAKETHIS) and no way to transfer a range within an article. That lack of robustness is a primary motivator for splitting large messages into series of separate articles.

The typical (default) article size for INN is 1MB, not 100KB, and that really only exists because of an implementation decision made over a decade ago based on typical OS limitations of the time.

Quite a few server operators misuse that article size limit as a way to cut down on binary traffic stored locally, but this tends to backfire and encourage posters to split their binaries into ever smaller chunks. It's the wrong tool for the job.

There is certainly no need for a new or revised protocol implementation, especially one designed with binary traffic in mind, to preserve the historic limitations.

Re: Microsoft Word .doc files?

[Anonymous Coward]

There are several features that can only be processed correctly through exhaustive testing because what their spec says isn't what Word does.

A standard specifies how to handle each special case in a way that guarantees interoperability. Organizations like IETF and CCITT have people who are smart enough to pull this off. Microsoft will never be able to, largely because they're unfit to compete incompatibility is the key to preventing the existence of competition in IT.

Re:Screw luddites

[Wolfier]

I'll make my arguments in point form so it'll be easier for you to reply.

1. HTTP was DESIGNED to transport HTML.

2. The Usenet Protocol was DESIGNED to transport plaintext. Newsreaders that read HTML off postings do it by ugly hacks, that don't always work.

3. I didn't say the web should return to pure ASCII, and Slashdot should return to ASCII either. But I know if Slashdot allows embedded objects, pictures, BLINKS, or Java applets within comments all of these comments will eventually modded to -2. Problem is, there is no limit what HTML you can post on Newsgroups.
If you can control your newsreader like you do with your browser, I'd not reject HTML-on-usenet as much.

4. If Usenet were invented today and included HTML, then I'll accept it and use it. Plain and simple - surprise eh? You know why?

BECAUSE the design and implementation of the protocol undoubtedly would be DIFFERENT than the original Usenet so they can handle HTML INHERENTLY.

If Usenet were invented today, it would be a DIFFERENT invention, not a "time-shift" of the same invention.

Anyway, do whatever you want. I was just saying Usenet in its form isn't suitable for HTML postings, that's all. Gotta go back and finish writing my ray tracer. Bye.

The ISO table for reacting to yEnc

[Rui del-Negro]

Level 1: [completely ignorant]

"Usenet? Yeah, I use the net. But I have no idea why Hank did what you say he did. Mimes always make me laugh."

Level 2: [watches movies but doesn't read books]

"Oh, newsgroup binaries, yeah, a friend of mine showed me one time. On my screen it just shows up a lot of gibberish. Must be because I don't have the right driver installed or something. I prefer ZIP files. Mimes? I just feel like punching them, don't you?"

Level 3: [pr0n addict]

"This yEnc thing sucks man, nowadays I can only download about 150 pictures of shaved lesbian asian schoolgirls a day. It's hardly worth unzipping my fly for. I wish they went back to the old format, whatever it was called. Oh, and I hate mimes."

Level 4: [pr0n and w4r3z addict]

"I can use yEnc fine so I think it's a great standard. I think the guy who released it is a hero (I bet ISPs hate him) and the people complaining that it breaks the standards and screws up some servers are just whiners. They can just change to a different server or a different ISP, or move to a different country or something. Also, I like posting in yEnc because it makes me look l33t and it impresses the girls. I don't care if 90% of people can't open the attachments and half the files get corrupted and have to be downloaded again and again; yEnc is 17 times faster that Base64 anyway. As to mimes, well, if I move my hand like this and use my tongue to push my cheek like this, what does it look like I'm doing? Eh-eh..."

Level 5: [programmer or some other kind of loon]

"It breaks existing transfer standards, it repeats all the mistakes of the previous methods, half the files being posted are corrupted and the actual gain in speed is less than 25% (do the maths for heaven's sake!). Let's do this right: create a proper MIME standard, add support for checksums and proper multi-part handling, automated processing, etc. Don't waste time implementing something that is broken from the start. I agree about the mimes, though; they should be shot."

Level 6: [Microsoft manager]

"It's the first public-domain standard that truly meets Microsoft's quality and reliability criteria. The fact that it breaks existing standards is irrelevant. Nothing can stand in the way of innovation. By the way, I'l like to take this opportunity to say a few words about our new product. It's called Microsoft Mime and it's a gesture recognition system for Windows XP. In two years we expect keyboards to be completely replaced by it."

A Developer's Perspective

[netdemonboberb]

I am posting this on Slashdot under the yEnc Article, news.software.nntp, and also will CC it to both Jeremy Nixon and also Jürgen Helbing

I like the idea of bringing down the overhead of binary messages. With probably over 100GB? per day of binaries on all the newsgroups, this probably means a drastic 40GB reduction (albeit probably temporary as Jeremy Nixon mentioned) which means a lot of saved bandwidth on servers and also faster downloads for end users our favorite digital media (whatever that may be ;-). I feel this is a good start - though maybe a little rushed. I have to thank Jürgen for bringing the inefficiencies of MIME to the forefront of discussion even though he might have created a little extra work for developers. I also agree with Jeremy that yEnc should be stamped out in place of moving this into MIME.

I would first like to say: From the involvement I have seen from Jeremy with Usenet (especially news.software.nntp) and nntp software, I would be insane to say that I know more than or even close to what he knows. In fact, I work on the browser portion of Mozilla (as a volunteer) and know little about the underlying code of the Mail-News portion (nor do I even use it because its still very beta, and doesn't do what I need from a Usenet client). I am also not very knowledgeable on the deep-down guts of Usenet. I am a regular user of Usenet though, and have a little knowledge about how the news system works, so I feel that gives me a right to talk about yEnc from a more general standpoint than one possible from someone with Jeremy's experience. But I've decided to start expanding my experience into Mail-News and I'm thinking about implementing yEnc for Mozilla (even though I have a million other projects I'm working on).

From Jeremy's article:

>Unfortunately, with the discussions being fragmented in many newsgroups, the people
>who will ultimately determine the acceptance (or non-acceptance) of yEnc -- the end-
>users -- are largely unaware of the issues and problems raised by this situation.

I respectfully disagree that it is the end-user's decision.It is developers' decisions whether or not it becomes popular. Otherwise, we will have people continue to "spam" the groups with yEnc messages supported by only a couple clients. The people that have those clients will be happy and continue to post in that format, leaving out the people using other clients from those postings. This will never end, and the only solution is that everyone switch to the same client, ignore the postings as much as possible, or quality clients (which hopefully will one day include Mozilla Mail-News) support yEnc (at least on a read basis) so that no one will be left out of yEnc. It almost seems like developers have no choice now if they want to stay competitive in the Usenet arena. Good or bad? I think good in the long run, although it is currently a headache for developers.

>My objection to yEnc is because it was done poorly, not because it was done by
>Jürgen, and I certainly have nothing against him. Had he done it right, I would be
>thanking him right now.

I agree that yEnc seems like a wimp compared to MIME. Of course, this isn't the end of the world and I think it might be a good impetus to whip some people into action. I imagine that Jeremy, with his experience, Jürgen, and others can get together and majorly revamp the MIME standard, throwing out 7-bit encoding, including every new technology that has been thought of since its inception, making it easily identifiable as something separate from the old standard, and renaming it.

> And the bandwidth savings? That's an illusion. A smaller encoding scheme gives
> us exactly one benefit: faster downloads and uploads for the users. It is not going
> to make Usenet smaller. It is not going to allow servers to increase retention. Do
> you really think people aren't going to post more, if they can do it faster? Of
> course they are. They're always going to post more, with or without yEnc. And,
> with yEnc, they are even more likely to post more, because posting the same
> amount of material will take a shorter time, and because people who can't use
> yEnc will ask for reposts in uuencode. [I actually see this happening]

> The growth of Usenet volume is more or less exponential, and has been for quite some
> time. So let's just say I'm wrong > about people and they really will post less. Let's say
> that, overnight, all of the binaries on Usenet start getting posted in yEnc, and people
> post exactly the same amount they would have posted with uuencode, resulting in less
> total volume. All you have done, in that far-fetched scenario, is create a one-time
> volume savings. Usenet will continue to grow at the same rate it has been growing, and
> after a few months, it will be just as large as it was before. And it will get bigger from
> there. So all you have done is moved the graph back by a few months. Big deal.

Personally, I think that Usenet has gotten way too large to have all the newsgroups (or a large portion of them) on every server and needs to be redesigned (possibly with a more distributed system). That's another issue though.

> Programmers of Usenet software now have to implement yEnc in their code.
> Not just once, either. The specification is, as I write this, up to version 1.3, and
> there will be future revisions. So everyone has to go back and update their code
> every time the spec is updated. And they don't just have to change it, they have to
> continue to support the older spec as well, because updates to a new version won't
> happen overnight. And because the spec is imprecise, programmers are forced to create
> and maintain even more ugly [ Pretty :-) ] code in their software, when they could be
> spending time making more worthwhile improvements. There is a good reason for
> new standards to be discussed at length and incorporate feedback from experts - so
> that you don't have to keep going back and fixing it. And, even when something better
> comes along, all that yEnc code can't just go away; it will still be there, and still have to
> be maintained. People won't stop using yEnc overnight. It would take years to become
> uncommon.

I'm willing to waste a little of my time implementing it if it will stir some people into action. We need some kind of standard out there that can warrant the throwing away of every other encoding standard out there.
Anyone willing to undertake such a project? If there is a standard that replaces all the current standards, I imagine that yEnc, UUEncode, Base64 would phase out over time. A more naive wish is there could be an industry-wide
agreement to rip out all the other standards from their code, but I know the chances of that happening are slim.

Anyway, I say for Jeremy the old adage, "If you want something done right, you have to do it yourself". Jeremy, why don't you organize a group to oversee the development of a new all-inclusive MIME
standard? Is there any point to leaving the 7 bit aspects of MIME in there? Why aren't they ripped out (A question from someone who hasn't looked at the mime standard :)?

> Now, he seems to be planning to update the spec to include a means of using yEnc
> with MIME, which is the way everyone has told him it should be done. But he says
> he's going to do it within a few weeks! You can't add something to MIME in a few
> weeks, and there are good reasons for that. So, in reality, what he may be planning to
> do is bypass the standards process and simply publish a specification.

> There are two small problems (two that I have found, at least) with the MIME spec
> which would need to be addressed before the creation of a yEnc transfer-encoding.
> Why was MIME ignored for yEnc in the first place? Jürgen has said that MIME is
> not suitable because not all newsreaders support it, or support it fully. Does this
> make sense? You can't use MIME because it's not universally supported, so instead
> you create something new which isn't supported by a single newsreader in existence?

It appears that things are moving too slowly for his liking. This makes me think of huge multinational corporations that just lumber along and don't get anything accomplished but arguing and fighting. This (along with squelching
all competition) causes a slowdown in the growth of technology. I have no clue if that is the case with the MIME group, but it appears that Jürgen would like things to go a little faster. I definately agree with him on this, since I am
getting sick of all the messed up multipart postings I see on Usenet, and if the new changes being proposed to MIME can fix that (MD5 checksums, etc), then I would like to see that implemented as soon as possible. Since I don't have
the experience to do that and would rather not throw myself into yet another project, I hope people with experience can get the ball rolling. From my look at Google Groups, it seems that these things have been talked about for ages.
Why can't we have a little less talk and a little more action? From my experience on the Mozilla project, I see that people can sit discussing something for years. Sometimes, it's better to take the bull by the horns and try to - even
if it doesn't satisfy everybody - actually do something. For me, this means implementing something that has been talked about and discussed about forever because no one wants to (or has the time to) sit down and come up with a detailed proposal on how it should be accomplished.

> If you agree with me, what can you do to help? If you are the author of a
> Usenet newsreader, you probably have to implement yEnc at least for decoding

Sounds like a very good idea. I will propose this for Mail-News.

yEnc bug on bugzilla.mozilla.org

I have been looking through the binary newsgroups (as I am a regular lurker on many groups) and have been seeing all the people moaning about yEnc, and I can't say I blame them, as I have been moaning silently, also. That's the main reason I am planning on implemting this for Mozilla. That would leave one other thing before I was willing to switch from Outlook Express: Multipart message decoding (Hopefully better than OE's wimpy support), for which I have written a lengthy preliminary spec but don't have time to implement myself - Multipart messages bug.

Being a Mozilla developer, I would like nothing more than to use Mozilla's own software for news, and I will be caught between a rock and a hard place if Mozilla supports yEnc and not multipart decoding, and Outlook Express
supports multipart decoding and not yEnc (at least until people stop posting yEnc messages).

Re:Screw luddites

[Reality Master 101]

HTTP was DESIGNED to transport HTML.

Actually, there's nothing about HTTP that's HTML specific, other than having a specific content-type.

The Usenet Protocol was DESIGNED to transport plaintext. Newsreaders that read HTML off postings do it by ugly hacks, that don't always work.

Roads were originally designed to carry horse traffic, but that doesn't mean we shouldn't adapt them as necessary when cars were invented.

If you can control your newsreader like you do with your browser, I'd not reject HTML-on-usenet as much.

Well, I can, since my newsreader uses the IE Browser object for displaying HTML. Still, I do sympathize with the fact that HTML can be abused on Usenet. I think that means we should have better newsreaders/browsers, not that HTML is intrinsically bad.

BECAUSE the design and implementation of the protocol undoubtedly would be DIFFERENT than the original Usenet so they can handle HTML INHERENTLY.

The only change that I can think of is that we wouldn't ship around messages with two copies, which would obviously be a plus. Still, it's not a perfect world and I think the advantages far outweight the disadvantages.

You're comparing apples and oranges

[Trepidity]

You're comparing a seven year old version of Microsoft Word to the current version of StarOffice. How about either comparing Word 95 to other word processors available in 1995 or comparing Word XP to the current version of StarOffice?

Re:You're comparing apples and oranges

[oltom]

I use Word because I write and it is about the most universally accepted program around. But, whether it's the first version or the latest, I have found more problems with Word than any of the other programs I have tried. I am not the most computer literate man in the world. I would have to be classified as a user, rather than as an expert. From my point of view, Word is something I use because the agents will accept its formay. It's too clumsy for comfort. Tom

Big Deal

[dJOEK]

B-news has been doing this for ages ...
but as we all know; /. isn't just America-centered, they just hate everything thats non-American

I agree with the article....

[kg4czo]

Actually, I only see that yEnc has taken off so fast by posters. It sucks for those of us who download on the daily bases with standards based clients that don't handle yEnc'd crap. Standards are just that. They allow all platforms to work with each other. You break, or even bend, a standard in a short period of time doesn't allow it to be adopted by all platforms fast enough. There are reasons for delayed standardization. One is to assure that it's truely needed and not just a bad version of what's already out there. Another is to assure everyone (programmers, users, etc.) get's a chance to have a say. Yet another would be to make sure the maturity of the product is where it needs to be. Need I go on? I'm not opposed to a new standard, but it needs to be standard before I will adopt it. Since it is a very immature product, tested as such and still changing, it would be wise to wait for a year or so until it has worked out most of it's kinks and finished it basic "standard. Until then, it's going to be useless and will be like a big ugly zit on UseNET. Later....

BiModem

[cpeterso]

I used to collect old-school transfer protocols, such as BiModem, Lynx, Puma, HSLink, Nmodem, Tmodem. I had about two dozen, but I can't remember the others..

Re:Screw luddites

[mdwh2]

Not to mention that proportional fonts are infinitely easier to read.

How is this relevant? text/plain posts show up in a nice easy-to-read proportional font on my news client. What's more, it's *my* choice of font, as opposed to what the sender thinks I should use..

Some suggestions for a REAL standard

[Rui del-Negro]

How exactly do you reduce something by 40% by moving from 6-bit to 8-bit encoding? I'm not a mathematician, but I believe 2 bits is 25% of 8, so Base64 / UUE has an overhead of 25% over the original data (plus carriage returns that add up to 2 bytes per each line of 60-80 characters, but those are also necessary in yEnc, so no change there). Add the fact that a couple of values need to be escaped and you end up with an overhead of about 1-3% in yEnc

That's at best 22-24% less than UUE / Base64. Definitely not 40%.

That means that instead of taking one hour to download a large file, you take about 50 minutes. Does that difference in time justify having to change every program out there that's designed to handle usenet messages? I'm not just talking about the news readers and browsers, I'm talking about indexers, search engines, on-line news services, etc.?

I don't think so.

The people posting in yEnc may think this is a great improvement, but 80 or 90% of the people downloading binaries simply ignore yEncoded messages. I think you will find yEnc actually was responsible for a decrease in usenet traffic, but it's not because of the reduced overhead; it's because a lot of people simply won't download something that's encoded with yEnc.

Usenet does need to be "upgraded", and that will inevitably cause some problems, but there's no point in going through it twice (once with a "hack" and then a few months later with the real thing).

A new standard should include at least:

• Error-checking.
• The ability to post multiple files, text blocks, etc. per message.
• The ability to identify which part of the original file a specific message corresponds to (for multi-part posts). This should be independent of the size of each chunk, so that the original file can be rebuilt from different-sized chunks as long as they cover all the original bytes. It should also be based on a CRC of the original file, to enable you to rebuild a file from chunks posted with different names (but made from the same original file). This information should be available without having to download the entire message.
• Support for filenames with spaces, any characters and any length.
• Support for 8-bit encoding (or as close to it as possible).
• Redundancy information to enable a file to be rebuilt even if not all parts are available (this would increase the overhead, so it should be optional).

Plus a couple of other things, but these are, IMO, the most important. There are already some standards (MD5, MIME, Unicode) in place that can be used for these things, although some of them would need some tweaking.

The new standard should be tested to make sure it does not create any major problems with existing software. If any relevant problems are found, the standard should not be made public until these problems are fixed (by releasing new versions of the problematic software, that are able to at least coexist with the new standard, even if not fully support it).

Just my 0.2...

P.S. - I'm not a programmer. I used to be, but I had some treatment and managed to recover (or so I thought, anyway)...

Re:Screw luddites

[Wolfier]

Roads were originally designed to carry horse traffic, but that doesn't mean we shouldn't adapt them as necessary when cars were invented.

It is exactly my point - posting HTML on today's Usenet is analogous to putting cars on roads built 100 years ago.

Roads built 100 years ago were built for horses. Roads built today are built for cars.

While engineers have adapted roads and bridges to cars by making their surfaces smoother and increasing their capability of carrying heavier loads, the Usenet hasn't undergone such adaptation yet.

If I want to drive a car on 100-year-old road I can, for example, get an AWD and drives like there's no road. In this case, it is the car that's adapting for the old road. However, no similar adaptation has been gone through HTML yet. (HEY!!! It triggers something - now I think using WML on Usenet is the best thing)

Still, it's not a perfect world and I think the advantages far outweight the disadvantages.

I agree, that the advantage outnumbers the disadvantages. But for me, the disadvantages associated with security and eyesores associated with spams far outweighs the advantages. So you see, whether it is more advantageous or not, it is in the eye of the beholder.

Have a nice day.

Jeremy, I suspect...

[bfe369]

...that one of the reasons that folks grabbed the (incomplete, non-standard) ball and ran with it is because it's the first real effort to *DO* something (in less than a decade's worth of jaw flappin') instead of just talk about it. Kinda like you, Russ, and the rest of nanap speculating endlessly about a good way to verify moderated content - all talk, and still nothing. 1,001 reasons why it shouldn't/wouldn't/couldn't be done, but nothing DONE.

yEnc IMO isn't a good implementation, I won't use it, and it probably shouldn't have ever been done as it has been done, but by god at least the ball got started rolling beyond the proof-of-concept stage, which is more than anyone else has bothered to do. The guy recognized (correctly) that going through the MIME standards process (just look at the history of that outfit) would end up just like nana* - all noise and no results.

Re:Counterpoints to all of Jeremy Nixon's main poi

[Harumuka]

Perhaps you should inform yourself. And I quote:

=yend is again similar to UUencode (end).
The combination =y at the line start cannot occur elsewhere
- because '=' is the escape char and 'Ctrl+Y' is not an escaped character.

Re:Counterpoints to all of Jeremy Nixon's main poi

[Anonymous Coward]

Jeremy Nixon was not saying that =y can occur in the encoded content which is what you seem to think he is saying. Rather, he is saying that =y can occur in USER data. Had you re-read that paragraph, you would have seen.

What's to stop me from writing in my message:

=ybegin

=yend

???

Absolutely nothing. And since yEnc uses binary data there is no easy way to find out if the content between an =ybegin and an =yend is valid. This is an exploit waiting to happen.

Hell, some clients are exploitable with the begin/end (uuencode) faking in message bodies such as Outlook and I'm sure there are others.

Re:Screw luddites

[pyramid termite]

Who cares about linking to Usenet posts? Just being able to link to web sites would be an advantage.

As someone said to me, "There is NOTHING about the web that guarantees a page "is there"."

The point is that it's a convenience, which is just as convenient in a Usenet post.

Some newsreaders actually change plain text urls to clickable links. Without inconveniencing the majority of Usenet readers by presenting a bunch of bracketed crap.

You know, you old school HTML people need to get with the program ... Why post HTML when you can post DIVX files of you talking and people can see you talk and stuff ... 90% of communication is nonverbal. You're making us miss so much!!

Re:Screw luddites

[mdwh2]

So should the web return to the days of pure ASCII?

Typically webpages have fairly complex formatting: multiple columns, or side bars for example. I have no problem if someone wants to use HTML on email/usenet where complex formatting is required - but most ppl just write in paragraphs no different to as if it were plain text.

Should Slashdot disallow any HTML postings? HTML is useful. Look at how it used on Slashdot -- italics, boldface, links, etc.

I personally don't see any great advantage in HTML posting - the main advantage is it's easier to mark quoted text as italics, but that's irrelevant on usenet where the newsreader will prefix quoted text with '>' automatically. And the main disadvantage of taking up more space is irrelevant, if the webpage as a whole is an HTML document anyway. Most news clients automatically provide links if you put a http:// in.

Not to mention that proportional fonts are infinitely easier to read.

What's hard to read are messages in some awkward font or colour. It's especially hard on the eyes when I'm reading a series of HTML messages, all in a different font, size and colour; it makes me think of one of those badly done newsletters where every article is in a different font..

How can you possible be in favor of HTML on the web, yet not in favor of HTML in Usenet? Put it this way: If Usenet were invented today and they included HTML, could you honestly say it would occur to you to say, "you know what would make this better? ASCII only!"

Well there are plenty of ppl who do think that hardcoded fonts/colours specified in HTML is a bad thing so there need not be any inconsistency at all here. HTML should be more for layout, which as I say, is rarely needed on usenet.

And yes, if usenet were invented today, I'd soon be asking for a way to get rid of the sender-specified fonts and colours, and whilst I realise sensible newsreaders will strip them out, I'd be wondering why not just send plain text in the first place.

Also NiceSTEP yEnc Decoder! (Java)

[nicestepauthor]

If you like a newsreader that doesn't support yEnc decoding yet and you run an OS that has Java JDK 1.2 or better you can use your newsreader to download the raw articles then decode the articles afterwards using the standalone decoder NiceSTEP yEnc Decoder. It has a nice drag and drop GUI and is available with source code or precompiled from http://nicestep.sourceforge.net

I've been using this in combination with Pan (before it had built in support for yEnc) for a few weeks now and it works great. And contrary to what others may say, downloads ARE significantly shorter with yEnc encoding.