Slashdot Mirror

I built something similar back in 2000/2001 (back then, such embedded boards sported a 200 MHz Cyrix GXm CPU — man were such embedded boards expensive!). Used that beast in my car as an MP3 player with 30 Gigs of HD space. Look for the string "MPorty" with your favourite search engine. Sadly, the two Linux magazine articles were taken offline since. I'm taking my old MPorty web page online for a while so if anyone's interested...

The general benefits of the suborbital path is that you spend most of your travel time coasting above the atmosphere, not using fuel at all,

There is a DBC implementation for CLOS, the Common Lisp Object System. It doesn't require a preprocessor, of course, Lisp is flexible enough to do such a thing portably.

"language of languages" case in point:

DBC in Lisp.

Except that functional and relational paradigms are the foundation stones of Lisp, instead of subsets or side-effects of something else. Sure, pure functional or pure relational styles can be expressed more easily with other syntaxes, but you quickly run into problems with multi-paradigm programming. The fact that Lisp can be so easily and cleanly extended (or transparently reduced into, via macros) to other paradigms ("structured" style control structures for programming efficiency, OO, and more recently constraints, parallel/distributed programming, and design-by-contract extensions to CLOS).

Right now. Common Lisp CLOS smokes eiffel. Particularly when you load a design-by-contract package into CLOS. Tee hee.

Seriously, Common Lisp's MOP (think java reflection on steroids) make it easily the most powerful OO language out there, extensible to incorporate pretty much any "OO" type construct

And (X)Emacs IS the king of IDEs (Shame it's not itself written in Common Lisp... yet.)

[Lisp dude ducks and runs away...]

This page lists an e-mail address for him, eilts@tor.muc.de. I hope that helps.

yEnc is yet another encoding method for binary files. Unlike Base64 or UUENCODE, it uses nearly all octet values, reducing the overhead from 33% to about 2%. Problems with yEnc No MIME or Back to the UUENCODE mess

A huge mistake is not to use MIME for yEnc. Let me explain why: In the pre-MIME era there was UUENCODE, which had several problems:

• There are no clear delimiters. UU-encoded files can start anywhere in a text message. As a result, some newsreaders incorrectly see attachments where there aren't any. This has been partly addressed by yEnc which uses =ybegin...=yend instead of begin...end. But there's still a chance that the markers appear in normal text.
• No labelling of file types (Content-Type in MIME).
• Charset recoding problems. Note that this is a much larger issue with yEnc than with UUENCODE. UUENCODE is only corrupted on gatways that can't handle some ASCII characters (esp. EBCDIC systems, which should now be extinct). For yEnc, every 8bit charset translation is fatal.
• No standard labelling of split messages. (This is addressed by yEnc, but only within the body.)

MIME provides a proven solution to these problems:

• It clearly labels all data out-of-bound, using Content-* headers. It clearly seperates text from binary data.
• Base64 uses an alphabet that most likely survives all charset translations.
• message/partial provides a standard way to split large messages. It even allows MTAs to split messages on the transport level (granted, that should NEVER happen on Usenet).

Back then, when UUENCODE was state of the art you could just cut and paste a UU-encoded file into your text message. With modern, MIME-aware newsreaders, this is no longer the case:

• Newsreaders might be tempted to apply Quoted-Printable (or Base64) encoding on the text. Yes, I've seen Quoted-Printable encoded UUENCODE attachments.
• Even if they can be convinced to use 8Bit (or Binary), they might suddenly do some charset recoding:
  • ISO-8859-15 and UTF-8 become more and more popular, especially due to the Euro Currency Sign. This means the charset has to be recoded from the systems native encoding (e.g. Windows-1252).
  • Even with plain old ISO-8859-1, DOS and OS/2 newsreaders have to recode from the DOS charset and Mac newsreaders have to recode from the Mac charset.
  Further, there's no way to include yEnc within UTF-8-encoded text, which is slowly becoming the worldwide standard: If you include the data as-is you'll have invalid UTF-8 sequences. (This also applies to other Multibyte Charsets, which are quite common in Far East countries.)
  (Note that this is no problem with UUENCODE or Base64, which only use an unproblematic ASCII subset.)

There are also some smaller problems which should be addressed:

• Using CRC32 as a checksum. There are much better hash algorithms like SHA-1 or MD5 available.
• Bad Extensibility and less features than MIME Content-*: Could be solved by integrating yEnc into MIME or vice-versa (e.g., Content-* headers could be allowed directly after =ybegin and before an empty line after which the binary starts.)

One of the solution, of course, is to embed yEnc into MIME. The first idea is to do that as a Content-Transfer-Encoding.
There have been some arguments against MIME, however, which should be addressed here.

The creation of new Content-Transfer-Encoding values is STRONGLY discouraged. (Quote from RFC 2045)

This is true and there's a very good reason for it: A lot of software needs to be updated to support it.
On the other hand, the situation is no better when MIME is not used: Users would still need new software. If the news client does not support the format, users can just export the message (nearly every newsreader can export a message in source format) and process it with external tools.
You have to ask whether a new, news-only encoding is a good thing. If yes, then it does not make a difference wheter it's embeded in MIME or not.

message/partial MUST not have binary content.

This is because it couldn't be recoded as neccessary on gateways. But with yEnc, recoding would only happen where the message would have been recoded anyway.
It only raises the question whether a news-only encoding is a good thing again. (With yEnc, recoding would only happen where the message would have been corrupted anyway.)

There's no per-part integrity checking for message/partial.

There is no reason why Content-MD5 could not be used on message/partial: RFC 1864 only disallows Content-MD5 on multipart/* and message/rfc822, not message/partial. (The reason for this is that these type can contain data that could be recoded at gateways. This would not happen with message/partial or, if it happens due to yEnc, the message would have been corrupted anyway.)

There's no easy way to find all parts from a single part (i.e. find the message ids of all parts) with message/partial

Neither is there with yEnc. All proposed solutions would both work with a non-MIME-yEnc and message/partial.

You can't use your standard newsreader by pasting the encoded data.

You haven't been scared enough by the yEnc-over-Quoted-Printable and Charset Fun chapter, have you?

Conclusion: yEnc as a MIME CTE is much better than yEnc without MIME. yEnc as a MIME Content-Type

The other solution, of course, would be to introduce yEnc as a MIME Content-Type. It then would have to use the binary 8bit CTE.

This would be a similar approach as used for application/mac-binhex40, which is also defined as a Content-Type. Also note that many compression and archive formats are a Content-Type.

As yEnc contains additional information (such as file name, markers, etc.) which a CTE usually does not, this seems to be the better solution.

Conclusion: yEnc should be a MIME Content-Type. Alternatives

Of course, one should ask what alternatives are there to yEnc (or any other super-Base64 encoding). Not using Usenet

Usenet is, even without the Base64 overhead, a horribly inefficient method to transfer large files. Because of the flood-fill mechanism, the articles are sent to all news-servers carrying the specified newsgroup, even if there's no user that wants the article there.

Peer-to-peer networks, such as Gnutella or Freenet are much more efficient and can transfer binary data as-is.

Conclusion: Binaries should not be sent over Usenet. This is not expected to happen any time soon, however. Use all MIME features

MIME already has most of the features necessary to send binary data over Usenet:

• An encoding: Base64.
• A standard to split messages: message/partial.
• An integrety check: Content-MD5 (MD5 is much better than CRC32.)
• Out-of-band labelling of data types.
• No mixing between text with charsets and binary data.

Of course, you would have to use all features provided by MIME to provide everything that yEnc provides (today):

• Use Content-MD5 on each embedded file.
• Use Content-MD5 on each message/partial
• Use Content-Disposition to transport file names and attributes.
• Use the number and total parameters on every part.
• Use the Message-ID convention described below.

Some features proposed for yEnc, such as assembling a file (message) from different sets of partial messages, could also be integrated into MIME - in a backward compatible way!

There is only one real argument agains MIME: efficiency. The Base64 encoding produces about 33% overhead.

Conclusion: MIME provides a proven solution to send binary data over Usenet. The only problem is efficiency. Link-level Compression

The 33% overhead can be more than nullified by using an compression method over bandwith-sensitive links.

There's a proposal for a MIME-aware compression scheme Assembly of partial messages

See my document about a Message-ID convention for partial messages. Conclusion

MIME already provides a solution to most problems that yEnc is supposed to solve. There's no reason to reinvent the wheel for yEnc, which causes some problems. Only the more efficient encoding remains. This problem can be addressed by introducing yEnc as a MIME compression scheme or by introducing a link-level compression/filtering.

Revisions
2002-03-19yEnc as MIME type would only require 8bit, not binary.

Some minor fixes. 2002-03-04Initial version.
Claus Frber <claus@faerber.muc.de>

But yEnc's bandwidth savings are real, which is a huge win for alt.binaries users. yEnc has been the most-requested feature for Pan over the last month. (0.11.2.90 supports it.) IMO yEnc is the format to use for multiparts right now.

Hopefully yEnc will motivate others to come up with a mime-friendly alternative encoding for Usenet. yEnc Considered Harmful is another yEnc opposition page that suggests mzip compression, but I haven't seen any public discussion of it yet.

If/when such a replacment comes along, Pan will support it too and add an are-you-sure dialog for yEnc postings.

But yEnc's bandwidth savings are real, which is a huge win for alt.binaries users. yEnc has been the most-requested feature for Pan over the last month. (0.11.2.90 supports it.) IMO yEnc is the format to use for multiparts right now.

Hopefully yEnc will motivate others to come up with a mime-friendly alternative encoding for Usenet. yEnc Considered Harmful is another yEnc opposition page that suggests mzip compression, but I haven't seen any public discussion of it yet.

If/when such a replacment comes along, Pan will support it too and add an are-you-sure dialog for yEnc postings.

It should be pointed out that this site, linked from yENC's own website, goes into more technical detail regarding the technical flaws of yENC. The fact that it's linked from yENC's own site is proof that the author is at least familiar with the concerns that people have with his implementation.

I personally still find it difficult to argue against the article author's point that THERE WAS NO RUSH to force yENC out the door in such an unpolished form. After so many years of waiting for something better, why ignore the recommendations of those you are trying to help?

< tofuhead >

More reasonable alternatives would include:

• Modula-3 , in which is written Spin.
• Or perhaps Oberon, which has been used to construct several OS-like environments.
• Or perhaps even Eiffel, whose Design By Contract approach makes claims that C++ can provide anything describable as rock solid look very silly.
• Based on the number of language compilers being built using ML, I'd think it to perhaps be a candidate. The ability to do heavy-duty static type inference would, not unlike with Eiffel, make claims of C++ being "rock solid" look pretty sad.

Yes, these languages don't have syntax that slavishly resembles C. But it's not as if the actual semantics of C++ or Java are actually that much like C...