Slashdot Mirror

Simply stating "Corporatization" is a massive mischaracterization and oversimplification of describing the situation.
Here is a peek in history from 2005 on the IETF mailing list itself and how IETF tried to sabotage the ratification of Zeroconf (Apple's Bonjour is the best known implementation of the Zeroconf protocol). This isn't simply "Corporatization" as both Apple and Microsoft are fighting and some in the the IETF actively trying to undermine it under the guise of simply offering alternatives (that nobody wants or plans to implement and is broken by design).

Stuart Cheshire is the creator of Zeroconf and calls them out directly on the IETF mailing list in 2005.
http://www.mhonarc.org/archive...
http://www.mhonarc.org/archive...
http://www.mhonarc.org/archive...
http://www.mhonarc.org/archive...

Simply stating "Corporatization" is a massive mischaracterization and oversimplification of describing the situation.
Here is a peek in history from 2005 on the IETF mailing list itself and how IETF tried to sabotage the ratification of Zeroconf (Apple's Bonjour is the best known implementation of the Zeroconf protocol). This isn't simply "Corporatization" as both Apple and Microsoft are fighting and some in the the IETF actively trying to undermine it under the guise of simply offering alternatives (that nobody wants or plans to implement and is broken by design).

Stuart Cheshire is the creator of Zeroconf and calls them out directly on the IETF mailing list in 2005.
http://www.mhonarc.org/archive...
http://www.mhonarc.org/archive...
http://www.mhonarc.org/archive...
http://www.mhonarc.org/archive...

Simply stating "Corporatization" is a massive mischaracterization and oversimplification of describing the situation.
Here is a peek in history from 2005 on the IETF mailing list itself and how IETF tried to sabotage the ratification of Zeroconf (Apple's Bonjour is the best known implementation of the Zeroconf protocol). This isn't simply "Corporatization" as both Apple and Microsoft are fighting and some in the the IETF actively trying to undermine it under the guise of simply offering alternatives (that nobody wants or plans to implement and is broken by design).

Stuart Cheshire is the creator of Zeroconf and calls them out directly on the IETF mailing list in 2005.
http://www.mhonarc.org/archive...
http://www.mhonarc.org/archive...
http://www.mhonarc.org/archive...
http://www.mhonarc.org/archive...

Simply stating "Corporatization" is a massive mischaracterization and oversimplification of describing the situation.
Here is a peek in history from 2005 on the IETF mailing list itself and how IETF tried to sabotage the ratification of Zeroconf (Apple's Bonjour is the best known implementation of the Zeroconf protocol). This isn't simply "Corporatization" as both Apple and Microsoft are fighting and some in the the IETF actively trying to undermine it under the guise of simply offering alternatives (that nobody wants or plans to implement and is broken by design).

Stuart Cheshire is the creator of Zeroconf and calls them out directly on the IETF mailing list in 2005.
http://www.mhonarc.org/archive...
http://www.mhonarc.org/archive...
http://www.mhonarc.org/archive...
http://www.mhonarc.org/archive...

I don't know if MHonArc is still being actively developed, but ListServ (my fav) has a decent web managment interface and MHonArc does a nice job of archiving posts in any number of methods. We implemented this in the day at Bell Labs for any number of internal mail lists (Bell Labs 'had' a lot of internal mail list activity).

http://www.mhonarc.org/

This also works really well for server processes that send automated email notifications with exit statuses, results, etc.

It should be pretty straightforward to get all the board members email accounts with the town/district and use these accounts for all official mail. For discussions, use a listserv that allowed self-registration (for reading, not sending to the list), plus auto-archive software such as Mhonarc. Forget FOIA requests, anybody can access any of the emails any time they wish. I don't think this can replace meetings, which would still be used to hear public input and to vote, but it would speed some communications. I don't have any ideas about how to deal with the members who are not online much, of course.

http://www.mhonarc.org/ is a perl script that will recurse though folders of emails, unpack the attachments and generate html versions of the mail, with links. A simple find can them delete the attachments, or you could just keep them as the result will be (mostly) smaller.

Okay, so most of the people here have wasted your time trying to convince you that "storage is cheap" or that there isn't a good reason to store all that e-mail, let alone try to organize it all. I'm with you, not them. It's the fricking 2000s. It should be easier to archive this stuff and organize it if you *want* to.

I've always wanted to do something about my messy mail archive of mbox files (dates back to the 1990s), but I dreaded the thought of coding something up from scratch given all the quirks of e-mail formatting. I had high hopes your post would elicit some sage advice from the readers of /., but so far I don't see much other than the good mutt+ruby solution. In frustration, I've started looking but I haven't found much either. For what it's worth, here's what I've go so far:

1) There are plenty of commercial solutions that promise to do everything for a low price (e.g., MailSteward for OS X looks pretty good and has a free trial up to 15000 messages). Maybe. But I'm cheap and will exhaust the fully free solutions before spending money. Most of them are more focused on mailbox conversion/migration (e.g., Emailchemy) than actual filtering/archiving.

2) Free / some assembly required:
archivemail - mostly for date-selection of messages and archiving/compressing. Doesn't help with attachments. Python.
archmbox - more capable than archivemail. Can do filtering based on date, header field matches, etc., copy selected messages and compress to archive. Perl. Closer.
MHonArc - converts mbox to HTML files with links to attachments. Meant for mailing list archiving, but it should work the same for a personal mailbox. Perl. There's also an OS X front end for it.

The HTML approach isn't ideal, but that could be a convenient way to browse through the archives (e.g., toss it all up on a password-protected web site and your mail archive is available anywhere, like your own personal and backed-up GMail), and a contributed program in the MHonArc distribution can turn an MHonArc archive back *into* an mbox file, which might let you do some modifications to the HTML files and linked attachments with scripts and then backconvert them after.

I haven't tested any of these, but I think I'll try MHonArc and see how it goes.

I actually use mharc, http://www.mhonarc.org/mharc/, on my private server to archive various public lists and work-related email since it provides searching capabilities.

For work-related email, I have nmh folders I file message to for the various projects I work on and then have a cron job that runs each night that uses mharc's mh-month-pack script to copy the mail into mharc's archive area.

This system provides me an MUA-neutral way to read and search email. Since mharc keeps an mbox formatted version of all data, I can import the messages to any MUA I want, however since I still have the nmh folder data, I've never had the need.

The other advantage of my setup is I can following mailing lists w/o cluttering my inbox. I have a separate mail account I used to subscribe to lists of interest, and I archive the messages on my private server for reading and searching whenever I want.

Er, someone has tried this before. Take a look at MHonArc.

I've been looking for a solution to this same problem for a long time. For now, I still have all my email from the last 10 years stored in my MS Entourage database. Scary. What I would like to do is convert them all to HTML and/or plain text so that it's searchable, and won't need a special app to view them. Who knows what formats will still be around in 10, 20, 30+ years? But I suspect HTML and plain text will survive.

What I don't like about the mbox archiving technique is that you have to import the mbox into an email app to read it. To me, that sounds like a chore because I'd want to keep those messages separate from my current email box, and then there's processing time it takes to import it, etc.

It seems to me that an ideal situation would be that the file system can search it (i.e. Spotlight in Mac OS X 10.4 Tiger) or use a web browser.

There is a solution for archiving emails to HTML: MHonarc. I've played with it in the past and it does its job fairly well. What annoys me about the script is how difficult it is to customize the output, especially if you want it to validate and use CSS.

There's nothing else out there that I could find to my satisfaction. Maybe some resourceful developers want to embark on a new OSS project??? :)

MIME isn't damage, MIME is a hack to fix the crippled SMTP message format. Maybe you are only interested in sending ASCII text messages-- and that's very hardcore of you and all-- but the rest of the world is interested in sending pictures, documents, text in languages other than English (well over a hundred, and you're fucking well right the "Standard" should support them), etc., and your underdeveloped message format just can't properly deal on its own. Maybe you should read up on the subject.

Text itself isn't a drawback-- XML is generally represented as text-- but a message format that is defined only for transmitting text just doesn't cut it now that we're out of Green Terminal Land and into the World Where People Use Computers to Do Stuff.

And you're missing the point of my remark about XML libraries. The problem is not that parsing email is hard, but that there's no standard for an internal representation of an email message, and if there was it would probably be completely non-interoperable with the rest of the world. XML has the DOM and SAX, among others. This means a whole world of functionality, in the form of libraries and technologies that understand XML via DOM or SAX, is available to the program author. You can transform the message into another format using XSLT, access and modify the message content and headers with XPointer, find references to and merge in external resources with XInclude, extend the message format using namespaces (thereby allowing anyone who doesn't care about your extension to safely ignore it), transform the message (with XSLT) into XHTML and provide rich formatting with CSS (both of which can be found in reusable libraries), and so on and so forth.

You use XML, you get all of the above essentially for free. You go with some application-specific grammar, and you can either limit your email to plaintext or you can reinvent all of those wheels. But I know how much you reet haxorz hate usability and interoperability... maybe we can hook you all up with some nice teletypes.

http://www.namazu.org/

Namazu is a open source full-text search engine. It has various document filters (HTML, Mail/News, PDF, MS Word, Excel, Powerpoint, man page, TeX, DVI, PostScript etc...) and mharc web-based mail archiving system adopts Namazu as its search engine.

In the specific area of online discussions in local communities we need your advice. Related discussions on this has occured on the Democracies Online Code Network e-mail list for civic-minded techies.

We use e-mail lists. They work. Our participants love them. They need to work better with the web. We do not need a web-based system that treats e-mail participants as second class citizens. Our thousands of users won't make the transition - and we are not going to sacrifice our sustainable non-profit model that has worked for eight years.

In an ideal world someone would create an e-mail/web system akin to a cleaner, crisper Yahoogroups but something better that you can host on your own domain.

What we have:
Mailman with additional archives using Mail-Archive. (We are moving our last few lists off Yahoogroups.)
Basic web pages with forum information, hundreds of Minnesota-specific political links, and special election/candidate link directories.

What we need in term of priority:

1. Advanced Web Archives and Subject Line Syndication - Improved web access to our e-mail forum archives including the ability to post via the web to -recent- messages by "no e-mail" members, the ability to automatically display via RSS the most recent subject lines from our various lists on our home page/other key web pages to posts in the archives. Hypermail, Mhonarc just don't cut it. They were great in their time, but we need something that takes advantage of MySQL, allows for linear display of posts in the same thread, and other tools. More on this ....

2. Member Preferences Page - A single page like Yahoogroups where someone can control their settings on the all the lists they subscribe to on our server. We'd also like to allow people to recommend new e-mail lists for their local communities and essentially reserve a spot by letting us know that they are interested in a specific city/county/region or statewide public policy issue. We do not open community discussions without at least 100 participants and have an extensive public outreach process that goes with each new lists (i.e. online and in-person recruiting). If we recruit 50,000 "e-citizens" across Minnesota we need to use technology to help shape our forum development priorities.

3. Member Directory with Archive Links - (Again, we are not interested/able to use a web-centric conferencing system) This is where the web can complement our e-mail environment. I'd like each member to have the option to share information about themselves (our rules for posting including signing your real name, we have to use personal accountability in our model for online political discourse or everything would be pure crap). I'd like each e-mail that goes through the list server to insert their member directory page URL. From the member-directory page I'd to present both the information provided by the participants but also links to their recent posts across our various forums. And perhaps ...

4. Participant Ratings - With unmoderated mailing lists, rating each post before it is delivered is impossible. Even if we moderate our lists, a multiple moderater bottle neck among our mostly non-techie audience would cause major delays in discourse. So ... one idea is to allow participants to optionally vote +1 substance, -1 for style for any post after it is distributed. We don't want to create a situation where people simply vote against people of other ideologies (we have a cherished and extremely rare cross-political spectrum audience) so some sort of forumula would have to be developed to give various weight to votes (i.e. repeat votes by one individual against another count less over time) and always bring the rating toward zero over time. Oh - why do this? While our unmoderated lists to have forum managers who have the power to sanction participants who violate our rules and guidelines, we ultimately believe that self-regulation, and group self-governance is our strength. We walk on a tight rope between chaos and control in order to keep and build our participatory civic audience based on our democratic and community purpose.

5. E-Newsletter Distributed Content Management System - We have currently have 4,000+ people on our general announcement list (over next five years we'd like it to raise it to 50,000 or 1% of Minnesotans). We are planning a once or twice monthly e-mail newsletter with various content sections. I'd like to give our volunteer editor the tools to allow other volunteers to submit content (i.e. event lists, Minnesota political history this month, quotes of the month from our forums) on a regular basis into key sections of the newsletter and assuming that some content will be to long for e-mail newsletter format, something that integrates with a longer web section. 6. Mailman Advancements? Or another list packages. As an organization we'd like the ability to send one message to everyone on one of our lists without double posting. For our volunteer list managers we need the ability to quickly delete all the non-member (mostly spam) posts in one or two clicks and not have to click and select every post. What list packages do people recommend?

If you actually read this far, you should join the DO-CODE e-mail list that I mentioned above.

Cheers, Steven Clift

One, create an intranet (yes, I hate that word too, but everyone knows what I'm talking about) webserver. Use something like MHonarc to archive the mail, and then install a search engine on the webserver.

Two, and I like this way better, write a couple of tools to handle this (perhaps based on MHonarc, heh) which will stick your mail in a database (Interbase, Postgresql, Mysql, whatever) and then do yet another web (or otherwise) application which will let you read them out and search them, interactively.

I personally hold onto my mail by renaming my mailbox (For instance, on the 240sx.org mailing list) to 240sx-preYYYYMMDD, then creating a new 240sx mailbox. I use Mozilla on win32 for email (despite the amazing crashiness - It feels like I'm using Corel software or something) and it has a fairly decent search facility.