Unarchiver Provides LGPL RARv3 Extraction Tool

An anonymous reader writes "Free software to support the RARv3 archive format has been listed on the FSF's High Priority Projects list for some time now. We've always had ways to create and extract free archive formats, using tools like GNU tar and Info-ZIP. The RARv3 format is proprietary, so we don't want it to replace these tools, but it's not uncommon to see it used for distributing multimedia files over the Internet. That means the lack of free software to extract RARv3 files has been sorely felt. We're happy to share the news that there's now a free software project to fill this gap, and we can mark this item as done. The Unarchiver is a small collection of software written by Dag Ågren."

Comments

Multimedia files?

[Anonymous Coward]

Also known as: Porno movies.

Re:Multimedia files?

[Anonymous Coward]

I thought those were flv files (Instant gratification while streaming etc)

Yay piracy!

[Anonymous Coward]

Compressing my HD rip to save 5MB on a 50GB download!

Re:Yay piracy!

[dougmc]

Compressing my HD rip to save 5MB on a 50GB download!

In the case of movies, it's not so much that it saves space, it's more that it breaks the large file up into more manageable chunks and it also gives you checksums to know if something got corrupted.

This isn't particularly important for distribution methods like bittorrent which provides it's own checksums and doesn't have problems with files over 2 or 4 GB, but for some other distribution methods it does make a big difference, especially when you throw par2 files into the mix for correcting problems.

Re:Yay piracy!

[blueg3]

There are perfectly cromulent tools for that without rar: you can just split a file with split and compute "checksums" with md5 and sha1.

It's of course completely unnecessary for BitTorrent, which has an excellent system for breaking files into pieces and hashing those pieces. In fact, you might as well make a system for piecing and hashing based on torrent metainfo files.

Re:Yay piracy!

[retchdog]

in unix, yes. in the windows world, there's only two levels of difficulty: drag-and-drop or impossible. most users have winzip or 7zip or whatever and pirates have traditionally favored rar. thus, the rar standard emerges and metastasizes so that programs like vlc support it natively. kind of silly, but it works.

if you want to cry, follow this link and count the number of shitty gui hacks that do nothing but "split" and "cat": http://www.google.com/search?q=split+file+windows

Re:Yay piracy!

[mikael_j]

The problem with RAR files and Bittorrent is "scene snobbery". Basically, people reason that since all scene releases are released in RAR archives then all media distributed should be distributed as multi-file RAR archives. This can be seen on invite-only torrent sites that demand that all upload regardless of origin be split into multiple RAR files.

And should someone point out that it's really only when downloading files using FTP and other non-checksummed protocols that this is necessary then they will be screamed down by the know-nothings. And then there's the whole thing where they seem to insist on using 25 or 50 MB files instead of larger chunks. If you're grabbing files from an FTP/HTTP server dedicated to sharing these large media files chances are that the server is able to push the files to you faster than say, 10 Mbps, and we'll assume that's as fast as your connection is, that means you can grab a 250 MB file in approximately 3m30s if we assume no overhead, if we a assume you have a regular uncapped g.dmt ADSL connection (8/0.8 Mbps) with the typical EoATM and TCP overhead for your transfers then we're still talking less than five minutes for a 250 MB chunk. Contrast this with people splitting things into 5 or 10 MB chunks back when a lot of people were still on modem connections, a single 5 MB chunk would take more than ten minutes on a good day...

Re:Yay piracy!

[king neckbeard]

multi-rar archives in torrents? Just thinking about that makes my blood boil.

Re:Yay piracy!

[jonwil]

The advantage of splitting into small files is that its easier to upload the content to hosting services like Rapidshare, Megaupload etc etc etc.

Re:Yay piracy!

[nedlohs]

In the windows world rar is one of those perfectly cromulent tools, why use something else when the current tool works just fine.

And given it's the piracy scene that the tool isn't free software is completely irrelevant.

I'm pretty sure BitTorrent isn't the primary mechanism of the initial distribution of pirated material. I guess maybe for the movie world where you don't need people who have learnt how to remove disk checks and other copy protection schemes and hence anyone can make the initial copy.

Re:Yay piracy!

[Anonymous Coward]

There are plenty of reasons to use RAR. For our windows friends It's the most reliable commonly available compression tool especially when it comes to recovering slightly damaged archives. It compresses better than most other tools available on the platform. It also has a pretty decent UI backing the windows version.
I'm not sure how v3 works. I havn't used it personally but I assume it's v3 cause it's even better.

Re:Yay piracy!

[Anonymous Coward]

Yep, you can do alot of this without rar. Although recovering a damaged bit of an archive takes a little more than a checksum.
It has it's strengths (recovery being a big one - cd's, floppies, hd's, dvd's all go a little bit bad usually before they become totally junk, very good compression).
It also has it's weaknesses (slow as hell, not 'standard')

It comes down to choosing the right tool for the job. And on a windows system, I'd use rar to archive something long before I'd look at zip or other tools on the platform.

Re:Yay piracy!

[Arlet]

You can't repair corrupted/missing archives with md5/sha1. The .rar format is perfect for usenet, where missing parts is very common.

Re:Yay piracy!

[wagnerrp]

If using a pirated piece of software (such as WinRar) is not a problem for such individuals, why go through the effort of writing an open source library as a replacement?

Re:Yay piracy!

[Anonymous Coward]

pirates have traditionally favored rar.

Your hearing may be faulty. Pirates have traditionally favored har and yarr!

Re:Yay piracy!

[Anonymous Coward]

And given it's the piracy scene that the tool isn't free software is completely irrelevant.

What about .rar compressed scanned (pirated) books on in DjVu format? I'd say they are critically important for free software hackers.

Re:Yay piracy!

[Anonymous Coward]

"People" is not a uniform group. One group is running Windows, and have no problems using a pirated copy of winrar, as much of their software is pirated anyway. When you're used to Winrar, you use it for everything, you don't use a different program for files that aren't pirated.

The other group is running an open source system, and want to be able to extract files compressed by the first group..

Re:Yay piracy!

[meerling]

Anyone who transferred files before the days of broadband favored rar because it made smaller files than zip or arc, and it also was quick to support multipart archives which was useful for those sites that limited how large a single file could be. Had no more to do with piracy than BBS or Internet. Of course the pirates like it too...

Re:Yay piracy!

[Anonymous Coward]

"Second: There also exists a reason for multi-rar torrents. I have seen on torrent sites archives where a part of the multi-rar has been corrupted (in many cases with a different size) so it passes crc-checks from rar, but creates a subtle corrupted file, that you do not at a first glace may see if it works or not.
In this cases you only have to try to fetch that part, and does not need to redownload the whole image."

A wellbehaving bittorrent client will do the same. I also had filesystem corruption (an xfs bug in an early 2.6 kernel). Downloaded the torrent again, pointed the save location to the place with the corrupt file, forced a recheck in azureus which found chunks of the image that were corrupt and only downloaded those blocks.

Re:Yay piracy!

[Anonymous Coward]

FTP is TCP, which is checksummed at the packet level. Broken packets will be discarded by the OS and retransmitted. If you need checksums on FTP, you're doing it wrong.

Re:Yay piracy!

[Anonymous Coward]

What it is good for is transforming the file into something that's not recognizable as the copyrighted material. It may be irrelevant when using BitTorrent, but there is a significant "scene" that utilizes RapidShare and its ilk to distribute files since they allow the uploader to make money. But those systems also have methods that allow copyright holders to file claims to get their content removed. As soon as they've done that, any upload with the same hash will be rejected. But if you break it up into a password-protected RAR archive, the hash will be different and there's no way for the upload host to reconstitute the file to check the hash. And, as a bonus, multiple files means multiple downloads and more money for the uploader.

There are search engines that allow you to find files uploaded to the various file hosting sites and you can find just about anything these days. The majority of that illegal content is RARed.

Re:Yay piracy!

[TheRaven64]

That's what par2 is for. It splits the file up and gives you a set of N files where you can reconstruct the original from N-M (depending on the redundancy settings) of them. Implementations available for most operating systems, including one GPL'd one. It's been a few years since I checked binary newsgroups, but par2 was pretty ubiquitous back then.

Re:Yay piracy!

[TheRaven64]

What it is good for is transforming the file into something that's not recognizable as the copyrighted material

Really? Given that no one other than pirates seems to use RAR, I'd have thought it was a red flag saying 'look, illegal stuff here!'

Re:Yay piracy!

[xaxa]

I'm sure Zip could do multipart archives -- that's how I used to transfer large files on Windows machines using only floppy disks.

Re:Yay piracy!

[xaxa]

PAR is used more often on Usenet (or it was last time I looked, which is about 2-3 years ago).

PAR leaves the original file untouched, and creates extra redundant data based on it.

Re:Yay piracy!

[fuzzyfuzzyfungus]

The most jarring place I've ever run into RAR is by a few companies, mostly random hardware OEMs for forgettable routers and such, who understand their legal obligations under the GPL; but clearly are totally disconnected from any cultural conventions of OSS. You go to their GPL page and boom, just a giant .RAR of the directory in which their firmware guy built the firmware.

Re:Yay piracy!

[grumbel]

There are perfectly cromulent tools for that without rar: you can just split a file with split and compute "checksums" with md5 and sha1.

RAR can not only have a checksum, but also optional redundancy, so that it can automatically fix a few flipped bits. RAR is also able to extract incomplete archives without problems unlike say 7zip where an incomplete archive becomes completely unusable. Unlike the gzip/tar mess RAR is also able to seek, thus you can extract the last file in the archive without first uncompressing, which is extremely useful when dealing with big archives. RAR also compresses extremely well.

Simply put, I haven't seen any other compression format that has all the features and robustness of RAR.

Re:Yay piracy!

[maxume]

7-zip will join split files. I think WinRar does too (I don't mean multi-part Rar files, I mean uncompressed chunks).

So the problem is more that people who have never thought about it think that Rar is the way to do it.

Re:Yay piracy!

[Hal_Porter]

Because the free software cultists can't read pirated coding books for religious reasons they don't grow up to be 3l33t coderz like the pirates, so they can't solve the problem.

Re:Yay piracy!

[Kjella]

Yes, but it's RAR + PAR, because this is another hack on top of a system never designed to do binaries. Long story short, any "large" file - originally 60kb and not much larger today - is split and transmitted as multipart binaries. This has nothing to do with RARs, every large file is that way. A lot of Usenet software won't download incomplete files because it means it should wait for more parts to arrive at the server. It doesn't know that with PAR files you actually have enough to recover the whole file anyway. So in practice many people download the *complete* RAR and PAR files they can get, then use the PAR files to recover the missing RAR files.

Maybe an illustration is easier:

Movie.avi [94/100]
Par1.par [7/10]
Par2.par [10/10]

Do we have enough parts? Yes (94+7+10 > 100), but many people can't save Movie.avi and make this work.

Movie.rar [10/10]
Movie.r01 [10/10]
Movie.r02 [10/10]
Movie.r03 [10/10]
Movie.r04 [4/10]
Movie.r05 [10/10]
Movie.r06 [10/10]
Movie.r07 [10/10]
Movie.r08 [10/10]
Movie.r09 [10/10]
Par1.par [7/10]
Par2.par [10/10]

Now people with "stupid" software download all parts except r04 as well as par2.par. We now have 90+10 parts and can recover r04. I won't bore you with old history, just say that everything is a hack to transfer over a system originally designed to only deal with ASCII text. It works, but it'll never be pretty.

Re:Yay piracy!

[EdZ]

Then don't even think about how much h.264 video is encoded 1-pass CBR to fit on a CD/single-layer DVD, rather than using a suitable CRF like a sane person. Or worse, how many people still encode with xvid for playback on anything other than a 10-year-old handheld player.

Re:Yay piracy!

[jedidiah]

So is a format where parts of the file can be missing and the rest will chug along just fine.

They use such formats for "broadcast". Being able to split and join them with "split" and "cat" and use the resulting fragments is a very handy thing.

Re:Yay piracy!

[jedidiah]

No matter how much you whine and get all "high and mighty" and "snooty" about it, it's really hard to argue against a 10x improvement in encoding speed when the target device doesn't really benefit from any of the "advantages" of the more computationally complex format.

Re:Yay piracy!

[advocate_one]

yup... I've seen those and also I've seen multi-rar files with par files as well in torrents!!!

Re:Yay piracy!

[Per Wigren]

All scene movie and music releases use rar -m0, aka "store mode", aka "without any compression at all".

Re:Yay piracy!

[Ash-Fox]

Another nifty thing is that RAR actually supports Unix permissions in it's container. So it's pretty useful on *nix systems when you need to archive a large set of directories, maintaining various permissions (like 'executable').

As you pointed out with seeking, this is useful for when you want to extract specific compressed 'backup' data quickly. The fact it maintains Unix file permissions makes it a viable solution on *nix.

Re:Yay piracy!

[Anonymous Coward]

I'll give you that tar is not great for random access -- it was designed for linear media after all -- but the rest of this is just an argument about how RAR does everything in one program, which many people (particularly UNIX users) see as a detriment, not a benefit.

For example, it's perfectly possible to create an archive, compress it, split it, create parity files, and calculate checksum. And if you do this with separate tools you can change the format of each intermediate step and/or their order to give you the desired behavior. For example, tar/cpio are very simple formats that don't require much memory to use and are compatible with linear-access storage, but you could easily replace a tarball with a loopback block device, or an ISO file, which are indexed and allow random-access. You could choose to compress the entire archive file after the fact for efficiency in space, or your could compress each component file individually for efficiency in single-file access speeds. You can choose gzip compression for speed or xz compression for size. You can split the file into parts and generate parity files after or before compression. You can generate checksums with whatever method(s) you want at any stage in the process. You could through in any number of encryption options at any stage in the process.

RAR can't do half of that. Even the parts it can do, it can't do with any flexibility or modularity. That makes RAR simple, but it also means that if RAR doesn't fit your use case you're just out-of-luck, and that RAR is nearly impossible to update if one of the parts is improved, at least without making it entirely incompatible with old versions. You can't pick and choose the parts that would work for your application, and you certainly can't meet any special needs or opportunities. For example, if I am writing to a tape drive, using an archive format with the index at the end is a *terrible* idea and much slower than tar or cpio. Or maybe I've got a custom compression algorithm that works for the particular type of data I'm sending -- RAR sure won't let me substitute that instead of its compression.

The idea of stringing together a particular set of archive/compress/split/parity/etc. operations to form a standard is not bad -- it's great to have compatible, easy-to-use systems. But putting all those together into a single, monolithic bit of software -- particularly a proprietary one -- is just 6 layers of vendor and format lock-in, which is rarely beneficial in the long term.

Re:Yay piracy!

[blueg3]

You can't, but there are perfectly normal and flexible tools for doing that, too.

MD5 and SHA1 are solely for detecting corrupted data (and also serve well as an index). You don't need to have a single tool do everything.

Re:Yay piracy!

[grumbel]

For example, if I am writing to a tape drive, using an archive format with the index at the end is a *terrible* idea and much slower than tar or cpio.

Yeah, but so is using a linear storage archive format for a random access media. Tar certainly has its uses, but that it is used for distributing source code and other stuff on Unix systems is really nothing more then a historical accident, a proper format designed for that use would look very different.

The simple fact is that all the shiny modularity of combining tar, gzip, bzip, par, gpg, cat, split and whatever to reproduce rar's features still doesn't give you basic random access to the archive, something I consider an absolute critical must-have feature for any archive format, as otherwise all the cool user friendly things, such as mounting an archive transparently into your directory tree become impossible or at least extremely slow when dealing with large archives.

I mean its not even rocket science to see the limitations of tar, take a directory of a few tens of gigabyte, tar.gz it, and then let tar show you all the files in the archive. Very basic usecase and tar utter fails at performing it at anything close to tolerable speed.

Re:Yay piracy!

[ashidosan]

Indeed. And then, when one of these services randomly decides they don't want these files (which happens often), you're stuck with no parts 14-17 of 31 of your download.

The advantage of splitting to small files USED to be to post them to Usenet.

Assuming Usenet ever existed, that is.

Re:Yay piracy!

[drooling-dog]

if you want to cry, follow this link and count the number of shitty gui hacks that do nothing but "split" and "cat"

Oh lordy, if anything typifies the Windows ecosystem for me, this has to be it. I can't count how many posts I've seen on Usenet discussing where to find shiny software that essentially does nothing but "cat file.avi.* > file.avi". I'm sure that such a thing must be possible from the command line, but the majority of Windows users are afraid to venture there.

Re:Yay piracy!

[mister_playboy]

Unlike the gzip/tar mess RAR is also able to seek, thus you can extract the last file in the archive without first uncompressing, which is extremely useful when dealing with big archives.

This is entirely dependent on how the archive was created. A "solid" archive offers better compression, but does not allow random access. Both 7z and rar support solid archive, the main difference is the default creation behavior: solid archives in 7z and non-solid in rar.

Re:Yay piracy!

[WuphonsReach]

PKZIP multi-part was extremely rudimentary and you ended up with stupidity (due to the underlying design of a ZIP file) where you had to insert the last disk first.

We used to do backups to multiple floppies with PKZIP, it was not pretty.

Re:Yay piracy!

[idontgno]

I'm sure that such a thing must be possible from the command line,

copy file.avi.* file.avi

or, if you are a bit paranoid and want to guarantee append order, you can fall back to the original (DOS 3.x) syntax:

copy file.avi.1+file.avi.2+file.avi.3 file.avi

But yeah, most Windows users aren't even aware of command prompt. Being an ancient and crusty user of DOS (back to 2.0) makes me a 133t h@x0r, apparently.

Re:Yay piracy!

[WuphonsReach]

PAR2 is still useful for archive media (CDs, DVDs, writable Blu-Ray, or removable disks). Since it doesn't care about the source file format, it works in more situations and the data can be left on the disk, usable as-is without decompression.

Combine PAR2 + dd_rescue and as long as the disk isn't completely non-functional, you have good odds of recovering data (if you have enough repair blocks). Setting aside 5-15% of the disk capacity for recovery data is not a big deal in most situations and will protect against most forms of loss where sectors go bad or bits get flipped.

The only downsides are (a) QuickPAR is not multi-threaded and (b) speed as modern CPUs can only calculate the recovery data at a rate of about 1 hour for 4GB of data with 20% redundancy and (c) the tools don't understand sub-directories. Unfortunately, the author of QuickPAR never managed to finish a PAR3 design which might have been a lot faster, or to fix the issues with sub-directory handling during repairs.

Re:Yay piracy!

[WuphonsReach]

The ballpark I used back in the 56k days was an average of about 15MB/hr over dial-up. 33.6k was 12MB/hr and 56k was 20MB/hr, but you usually got speeds / throughput somewhere between those two numbers. In comparison, a T1 (1.5Mbps) can push about 500MB/hr in each direction.

Re:Yay piracy!

[MrNemesis]

It goes even further than that - back when I was first getting into and understanding computers, I saw a bunch of warez on WinMX labelled as "packaged with Wrapster", stating you needed to go and download wrapster (which, I'm sure you're surprised, was 90% malware, 9% GUI and 1% features) to unpack them.

All wrapster did was chop off the .zip or whatever file extension and append .mp3 instead (since apparently you could only share MP3 files on napster), simply because default windows settings remove your ability to see, and thus change, the file extension... because that might "render the file unusable" in microsoft's words.

I keep cygwin + filemagic on my windows computer at work and even fellow team members are astonished when I "recover" files that have had their extensions lopped off. In my first job I astonished my boss by "recovering" about 3 million TIFF files across hundreds of directories that had had all their file extensions chopped off (his beloved "mass file renamer" wouldn't work on files without an extension). As the saying goes, linux makes the easy things hard, the hard things easy and the impossible things possible. As the other saying goes, those who do not understand UNIX are condemned to reinvent it, poorly. Both of these are tear-inducingly true on such simple operations.

Re:Yay piracy!

[fast turtle]

Strange comments about 7zip being unable to deal with damange/incomplete archives because I was viewing/extracting single files from incomplete achives just a couple days ago. So long as the part is complete, 7zip can deal with it and no even WinRar doesn't deal well with >100 percent downloaded part because normally critical data is missing to even open the archive.

Re:Yay piracy!

[wolrahnaes]

In the case of movies, it's not so much that it saves space, it's more that it breaks the large file up into more manageable chunks and it also gives you checksums to know if something got corrupted.

But we already have things for that which do it better. Bundled checksum files in the normal SFV, MD5, or SHA1 formats handle detecting corruption and PAR2 can not only detect but also repair files. Hell, PAR2 can even split and reassemble files when needed (though it is NEVER needed, since every protocol in use today either transfers things in chunks no matter what (BitTorrent, Usenet) or supports retrieving arbitrary portions of a file (FTP, HTTP) to retry a damaged chunk.

Basically there is absolutely zero reason for the continued use of split RAR files, the "scene" continues to use them solely on inertia.

I wish there were better legitimate ways to do video-on-demand. Dealing with the pirate scene and their stupid standards is probably the worst part of having a decent HTPC setup these days. They'll go out of their way to demand a specific type of encoding solely to retain compatibility with a hardware Xvid player only made for three months in China, but they won't give up on useless extra layers of archive formats.

Re:Yay piracy!

[wagnerrp]

I'm appalled by the whole concept of choosing your file size to fit on a CD or single layer DVD. Considering hard drive storage is cheaper than both of those recordable mediums, why bother?

Re:Yay piracy!

[bluefoxlucid]

Actually TAR is used for the distribution of source code and packaged binaries (rpm, deb), which is excellent: you have no need to do anything except 1) ball these up into a single blob; and 2) dump absolutely everything to the hard disk. These are not things that you pick apart; they're delivery vehicles, like freighter trucks. For archives that you want to mill through, zip/7z archives are better.

Re:Yay piracy!

[WhiteDragon]

if you want to cry, follow this link and count the number of shitty gui hacks that do nothing but "split" and "cat"

Oh lordy, if anything typifies the Windows ecosystem for me, this has to be it. I can't count how many posts I've seen on Usenet discussing where to find shiny software that essentially does nothing but "cat file.avi.* > file.avi". I'm sure that such a thing must be possible from the command line, but the majority of Windows users are afraid to venture there.

sort of. COPY /B exists, but you have do do:

copy /b file.avi.001 + file.avi.002 + file.avi.003 + file.avi.004 + file.avi.005 file.avi

Re:Yay piracy!

[WhiteDragon]

I'm sure that such a thing must be possible from the command line,

copy file.avi.* file.avi

or, if you are a bit paranoid and want to guarantee append order, you can fall back to the original (DOS 3.x) syntax:

copy file.avi.1+file.avi.2+file.avi.3 file.avi

But yeah, most Windows users aren't even aware of command prompt. Being an ancient and crusty user of DOS (back to 2.0) makes me a 133t h@x0r, apparently.

I wasn't aware that you could use wildcards for that. Good to know.

Re:Yay piracy!

[nabsltd]

I can't count how many posts I've seen on Usenet discussing where to find shiny software that essentially does nothing but "cat file.avi.* > file.avi".

Since what you propose would generate an AVI file with an invalid index, I can see why people would want something that worked correctly.

Re:Yay piracy!

[nabsltd]

Or worse, how many people still encode with xvid for playback on anything other than a 10-year-old handheld player.

My home media streamer doesn't understand anything H.264, so I use plain MPEG-4 (usually XVID). My encodes give me an SSIM of about 99.6% at reasonable bitrates (around 4GB for most movies), so I don't think I'm losing much from my original Blu-Ray source.

Re:Yay piracy!

[drooling-dog]

Since what you propose would generate an AVI file with an invalid index

Nope, works fine. The input files aren't stand-alone AVIs, but a single large one that's been split up. That is, file.avi.001, file.avi.002, etc., and not file001.avi, file002.avi,....

Re:Yay piracy!

[Hatta]

I do the same thing here. We have microscope software, which for some reason does not add the TIFF extension to its output. Perhaps because it's OSX, and extensions aren't necessary for associations there.

Every month or so someone will come down and use the microscope not knowing of this quirk. After a few hours they come to me freaking out about their files. No problem, I pop open a terminal and do a little one line for loop in bash. Had one middle eastern lady here just about flip when she saw it. I don't think she could have been more amazed if I walked on water.

Re:Yay piracy!

[EdZ]

What the heck sort of media streamer are you using that can decode 1080p30, but not h.254? I wouldn't trust SSIM too much for anything other than comparing the output of the same encoder of the same source, and I can't imagine 1080p30 encoded as 6 mbps MPEG-4 ASP looking anything other than a blocky mess.

Re:Yay piracy!

[X0563511]

... which, you should mention, does not require or necessarily even suggest that the compressed files in question are pirated.

Re:Yay piracy!

[X0563511]

You might be interested that PAR2 uses Reed-Solomon encoding, which is what optical disks use for their bit-level error correction.

Re:Yay piracy!

[retchdog]

yeah and my point was that this gets built-up, so that eventually vlc supports split rars, which means that the megauploadblogs use rar and eventually rar actually IS the way to do it although not for any good reason.

Re:Yay piracy!

[MrNemesis]

Any sufficiently advanced bash one-line for-loop is indistinguishable from magic :)

Re:Yay piracy!

[metamatic]

Also don't think about how many people take H.264 MPEG-4, and shove it in an MKV container, when if they used the standard MPEG-4 container it would be playable on far more devices.

(I can see an argument for MKV if your payload isn't patent-encumbered, but for MPEG-4? Puhleeze.)

Re:Yay piracy!

[WuphonsReach]

Yes, but the issue with the CRC on CDs is that often any damage is only detected *after* the RS encoding on the CD sectors has already failed. At which point, you're basically hosed unless you have a 2nd copy of the data that you know is good. Very few users will use CD/DVD testing software that is smart enough to check how much actual damage is being hidden by the RS coding on the disk itself.

The advantage of PAR, PAR2 and other parity-like systems is that they sit on top of that layer and the user can take action after the corruption is discovered. Plus you can tune the ratio of data:parity to match your paranoia level (mine is about 10% parity on top of the data).

In addition, optical media suffering from rot tend to fail linearly. First it will be a few sectors, then a few more, etc. Which gives you a recovery window of sorts where if you catch it early enough, before the damage exceeds the amount of parity data that you have, you can recover all of the data.

Re:Yay piracy!

[nabsltd]

What the heck sort of media streamer are you using that can decode 1080p30, but not h.254?

An old one.

Seriously, HD video has been around for a long time, but AVC and other modern codecs are relatively recent. Any media streamer over 3 years old would have been designed 4 years ago, and at that point in time hardware H.264 was expensive, while hardware XVID/etc. was relatively cheap. But, HD broadcastin the US would have been 5 years old, so supporting higher resolutions wouldn't have been a big deal.

I can't imagine 1080p30 encoded as 6 mbps MPEG-4 ASP looking anything other than a blocky mess.

I haven't found any 1080p30 Blu-Ray encodings, although there are a few 29.970fps 1080i with pulldown. eac3to will convert these into 23.976fps progressive, just like the majority of Blu-Rays. I suppose that there are some things like the nature HD on Blu-Ray that are 30fps progressive. Anyway, you'd be surprised what a two-pass encoding can do for MPEG-4 ASP. Many static scenes can get by with as little as 500kbps, which allows a large bit pool for tough-to-encode scenes, while still keeping the average bitrate low. It would be nice if XVID had the equivalent of CRF, but I suspect that x264 is one of the few implementations of "constant quality" for H.264. The encodes for most Blu-Ray discs look to be one-pass average bitrate, based on the relatively small differences between the min and max bitrates, as well as the small differences from second to second.

Seems like the distributor needs to be slapped

[wagnerrp]

Why would you use a proprietary format to store openly distributed files?

Re:Seems like the distributor needs to be slapped

[MightyMartian]

Because, you see, some children have two daddies or two mommies instead of a mommy and daddy.

Re:Seems like the distributor needs to be slapped

[Bacon Bits]

Maybe because it's the best tool for your job? Politics isn't the only reason to pick your software. Or maybe someone has to deal with files from *gasp* someone else!

Re:Seems like the distributor needs to be slapped

[mug funky]

because most PC users, when faced with a .7z file, will download winrar.

and when that happens, it's extra effort for bugger-all (none?) extra compression to make your own archives in 7z. so they end up in rar by default.

Re:Seems like the distributor needs to be slapped

[gmhowell]

Because, you see, some children have two daddies or two mommies instead of a mommy and daddy.

And sometimes, nerds like to watch movies where both mommies and both daddys are in the same room giving each other special hugs. But they like to watch them for free, so they wind up downloading the movie as a 50 part .RAR file.

Re:Seems like the distributor needs to be slapped

[slashdot_commentator]

Odd. I don't understand why people even use RAR anymore when there's 7z. I don't see this horde of clueless users dying for shredded archiving. And as far as I know, RAR isn't this gratifying user experience.

Re:Seems like the distributor needs to be slapped

[Anonymous Coward]

And sometimes, nerds like to watch movies where both mommies and both daddys are in the same room giving each other special hugs. But they like to watch them for free, so they wind up downloading the movie as a 50 part .RAR file.

Just wait till you get to the ones with two daddies and two mommies and everyone is getting special hugs. That's when it gets kinda juicy.

Re:Seems like the distributor needs to be slapped

[Loopy]

http://www.compression.ca/act/act-win.html

Re:Seems like the distributor needs to be slapped

[wagnerrp]

The closed source and payware distributors use executable installers. The Linux community uses tar, compressed using gzip, bzip, or lzma. The legitimate Windows community uses WinZip (because they don't know otherwise) or 7-Zip (because it's free). The hugely overwhelming use of WinRar is just to split files for distribution on antiquated mediums like FTP and Usenet, which don't natively support multi-part uploads and downloads, and have file size limits. Since this is generally multimedia content that cannot be further losslessly compressed, these archives use store mode, and the only thing the rar format is used for is integrated checksumming, which rar is just one of many formats that will perform that job equally well.

Content distributed in that manner is almost always in violation of copyright, which makes the entire concept of developing an open source library to access a proprietary format, rather than simply using another format, or pirating WinRar as well, is comical. It's either a waste of effort when other tools are available, or a downright confusing inconsistency in ethics on someones part.

Re:Seems like the distributor needs to be slapped

[wagnerrp]

So why not use one of the many other open formats that support multi-part checksummed files, or pirate WinRar just like they did the porn.

Re:Seems like the distributor needs to be slapped

[wagnerrp]

I don't understand what relevance a 9yr old compression test is.

Re:Seems like the distributor needs to be slapped

[Anonymous Coward]

Because these are Windoze users and they can't tell the difference between sharewarez that they forget to pay for and open source.

Re:Seems like the distributor needs to be slapped

[jonwil]

I find RAR files all the time, even small ones used for small files.

Lots of people just use RAR because they have WinRar (usually pirated) and its seen as "better"

Re:Seems like the distributor needs to be slapped

[SuricouRaven]

For the same reason they still use mp3 when there are several alternatives that provide superior quality at any bitrate. It's established now. Once a technology is established, it's very hard to get rid off.

Re:Seems like the distributor needs to be slapped

[mug funky]

the one time i tried a 7z only client (i presume the one made by the 7z people), it sucked badly.

you don't know how often you use shell integration until you lose it, or have to manually add it back in.

it's probably much better now, but i wouldn't know because it's so much quicker to install winrar.

Re:Seems like the distributor needs to be slapped

[SeaFox]

Except Winrar isn't free (in any sense). The shell integration has been part of the installation options in 7-zip for years now. If you don't have a need to create rar archives, only open them. there really is no reason to give Rarlab your money or make it a pirated piece of software on your system.

Re:Seems like the distributor needs to be slapped

[wagnerrp]

WinRar does better for some filetypes, 7-Zip does better for other filetypes, but in general they're pretty comparable. Given the option between programs comparable in capability and usability, one being free and the other costing $30, why would you choose to pirate the payware one? Is there some stigma that the program you refuse to pay for is somehow inherently better?

Re:Seems like the distributor needs to be slapped

[lisaparratt]

Because the files I receive from other people are in Rar format, I need to decompress them, and my power to dominate their puny, weak minds with my glowing neon brain-waves, and force them to resend it as a .tar.xz only works in person?

Re:Seems like the distributor needs to be slapped

[thegarbz]

Why would you use a proprietary format to store openly distributed files?

More to the point, why would I care as long as I can open said file?

Re:Seems like the distributor needs to be slapped

[Anonymous Coward]

and those families should be put down because they'll spread teh gay.

Re:Seems like the distributor needs to be slapped

[gmhowell]

And sometimes, nerds like to watch movies where both mommies and both daddys are in the same room giving each other special hugs. But they like to watch them for free, so they wind up downloading the movie as a 50 part .RAR file.

Just wait till you get to the ones with two daddies and two mommies and everyone is getting special hugs. That's when it gets kinda juicy.

Yeah, but I gotta draw the line at 50 daddies and one mommy.

Re:Seems like the distributor needs to be slapped

[internettoughguy]

But 7zip can decompress rar files.

Re:Seems like the distributor needs to be slapped

[martin-boundary]

Yeah, but I gotta draw the line at 50 daddies and one mommy.

You gotta draw the line much earlier than that (if you want an orderly procession).

Re:Seems like the distributor needs to be slapped

[Anonymous Coward]

Because non-techie people associate filetypes with programs 1:1, and WinRAR is a usable program whereas the 7-zip GUI on Windows sucks big time and hooks itself into everything and bloats the system in an unconfigurable way. I tried the 7-zip Windows GUI and uninstalled it right away. WinRAR is a bit retro in its design but it does the job.

What's probably even more important here is tradition. And "scene rules". There's a cargo cult about splitting releases up into dozens of small partial archives as if we were still on Usenet, and then people pack those into bigger archives and split those up again because of filehoster limits. It's a mess, and it's all done in RAR, nothing else.

There are also de-facto standards that require RAR, such as for vobsub files.

Re:Seems like the distributor needs to be slapped

[dargaud]

It's been a while, but isn't 7-zip limited to decompressing rar files only ?
And also, isn't there optional redundancy in split rar files ? i.e. if you compress a (set of) file(s) into several split rars and you loose one or two of them you can still decompress the original file(s). Very convenient when archiving on CDs or, cough, cough, Usenet.

Re:Seems like the distributor needs to be slapped

[drinkypoo]

BBSes kept using ZIP well after RAR came out because they had scripts that would embed their taglines into the ZIP files that passed through their board, thus incriminating the owners in criminal copyright infringement. I'm not sure why this was considered to be a feature, but it was and probably still is. However, the scripts were eventually updated to support RAR which was carried through the society because the boards were run by nerds or kiddies who wanted to think they were nerds and both groups are willing to move to a technically superior solution. Now we have RAR and no compelling reason to replace it... not a reason big enough for the whole internet anyway

Re:Seems like the distributor needs to be slapped

[icebraining]

So why not use one of the many other open formats that support multi-part checksummed files

If you're downloading the file, chances are you didn't create it.

or pirate WinRar just like they did the porn.

That's what everyone did. Well, actually they don't need to pirate in the usual sense of the word, you can download it from the developers' site, but you're supposed to stop using it after 40 days.
They're actually pretty nice, the software doesn't stop working, the GUI version just starts nagging you and the CLI doesn't say anything at all.

Re:Seems like the distributor needs to be slapped

[Rockoon]

Maybe because it's the best tool for your job?

Since when is rar the best format for any job? I swear that you pirates are completely ignorant. Your obsession with rar is a self destructive cycle that will forever have you passing around significantly larger files than necessary.

1995 called and it wants its inefficient proprietary archive format back.

Re:Seems like the distributor needs to be slapped

[Ash-Fox]

Now we have RAR and no compelling reason to replace it... not a reason big enough for the whole internet anyway

Honestly, I keep encountering .7z files all over the place now.

Re:Seems like the distributor needs to be slapped

[Rockoon]

It isnt about opening the file. Its about creating it to begin with. Why would you ever choose an inferior technology like rar?

Its like storing your photos in gif format just because those files are smaller than bmp, completely ignoring the fact that both of those options are shitty inferior choices.

The fact that rar is not only a shitty inferior choice, but also a proprietary one, makes the gif analogy perfect.

Re:Seems like the distributor needs to be slapped

[jedidiah]

...except you don't have to worry about whether or not your car or some cheap player from China supports rar.

A "pirate media" format can change whenever the packagers fancy changes. It doesn't have to be set in stone.

You don't use ARC any more do you? So obviously change is possible in this area.

Re:Seems like the distributor needs to be slapped

[Per Wigren]

Yes, RAR (v3) has that functionality but just about nobody uses it. People use separate PAR (v2) files instead.

Re:Seems like the distributor needs to be slapped

[tepples]

my power to dominate their puny, weak minds with my glowing neon brain-waves, and force them to resend it as a .tar.xz only works in person

My experience is different. I manage to get people to cooperate once I explain politely that I lack software to read the file and ask to resend in Zip or 7-Zip format.

Re:Seems like the distributor needs to be slapped

[Anonymous Coward]

I draw the line at 2 mommies, one cup.

Re:Seems like the distributor needs to be slapped

[Just Some Guy]

Because a lot of sysadmins didn't think to install an unRAR plugin for ClamAV on the mailserver. If you can put lindsaynaked.jpg.exe inside a RAR file so that it doesn't get scanned, then there's a better chance of getting it onto the corporate LAN.

Re:Seems like the distributor needs to be slapped

[thegarbz]

Why would you ever choose an inferior technology like rar?

Citation needed. Google searches seem to conclude that RARv3 is bested only by 7-zip and then only enough to make the difference completely irrelevant in the real world.

Re:Seems like the distributor needs to be slapped

[grumbel]

Odd. I don't understand why people even use RAR anymore when there's 7z.

Because 7zip is kind of crap really. Not only is it quite slow, it also has absolutely no redundancy or recovery build in what so ever. A single flipped bit will quite literally destroy your whole archive, same with an incomplete 7zip that is missing some bytes at the end. RAR on the other side doesn't mind a flipped bit, with build in recovery data it can fix that and even without it that data will only affect a single file, all the other files and what is left of the damaged file can be extracted without a problem.

7zip for me is a classic case of completely over engineering in one direction (good compression) while completely forgetting the other one that is much more relevant for real world use: People want to get data that they put into an archive back out again.

Re:Seems like the distributor needs to be slapped

[WuphonsReach]

7Z has all the shell integration that WinRar has (and has for a number of years). We switched away from WinZIP / WinRAR as soon as that happened. As a bonus, it's one less set of licenses that I have to track.

(Plus the 7Z package is open-source, so it's widely available.)

Re:Seems like the distributor needs to be slapped

[wildstoo]

I think you presumed wrong. 7-zip file manager has had that feature for a loooooong time.

http://www.7-zip.org/ is the official 7-zip distribution site. If you get it from elsewhere, good luck.

The client is plain, straightforward and easy to set up, and it can decompress many compression formats (including RAR).

Full Windows shell integration built in.

Re:Seems like the distributor needs to be slapped

[petermgreen]

Except Winrar isn't free (in any sense).

It's free in the sense that you can download a fully functional copy from the developers website. You are supposed to register it after a trial period but the devs don't actually make any significant attempt to enforce that.

Re:Seems like the distributor needs to be slapped

[mister_playboy]

The plug-in/add-on that allows 7z to decompress rar v3 is not free (as in freedom) software.

Re:Seems like the distributor needs to be slapped

[mister_playboy]

I would rate WinRAR's GUI as being a bit nicer than 7z.

Re:Seems like the distributor needs to be slapped

[Sloppy]

Why would you use a proprietary format to store openly distributed files?

More to the point, why would I care as long as I can open said file?

Many possible reasons.

1. Premise rejection: Because you can't open the file. Proprietary software tends to be ported to far fewer platforms than Free software (is there a PPC NetBSD port of WinRar? I don't know) so the proprietary software may not be available to you. (Related to this: because running software inside of emulators is cumbersome and inconvenient, even when it technically can do the job. The idea of running a Windows archiver in Wine where it can't be scripted as part of the bigger get-files-from-Usenet picture is laughably luddite.)
2. Safety: who knows what else this proprietary software does or is even unintentionally capable of doing? Theo de Raadt aside, who has audited it, even superficially and informally? If you can't answer that question (as is often the case unless you do the auditing yourself) then consider what the upper limit may be.
3. Maintenance: What can you do about bugs? If the software has a problem then you're not ever going to get maintenance until the vendor decides they have reason to fix that bug. With Free software, bug fixes and feature additions can happen independent of any particular central authority. (Sometimes this is a very big deal. Honestly, I think this reason is the #1 factor in how I became a Linux user. I got burned so many times, both at home and professionally. Never Again. I will give up computers altogether and live in a shack in Montana, before I ever return to that level of frustration and utter helplessness. I don't know how anyone can stand it.)
4. Competition: Remember that the whole reason you're using this proprietary software, is that you somehow got stuck with a proprietary file format. Whenever that happens, it is usually the case that that particular proprietary application is the only application that can deal with the file. That means that if you ever want a particular feature or bugfix, not only do you not get maintenance (as mentioned above) but you may not even have the option to jump ship to a competing application. Free software, on the other hand, tends to default to writing non-proprietary formats, and because of that, there tend to be multiple implementations (whether they're forks of an ancestor or completely unrelated), so if you decide you don't like that application, there's probably another one which is interoperative with your data. This is why it's big news (I guess) that someone has created an application that can read rar3 files, whereas it wouldn't be news at all if someone created an application that can read tar or zip files.

This isn't to say that Free software is always something users must insist on, but it nevertheless does have practical non-religious advantages. (Well, non-religious unless you think platform diversity, security, maintenance availability and competition are all religious values.)

If you think you've found a case where those advantages are outweighed by other factors, or happen to be met by some particular proprietary application, fine, I won't preach at you and will leave RMS to do that job. ;-) But nevertheless, using proprietary formats for interchange with other users really is a dumb idea. And considering the particular application we're talking about here, it's a doubly-dumb idea because the proprietary format we're talking about just doesn't have any significant advantages. Whoever is writing rar3 files is clueless.

Re:Seems like the distributor needs to be slapped

[wagnerrp]

So it's nagware. Why put up with nagware when there are free alternatives that work just as well?

Re:Seems like the distributor needs to be slapped

[WorBlux]

Well no, some portable music players only support .mp3 with the stock firmware. Also .mp3 is lossy so converting to a different codec except a loseless on would make the file lossier.

Re:Seems like the distributor needs to be slapped

[drinkypoo]

Sure, they're becoming more common, I've distributed some stuff in 7z, but unless compression really counts I just use ZIP, because everyone and their mom has an unzip utility, but not everyone has 7zip.

Re:Seems like the distributor needs to be slapped

[Anonymous Coward]

A single citation, FreeARC, of one of many compressors that put rar to shame on every metric that you somehow amazingly (dishonestly! asshole) could not find with google.

Re:Seems like the distributor needs to be slapped

[Anonymous Coward]

7-Zip wins if there is redundancy between different files in the same archive. No other archive type eliminates that equally well. RAR wins in almost all other cases (that contain more than just text files).

Re:Seems like the distributor needs to be slapped

[Anonymous Coward]

There is a simple reason why rar is so popular. It is possible to open a partially downloaded rar archive.

If you're downloading something large or the download has failed, you can open the partially downloaded rar to check on what's there - doing that with 7z or zip is not really very possible.

Re:Seems like the distributor needs to be slapped

[slashdot_commentator]

1) 7zip is THE application for lossless, maximal compression. Speeding up 7z is a simple matter of setting for increased speed (decreased compression).

2) Error correction & recovery is done with parchive, not RAR. Even if winrar has a tweak that works in conjunction with parchive (which I am unaware of), rar archives are not more error resistant than 7zip.

Re:Seems like the distributor needs to be slapped

[grumbel]

From the rar man page:

              rr[N] Add data recovery record. Optionally, redundant information (recovery record) may be added to an archive.

No messing around with par needed, rar can do recovery by itself. And even without recovery records rar has no problem extracting files on an incomplete or damaged rar file, some of the extracted files might of course be broken when you extract them, but you can still get the rest.

As for 7zip, see the Wikipedia page, if you don't have the full .7z file in perfect condition you are in trouble, having only half the file leaves you with nothing to extract. If there is a way around that feel free to update the Wikipedia page.

Re:Seems like the distributor needs to be slapped

[metamatic]

Same reason people use a crappy open format (MKV) to store patent-encumbered encoded data (MPEG-4 H.264, AC3, MP3 and AAC).

That is, they're idiots.

Re:Seems like the distributor needs to be slapped

[SeaFox]

It's free in the sense that you can download a fully functional copy from the developers website. You are supposed to register it after a trial period but the devs don't actually make any significant attempt to enforce that.

The fact they don't make the software cease to function completely after the trial period doesn't make it freeware magically. You're still using the software without paying for it.

Re:Seems like the distributor needs to be slapped

[robsku]

Why would you ever choose an inferior technology like rar?

Citation needed. Google searches seem to conclude that RARv3 is bested only by 7-zip and then only enough to make the difference completely irrelevant in the real world.

Not sure if it's RARv3 that I've tested against tar+gzip/bzip2 but I remember that at least bzip2 compression won rar when I last have tried it...

Re:Seems like the distributor needs to be slapped

[thegarbz]

The onus is on you to provide citation not on me to find it.

So now we have ARC. Some compression algorithm that doesn't even show up in a quite logical google search, that most people haven't heard of and isn't supported by any common program.

Better still you call me a dishonest arsehole for not using great googlefu, yet you still haven't provided a citation. www.freearc.org doesn't even exist.

Stop wasting my time idiot.

Excellent!

[Anonymous Coward]

Finally there is a Richard Stallman approved way of extracting my pirated pornography, movies and TV shows on my Linux box.

Using a proprietary tool on an OSS system is so unethical...

Re:Excellent!

[Anonymous Coward]

What about Creative Commons licensed porn? Is that on that FSF high priority list?

On second thought, I hope Stallman's NOT working on that. In fact, how do I unthink that mental image?

Re:Excellent!

[Anonymous Coward]

Finally there is a Richard Stallman approved way of extracting my pirated pornography, movies and TV shows on my Linux box.

Without a GUI. But maybe the FSF thinks that is a feature. These newfangled clickypointy interfaces are the devil's playground.

Re:Excellent!

[icebraining]

What about Creative Commons licensed porn?

http://www.freedomporn.org/

The mission of Freedom Porn is to empower and engage individuals to create and share ethical porn as a means of advancing sex-positivism and sexual freedom.

We advocate safer sex and consensual sex, and feminism is inseparable from our mission. We also fight for freedom of speech, privacy, and free culture. As such, we are the world's first porn repository of entirely free cultural works. All videos are in free formats, either WebM or Ogg Theora, and we encourage the use of free software. We run on donations, so please contribute!

That's the problem, really?

[tftp]

it's not uncommon to see it used for distributing multimedia files over the Internet. That means the lack of free software to extract RARv3 files has been sorely felt.

99.999% of those "multimedia files" are proprietary, commercial materials. The archiver is probably the least concern here.

Re:That's the problem, really?

[Arlet]

The archiver is probably the least concern here.

It is if you're a Linux distributer, and you want to include the archiver as a standard package.

Also, with the source code available, you could modify the archiver. For instance, you could make/modify an open media player with the capability of automatically downloading Usenet archives, unpacking those, and playing the media file.

Re:That's the problem, really?

[Anonymous Coward]

For instance, you could make/modify an open media player with the capability of automatically downloading Usenet archives, unpacking those, and playing the media file.

As a matter of fact, TheUnarchiver itself can do something similar to what you suggest. When you have it and Xee (an image viewer from the same developer) installed on a mac, you can browse images inside of any archive supported by TheUnarchiver without needing to expand the archive to disk.

There are already "free" unrar apps

[1u3hr]

I assume theyr'e using "free" in the not-as-beer sense, since there are plenty of free apps that can unrar , 7-Zip for example. True, these all seem to use code available from RAR that isn't "free" . E.g., from the 7-Zip licence:

unRAR restriction
The decompression engine for RAR archives was developed using source
code of unRAR program.
All copyrights to original unRAR code are owned by Alexander Roshal.

RAR is pretty much the default foprmat on Usenet binary groups, for instance.

Re:There are already "free" unrar apps

[1u3hr]

PS: It'd be real news if there was a free app to CREATE RAR archives.

Re:There are already "free" unrar apps

[Anonymous Coward]

I've found that the rar implementations in ubuntu were unable to uncompress rar files if i tried to open the 2nd or 3rd file. If I opened anything but the first file it resulted in a corrupted output... hopefully that has been fixed.

Re:There are already "free" unrar apps

[mikael_j]

The main problem with the implementations that are available seems to be that they are very slow for larger files split across multiple rar files. I even remember reading some posting about this somewhere where the gist of the message was "the portable *nix version of rar/unrar was developed to be portable, not efficient. Try using the Windows version if you think it's too slow.".

Re:There are already "free" unrar apps

[jonwil]

Now that there is LGPL code that can decompress RAR files (code that is presumably untainted by the restrictions on the "unrar" code used by most archivers to uncompress RAR files) someone can use that code as a reference for how the compression works.

Re:There are already "free" unrar apps

[Anonymous Coward]

Are you running 64-bit ubuntu? I've only ever had that problem on 64-bit linux, never on 32-bit linux. I'm not sure if the free-as-in-beer code by the original RAR guy was even intended to be portable to 64-bit systems, or if this is just a bug that he hasn't got around to fixing.

Re:There are already "free" unrar apps

[Anonymous Coward]

This free software version of UnRAR is based on an old version of RARLAB's UnRAR with permission from author Eugene Roshal.[1] It is licensed under the GPL. It does not support the RAR3 format.

Reference: http://en.wikipedia.org/wiki/Unrar

Re:There are already "free" unrar apps

[Goaway]

It would be fairly easy to write that. Getting it to be as efficient as the original rar archiver might be tricky, but basic compression is about as easy as it is for zlib's Deflate.

Re:There are already "free" unrar apps

[Anonymous Coward]

Copyright doesn't matter, what matters is the license. You omitted the relevant part:

The license for original unRAR code has the following restriction:

            The unRAR sources cannot be used to re-create the RAR compression algorithm,
            which is proprietary. Distribution of modified unRAR sources in separate form
            or as a part of other software is permitted, provided that it is clearly
            stated in the documentation and source comments that the code may
            not be used to develop a RAR (WinRAR) compatible archiver.

So unRAR is sort of free but don't you dare look at the source with the wrong thoughts (writing a packer). I wonder how the LGPL can be compatible with such a restriction.

Re:There are already "free" unrar apps

[icebraining]

There is an OSS version (unrar-free), which only decompresses RARv1&2. Only the non-free package (unrar) can decompress RARv3 files.

Re:There are already "free" unrar apps

[Anonymous Coward]

Really 7zip is a better split-file type archive format, it's just that RAR is entrenched so it will be a while before it changes. Plus 7zip really is open.

Before RAR the primary format was ARJ so changes do happen, just slowly. Unless something better than 7zip comes along or RAR gets opened then eventually 7zip will probably become the de facto standard.

Re:There are already "free" unrar apps

[Anonymous Coward]

This not obvious, and probably not true. Compression is a much harder problem than decompression.

I remember...

[Super Dave Osbourne]

Back in the day, OpenIt! :) I think I even wrote some of that code. How times change, but ObjC still is the best damn mistake to ever happen in programming.

Re:I remember...

[Anonymous Coward]

What the fuck are you talking about?

The Unarchiver

[PhunkySchtuff]

I'll put in a big thanks for The Unarchiver.

I deploy it as my standard unarchiving utility on all desktops I manage. It replaces the Mac OS X built-in BOMArchiveHelper which isn't as smart about handling extracting multiple files at once, and it handles a vast range of file formats that you'd otherwise have to resort to the command-line to deal with. News of it adding RARv3 is the icing on the cake - not that I've encountered a RARv3 file, but because now I don't need to worry if I do as my standard utility will deal with it.

Big double-thumbs up to Dag Ågren. Cheers.

Re:The Unarchiver

[Lord_Jeremy]

Indeed. I discovered this amazing gem of a tool a couple years ago and I've loved it ever since. I've been familiarizing myself with the source lately with the intent of implementing a FUSE filesystem layer for at least viewing archives. Glad to see TU is getting a decent amount of attention.

Re:The Unarchiver

[Anonymous Coward]

Dag Ågren. Cheers.

Ah. Å is Å An html entity. For a second I thought /. finally had unicode.

Catch 22

[Anonymous Coward]

Unfortunately for free software purists, the author compressed for distribution all binaries and source code of the The Unarchiver using RARv3.

Speaking of unarchivers...

[antdude]

Hi.

Last week, I downloaded 4.2 GB http://dl.godsandheroes.com/Gods&Heroes_0.10.30.0a.zip file, but I am having problems using command line's unzip to test and extract this file:

$ unzip -t GodsAndHeroes_0.10.30.0a.zip
Archive: GodsAndHeroes_0.10.30.0a.zip
warning [GodsAndHeroes_0.10.30.0a.zip]: 131165639 extra bytes at beginning or within zipfile
    (attempting to process anyway)
error [GodsAndHeroes_0.10.30.0a.zip]: start of central directory not found;
    zipfile corrupt.
    (please check that you have transferred or created the zipfile in the
    appropriate BINARY mode and that you have compiled UnZip properly)

$ unzip -v
UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip/ ;
see ftp://ftp.info-zip.org/pub/infozip/UnZip.html for other sites.

Compiled with gcc 4.4.3 for Unix (Linux ELF) on Feb 21 2010.

UnZip special compilation options:
                ACORN_FTYPE_NFS
                COPYRIGHT_CLEAN (PKZIP 0.9x unreducing method not supported)
                SET_DIR_ATTRIB
                SYMLINKS (symbolic links supported, if RTL and file system permit)
                TIMESTAMP
                UNIXBACKUP
                USE_EF_UT_TIME
                USE_UNSHRINK (PKZIP/Zip 1.x unshrinking method supported)
                USE_DEFLATE64 (PKZIP 4.x Deflate64(tm) supported)
                UNICODE_SUPPORT [wide-chars, char coding: UTF-8] (handle UTF-8 paths)
                LARGE_FILE_SUPPORT (large files over 2 GiB supported)
                ZIP64_SUPPORT (archives using Zip64 for large files supported)
                USE_BZIP2 (PKZIP 4.6+, using bzip2 lib version 1.0.5, 10-Dec-2007)
                VMS_TEXT_CONV
                WILD_STOP_AT_DIR
                [decryption, version 2.11 of 05 Jan 2007]

UnZip and ZipInfo environment options:
                      UNZIP: [none]
                UNZIPOPT: [none]
                  ZIPINFO: [none]
            ZIPINFOOPT: [none]

What's wrong? I even tried it on my friend's 64-bit Fedora box with the same unzip version with the same results. :( WinZip v12.1 in old, updated 32-bit Windows XP Pro. SP3 had no problems!

Is unzip too old? Is there another one to use?

Thank you in advance. :)

Re:Speaking of unarchivers...

[Anonymous Coward]

My god man. This is slashdot, not random tech support. Surely a 5-digit should know better than this.

Unarchiver?

[Anonymous Coward]

De-archiver perhaps, at a pinch, would be an acceptable noun.

But "unarchiver" does not make sense as an English word, meaning literally "not archiver". There is no means by which English could be contorted to create the verb "unarchive" to mean "remove from an archive".

Re:Unarchiver?

[king neckbeard]

To unearth is to remove from the earth. One that unearths would be an unearther. To unarchive is to remove from an archive. One that unarchives would be an unarchiver.

Re:Unarchiver?

[Anonymous Coward]

Looks like you need an untroller.

Re:Seriously, just download WinRAR.

[king neckbeard]

The importance is that this is free as in freedom software. Ubuntu, gNewSense, and Debian can all legally ship this out of the box.

"Legally"? Is this a new word?

[Anonymous Coward]

I'm a pirate, so I really have no idea what the hell you are jabbering about. I don't need you talking no trash, I do all the work around here. When I see something I want, I TAKE IT!!!! Capiche?

fuhgetaboutit

Thank you for the tip... Amiga formats, too

[chiark]

To slashdot, thanks for bringing this to my attention! I have far too much data in Amiga archives / disk image formats that previously I had to fire up an emulator to access. With this, and xnview / nconvert along with other tools, I can access my old stuff whenever I want.

The fact it is open source, and deals with rars, and almost every other archive well is an added bonus!

Of course, huge thanks to the author. Donation on its way.

...and I've just noticed that the author provides a lightweight IFF viewer. What a star!

Already had source code

[peppepz]

You could already download the UnRAR source code from the RAR web site itself; if all you want to do is to extract RAR files, its license doesn’t look too evil (I'm quoting the most "problematic" part):

The UnRAR sources may be used in any software to handle RAR archives without limitations free of charge, but cannot be used to re-create the RAR compression algorithm, which is proprietary. Distribution of modified UnRAR sources in separate form or as a part of other software is permitted, provided that it is clearly stated in the documentation and source comments that the code may not be used to develop a RAR (WinRAR) compatible archiver.

OK, it's no GPL, but still I'd say that it puts "open source" RAR support in a better position than other high priority GNU projects such as Flash support, where your only chance to have a good experience is to use binary-only code.

Re:Already had source code

[Anonymous Coward]

That's distributing code. There are two parts to being open and distribution. Devs want access to the code, users want the binary installed as a package.

I'm surprised RAR is still relevant. People using it are capping/ripping video that they don't own copyright to, and there's no need to split the file into small chunks now bittorrent is used. Useness for binaries is all but dead. Torrents have built in error checking, so the need for parts and recovery is redundant in the process.

Re:Already had source code

[mister_playboy]

People using it are capping/ripping video that they don't own copyright to, and there's no need to split the file into small chunks now bittorrent is used. Useness for binaries is all but dead. Torrents have built in error checking, so the need for parts and recovery is redundant in the process.

Torrents are really yesterday's news at this point. Uploaders have moved to filehosts like FileServe, Oron, et al because they can make money uploading content and because you can upload many more files when you rely on a third party to do the heavy lifting of distribution. It's just icing on the cake that the legal authorities focus on torrent sites while allowing filehosts to operate in the free and clear, despite the fact that the commercial, direct distribution nature of filehosts really ought to make them far more legally culpable than torrent site operators.

Re:Already had source code

[Hatta]

Do any of these file hosts provide FTP access or something actually usable? As long as they require you to click on ads or wait 40 seconds before downloading one part of a 60 part archive, they'll never replace torrents. Even USENET is more convenient than this crap.

Re:Already had source code

[mister_playboy]

For free accounts, FileServe and FileSonic both allow FTP uploading. FileSonic's FTP is buggy as hell, but FileServe's works very well and makes uploading a snap.

I think FileServe represents the best compromise between the needs of uploaders and leeches, giving good service to both group. Leechers love Mediafire and Megaupload, but the only uploaders who need to use these hosts are either the very principled or the very naive since they don't offer any money (in theory MU offers cash rewards, but the bar is set so high as to be nearly unreachable).

I am a big proponent of torrents, but torrent sites are falling by the wayside... for example, the 3 biggest porn trackers (Empornium, PureTnA, Pornolab) are all currently defunct, and very few new porn sites choose to run a tracker. It's all about starting your own blog, harvesting links, and posting them on hosts that pay cash for downloads.

Re:Already had source code

[Hatta]

For free accounts, FileServe and FileSonic both allow FTP uploading.

What about downloading? I can pull a 450 GB torrent off of my favorite tracker in a couple of days with about 10 seconds of my attention. Or better yet, I can point a script at an RSS feed that watches for keywords, downloads matching .torrents, launches the torrent, and has the file appear in XBMC with no user interaction at all. Can FileServe compete with that?

Leechers love Mediafire and Megaupload

I can't stand either of them.

I am a big proponent of torrents, but torrent sites are falling by the wayside... for example, the 3 biggest porn trackers (Empornium, PureTnA, Pornolab) are all currently defunct,

And yet there are still many excellent trackers for music, movies and video games. I haven't lost a tracker I value since Oink went down. What's more, the quality is great since people are uploading out of a spirit of sharing instead of profit.

As long as these file sharing services are less convenient for downloaders than torrents are, torrent sites will continue to thrive. If profiteering uploaders go somewhere else, good riddance. We don't need what they have to offer anyway.

Most distro's already include unrar don't they?

[timbo234]

On the 2 distros I've used most in the last 8 years (Mandriva and Opensuse) unrar is already included (in Mandriva from the PLF repo, in Opensuse from the non-oss repo). So what's the advantage of this new program?

Re:Most distro's already include unrar don't they?

[thegarbz]

UnRAR is a binary. This is why it's in the non-oss repo in Opensuse. The advantage is that this one is under an open source license.

Re:Most distro's already include unrar don't they?

[arkhan_jg]

unrar is by rarlab, the same company that makes winrar and is run by Alexander Roshal - and licences the format from Eugene Roshal, who invented it. It's shareware, and can only decompress files, not create them. Rar and winrar, that can create archives, are closed source.

This one is open source, and thus can be incorporated more easily into other open source apps for extraction of the current v3 of rar files; previously, unrarlib could do v1 and v2, but not v3.

Re:Most distro's already include unrar don't they?

[wastedlife]

It really depends on the "unrar" that you are comparing it to and how much you care about using open software.

GNA unrar is based off of an old version of RARLAB unrar, and does not support RARv3. It is GPL licensed.

RARLAB unrar, while the source is provided, has restrictions preventing it from being considered "open". One of these restrictions is that it cannot be studied to recreate the RAR compression algorithm.

Since this new implementation is open, the code could be studied to make an open source application to create RAR files, instead of just extract them. This is not allowed with the unrar source.

Note: Some of the above info was sourced from Wikipedia.

Re:Most distro's already include unrar don't they?

[Anonymous Coward]

thegarbz, you are mistaken, rar is the one which is available only as a binary. The source code for unrar has been open from the beginning. Being in the non-oss repo merely indicates that the licence is not one of the OSI approved texts.

That being said, I agree with the parents' conclusion that having an rar unarchiver going from a very much MIT-like (with obnoxious "RAR is a proprietary algorithm" clause) to LGPL is not an improvement that makes me jump off my chair.

- * - * - * - * -

Dag, I love you a million times owing to your message board software, but wake me again when you have an rar archiver.

Re:Most distro's already include unrar don't they?

[Anonymous Coward]

arkhan_jg, shareware is not the right word. Its meaning refers to software distributed as binary only, like rar/WinRAR.

But unrar is much more liberal than that, it is effectively open source under a licence similar to the MIT licence, the only difference is an obnoxious "RAR algorithm is proprietary" clause.

Irony

[ianare]

So now we have free (libre) software for extracting RAR files. Great ! You just need to run it on a proprietary operating system ...

Re:Irony

[Goaway]

The command-line version runs on Linux, OS X and Windows. Probably on BSDs too, but I have not been able to test this.

Re:Irony

[Anonymous Coward]

What's the problem here?
You're free (as in freedom) to run it wherever you like.

Or you're under the assumption that this only runs on Windows?
It's an FSF project, for Pete's sake.

Re:Irony

[Goaway]

It's definitely not an FSF project, but it does run on various platforms, including Linux.

Re:Seriously, just download WinRAR.

[icebraining]

You should read the license of the software you download. You can only use it legally for 40 days.

FSF's High Priority List

[simpz]

I'd have thought things like a tool that can read Visio files would be on that list ?

Re:FSF's High Priority List

[RockDoctor]

I keep on hearing people mentioning Visio as if it's something important. Never seen it or heard of it at work ... what does it do? Ah, it's a vector graphics programme. OK, that makes it a Class 3 problem : not a small problem, or a big problem, but someone else's problem.

Actually, technically it probably is to a degree my problem - a major part of my work is in producing what are essentially special purpose vector drawings, fed off a large database. But again, since work supplies the software for me (or if a client wants to use something else, they provide it, a dongle, and a manual), that's not a problem.

How long ago was this?

[tepples]

the one time i tried a 7z only client

How long ago was this? When I switched to 7-Zip nearly four years ago, it already had the same sort of shell integration that WinRAR had.

STOP USING RAR

[Anonymous Coward]

Seriously, stop using RAR, uploaders. It baffled me when you did it 10 years ago (and WinRAR was the only software available for it), and still does now.

Oops

[drooling-dog]

Oops - close quote on that first line.

It's about the v3, stupid.

[Anonymous Coward]

A lot of RAR files can already be extracted by a FOSS implementation of unrar (the package is unrar-free in Debian/Ubuntu). This just provides feature parity with the proprietary unrar (even if it does require OSX - maybe a GNUstep port in the works?).

think I was one of the lucky few

[Anonymous Coward]

I found the rar in some old debian archive (unrar is everywhere) in an old ftp site, I think it was after I had googled, then drilled down to some cache somewhere, and I was able to snatch it right before RAR ceased being free/open. It might have been a BSD port, but the tarball compiled without a problem. I don't know if it has been removed since. Now, I would tell that story to interviewers who ask, What do you consider your greatest accomplishment? but then it would always lead to more questions on a variety of subjects, always with that same quizzical look on their faces.

I think it was so I could enjoy a season's worth of Daria episodes. MTV has finally released a boxed set, BTW.

Transcode when filling the player

[tepples]

If it's for a portable music player, you probably want to transcode your 192 kbps m4a/ogg rips down to 96-128 kbps mp3 when copying the music onto the player so that you can fit more music at once. The lower bitrate adds noise, but in my experience, portable music players are used in noisy environments anyway.

Can't believe this hasn't been posted yet

[Anonymous Coward]

Obligatory xkcd
RARv3 unpacker, who uses that?
Let's support more CPUs!

Seeing all the comments complaining about this made me think of this.