Slashdot Mirror
Linux/Mac/Windows File Name Friction
lessthan0 writes "In 1995, Microsoft added long file name support to Windows, allowing more descriptive names than the limited 8.3 DOS format. Mac users scoffed, having had long file names for a decade and because Windows still stored a DOS file name in the background. Linux was born with long file name support four years before it showed up in Windows. Today, long file names are well supported by all three operating systems though key differences remain. "
Long filenames aren't all they are cracked up to be. I got made fun of once for using one. I can remember it so clearly now, we were in music theory class in high school and we had to use Finale on a Mac (OS 7 at the time) for our composition projects. I named one of my projects something like "Suso's Music Theory assignment number 4 for Mr. Becker 1993-9-24.mus" and saved it. A week later I was on the same Mac and noticed a file that wasn't mine called "Making fun of people who use really long filenames for their music theory assignments.mus". Nobody was admit to doing it but I knew who it was. I was devastated and never felt comfortable again in that class.
Now I'm scarred for life. I should have listened to my parents and gone with 8.3.
Although it was cast as a negative, I always enjoyed being able to use the ~1 (and ~2, ~3, etc) for long names in MSDOS. In my mind Program Files is progra~1 and Microsoft is micros~1.
But in this particular case, the summary has as much meat as the story, with the added benefit of saying it in a paragraph instead of several ( and even that's too long ).
For those of you who haven't read it, here it is: Windows, Linux and Mac OS X all support long file names, albeit differently. Linux is case sensitive, the others are not.
Tada! Two sentances. I imagine, were I a perl coder, I could have done it in half of one, but there you go.
Mod me down with all of your hatred and your journey towards the dark side will be complete!
Why are computer file names and conventions and protocols so messed up? It's bizarre -- and Microsoft has been one of the worst offenders with one of the most powerful positions and opportunities to make it a better filename-naming world.
I had worked in the DOS world long ago, and I'd always been frustrated with not only the restriction of the 8.3 naming convention, but the added imposition of:
Many years later, I had opportunity to consult in the Windows/DOS world after having worked in the Unix world for over a decade -- figured Microsoft had had enough time and money to work out the kinks in what had obviously been an early-technology constraint for the brain dead old DOS naming restrictions. Not. Sigh.
And then the transition was a nightmare, whoever conjured up the VFAT naming format and the "tilde" mapping backwards compatibility to FAT names should have been shot. A golden opportunity lost.
And then everything swings completely the other direction where anything goes. This may curry favor with users, but wreaks havoc on billions of lines of code which all of a sudden choke on what had been simple parsing routines -- fixable, but at great expense. I still think this was a paradigm shift that somehow could have accommodated the user space/community but still allowed some sanity in the machine world.
But layered on, or dovetailed into that quagmire is the Microsoft insistence they "know better than thou"... and the condescending insistence of dragging the ".3" extension nightmare into the new rules for file naming. Would have been okay to "allow" ".3" naming, but to impose the bizarre rules and behaviors Microsoft has? (How many of you have files named picture.jpg.jpg.jpg out there?)
Options to show extension, defaults to hide extensions, and continued reliance and semantics applied to extensions continue to make the filenaming world a landmine field.
And, Microsoft dares to allow mixed case naming, but does case insensitive handling of file names... don't even get me started about some of the bizarre results and buggy behavior I've traced to that. I only wish I'd had a chargeback code for all of the time I've spent fixing and debugging systems that all come back to the file naming. Sigh, again.
All of this isn't to let Unix and Unix style file naming skate. I've had problems, though fewer, there. But, at least it's seemingly (to me) more consistent and predictable, though there has been what I call "Windows" creep in that there have appeared some apps that somehow think managing and imposing "transparently" the extension to "file type" mapping is a good thing (it's not).
(One of the funniest Unix debacles I experienced was debugging a groups application -- they were moving files around and losing all but one each processing cycle... turned out they were remote copying from one Unix that had 14 (or more, can't remember) char limit on file names to an old SunOS system that allowed only 11. The remote copy that moved files from one system to the other for subsequent processing did so without complaint, the receiving side silently truncated the incoming files -- which were identical in name through 11 chars... essentially copying the incoming files over and over again on top of the same file... Sigh and sheesh!)
Too bad the article didn't mention what happens when you copy long filenames over the network. All kinds of crazy things happen in all kinds of client/server combinations.
Try copying a 40 character file from a windows server to a OSX client. What happens? Well... it depends if you used appletalk or SMB to connect with.
What about OSX server -> a windows client... depends on the version of windows AND OSX of course.
I've had nightmares.
Wanna be safe most of the time:
1) No spaces
2) Under 32 character filenames
3) Alphanumeric, underscore, period, or hyphen ONLY
4) Only a single period allowed.
I've got a CD-ROM that is unreadable under Windows XP because a Mac user put the files in a directory containing a '>' character.
If I can turn off Joliet comprehension I'll have access to the files in their original ISO9660 8.3 glory.
It's unfortunate that Microsoft's Joliet driver doesn't realize it's presenting names the OS can't tolerate. Otherwise it could replace the forbidden characters with % escapes before returning them to the OS. Or, alternately, handing the ISO9660 name to the OS if the Joliet name was forbidden by Windows' rules.
Did you somehow miss the link? It basically said to remove files with a preceding '-' (-filename) you do 'rm -- -filename' or 'rm ./-filename'. And to remove a file with unprintable characters try 'rm file?with?unprintable?characters'.
It's like sex, except I'm having it!
So, your OS supports long filenames, huh? Then why doesn't the vendor use them for all the cryptically named shared libraries, scripts, etc. that clutter up any modern os system directory?
They way I look at it, the day I look at something like "d3d8.dll" or whatever drek is fermenting in \WINDOWS32\ and it is actually named with a descriptive filename, then that OS will truly support long filenames.
Not sure where the Linux crown compares, but OS X is getting better with each revision. Classic Mac OS had this one down (mostly) cold.
The white zone is for loading and unloading only. If you need to load or unload go to the white zone. It's a way of life
Why not simply follow the POSIX standard*? You can avoid a lot of hassles that way. Isn't that why we have standards?? I know, it doesn't resolve the conflict with Windows case "insensitivity", but ... it does provide interoperability between POSIX-compliant OSes.
* upper/lower case alphabetic characters, numeric digits, underscore, dash, and period.
From the wikipedia entry on NTFS:
"Though the file system supports paths up to ca. 32,000 Unicode characters with each path component (directory or filename) up to 255 characters long, certain names are unusable, since NTFS stores its metadata in regular (albeit hidden and for the most part inaccessible) files; accordingly, user files cannot use these names."
The article incorrectly states "Windows file names can be up to 255 characters, but that includes the full path. A lot characters are wasted if the default storage location is used: "C:\Documents and Settings\USER\My Documents\"." I will grant that this may have been a limitation in the past, but XP has had NTFS from the start, and NTFS is by far the most common windows FS today.
Have you looked at http://www.chipx86.com/wiki/Leaftag
El Tonerino
Amiga has had long filename support since it was first released in 1985.
Dan East
Better known as 318230.
C:ONGRTLNS.W95
If Jesus wants me it knows where to find me.
As a quick tip, "rm -- filename" would have worked; it makes rm not parse the filename in any way whatsoever.
Trust the Computer. The Computer is your friend.
That is what the -- option is for. It signifies that there will be no further options, so anything following it that starts with '-' will be interpreted as a filename. rm -- -funny-named-file will do the trick.
Bogtha Bogtha Bogtha
rm /home/someuser/-file
Or even
rm ./-file.
Ideology: A tool used primarily to avoid the bother of thinking.
*nix is much longer and able to go much deeper in the path.
I know there's a joke in there somewhere...
Here we go again!
File names aside, is there a good way to "tag" files (generic metadata) on Windows or Linux?
On NTFS, you can use ADS (Alternate Data Streams) to store metadata about a file, though I don't know of any software that can read such data in a consistant manner - Not to mention, just about every malware scanner out there will flag such files as suspicious.
On Linux, it very much depends on the FS you choose, though again, support for file metadata remains about as standardized as snowflakes.
They have a whole block on "Avoid using these characaters for maximum portability".
But, where's the exclamation mark? TONS of Windows people (including me) use exclamation points as the first character to put files/directories to the top of the list. Linux constantly chokes on these characters. But, no mention of it at all in this article.
Well, if MS hadn't decided to use \ as a directory separator, you could just backslash it. But no. MS had to be retarded.
Goten Xiao
No, they go in "C:\Program Files" and the Registry and one or more users' "C:\Documents and Settings\%USERNAME%\Application Data" folder.
No, they go in "/Applications" and "/Library" and one or more users' "~/Library". Also, by the way, OS X does have /bin and /usr/bin and all the other UNIX standard folders; they're just hidden from the finder.
No, on Linux they go in "/usr/local/bin" and "/usr/local/etc" and one or more users' "~" only, because "/bin" and "/usr/bin" are reserved for bits of the OS itself (equivalent to "C:\Windows" and "/System").
"[Regarding the 'cloud,'] ownership was what made America different than Russia." -- Woz
cmd.exe's completion is a bit strange if you're used to bash, but once you get to know it, you can get around quite well.
Example:
Before Windows XP, you had to activate the tab character by changing a registry key. XP has this set by default.
WWTTD?
Yes and no. That was a limitation of Windows 9x (a holdover from DOS and Unix), and still exists in the ANSI versions of the NT APIs. However, the native NT Unicode APIs support 32k characters for the path. I don't know if there's a 255 char limit on individual names for NT, off the top of my head. Though it's possible that the number of programs still using the ANSI APIs (since the Unicode version only works on NT, but the ANSI version works on 9x as well) may impose an artificial limit of 255 char paths on your file system.
You have tried to support your argument with faulty reasoning! Go directly to jail; do not pass Go, do not collect $200!
Windows here suck the most. NTFS is good, but all the backward compatibility cruft just drags the FS down.
Once under Windows, I have spent about half an hour with Explorer refusing to copy one one. Explorer was insisting that "File No Found". Text file was there and perfectly editable by notepad. I needed about 30 minutes to observe that Explorer was giving error on only on file of whole directory and that file have had the longest name. ZOMG!!! They still have cap 255 bytes on path(!) length!!!
Welcome to 3rd millenum, Microsoft! Where do you want to go today?!?
End-Of-Sarcasm.
Realistically, I never had a single problem with Mac or Linux when handling file names. But when you get to Windows, you start getting the annoying messages from explorer "invalid character" with attached long list of characters they do not allow to use. All the time. And I'm not talking about bunch of non-Unicode applications (for example Adobe Acrobat Reader) which cannot open file with name containing international characters. Could it be any dumber?
All hope abandon ye who enter here.
I just hit the file name issue trying to sync some stuff between unix / Windows XP using rsync.
The case insensitivity was annoying and the limited char set on XP was no good.
Again, you would think they would have fixed this on XP.
I want documents not files. Sometimes multiple files make up one document (webpage + stylesheet + media), sometimes there are multiple documents in one file (zip).
When will anyone come up with a persistant storage system which allows me to make random tags to documents and groups of documents. Drop the folders and give me 'search queries' on content and tags. Automatically save all data and don't bother me with giving it a name... When it's important I will give it the proper tags until then just remember it for me.
Do I have to name the paper before printing?
What I cannot create, I do not understand
Or perhaps more simply:
./-annoying_file
rm
---GEC
I'm but the humble pupil, seeking to snatch the scratchbuilt pebble from the master's fully articulated hand
Dos 1.0 had no directories, and arbitrarily used / for options (remember DIR /P ?). When 2.0 came out, to preserve backwards compatibility they kept / for options, so decided on \ as the directory separator. Modern dos/windows can handle / for directories fine, but they need to still support \ for - you guessed it - backwards compatibility.
I am trolling
Is this anything other than an attempt to dis Windows for no other reason than 'Because'?
I think it is a valid issue. There are some files in a CVS module I simply cannot use on Windows because the filesystem chokes when CVS tries to write them in Windows and the rest of the CVS commit is aborted. It is a huge pain in the ass, even though these files do not contain any capital letters. This happens with ever CVS client on Windows, even Cygwin. MS needs to get off their butts and fix this crap once and for all.
That is a directory with no files but several other directories in it...
In Terminal.app, you can create file names with colon, but such character is mapped to a forward slash when seen in Finder. On the other hand, you can use forward slash in Finder, and it is mapped to a colon in the command line.
Historically, Mac OSes use colon to separate folder names in a path.
There is a subtle restriction in HFS+. All files in HFS+ have their names in normalized unicode, and in order to normalize in the first place, file names must be in valid UTF-8 encoding. You cannot use random character string for file names.
There is no such restriction for UFS on Mac OS X. I think UFS supports roughly the same characters as in BSD and Linux and any other Unices. If you're transferring files from Linux with names in a legacy encoding, you can create a UFS disk image and convert file names to UTF-8 before copying them to HFS+.
I once had a signature.
There's a whole new dimension of fun when your file names include non-Roman characters, such as Japanese.
First of all, there is the matter of which encoding the file names are in. Lots of Japanese Windows installs and their utilities still use Shift-JIS for file names. OS X, on the other hand, uses Unicode, and typically expects UTF-8 for file names from programs. In fact, it not only expects it, it enforces it, returning an error when attempting to use a file name which is invalid UTF-8.
Many command utilities that deal with archive files utterly fail on OS X when given archives using Shift-JIS file names, and many others improperly translate it as 8-bit ISO Latin I. A few (such as the command line RAR archiver) are actually smart enough to make a system call to translate the file name from Shift-JIS to UTF-8.
And then there is the issue of Shift-JIS MP3 tags. If you open those with iTunes, not only do they get interpreted as ISO Latin I, but irreversably so if you do something that writes them back to the .mp3 file. (They get written back as a UTF-8 representation of the ISO Latin.) I've had luck in the past using a hex editor and SimpleText in Classic to convert them with much work, but I'm not sure what I'll do with the new Intel Macs that don't support Classic.
--
"Open source is good." - Steve Jobs
"Open source is evil." - Microsoft
I know this might sound a bit offtopic, but since the post mentioned windows filesystems, I felt it might be a good place to throw this question...
Not many people know or have even used this, but NTFS has support for multiple streams of data in a single file, which is something that borrows concepts from object-oriented filesystems. This is scarcely, if at all, included in the regular windows documentation (it is documented in the MS knowledge base http://support.microsoft.com/kb/105763/). I thought it to be a nice idea for, say, media files, to store the audio in one stream the the video in another, or adding subtitles or metainformation in different streams in a very standard way. But for some reason nobody used that, not even Microsoft who designed the feature.
Does anybody have a clue as to why this has not been used?
www.meneguzzi.eu/felipe
My favorite shell-expansion moment: when I was a new Unix user long, long ago (freshly coming over from VMS), I wanted to remove one funny-named file in a directory. I discovered that rm had this cool switch "-i" that would prompt for removal on each file. Great! I'd just say "yes" to the file named *, or whatever I'd accidentally created. So, being a VMS user (and thus used to switches that went anywhere on the command line), I typed this:
$ rm * -i
Have you read my blog lately?
The purpose of the "OS" (its actually not the OS here, but lets use that term to make the following discussion clear) is to provide the set of tools needed to implement your "paradigm" (again, not true, but it will do).
Your way of thinking.
As it turns out, having multiple "files" composing a "document" is easily mapped in a hierarchical layout. As a simple idea, put all the files into a node and call that node the name of the document.
The "OS" should not impose upon the applications, but should provide ready services that map well into what the application(s) want.
Unix further provides "hard" and "soft" links to allow you to do (for example) sharing. As an example; you have a boilerplate logo image. It can be hard linked into your documents.
"Random" (I do not think you really want random) can be accomplished with soft links.
Content searching? Either "find" or "grep" will do (ok, for up to several hundred megabyte of content -- and if you have hit THAT limit, let me know -- its a separate discussion).
You will have noticed that I have (so far) eschewed GUI tools in this discussion. The blatent omission of find/grep and other tools has mystified me. Either it is hard to do (semantic mapping of symbolic language to pictures) -- which is true, or the GUI designers are deliberatly dumbing things down.
It would be nice to have an "assembler" in file open boxes: I would like to be able to say "Please open a file containing project in the name, whose contents include September 10, which was last modified in 2002, of type ASCII text".
Now, all the tools to do this are included in the "CLI" interface: find, grep, ls, file. But, when we hit the GUI, these tools vanish. "NO SYMBOLIC REASONING FOR YOU - STICK TO THE CONCRETE" is the slogan.
Since the "classic" Unix GUI is basically X supporting XTERM, which in turn launches applications, the CLI is still there. But in modern Linux, Unix, OS X environments, most users are never exposed. And, in turn call for more "paradigms" to be created.
And, this is HARD. Witness Microsofts failure with "WinFS". Witness that the largest jump has been to Plan9, which extends the Unix way (by putting more stuff under this control). Witness the success of mapping things like "/proc". There really HASN'T been a new "paradigm" that offers more.
The problem is that trying to utilize the filesystem is lost in the GUI translation. Apple indexes files, by content, for GUI consumption. This is NOT a new breakthrough -- Unix has had "locate" which is most of the way there for ages. Indexing by content? Again, not new. Merging these ideas is fair, and I wish that Apple had based the kit on CLI for maximum portability.
Ratboy
Just another "Cubible(sic) Joe" 2 17 3061
Bring on the modding, my karma can take it.
Ah the magic words for +5
Personally I like the Linux (Unix) method. Nothing wasted. But, have we reached the point yet were we can abstract the filesystem away from users? It's nice for an admin to be able to poke around and fix problems, but can we come up with one set of rules on filenames (Linux please!) and then have the OS read the FS and put a pretty picture of those rules in front of the common user? One of my biggest wonders in computers has been the fact that the underlying FS determines what file names and such I have to juggle. That's kinda annoying. Why haven't we come up with an API to abstract this yet?
AB HOC POSSUM VIDERE DOMUM TUUM
Despite your naive assumption that something with "16" in it's name is better than something with "8", the facts are that UTF-16 cannot handle as many characters:
UTF-16 as originally designed handles 0xffff characters.
Because that was not enough characters, UTF-16 was modified to have "surrogate pairs". Usually claimed to now handle 0x10ffff characters, but in fact they fail to subtract the surrogate half-characters (0x800). Also this deleted the only plausable claim that UTF-16 is better than UTF-8, in that characters all are the same number of bytes long (it is in fact worse, because the variable-length characters are much more rare, so bugs in handling them are much less likely to be detected and thus more catostrophic when they do happen).
UTF-8 as originally designed handles 0x7fffffff characters.
Because of the UTF-16 braindamage, the standards for UTF-8 were modified to say that all encodings after 0x10ffff are illegal, so literally UTF-8 was downgraded to match UTF-16. It still is false to say that you can losslessly translate from UTF-8 to UTF-16, due to the surrogate pairs, so they are not equivalent even with this limitation.
The one positive benifit of the "Unix wars" was that it stopped a whole lot of politically-correct idiots from forcing "wide characters" on everybody, and thus Plan9's UTF-8 could take hold. Unfortunatly Microsoft completely ignored all the proof that wide characters were a very bad idea and went and did it themselves in Windows. Still not as bad as if Unix had done it too...
...you suck at scripting.
...
Typical shell scripting idioms like:
for $each in *glob.pattern* ; do
command "$each"
mv "$each" "$(echo \"$each\" | sed -e s/stuff/replace)"
done
The only extra quoting necessary is in commands with variable substitution. And (while it may seem confusing), that syntax works even when the filenames have quotes internally. The double quotes identifies the contents to be treated as a single token with interpolation to be performed before passing on to ``command'', which is what you wanted.
Also the $() syntax is your friend. But remember to give it ""s too, you don't want it to expand it AND THEN tokenize it.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
Long file names in windows is kinda hokey. If you are at the command line, then you are stuck with the 8.3 format. Ending a directory name in "~1" is not my idea of long file name support.