That way, if he loses in court, he can get Scotty to beam some tribbles down to the court room.. and in the ensuing mayhem, escape through the gate to Abidos.
Or, it could be the third option, which is actually the most common IME - a pulse-width modulator. This is why fluorescent lighting ballasts don't work with dimmers. They need more than a fraction of a cycle to start up the discharge arc.
Similar issues exist without problems when mmap()ing files from NFS. You cannot update just a few bytes with NFS, you have to write the whole disk block out.
I'm fairly confident that the current "standard" way to implement mmap at the moment is to update the pages, mark them dirty, and let the VM subsystem write them to disk.
I haven't had to look at mmap's implementation in a long time, though.. but IIRC Rich Teer and/or Adrian Crocroft had good articles about it a few years back.
Obviously, I arrive at this problem space with a huge Solaris influence. But this is a well-understood problem, and I don't think Sun's implementation is particularly revolutionary.
Quick point of order, there is no requirement to use GNU tar with gzip!
gzcat filename | tar -xvf -
will work exactly the same as
tar -zxvf filename
That said, point taken -- my gzcat (Solaris 9 sparc stripped no non-standard DSOs) weighs in at a hefty 63K. I could probably get it down 30% from there with TFS and a C compiler by throwing out some features, but it would take a couple of days, and really, there would be no point.
> Unfortunately, in the context of Unix, there's nothing 'portable' about using gzipped files.
You're absolutely correct, but missing the mark.
The moronic FA author was trying to portably gunzip files. Given those requirements, gzcat is a reasonable prerequisite. Thus providing gzcat with arguments which would be portable to zcat is unnecessary, as zcat could not complete the task required in the first place.
BTW, I don't know what kind of machines you admin, but I have the luxury of administering primarily sparc/solaris boxen these days. This means that I can ship a Solaris 2.5.1 binary version of GNU tar and gzip with my install packages, and it will run on any of my machines, assuming they are not more than 11 years old. Sun ABI compatibility is quite good. I could even ship 2.4 binaries without issue, extending my box-age to 12 or 13 years, but that seems kind of pointless since all the boxes are less than 5 years old, with the exception of the workstation I use to build Solaris 2.5.1 binaries (an ultra 5 running 2.5.1 hardware 11-97).
I suppose I'm going to have issues at some point the future, though, as we are considering moving to Opteron machines. (If they would rip out the BMC and replace it with OpenBoot, we would have moved long ago). In that case, I'll probably have to snag gzip and tar from an old Solaris 7/x86 box I still have around.
...unfortunately, sudo did not exist when I started with UNIX.
Well, more specifically, it did not exist outside of CU-Boulder, and certainly did not ship with the version of slackware (which I downloaded at school and brought home on FLOPPY DISKS because 2400bps was too slow) I was using at the time.
Point taken, though -- sudo *is* a valuable tool, I should really integrate it into my daily habits... but I've got the "su XXX ^D" pattern pretty much hardwired into my mental pathways.
I am no SMB expert; in fact, I would NEVER think of a windows machine as being a "server" to a UNIX anything.. but (and I am also not a Linux expert, rather a UNIX guy), I seem to recall that there is an option for mount_smbfs to not do upper/lower case crapfulness. There certainly is in the other direction (windows machine mounting partition from a real OS).
If there isn't, it actually wouldn't be too hard to fix what you want, assuming that your smbfs is implemented a fairly normal unixish vfs switch. (If you don't know what this is, you really gotta read the samba docs, they did a really nice job explaining this concept).
Take a look at the rename system call which is hopefully addressed by this switch, detect the condition, and repair it.
Hmm. Scratch that idea, a little bit of debugging reveals that it's a red herring:
...The problem here is that mv checks for identical-ness before calling rename() -- it's probably checking the inode numbers in lstat. (In fact, I just manned rename, it doesn't even have an errno for same-name).
SO. Looks like the solution is to rewrite mv. That should actually be easier!
Find the lstat checks (probably actually calls to stat). If the inode number comes up the same, see what type of filesystem the files are on (statvfs?). If it's an smbfs, open your new file, carefully, like this:
open(path, O_WRONLY | O_CREAT | O_EXCL, 0600)
If THAT succeeds, go through to the rename call, otherwise it really *is* the same file.
I think that will work.
It would probably piss off POSIX though. Fixing the mount options of the smbfs would be more portable.
Why is it that you perl weenies always bitch about how hard it is to solve problems in shell... Then implement the same solution, only with bugs?
My perl is weak, but I believe your solution fails address the cases where filenames have spaces, or where then are subdirectories in the directory hierachy which should be left alone.
Never mind the fact that you're escaping to the shell TWICE in one line of perl to the damned job done. Not to mention spawning a new shell for each and every file to be renamed!
.... hopefully re-writing it like this (in a real programming language) makes your problems more obvious.
A note for the pedants: I have not tried to compile the above C program, and so definately do not warrant that it will not shoot off your foot!
Note of course, that there are a few minor I/O bugs in this program that are not present in yours. I would normally never use a non-abstracted read() statement from a pipeline and assume it's going to completely fill the buffer. (Although it would in 99.999999% of reasonable cases). I don't know what perl does in these odd situations (for example, a pipeline stall, EAGAIN, ENOLCK, EIO, or (especially) EINTR).
Oh, for the love of God, stop bitching about how hard this is to do under UNIX without package x.y.z installed and imagine doing it under Mac OS9 or Windows.
Break it down into its constituent parts: Iterate, rename. Whoo! Simple now, eh?
# cd cameraDir # find . -type f -prune | while read file > do > mv "$file" "`echo \"$filename\" | tr '[A-Z]' '[a-z]'`" > done #
... should do the trick. If you don't have tr, you can use sed with the y command. But I can't think of the last time I saw a box w/o tr.
This jackalope probably once something that LOOKS like this, didn't understand it, and is now spouting it as "Wisdom".
He probably saw somebody type:
ls -1 | xargs
Which, of course, is the same thing as ls | xargs on a non-broken ls.
By non-broken, I mean that ls -1 is the default behaviour of ls when isatty(STDIN_FILENO) or isatty(STDOUT_FILENO) return non-zero.
Or, perhaps, he's planning to use xargs to parse, say, filesizes out of ls? But that would be incredibly annoying, especially if your filenames contained spaces.... or something.
I dunno. I have always found xargs in newb books, but I have VERY RARELY used it in the real world. I am a big fan of using find to key iteration.
find dir -type f | while read file; do echo "$file"; done
Want the filesize instead?
find . -type f -ls | while read line; do set -- $line ""; echo $7; done
You could also use ls similarly, but ls gets slow for LARGE directories, as it must sort the results. Find will just traverse the directory, spitting stuff out in disk order, meaning you can do processing on each file while the next directory entry is being read. I think ls (at least on Solaris) must be implemented with bubblesort, it's SLOW for any N over 100,000 or so, no matter how big the box is.
FWIW, GNU has their own version of getopt, for handling long arguments.
*shrug*
I've never actually used it, except maybe without knowing about it.
The odd case where I need to handle -- arguments, I just use string-to-character lookup function and shortcut the getopt loop with a goto during the the '-' case.:)
I think you and I are from the same school of sysadmin-dom. But you'd never catch me using mc. Wuss!
Here is a common pattern I use:
a="`echo filepattern`" echo $a ls -lrt $a | head ls -lrt $a | tail export a su echo $a rm $a ^D
This is quick to type and accurate for certain VERY sensitive directories where a) errors are unforgiveable b) MANY files exist c) Files DO NOT contain spaces
> % for i in foo bar baz; do echo somecommand $i; done
for i in foo bar baz; do echo somecommand "$i"; done
Quotes make this quite a bit safer, in case foo, bar, or baz have space in them. And don't forget this construct:
find/dir filepatter\* | while read file; do echo "$file"; done
The backslash, BTW, is more important that most junior sysadmins think. But it is only used if something in your local directory matches filepatter*....
I will almost certainly not be a 7 times speed difference on a modern UNIX platform. Most of that 0.084 seconds is probably taken up by fork..exec and pipe connecting.
Depending on the kernel, cat, and grep implementations, it's actually possible that it would be faster with the extra cat; but that's unlikely IME. The big thing, however, is that idiot boy committed the CS measurement faux-pas of the un-leet who understand nothing.... he applied ONE measurement to an entire class of problems, and that measurement was made only to back up his intuition and get people like you, to make wildly insane assertiations based upon single-datum linear extrapolation.
Incidentally, don't talk to me about non-modern UNIX platforms w.r.t. your example, because they would all break 48 gigs before your file made it into their filesystems anyhow.
On the otherhand, I would never actually write that, because if I had the archive in place, I'd just transfer it with scp and untar it myself on the remote end. Unless, of course, cat in the example is just a place holder for 'arbitrary cool shit in the pipeline'.
HEY! PYTHON WEENIE! YEAH YOU, UP THERE!
Let's see you do this in your bloatware:
(cd/updateDir && find . -newer timestamp -type f | tar -T - -zcf -) | ssh user@foo 'cd/stagingDir && tar -zxvf -'
Incidentally, dropping the "z" flags and adding "-C" to ssh will make this totally cross platform, even to non-gnu-land, back as far as ssh 0.99 without significant penalty or performance difference. A reasonable alternative, before about 1992, would have been:
(cd/updateDir && find . -newer timestamp -type f | tar -T - cf -) | compress | rsh -l user foo 'cd/stagingDir && compress -d | tar -xvf -'
Moo hoo hahaha
What would you python weenies do if confronted with an AIX 2 or a SunOS 4 box? Go home and backport the behemoth? Any one of my sysadmins -- who have never used either of those OSs but know shell -- could solve that problem in five minutes flat.
Slashdot Mirror
He should hire James Spader AND William Shatner.
That way, if he loses in court, he can get Scotty to beam some tribbles down to the court room.. and in the ensuing mayhem, escape through the gate to Abidos.
Or, it could be the third option, which is actually the most common IME - a pulse-width modulator. This is why fluorescent lighting ballasts don't work with dimmers. They need more than a fraction of a cycle to start up the discharge arc.
That's bitten me a few times with AOhelL users... their proxy sometimes dynamically changes.
Mind you, nobody's bitched about it in a couple of years now. Maybe their proxies are smarter now?
MmmMmmmm.
Eatin' for free tonight!
Yummy cookies.
Whatabout sun cluster 3? I particular, for the global filesystem?
Last I looked, they said, "NO! NO ZFS! IF YOU TRY, THE MOONS OF URANUS WILL CRASH INTO PLUTO!"
Depending on the checksum type, it is indeed possible to recover corrupted data. For a concrete example, consider the case of RAID 5.
Right.
Similar issues exist without problems when mmap()ing files from NFS. You cannot update just a few bytes with NFS, you have to write the whole disk block out.
I'm fairly confident that the current "standard" way to implement mmap at the moment is to update the pages, mark them dirty, and let the VM subsystem write them to disk.
I haven't had to look at mmap's implementation in a long time, though.. but IIRC Rich Teer and/or Adrian Crocroft had good articles about it a few years back.
Obviously, I arrive at this problem space with a huge Solaris influence. But this is a well-understood problem, and I don't think Sun's implementation is particularly revolutionary.
Dude, his post is 100% on the money.
The problem is that you apparently missed his point entirely.
Quick point of order, there is no requirement to use GNU tar with gzip!
gzcat filename | tar -xvf -
will work exactly the same as
tar -zxvf filename
That said, point taken -- my gzcat (Solaris 9 sparc stripped no non-standard DSOs) weighs in at a hefty 63K. I could probably get it down 30% from there with TFS and a C compiler by throwing out some features, but it would take a couple of days, and really, there would be no point.
> Unfortunately, in the context of Unix, there's nothing 'portable' about using gzipped files.
You're absolutely correct, but missing the mark.
The moronic FA author was trying to portably gunzip files. Given those requirements, gzcat is a reasonable prerequisite. Thus providing gzcat with arguments which would be portable to zcat is unnecessary, as zcat could not complete the task required in the first place.
BTW, I don't know what kind of machines you admin, but I have the luxury of administering primarily sparc/solaris boxen these days. This means that I can ship a Solaris 2.5.1 binary version of GNU tar and gzip with my install packages, and it will run on any of my machines, assuming they are not more than 11 years old. Sun ABI compatibility is quite good. I could even ship 2.4 binaries without issue, extending my box-age to 12 or 13 years, but that seems kind of pointless since all the boxes are less than 5 years old, with the exception of the workstation I use to build Solaris 2.5.1 binaries (an ultra 5 running 2.5.1 hardware 11-97).
I suppose I'm going to have issues at some point the future, though, as we are considering moving to Opteron machines. (If they would rip out the BMC and replace it with OpenBoot, we would have moved long ago). In that case, I'll probably have to snag gzip and tar from an old Solaris 7/x86 box I still have around.
I'm a UNIX guy who's been using UNIX and derivatives for something like 2/3 of my life.
This post is bang on the money, and, in my estimation, approximately 3113.7% more useful than TFA.
Howabout WABI?
...unfortunately, sudo did not exist when I started with UNIX.
Well, more specifically, it did not exist outside of CU-Boulder, and certainly did not ship with the version of slackware (which I downloaded at school and brought home on FLOPPY DISKS because 2400bps was too slow) I was using at the time.
Point taken, though -- sudo *is* a valuable tool, I should really integrate it into my daily habits... but I've got the "su XXX ^D" pattern pretty much hardwired into my mental pathways.
If there isn't, it actually wouldn't be too hard to fix what you want, assuming that your smbfs is implemented a fairly normal unixish vfs switch. (If you don't know what this is, you really gotta read the samba docs, they did a really nice job explaining this concept).
Take a look at the rename system call which is hopefully addressed by this switch, detect the condition, and repair it.
Hmm. Scratch that idea, a little bit of debugging reveals that it's a red herring: ...The problem here is that mv checks for identical-ness before calling rename() -- it's probably checking the inode numbers in lstat. (In fact, I just manned rename, it doesn't even have an errno for same-name).
SO. Looks like the solution is to rewrite mv. That should actually be easier!
Find the lstat checks (probably actually calls to stat). If the inode number comes up the same, see what type of filesystem the files are on (statvfs?). If it's an smbfs, open your new file, carefully, like this:
open(path, O_WRONLY | O_CREAT | O_EXCL, 0600)
If THAT succeeds, go through to the rename call, otherwise it really *is* the same file.
I think that will work.
It would probably piss off POSIX though. Fixing the mount options of the smbfs would be more portable.
My perl is weak, but I believe your solution fails address the cases where filenames have spaces, or where then are subdirectories in the directory hierachy which should be left alone.
Never mind the fact that you're escaping to the shell TWICE in one line of perl to the damned job done. Not to mention spawning a new shell for each and every file to be renamed!
You have written the equivalent of this program: .... hopefully re-writing it like this (in a real programming language) makes your problems more obvious.
A note for the pedants: I have not tried to compile the above C program, and so definately do not warrant that it will not shoot off your foot!
Note of course, that there are a few minor I/O bugs in this program that are not present in yours. I would normally never use a non-abstracted read() statement from a pipeline and assume it's going to completely fill the buffer. (Although it would in 99.999999% of reasonable cases). I don't know what perl does in these odd situations (for example, a pipeline stall, EAGAIN, ENOLCK, EIO, or (especially) EINTR).
Oh, for the love of God, stop bitching about how hard this is to do under UNIX without package x.y.z installed and imagine doing it under Mac OS9 or Windows.
... should do the trick. If you don't have tr, you can use sed with the y command. But I can't think of the last time I saw a box w/o tr.
Break it down into its constituent parts: Iterate, rename. Whoo! Simple now, eh?
# cd cameraDir
# find . -type f -prune | while read file
> do
> mv "$file" "`echo \"$filename\" | tr '[A-Z]' '[a-z]'`"
> done
#
IMNSHO, wildcards should only be used where shell completion won't work at all. This is part of instilling good habits. :)
/bin/test on most unices).
:)
/dev/null.
I prefer [ -e $DIR ] || mkdir -p $DIR, but this is exactly the same solution as yours. (/bin/[ is a soft link to
And I think you meant 2>/dev/null >/dev/null.
Your solution backgrounds the command and redirects stdout to
Well, assuming you're not using bash. That's the case with sh and ksh.
Entertainly enough, the linux b0yz of the late '90s thought that the dash was superfluous everywhere, and so made ps bitch if you used it.
.bashrc
:)
echo 'export I_WANT_A_BROKEN_PS=fu' >>
oughtta fix that for ya.
Now, if only I could figure out how to quickly fix the linux xterm termcap on every linux box I use....
This jackalope probably once something that LOOKS like this, didn't understand it, and is now spouting it as "Wisdom".
He probably saw somebody type:
ls -1 | xargs
Which, of course, is the same thing as ls | xargs on a non-broken ls.
By non-broken, I mean that ls -1 is the default behaviour of ls when isatty(STDIN_FILENO) or isatty(STDOUT_FILENO) return non-zero.
Or, perhaps, he's planning to use xargs to parse, say, filesizes out of ls? But that would be incredibly annoying, especially if your filenames contained spaces.... or something.
I dunno. I have always found xargs in newb books, but I have VERY RARELY used it in the real world. I am a big fan of using find to key iteration.
find dir -type f | while read file; do echo "$file"; done
Want the filesize instead?
find . -type f -ls | while read line; do set -- $line ""; echo $7; done
You could also use ls similarly, but ls gets slow for LARGE directories, as it must sort the results. Find will just traverse the directory, spitting stuff out in disk order, meaning you can do processing on each file while the next directory entry is being read. I think ls (at least on Solaris) must be implemented with bubblesort, it's SLOW for any N over 100,000 or so, no matter how big the box is.
FWIW, GNU has their own version of getopt, for handling long arguments.
:)
*shrug*
I've never actually used it, except maybe without knowing about it.
The odd case where I need to handle -- arguments, I just use string-to-character lookup function and shortcut the getopt loop with a goto during the the '-' case.
I think you and I are from the same school of sysadmin-dom. But you'd never catch me using mc. Wuss!
/dir filepatter\* | while read file; do echo "$file"; done
Here is a common pattern I use:
a="`echo filepattern`"
echo $a
ls -lrt $a | head
ls -lrt $a | tail
export a
su
echo $a
rm $a
^D
This is quick to type and accurate for certain VERY sensitive directories where
a) errors are unforgiveable
b) MANY files exist
c) Files DO NOT contain spaces
> % for i in foo bar baz; do echo somecommand $i; done
for i in foo bar baz; do echo somecommand "$i"; done
Quotes make this quite a bit safer, in case foo, bar, or baz have space in them. And don't forget this construct:
find
The backslash, BTW, is more important that most junior sysadmins think. But it is only used if something in your local directory matches filepatter*....
I learned that the hard way, too, back in my first compiler construction course...
# rm file.y file.l
# ls
file.y.out file.y.cc file.o blah blah
you get the picture
# rm *
Permission Denied: You were building as root, shithead
# su -
Password:
# rm *
<whirr whirr>
Me: Why is my harddrive going crazy?
<whirr whirr>
Me: oooooh SHIT!
^C^C^C^C^C^C^C^C^C^C^C^C^C^C
CTRL-ALT-DEL
System: INSERT BOOT DISK
Me: Hello? Faculty of Arts and Sciences? Yes... This is student 338-xxxx. I'd like to drop CISC 458..
(Glad I re-took it the next year, though. VERY good course, better prof too!)
I will almost certainly not be a 7 times speed difference on a modern UNIX platform. Most of that 0.084 seconds is probably taken up by fork..exec and pipe connecting.
Depending on the kernel, cat, and grep implementations, it's actually possible that it would be faster with the extra cat; but that's unlikely IME. The big thing, however, is that idiot boy committed the CS measurement faux-pas of the un-leet who understand nothing.... he applied ONE measurement to an entire class of problems, and that measurement was made only to back up his intuition and get people like you, to make wildly insane assertiations based upon single-datum linear extrapolation.
Incidentally, don't talk to me about non-modern UNIX platforms w.r.t. your example, because they would all break 48 gigs before your file made it into their filesystems anyhow.
Yeah!
Can you show me point 0 8 4 of a second? I didn't think so!
Don't blame me, I'm not a mathematitian!
You should be using gzcat, not zcat, anyhow. zcat is only portably able to be compress -d.
/updateDir && find . -newer timestamp -type f | tar -T - -zcf -) | ssh user@foo 'cd /stagingDir && tar -zxvf -'
/updateDir && find . -newer timestamp -type f | tar -T - cf -) | compress | rsh -l user foo 'cd /stagingDir && compress -d | tar -xvf -'
gzcat will never be broken in the way described, hence the following is fine and portable IME:
gzcat arc.tar.gz | ssh user@foo 'cd tmp/a/b/c && tar -xvf -'
HOWEVER, I find that even vaguely modern CPUs are much faster at gunzipping than typical internet speeds. So, I would use this myself:
cat arc.tar.gz | ssh user@foo 'cd tmp/a/b/c && gzcat | tar -xvf -'
On the otherhand, I would never actually write that, because if I had the archive in place, I'd just transfer it with scp and untar it myself on the remote end. Unless, of course, cat in the example is just a place holder for 'arbitrary cool shit in the pipeline'.
HEY! PYTHON WEENIE! YEAH YOU, UP THERE!
Let's see you do this in your bloatware:
(cd
Incidentally, dropping the "z" flags and adding "-C" to ssh will make this totally cross platform, even to non-gnu-land, back as far as ssh 0.99 without significant penalty or performance difference. A reasonable alternative, before about 1992, would have been:
(cd
Moo hoo hahaha
What would you python weenies do if confronted with an AIX 2 or a SunOS 4 box? Go home and backport the behemoth? Any one of my sysadmins -- who have never used either of those OSs but know shell -- could solve that problem in five minutes flat.