--NOTE: If you're on Slashdot, and you haven't encouraged *all* of your friends and family to install GWX Control Panel (or the GRC tool) on their Win7/Win8 boxes -- you need to step up your game.
--ZFS is much more flexible with mirrored pairs. Build a 5- or 6-disk RAIDZ2, and you have to add the same number of drives ( with the same capacity, e.g. 6x1TB + 6x1TB ) to double the RAID capacity properly and keep your I/O throughput sane.
--However, if you start with a mere 2-disk mirror, you can keep adding mirrored pairs much more easily. Example: Start with 2x1TB WD RED NAS drives in a mirrored pair. Create a mirrored zpool with both disks. You start with ~1TB of redundant mirrored storage, with ~2x the speed of 1 drive for read operations since ZFS can pull blocks from either disk.
--Now add another 2x1TB mirrored pair to the zpool (there are plenty of articles online on how to do this, but I will refer to the official docs): http://docs.oracle.com/cd/E192...
--Wait for resilver to complete: ' zpool status ', and now you have a RAID10 with ~2TB of redundant writable space. You can't survive losing *both* disks in a given mirror "column", (SO BACKUP!) but you *could* lose 1 disk from either mirror set and still be up and running with a "degraded" pool.
--Keep the drive sizes reasonable, and recovery time will be minimized since ZFS only has to resilver the "used" blocks in the filesystem, not the whole disk. (Another big advantage over RAID5/RAID6.)
--Caveat: You can't "shrink" the pool without destroying it and recreating it, but overall that's pretty easy to get used to once you consider all the benefits of the filesystem. Plan your pool capacity accordingly, and allocate entire GPT-labelled disks to the pool. You can expand on-the-fly with negligible "downtime" -- (the pool is still quite usable, but I/O operations will slow down the resilvering.)
- Interactive: o Use Midnight Commander. Works from a text terminal, no GUI needed. Insert to mark files/dirs, F8 to delete. Never had a misfire, even as root.
- Script (for the paranoid, and want a log): IF you have a known dir " destdir=/tmp/blah " with a subdir "/tmp/blah/1 " $ cd $destdir && cd 1 && cd.. && rm -rv 1/* |tee/tmp/rm.txt
- REALLY safe rm, with find: # find multiple (known) names of files > 40 days old in a given directory and delete them bkpath="/mnt/bkpdrive" pathh="$bkpath/work/bkpsys-laptop-p2400-thinkpad-xubuntu-14-04-LTS--64--sda7" cd $pathh && \
find $pathh/* \( -name "bkp*gz" -o -name "bkp*bz2" -name "bkp*lzop" -o -name "flist*" \) -type f -mtime +40 -exec/bin/rm -v {} \;
--If you need to delete multiple levels of subdirectories, or dotfiles -- do it interactively, and use MC.
--You do know you can override the alias by calling the path+exec?
$ alias hostname alias hostname='hostname | cut -d '\''.'\'' -f 1'
$ hostname # calls the alias thisbox
$/bin/hostname # calls the exec direct thisbox.fully.qualified.domain.name
--Plus, bonus:
$ for i in.profile.bash_profile.bashrc/etc/profile/etc/bashrc; do echo $i; grep 'rm -i' $i; done $ grep -R 'rm -i'/etc/bash/*
> Command-line tools that ask for confirmation suck for scripting. Especially if those prompts only occur under specific conditions (such as confirm overwrite).
--Agreed, but you can usually get around those by ' echo y| annoyingscript ', unless you have sudo in the mix... Then you might have to (shudder) use parenthesis or something;-)
--Caveat: ' convert ' requires the " imagemagick " package to be installed; which admittedly most desktop/multimedia-oriented distros provide with the default install these days...;-)
--Still a perfectly good commandline solution tho. To folks like me and thee, arguably SIMPLER than using a GUI frontend for it. But some folks aren't commandline-oriented, unforch. Blasted kids, probably grew up with Win XP instead of DOS...
--Ummm, try using a better terminal emulator? XFCE4 terminal and LXterminal both have Edit menus where you select w/ the mouse and Edit\Copy and paste text. I use Linux as my primary operating system at work and have no trouble copying/pasting with JIRA tickets.
> Also, I have an old laptop that is still fully functional. Sad thing is, I have to rely on distros from Jurrassic Park that are no longer maintained. So I am stuck with outdated and buggy components. Any attempt to update causes the laptop to no longer boot spitting out useless error messages.
--This is an honest attempt to help; can you post the laptop make/manufacturer and specs? Feel free to email me, just put your Slashdot username in the subject with the details.
--If you can get System Rescue CD or Knoppix to boot on your laptop, that is a good start...
--I have some experience with getting older laptops working well with Linux. I have a single-cpu 32-bit Dell with 1GB RAM and a 100GB HD running fine, and also just finished getting an old Vista-era Dell laptop with 1xCPU ((64-bit)) with 2GB RAM running a ZFS 2x500GB external SATA RAID1 mirror with an SMB share.
--BTW, as long as your CPU is 64-bit, getting getting ZFS+Samba working is not hard. You may have better luck with straight Xubuntu 14.04-LTS.
--UPDATE: The VM shit itself and wouldn't reboot after converting the disk from Flat to Growable. Reinstalled, this time with a UFS instead of ZFS root, and the 'zpool' command ISN'T EVEN AVAILABLE.
--Judgement: This is Alpha-level software, not suitable for general distribution yet. No real homepage for the distro except for Sourceforge and no support forum. AVOID unless you like frustration.
--1st impressions: Buggy desktop - hangs when I do Edit \ Prefences in the xfce4 Terminal or the XFCE Panel window, **no text virtual consoles** on screens 1-4... Only 1 virtual desktop by default?? Strange choice for X-windows...
+ installed pkgs joe mc screen netcat == OK
PROTIP: + Installing " lxde " display manager and running " lxrandr " to set the screen resolution seems to get around most of the bugginess so far. Also using LXTERMINAL instead of xfce's terminal.
+ Apt-get update/upgrade works with my proxy settings remembered from the install, which is nice. I was able to install Firefox web browser, but ' apt-file update ' doesn't do anything useful.
--If anyone wants a working Vmware VM of UbuntuBSD, feel free to email me.
--UPDATE: I rebooted the VM and ran the install again, deleted the ZFS config and redid it, proxy went OK (so I didn't waste a lot of bandwidth re-downloading packages) and did the same software selection (1st option, +SSH, +Minimal Xubuntu install) and this time Grub installed OK and I booted into the UbuntuBSD display manager.
--Basically I gave (sda is 15GB) sda1 300MB UFS/boot, sda2 700MB swap, and sda3 ZFS. When I hit "Configure ZFS", I named the default pool "zroot" and configured the Logical volume to have 100MB less than the default displayed size (IIRC.)
--When Selecting and Installing Software, I was able to use my local Squid proxy but the step kept bombing out. I ended up selecting the 1st option + OpenSSH + Minimal Xubuntu install to get it to work. (Had to skip Samba + Print server + Xubuntu desktop.)
--And now installing Grub has failed at "update-grub".
--Don't think this is ready for general release yet. And the distro REALLY needs a Forum where people can login and post.
--Unfortunately, having given kfreebsd a real shot for about 2 years a while back (installing it in a VM and contributing bug reports), I'm not really surprised by my results so far. I continue to hope for the best though, maybe eventually kfreebsd will get to be as stable as everything else but it needs a really dedicated development team (and maybe a paying sponsor.)
> Because ZFS on linux is not yet ready for serious use
--That is not my experience. Killer-app for Linux is ZFS+Samba, and I have had that going with decent performance (for home / small business use, at least) for the last 2-3 years.
--This is with an older quad-core 2.4GHz COTS PC box with 6GB DDR2 RAM running Xubuntu 14.04--64--LTS and standard SATA-2 hard drives connected to (2) inexpensive 4-port PCIe cards. With standard 1500-byte Ethernet frames, I can almost saturate the link (~100-120MB/sec) running an FTP transfer from a RAIDz10 with noatime (2x2 disk pool.)
--The limiting issue is usually the write speed on the receiving side, unless it's going to SSD. (Writes to the pool are admittedly a bit slower IIRC, but zfs *is* doing auto-checking for every write.) But in general, **you don't run ZFS for max speed**, you run it for Reliability -- and features like "copies=2" + snapshots + filesystem-level compression. As well as the occasional quick RAID rebuild.
--What some folks might want to do to speed things up is some research; I have some custom stuff going on in/etc/rc.local (sysctls to speedup I/O, blockdev --setra 8192/dev/sd* , Gig ethernet speedup script, rmmod unused modules, stop all unneeded services, etc. Optimizations are available upon request.) Bog-standard Linux install is not optimized for speed, usually. Plus if you're running all your pool drives off motherboard SATA ports, it can limit your speed. On my rig, the "mirror" drives in the pool are connected to the 2nd SATA card.
--Also - if you want best speed from ZFS, most advice is to run mirrored pools - *not* RAIDZ. I switched out my original 500GBx6 RAIDZ2 to equivalent RAIDz10 (with 2x2 newer 2TB WD Red drives, which aren't even considered hi-speed drives) and the pool performance improved. If I expand the pool to 6 drives, it should pick up even a little more speed. If I wanted to spend more money, I could put a SAS card in and run that instead of SATA.
--An interesting feature of ZFS, BTW - I started out with 2x2TB drives, unmirrored, making a non-RAID writable space of ~4TB. After saving up for a couple of months and buying 2 more drives, I was able to add mirrors to both disks in the pool ON-THE-FLY and converted the pool in-situ to RAIDz10. I did a "burn-in" test (R/W all sectors) on the new drives 1st, and I consider WD to be pretty reliable to begin with, but still - I don't know any other filesystem that you could do that with.
> It is never going to run on a Pentium M or D machine
--I actually had the Win10 Tech Preview running on an old Pentium-M laptop last year (single core, 2GHz, 1GB RAM, 100GB IDE HD.) Finally wiped out that install/test last month (after getting sick of all the obvious spyware builtin to Win10) and upgraded from an old Crunchbang Linux to Antix. Repurposed the laptop as a music jukebox with VLC and Deadbeef.
--Your efforts are appreciated. Keep up the good work:-)
--Aside: A few weeks back I wanted to get in contact with you to recommend something, but it didn't coalesce - so at the risk of going a bit offtopic I will post it here. You may want to see if you can work a deal with O'Reilly books to sell at a discount to Slashdot users. It's tech-related and would seem to be a good match for the site. Just a thought.
> Nobody's ever changed anyone's mind just because of a few pixels on the screen and most folks are not actually interested in holding their views up for scrutiny and changing them when new information comes to light.
*raises hand* I have actually had my mind changed by something I read on the web; whether an article or poster on a site like slashdot.;-)
Slashdot Mirror
--Say what now?
--NOTE: If you're on Slashdot, and you haven't encouraged *all* of your friends and family to install GWX Control Panel (or the GRC tool) on their Win7/Win8 boxes -- you need to step up your game.
/ not targeting parent poster, just sayin'
--ZFS is much more flexible with mirrored pairs. Build a 5- or 6-disk RAIDZ2, and you have to add the same number of drives ( with the same capacity, e.g. 6x1TB + 6x1TB ) to double the RAID capacity properly and keep your I/O throughput sane.
--However, if you start with a mere 2-disk mirror, you can keep adding mirrored pairs much more easily. Example: Start with 2x1TB WD RED NAS drives in a mirrored pair. Create a mirrored zpool with both disks. You start with ~1TB of redundant mirrored storage, with ~2x the speed of 1 drive for read operations since ZFS can pull blocks from either disk.
--Now add another 2x1TB mirrored pair to the zpool (there are plenty of articles online on how to do this, but I will refer to the official docs):
http://docs.oracle.com/cd/E192...
--Wait for resilver to complete: ' zpool status ', and now you have a RAID10 with ~2TB of redundant writable space. You can't survive losing *both* disks in a given mirror "column", (SO BACKUP!) but you *could* lose 1 disk from either mirror set and still be up and running with a "degraded" pool.
--Keep the drive sizes reasonable, and recovery time will be minimized since ZFS only has to resilver the "used" blocks in the filesystem, not the whole disk. (Another big advantage over RAID5/RAID6.)
--Caveat: You can't "shrink" the pool without destroying it and recreating it, but overall that's pretty easy to get used to once you consider all the benefits of the filesystem. Plan your pool capacity accordingly, and allocate entire GPT-labelled disks to the pool. You can expand on-the-fly with negligible "downtime" -- (the pool is still quite usable, but I/O operations will slow down the resilvering.)
--Take a look at the OpenBSD codebase and learn how to code without leaving a bunch of security holes.
--Yep. :-D
/ n00b :b
--I used to run xmms back in the day; ended up replacing it with deadbeef. I still have an alias xmms=deadbeef to start it tho, it's easier to type ;-)
--$22.50 for 1 movie ticket? Dafuq do you live, New York? You never go to a matinee??
--HOWTO safe rm on Linux (at least):
- Interactive:
o Use Midnight Commander. Works from a text terminal, no GUI needed.
Insert to mark files/dirs, F8 to delete. Never had a misfire, even as root.
- Script (for the paranoid, and want a log): /tmp/blah/1 " .. && rm -rv 1/* |tee /tmp/rm.txt
IF you have a known dir " destdir=/tmp/blah " with a subdir "
$ cd $destdir && cd 1 && cd
- REALLY safe rm, with find: /bin/rm -v {} \;
# find multiple (known) names of files > 40 days old in a given directory and delete them
bkpath="/mnt/bkpdrive"
pathh="$bkpath/work/bkpsys-laptop-p2400-thinkpad-xubuntu-14-04-LTS--64--sda7"
cd $pathh && \
find $pathh/* \( -name "bkp*gz" -o -name "bkp*bz2" -name "bkp*lzop" -o -name "flist*" \) -type f -mtime +40 -exec
--If you need to delete multiple levels of subdirectories, or dotfiles -- do it interactively, and use MC.
--You do know you can override the alias by calling the path+exec?
$ alias hostname
alias hostname='hostname | cut -d '\''.'\'' -f 1'
$ hostname # calls the alias
thisbox
$ /bin/hostname # calls the exec direct
thisbox.fully.qualified.domain.name
--Plus, bonus:
$ for i in .profile .bash_profile .bashrc /etc/profile /etc/bashrc; do echo $i; grep 'rm -i' $i; done /etc/bash/*
$ grep -R 'rm -i'
> Command-line tools that ask for confirmation suck for scripting. Especially if those prompts only occur under specific conditions (such as confirm overwrite).
--Agreed, but you can usually get around those by ' echo y| annoyingscript ', unless you have sudo in the mix... Then you might have to (shudder) use parenthesis or something ;-)
--XLERATOR hand dryers FTW. I love those things.
/ dyson is obviously a poor alternative
--Caveat: ' convert ' requires the " imagemagick " package to be installed; which admittedly most desktop/multimedia-oriented distros provide with the default install these days... ;-)
--Still a perfectly good commandline solution tho. To folks like me and thee, arguably SIMPLER than using a GUI frontend for it. But some folks aren't commandline-oriented, unforch. Blasted kids, probably grew up with Win XP instead of DOS...
> What exactly does Linux do for my wife that Windows 10 does not?
--Well, unless you're using an older version of Ubuntu Unity or something, LINUX DOESN'T SPY ON HER... Can't say the same for Win10.
--Ummm, try using a better terminal emulator? XFCE4 terminal and LXterminal both have Edit menus where you select w/ the mouse and Edit\Copy and paste text. I use Linux as my primary operating system at work and have no trouble copying/pasting with JIRA tickets.
> Also, I have an old laptop that is still fully functional. Sad thing is, I have to rely on distros from Jurrassic Park that are no longer maintained. So I am stuck with outdated and buggy components. Any attempt to update causes the laptop to no longer boot spitting out useless error messages.
--This is an honest attempt to help; can you post the laptop make/manufacturer and specs? Feel free to email me, just put your Slashdot username in the subject with the details.
--If you can get System Rescue CD or Knoppix to boot on your laptop, that is a good start...
--I have some experience with getting older laptops working well with Linux. I have a single-cpu 32-bit Dell with 1GB RAM and a 100GB HD running fine, and also just finished getting an old Vista-era Dell laptop with 1xCPU ((64-bit)) with 2GB RAM running a ZFS 2x500GB external SATA RAID1 mirror with an SMB share.
--BTW, as long as your CPU is 64-bit, getting getting ZFS+Samba working is not hard. You may have better luck with straight Xubuntu 14.04-LTS.
--News article reporting a new Web-based viral exploit for the Win10 "bash" shell in 5..4..3..
--UPDATE: The VM shit itself and wouldn't reboot after converting the disk from Flat to Growable. Reinstalled, this time with a UFS instead of ZFS root, and the 'zpool' command ISN'T EVEN AVAILABLE.
--Judgement: This is Alpha-level software, not suitable for general distribution yet. No real homepage for the distro except for Sourceforge and no support forum. AVOID unless you like frustration.
--1st impressions: Buggy desktop - hangs when I do Edit \ Prefences in the xfce4 Terminal or the XFCE Panel window, **no text virtual consoles** on screens 1-4... Only 1 virtual desktop by default?? Strange choice for X-windows...
+ installed pkgs joe mc screen netcat == OK
PROTIP: + Installing " lxde " display manager and running " lxrandr " to set the screen resolution seems to get around most of the bugginess so far. Also using LXTERMINAL instead of xfce's terminal.
+ Apt-get update/upgrade works with my proxy settings remembered from the install, which is nice. I was able to install Firefox web browser, but ' apt-file update ' doesn't do anything useful.
--If anyone wants a working Vmware VM of UbuntuBSD, feel free to email me.
--UPDATE: I rebooted the VM and ran the install again, deleted the ZFS config and redid it, proxy went OK (so I didn't waste a lot of bandwidth re-downloading packages) and did the same software selection (1st option, +SSH, +Minimal Xubuntu install) and this time Grub installed OK and I booted into the UbuntuBSD display manager.
--Setting up root on ZFS isn't exactly intuitive, not sure if I could duplicate what I ended up doing...
REF: http://www.tumfatig.net/201205...
--Basically I gave (sda is 15GB) sda1 300MB UFS /boot, sda2 700MB swap, and sda3 ZFS. When I hit "Configure ZFS", I named the default pool "zroot" and configured the Logical volume to have 100MB less than the default displayed size (IIRC.)
--When Selecting and Installing Software, I was able to use my local Squid proxy but the step kept bombing out. I ended up selecting the 1st option + OpenSSH + Minimal Xubuntu install to get it to work. (Had to skip Samba + Print server + Xubuntu desktop.)
--And now installing Grub has failed at "update-grub".
--Don't think this is ready for general release yet. And the distro REALLY needs a Forum where people can login and post.
--Unfortunately, having given kfreebsd a real shot for about 2 years a while back (installing it in a VM and contributing bug reports), I'm not really surprised by my results so far. I continue to hope for the best though, maybe eventually kfreebsd will get to be as stable as everything else but it needs a really dedicated development team (and maybe a paying sponsor.)
> Because ZFS on linux is not yet ready for serious use
--That is not my experience. Killer-app for Linux is ZFS+Samba, and I have had that going with decent performance (for home / small business use, at least) for the last 2-3 years.
--This is with an older quad-core 2.4GHz COTS PC box with 6GB DDR2 RAM running Xubuntu 14.04--64--LTS and standard SATA-2 hard drives connected to (2) inexpensive 4-port PCIe cards. With standard 1500-byte Ethernet frames, I can almost saturate the link (~100-120MB/sec) running an FTP transfer from a RAIDz10 with noatime (2x2 disk pool.)
--The limiting issue is usually the write speed on the receiving side, unless it's going to SSD. (Writes to the pool are admittedly a bit slower IIRC, but zfs *is* doing auto-checking for every write.) But in general, **you don't run ZFS for max speed**, you run it for Reliability -- and features like "copies=2" + snapshots + filesystem-level compression. As well as the occasional quick RAID rebuild.
http://open-zfs.org/wiki/Featu...
--What some folks might want to do to speed things up is some research; I have some custom stuff going on in /etc/rc.local (sysctls to speedup I/O, blockdev --setra 8192 /dev/sd* , Gig ethernet speedup script, rmmod unused modules, stop all unneeded services, etc. Optimizations are available upon request.) Bog-standard Linux install is not optimized for speed, usually. Plus if you're running all your pool drives off motherboard SATA ports, it can limit your speed. On my rig, the "mirror" drives in the pool are connected to the 2nd SATA card.
--Also - if you want best speed from ZFS, most advice is to run mirrored pools - *not* RAIDZ. I switched out my original 500GBx6 RAIDZ2 to equivalent RAIDz10 (with 2x2 newer 2TB WD Red drives, which aren't even considered hi-speed drives) and the pool performance improved. If I expand the pool to 6 drives, it should pick up even a little more speed. If I wanted to spend more money, I could put a SAS card in and run that instead of SATA.
--An interesting feature of ZFS, BTW - I started out with 2x2TB drives, unmirrored, making a non-RAID writable space of ~4TB. After saving up for a couple of months and buying 2 more drives, I was able to add mirrors to both disks in the pool ON-THE-FLY and converted the pool in-situ to RAIDz10. I did a "burn-in" test (R/W all sectors) on the new drives 1st, and I consider WD to be pretty reliable to begin with, but still - I don't know any other filesystem that you could do that with.
> It is never going to run on a Pentium M or D machine
--I actually had the Win10 Tech Preview running on an old Pentium-M laptop last year (single core, 2GHz, 1GB RAM, 100GB IDE HD.) Finally wiped out that install/test last month (after getting sick of all the obvious spyware builtin to Win10) and upgraded from an old Crunchbang Linux to Antix. Repurposed the laptop as a music jukebox with VLC and Deadbeef.
--Your efforts are appreciated. Keep up the good work :-)
--Aside: A few weeks back I wanted to get in contact with you to recommend something, but it didn't coalesce - so at the risk of going a bit offtopic I will post it here. You may want to see if you can work a deal with O'Reilly books to sell at a discount to Slashdot users. It's tech-related and would seem to be a good match for the site. Just a thought.
--Look into the Cubietruck. I've been running it for the last ~3 years or so as a standalone Linux+Squid cache with an SSD drive attached.
http://www.amazon.com/Cubieboa...
http://www.amazon.com/Eleduino...
^^ I haven't used this one personally, but looks like it may be useful for you...
--REFS:
http://cubieboard.org/
http://www.cubietruck.com/
http://www.cubieforums.com/
> Nobody's ever changed anyone's mind just because of a few pixels on the screen and most folks are not actually interested in holding their views up for scrutiny and changing them when new information comes to light.
*raises hand* I have actually had my mind changed by something I read on the web; whether an article or poster on a site like slashdot. ;-)
/ see sig
--May I ask what cloning software you use / recommend for HD -> SSD Windows 7 / Win10?