Slashdot Mirror

openwlanmap.org uses it to display maps of wifi-war-drived-data when you submit any. I scanned wifi-access points while driving to France for a holiday; amazing how many access points you detect even in the middle of nowhere!

Also my toy-project O2OO uses its api (very simple to implement!) to draw car-sensor data of a trip you made on a map. Nice to see how e.g. the load of the engine changes when taking a corner or driving uphill ("duh" I hear you say, but it is nice to see how much it changes).

Entropy Broker: A project to allow one computer to act as a randomness server to others. Could use any actual hardware machine to seed any number of VMs.

See also the comments in the LWN article, where someone with the user name "starlight" simply sends random data over SSH and then the receiving computer uses rngd to mix that data into the entropy pool. Simple, and simple is good.

https://lwn.net/Articles/546428/

http://www.vanheusden.com/entropybroker/

I think it is past time for CPUs to provide hardware random numbers. Via CPUs have done this for years, but Via CPUs are just too slow for most uses. (I used to run my mail server on a Via C3... I am a lot happier now that my server runs on an AMD low-power dual-core.)

Recent Intel chips do have some sort of random number generator (RdRand).

Hardware RNG accessories are available but expensive.

There is the LavaRnd project, which I think is really darn cool. However, I downloaded the source code, and it hasn't been updated since 2003... a decade later, GCC won't even compile the code. (GCC now issues warnings about some of the code and they set the "treat warnings as errors" flag. I didn't experiment with disabling that flag and trying the code out.) Also, the supported hardware list is a short list of decade-old webcams.

(Note: this would be a good project for a high school student or college student who knows C: update LavaRnd so it builds with GCC or Clang, get it working with at least one currently-available webcam, and write a report about it.)

The Raspberry Pi has a hardware RNG as part of the system-on-a-chip, and Linux on the Pi supports it. You could set one up as a randomness server to your VMs, and that would be quite inexpensive. At least the VMs could reseed their PRNGs with random values pulled from the Pi.

http://vk5tu.livejournal.com/43059.html

If you have a sound device, Audio Entropy Daemon may work.

http://www.vanheusden.com/aed/

P.S. Haveged looks interesting... I just discovered it and I don't know how well it actually works.

https://www.digitalocean.com/community/articles/how-to-setup-additional-entropy-for-cloud-servers-using-haveged

I was doing similar research a few days ago.

Some of these are already mentioned...

• Lessfs - v1 is stable, v2 is pre-alpha/alpha. http://www.lessfs.com/
• Blackhole - http://www.vanheusden.com/java/BlackHole/ - requires Java, which seems like a bad idea to me for a block level device, but I haven't tested it yet.
• SDFS from OpenDedup - http://code.google.com/p/opendedup/ - http://www.opendedup.org/ - looks very promising, but may have stalled
• Dedupfs for Ext3 - http://pages.cs.wisc.edu/~kosmatka/dedupfs/
• ZFS. You know about that.
• DragonFly/Hammer - http://www.dragonflybsd.org/hammer/ includes dedup. Competitor to ZFS and Btrfs, also using Btree. Includes block level dedup, but I'm not sure if it's fixed block or not. Suspect it is fixed.
• Btrfs - there's a patch. Not sure if it's in mainlined yet. But without fsck btfs is not trustworthy enough. That's coming soon, but has been for a while. In case you read this as being negative about btrfs, it's not; it's an awesome file system, combining modern ideas and an excellent implementation, but it's still at testing stage for critical data.

Other stuff:

• Dext2 - an idea. No code. http://code.google.com/p/binarywarriors/
• BackupPC, the next version may have block level dedup, it's been suggested/requested. Numerous people pointed out the hard linking scheme it uses. I'm backing up VM images, which is what started me on this block-level dedup search, and when you have a small change in a 60BG file, it's a new file. (Yes, I have thought of schemes to split them.)
• Bacula have been experimenting with block level dedup, fixed and sliding. May be in future versions.
• Bup - https://github.com/apenwarr/bup has many of the ideas. It's not a file system, but could be reconstructed, I think. Based on Git store. I recommend reading http://apenwarr.ca/log/ which has more, and is entertaining. I think this is an excellent approach. Read back in his blog for details on bup ideas.
• SquashFS - for static data.
• Epitome - http://www.peereboom.us/epitome/man/ - for static data too, I think. Not fully investigated.
• I know I saw at least one Google Summer of Code submission about dedup. Haven't followed it up yet, and couldn't find the tab in my browser.
• Interesting conversation - http://news.ycombinator.com/item?id=2932335

By fixed block I mean that the file system does not search out shared data when the blocks are not on block boundaries. So if you add one byte to the beginning of a 10 GB file, and that has the unfortunate consequence of rippling up through all the blocks that make the file, then there will be no block level sharing with the original file. Of course that's a pathological case, but you get the idea.

Original poster, perhaps you could keep us informed of your findings? There's at least me who is also interested.

Slightly related: it would be nice if someone wrote a program that lets you create 3d models for e.g. blender or povray using a kinect. I wrote the beginning for that ( http://www.vanheusden.com/kinect2povray/ ) but don't have the time to extend it so that it combines multiple angles.

I'm sorry to spam an url, but with this program I exactly tried to recreate that old-school feeling: http://www.vanheusden.com/banihstypos/ It is a program I used to play on my MSX home computer, 25 years (or so) ago. For extra nostalgia feeling it doesn't use any fance X11/SDL/whatever but runs on the console.

re:"Interesting problem for dd-wrt"

Agreed.
We are throwing efforts at both the mainline kernel and openwrt.
Openwrt is foundational for dd-wrt and several other (commercial) distributions of Linux on the router. I have a large set of debloated routers already, I'm just awaiting further work on the eBDP algorithm to make better....

http://www.bufferbloat.net/projects/bloat/wiki/Experiment_-_Bloated_LAGN_vs_debloated_WNDR5700

re: "using pings"
httpping is a much saner approach than ping, in many cases. Get it from:

http://www.vanheusden.com/httping/

re: RED & AQM

SFB and CHOKEe are in the debloat-testing kernel, as is eBDP.
RED 93 isn't going to work. nRED may. Experimentation and scripts highly desired. See the bloat and bloat-devel mailing lists for discussions.

Also:

http://www.bufferbloat.net/projects/bloat/wiki/Dogfood_Principle

Also:

I've seen some VERY interesting behavior with tcp vegas over bloated connections.

http://www.bufferbloat.net/projects/bloat/wiki/Experiment_-_TCP_cubic_vs_TCP_vegas

Hmmm, if I remember correctly I already used that algorithm in 2005?

soundsort

(prior art ftw)

Yes, you can.

See my website: Making your Flukso log to a database/RRD tool

I designed a Nagios interface especially for Control Rooms. My program can be run on a large screen hanging on the wall and then display a list of problems. Of course has a nice web-interface for remote configuration ;-)

It is called CoffeeSaint.

This problem has been solved: use EntropyBroker: a physical machine gathers entropy data and distributes this to the virtual machines. If I remember correctly KVM has a special driver for feeding the VM with entropdata from the host system.

.. or one term running multitail.

The only feed I watch is the local Nagios status using nagserv.

There's project dedicated to this on Linux, http://nbd.sourceforge.net/.

If there's nothing similar for windows, you might be able to run it through cygwin.

Actually, this claims to run on Windows: http://www.vanheusden.com/Loose/nbdsrvr/

I also perform these analysis (well, a little less elaborate) on the outcomings of the Dutch "Staatsloterij". One can find these here (updated every month).

If that is worry to you, use my program cryptosync. It compresses and encrypts each file and also encrypts the file- and pathname. All files stay individual files so that you don't need to transfer a tar file of gigabytes.

audio-entropyd: get random data from an audio source
videwo-entropyd: get random data from a video source (webcam, tv-card, etc.)

audio-entropyd: get random data from an audio source
videwo-entropyd: get random data from a video source (webcam, tv-card, etc.)

Multitail!
MultiTail lets you view one or multiple files like the original tail program. The difference is that it creates multiple windows on your console (with ncurses). It can also monitor wildcards: if another file matching the wildcard has a more recent modification date, it will automatically switch to that file. That way you can, for example, monitor a complete directory of files. Merging of 2 or even more logfiles is possible. It can also use colors while displaying the logfiles (through regular expressions), for faster recognition of what is important and what not. Multitail can also filter lines (again with regular expressions) and has interactive menus for editing given regular expressions and deleting and adding windows. One can also have windows with the output of shell scripts and other software. When viewing the output of external software, MultiTail can mimic the functionality of tools like 'watch' and such. http://vanheusden.com/multitail/

So that was the hardware. What about some software to use this source? I.e.: gather the pictures, do some hasing, etc? This would be a nice addition to audio-entropyd.... :-) http://www.vanheusden.com/aed/ http://www.vanheusden.com/ved/ perhaps?

So that was the hardware. What about some software to use this source? I.e.: gather the pictures, do some hasing, etc? This would be a nice addition to audio-entropyd.... :-) http://www.vanheusden.com/aed/ http://www.vanheusden.com/ved/ perhaps?

If you find building something yourself to much of a hassle and you have either a webcam or a soundcard lying around, you could give audio-entropyd or video-entropyd a try.

If you find building something yourself to much of a hassle and you have either a webcam or a soundcard lying around, you could give audio-entropyd or video-entropyd a try.

Why wasn't multitail mentioned? MultiTail lets you view one or multiple files like the original tail program. The difference is that it creates multiple windows on your console (with ncurses). It can also monitor wildcards: if another file matching the wildcard has a more recent modification date, it will automatically switch to that file. That way you can, for example, monitor a complete directory of files. Merging of 2 or even more logfiles is possible. It can also use colors while displaying the logfiles (through regular expressions), for faster recognition of what is important and what not. It can also filter lines (again with regular expressions). It has interactive menus for editing given regular expressions and deleting and adding windows. One can also have windows with the output of shell scripts and other software. When viewing the output of external software, MultiTail can mimic the functionality of tools like 'watch' and such.

If the badsectors on medium a are not on the same place as on medium b, you can use recoverdm to merge those into one correct image.

Sorry for plugging my own project but I think MultiTail can be very usefull for a lot of admin tasks. MultiTail lets you view one or multiple files like the original tail program. The difference is that it creates multiple windows on your console (with ncurses). It can also monitor wildcards: if another file matching the wildcard has a more recent modification date, it will automatically switch to that file. That way you can, for example, monitor a complete directory of files. Merging of 2 or even more logfiles is possible. It can also use colors while displaying the logfiles (through regular expressions), for faster recognition of what is important and what not. It can also filter lines (again with regular expressions). It has interactive menus for editing given regular expressions and deleting and adding windows. One can also have windows with the output of shell scripts and other software. When viewing the output of external software, MultiTail can mimic the functionality of tools like 'watch' and such. For a complete list of features, look here. Multitail can be found here: http://www.vanheusden.com/multitail/.

Sorry for plugging my own project but I think MultiTail can be very usefull for a lot of admin tasks. MultiTail lets you view one or multiple files like the original tail program. The difference is that it creates multiple windows on your console (with ncurses). It can also monitor wildcards: if another file matching the wildcard has a more recent modification date, it will automatically switch to that file. That way you can, for example, monitor a complete directory of files. Merging of 2 or even more logfiles is possible. It can also use colors while displaying the logfiles (through regular expressions), for faster recognition of what is important and what not. It can also filter lines (again with regular expressions). It has interactive menus for editing given regular expressions and deleting and adding windows. One can also have windows with the output of shell scripts and other software. When viewing the output of external software, MultiTail can mimic the functionality of tools like 'watch' and such. For a complete list of features, look here. Multitail can be found here: http://www.vanheusden.com/multitail/.

I bought a couple of temperature-sensors at cleware. They connect through USB and work really well. Wrote a tool for it for easy controlling their devices: clewarecontrol. They also have humidity sensors.

For the situations where /dev/random can't feed enough random-data there are 2 tools for seeding the kernel PRNG: audio-entropyd (which I'm maintaining) and video-entropyd (which I developed myself).
Audio-entropyd takes a stereo audio-device and calculates random bitstreams of what it 'hears' and feeds that to the kernel. video-entropyd is the same altough it retrieves images from a video4linux-device and feeds that (after processing) to the kernel.

For the situations where /dev/random can't feed enough random-data there are 2 tools for seeding the kernel PRNG: audio-entropyd (which I'm maintaining) and video-entropyd (which I developed myself).
Audio-entropyd takes a stereo audio-device and calculates random bitstreams of what it 'hears' and feeds that to the kernel. video-entropyd is the same altough it retrieves images from a video4linux-device and feeds that (after processing) to the kernel.

I found a tool for reading rss-feeds with a bayesian filter on it so that you only get to see the items you're interested in. After learning 100-200 items it works pretty wel. The program is called 'sux0r' and can be found here.
For those who like to give it a try, check my site.

Sorry for that spam but I think this program might actually be relevant: mboxstats generates a statistical report of a mailbox with information like who wrote the most messages, at what time are the most messages written, what is the most used subject, etc. etc.

If you need something that colorizes and/or does regular expression filtering, merging with other (log-)files, multiple windows, etc. etc. then maybe multitail might come in handy.

Initially I wanted to integrate btail into multitail, but multitail is bloated enough already :-)

Hi, You might want to give my program recoverdm a try. It is especially aimed at bad CD-roms, but is also usable for disks (harddisks/floppydisks/zipdisks). It repeatingly tries to read the sector and then uses statistics which outcome is supposed to be the most likely.

When it comes to monitoring logfiles, I prefer my own program MultiTail big time. It took quiet a lot of effort to develop it, but I think I included all functionality that I could think off.
I use it always and install it everywhere. But then, I might be a little biased :)

Altough this might sound like an ad (it is not - it is not commercial) one might take a look at 'libsd': libsd makes ALL applications on your system do a secure delete without changing a single line of code.
It does this by intercepting calls like 'unlink' (delete files) and 'truncate': before deleting or truncating a file, the previous contents is first overwritten with garbage which is forced to disk.
So if you use this library and you delete a file with a password in it, that password should not be recoverable (altough it might still reside in your swappartition...).

To prove that; I once wrote an MSX emulator in GFA basic :-) (on an Atari ST)
I emulated all chips (Z80, videoprocessor, etc.) and ran on average 1,5% of a real MSX (which runs at 3MHz, the Atari ran on 8MHz).
You can find it here.

It really puzzles me why this evolution-program is announced on slashdot.
Ok, the linux-kernel I can understand, but some program clearly not every linux-user is using? This isn't freshmeat.net!

-- Flok 3.0.2

A patch for 2.4.21 can be found here:
http://www.vanheusden.com/Linux/lsecpatch.diff.gz

Just to clarify what this guy is saying:

1) Make all your machines NBD servers. NBD for Linux, NBD for Windows. NBD stands for "network block device" and allows a client to use a server's block device.
2) Set up a master client/server (using Linux or something else with a decent software RAID stack). This machine will be the only NBD *client*, and it will use all the NBD block devices exported by the rest of your network.
3) On the master set up in 2), create a Linux MD RAID array overtop all the NBD devices that are available.
4) Create a filesystem on the brand-spanking-new multi-machine RAID array.
5) Export it back to the other machines via Samba or NFS or AFS or what have you.

Why does only one machine (the "master server") access the NBD devices, you ask? Because for a given block device, there can only be one client accessing it safely. Thus, if you want to make the RAID array available to anything other than the machine which is *running* the array off the NBD devices, you need to use something which allows concurrent access; something like NFS, Samba, or AFS.

Hope that clears it up a bit.

And since the guy is also using windows-boxes, an NBD-server for windows can be found here:
http://www.vanheusden.com/Loose/nbdsrvr/
This version enables you to also export partitions/disks.

It's here! And it's for free!
Now you can recover those heavily scratched CDs yourself and the best part is: you can have the tool for doing so for free! It's actually released under GPL!

So don't longer hesitate, download recoverdm and recover your precious data!

Free!

You can also setup your own alternative.
You just need a large group of people willing to co-operate and then setup a web of "cloudish" proxies. This cloudish-proxy can be retrieved from: www.vanheusden.com/cloudish/

You can also setup your own alternative.
You just need a large group of people willing to co-operate and then setup a web of "cloudish" proxies. This cloudish-proxy can be retrieved from: www.vanheusden.com/cloudish/

You can also setup your own alternative.
You just need a large group of people willing to co-operate and then setup a web of "cloudish" proxies. This cloudish-proxy can be retrieved from: www.vanheusden.com/cloudish/

This lavarnd looks a bit as (as we say in Dutch:) 'old wine in new bottles': video_entropyd is quiet a while available for download.

While you're at it, take a look at audio_entropyd as well: it generates random-values from unconnected inputs of your soundcard.

This lavarnd looks a bit as (as we say in Dutch:) 'old wine in new bottles': video_entropyd is quiet a while available for download.

While you're at it, take a look at audio_entropyd as well: it generates random-values from unconnected inputs of your soundcard.

generally, if i haven't reloaded the file or quit vim, i'll do a 20u and playback changes with ctrl-r. with screen and vim's split windows i can generally keep nearly every file open that i was working on before i went home.

one screen window for model, one for view, one for controller, one for unit tests, one for templates, etc... then another putty multitailing logs, set transparent and on top with vitrite.

then i hope and pray that i don't get laid off.

If you're interested in something like Twin: you might like MultiTail: it enables you to view multiple files and/or the output of multiple commands in one terminal-window.
Link: http://www.vanheusden.com/multitail/

Description:
MultiTail lets you view one or multiple files like the original tail program. The difference is that it creates multiple windows on your console (with ncurses). It can also use colors while displaying the logfiles, for faster recognition of which lines are important and which are not. It supports regular expressions. It has interactive menus for editing given regular expressions and deleting and adding windows. One can also have windows with the output of shell scripts and other software. Also multiple files can be merged into one window.