Slashdot Mirror

The rsync algorithm docs say that [...] the much more expensive strong checksum must be calculated for each block where the weak checksum matches.

So the strong checksum (MD5) must be cached for a lot of blocks (typical application: files are nearly identical), and block size is usually 500 - 1000 bytes. Is 1/30th the size of the original (500 bytes vs. 128 bits) acceptable for the checksum cache?

Agreed. All the technical issues are solveable, and in fact have been solved at some level; and the namespace would not significantly change in size regardless of structure (unless it shrinks majorly, as name-squatting becomes far less profitable).

The problems are non-technical. One is the FUD being spread about the technical issues (I am *really* tired of hearing all this nonsense about a supposed technical need for artificially scarce namespace) and another is the problem of rule creation and enforcement.

ICANN and the Department of Commerce control the rule-making and enforcement process (by holding the root nameservers hostage). It seems that they will not allow a better system to evolve.

Internet users' best bet is probably to end-run ICANN. Just as we can use samba and samba-tng to defeat Microsoft's attempt to dominate our networks, we can use OpenNIC and friends to obviate ICANN.

PS: I included the definition of obviate because I got savaged for using it in a post once. I can use the most arcane networking terms imaginable and nobody complains, but use a slightly offbeat non-technical word and everyone's suddenly too busy flaming me to look it up. ;^>

Samba-TNG was originally an unstable CVS branch, run by people from the Samba team. However, the project has now forked, and is developed by a separate group. It's vaguely similar to XEmacs vs GNU Emacs, although the details are very different.

If you want the unstable version of Samba, try the Samba 3.0 alpha snapshots. Many of the domain integration features will be in this development series. If I understand correctly, some of the code is reused from Samba-TNG (both projects are GPL'd), but most is rewritten.

As Andrews says in the open letter, diversity is good: you can try -TNG and 3.0a and see which one suits you.

Samba-TNG was originally an unstable CVS branch, run by people from the Samba team. However, the project has now forked, and is developed by a separate group. It's vaguely similar to XEmacs vs GNU Emacs, although the details are very different.

If you want the unstable version of Samba, try the Samba 3.0 alpha snapshots. Many of the domain integration features will be in this development series. If I understand correctly, some of the code is reused from Samba-TNG (both projects are GPL'd), but most is rewritten.

As Andrews says in the open letter, diversity is good: you can try -TNG and 3.0a and see which one suits you.

Samba-TNG was originally an unstable CVS branch, run by people from the Samba team. However, the project has now forked, and is developed by a separate group. It's vaguely similar to XEmacs vs GNU Emacs, although the details are very different.

If you want the unstable version of Samba, try the Samba 3.0 alpha snapshots. Many of the domain integration features will be in this development series. If I understand correctly, some of the code is reused from Samba-TNG (both projects are GPL'd), but most is rewritten.

As Andrews says in the open letter, diversity is good: you can try -TNG and 3.0a and see which one suits you.

Samba-TNG was originally an unstable CVS branch, run by people from the Samba team. However, the project has now forked, and is developed by a separate group. It's vaguely similar to XEmacs vs GNU Emacs, although the details are very different.

If you want the unstable version of Samba, try the Samba 3.0 alpha snapshots. Many of the domain integration features will be in this development series. If I understand correctly, some of the code is reused from Samba-TNG (both projects are GPL'd), but most is rewritten.

As Andrews says in the open letter, diversity is good: you can try -TNG and 3.0a and see which one suits you.

Andrew Tridgell (original Samba developer) thinks the fork is a good opportunity to reinvigorate the project and try other things.

Before anyone gets off on a huge rant about this fork being pointless/harmfull/etc, read this - it's a statement by Andrew Tridgell, saying that he is "delighted" about the fork...

ahh the wonders of CUPS and SAMBA. Just download SMB Browser and you are in the run! (Assuming he has Jaguar, remember to click Windows sharing)

Well, because for the particular case of building software, distcc probably does a better job than OpenMOSIX -- although to be fair nobody has done back-to-back tests yet. Have a look on the web site for more details, but briefly:

distcc is easier to install: you don't need a kernel patch, you don't need root, you don't even need all the machines to be running Linux or the same CPU architecture.

distcc is optimized for compilation: it's not clear that MOSIX's process migration/IO redirection stuff is really very efficient for this type of work. For some projects, distcc gets over 90% of the theoretical max scalability, which is pretty good.

A small, specific tool generally beats a complex generalized one. OpenMOSIX is very cool, but if you want faster builds easily distcc is the way to go.

Well, on the whole, fairly uninteresting.

But in case people don't know about it. distcc, the clustering solution he used rocks! It's just a wrapper for gcc, and works on most platforms. We use it here at work on Mac OSX!

http://distcc.samba.org/

super easy to set up too!

When you've checked out the sources once, it's enough to do "cvs -z3 update -dP" in the ffmpeg and MPlayer directories. It will only fetch the files that have been updated since last time. BTW I'm using distcc to split the compilation work between my two machines.

KDE developers put their computers through a lot of work.

Perhaps they should consider using ccache to avoid recompiling unchanged sources and distcc to make use of the almost-inevitable collection of low-powered machines every developer seems to have?

KDE developers put their computers through a lot of work.

Perhaps they should consider using ccache to avoid recompiling unchanged sources and distcc to make use of the almost-inevitable collection of low-powered machines every developer seems to have?

I found this in one of the posts, looks promising.
Cached compliation, basicly like pre-compiled headers on a per make run basis. No changes required.

Building KDE on my modern desktop (1.4 GHz Athlon, 512 MB RAM) takes 6-8 hours. Many developers are working on systems which cannot fully build KDE in under 24 hours

Given that most source files don't get changed every build, the major problem is going to be disk space. We need to donate some of those old 10 Gig disks that are too small for our MP3 (sorry, Ogg) collections now.

Oh, and we should give them all a pointer to Compiler Cache while we're at it.

Must install that on my own system, it look sweeeeet.

I hope you're not suggesting that:

a) Australians can't be nerds or geeks
b) that finding little Johnny's email address isn't a valid exercise for said nerds and geeks

Maybe there are just a lot of us around. And, after all, where does Samba come from?

Alister

You could argue that including only the very specific headers you need for each source file is the best way to go, but I think it is a reasonable trade-off to include all these static system/library headers in a precompiled header, then to re-reference the specific headers in the user source code to indicate the dependency explicitly. I totally agree with the stuff in the book about people binding modules together too tightly through inter-dependencies in user headers though (although I'm not convinced by everything he talks about :)

I usually chuck almost all the stl and alot of boost into my pre-compiled headers when setting up a build, which cuts the full rebuild time at least in half, usually more. I'm always reluctant to do a full rebuild of the same module under g++ as I know it'll take a long time (comparatively). Admittedly the larger each source file is, the less the benefit, but I tend towards lots of small to medium sized source files anyway.

I'm not sure how the gcc version will do it, but the msvc (boo, hiss etc.) version actually takes a memory dump of the parsed code tree when pre-compiling, then just copies this back into memory for successive compilations, so the speed increase is dramatic. Hopefully the gcc version does something similar (or better). Be nice if it had something similar to ccache built in as well.

Yes, the do have SIDs. However, mapping SIDs to UIDs is not an easy job especially since SIDS are really a GUID (like {0000002F-0000-0000-C000-000000000046}) which are too long to be cleanly mapped in a UID. Take a look at the Samba's Winbind project here for some more details about the problems involved.

The end result is that there isn't a clean mapping between the two domains. And the implementations of NFS for windows that I have seen have been hackish at best and disfunctional at worst.

Using the windows protocols (CIFS or whatever it is being called these days) may be the best option.

A recent Microsoft-sponsored study by researcher IDC concluded that servers based on Microsoft's Windows 2000 were cheaper to own and operate when used for networking, storing and sharing files, printing and security

What's interesting is that open source has the lower TCO for implementing open standards (e.g., web serving). In other words, by using open source to implement open standards both saves me money and gives me greater freedom.

Ultimately, there's a tipping point when enough hardware is running open source that it becomes worthwhile to move to non-microsoft standards. Then the game then becomes unfavorable to microsoft. The real TCO analysis should be done on companies that have implemented pure open source solutions without having to insert microsoft products in the mix. There are now examples out there in retail and the city of
Largo, Fl. We should see analyses of those.

If you do not do the following, please use -MM. It autogenerates dependencies so that when you change a header, affected object files are autorecompiled. It makes everyone's life better.

C++ is still hideously slow to compile with g++.

ccache should be a standard developer tool. Tremendously helpful. Speeds up builds after "make clean"s by a huge amount.

The ecomomical answer isn't to invest tons of money in developing faster CPUs, it's to throw available CPU cycles at the problem -- stealing cycles from other systems.

If you use a un*x varient, check out distcc
http://distcc.samba.org
It should make your day much more productive.

Want to know more about it?

Of course my MP3 collection is quite a bit larger than my /home.

Xix.

I agree with most of your post, but I'll bite on semantics because I have some links to dump...

"While various linux filesystems support ACLs, no one is using them yet."

Nobody? Then how come Samba has had support for them in the server (with kernel support) and the client since version 2.2.0?

For filemanagers, what is it that exporer.exe can do that nautilus or konqueror can't? I mean except freeze up when you unplug the network cable... I'm puzzled.

I agree with most of your post, but I'll bite on semantics because I have some links to dump...

"While various linux filesystems support ACLs, no one is using them yet."

Nobody? Then how come Samba has had support for them in the server (with kernel support) and the client since version 2.2.0?

For filemanagers, what is it that exporer.exe can do that nautilus or konqueror can't? I mean except freeze up when you unplug the network cable... I'm puzzled.

I agree with most of your post, but I'll bite on semantics because I have some links to dump...

"While various linux filesystems support ACLs, no one is using them yet."

Nobody? Then how come Samba has had support for them in the server (with kernel support) and the client since version 2.2.0?

For filemanagers, what is it that exporer.exe can do that nautilus or konqueror can't? I mean except freeze up when you unplug the network cable... I'm puzzled.

here is a complete howto...

creating and adding a samba BDC. samba used as your pdc and your bdc's.

Like an eraser, or a bottle of tippex, that's what the "undo" button is there for. All this means is that the save process has to be a bit more sophisticated and store the last n changes.

A more sophisticated file system could help us there. During the day, we rsync the development areas every 15 minutes. It takes a trivial amount of space and CPU time. Yet for years I was stuck in the metaphor of doing nightly backups and telling folks they couldn't get back the files they changed in the morning.

The point is that saving files or versions in case we stuff things up shouldn't be our problem. We should have 'hard' commit points (this is a published document/reviewed code). Between then 'soft' checkpointing could be managed by the OS.

NTLM auth is bug 23679, and is scheduled for Mozilla 1.3 alpha

That's good news considering any Java server can now negotiate NTLM Auth using jCIFS and use the credentials to access SMB resources.

Your mention of dependencies made me think of ccache which is memoization for the C compiler (based on the older 'compilercache'). It means you can just recompile all the files, all the time, and while the performance won't be as good as having a makefile which only does the minimum compilations necessary, it will be almost as fast and a lot less hassle to maintain.

Another AC pointed out that name mangling stops you mixing C++ object files from different compilers. I just echo that here because nobody has so far moderated up his post.

Here's the point: if I develop confidence in bogofilter and build a good database, then I can use it as an ingress filter on some large (>8000 members) mailing lists that I run. Every spam that gets through there costs us bandwidth, and wastes the time of every subscriber. In addition, spam messages which do get through to the list typically generate additional traffic, either by people complaining about spam, or in bounces from recipients whose servers reject it.

So the return from a good spamstopper is potentially enormous.

Any messages which are false positives can be handled by some other means; e.g. forwarding to a postmaster for review.

I'm building a heartbeat cluster to serve WebGUI pages and files via samba.

This going to be presented at a congress for the Netherlands Network User Group November 13th (a mostly Novell and Microsoft NT association).

I have been looking for a solution to mirror files between the two cluster nodes. SCSI is just too expensive for this, since low cost is one of the requirements. I've been trying to compile DRDB on my gentoo 1.3 systems but the 2.4 kernel isn't supported by the default DRDB distibution yet.

Does anyone know about any other projects like these that actually work?

I thought that recent gzip compressed files were "rsyncable", in that the blocks they consist of are designed to remain as invariant as possible given the slight differences in content of the files. See, for example, this patch.

If the internal temperature on your TiVo reaches a certain point, you're greated with an image of the TiVo dude in flames, with a message "Your TiVo is on fire! Call 911 now!". http://tivo.samba.org/download/belboz/firegood.jpg .

This is in the FAQ: I suspect OpenMOSIX will not speed up compilation very much, because small bursty tasks, like compilers, probably perform poorly on MOSIX's process-migration architecture. As I understand it, it should work much better for interactive use or scientific computing where the tasks can usefully stay on each machine for a long time.

But if you want to post some comaritive performance measurements I'm sure everybody would find it interesting.

using distcc and ccache

from the above link:


distcc & ccache
Has anybody yet thought of integrating distcc & ccache?

Yes, of course. They work pretty well as separate programs that call each other: you can just write CC='ccache distcc gcc'. (libtool sometimes has trouble with that, but that's a problem that we will fix shortly and it is a separate question.)

This is very nearly as efficient as making them both part of a single program. The preprocessor will only run once and the preprocessed source will be passed from ccache straight to distcc. Having them separate allows them to be tested, debugged and released separately.

Unless there's a really strong argument to do otherwise, having two smaller programs is better practice, or at least more to my taste, than making one monolithic one.

Distcc has a problem with gdb.
It appears when a file is pre-processed on the host machine gcc does not record the directory to the pre-processed output file. When the pre-processed file is farmed out to a remote gcc build machine the remote gcc compiler (not knowing any better) compiles the file and records the remote machine's directory to the object file. Now when a user tries to debug the program gdb cannot find the source directories.
Unfortunately, this "debugging" bug has to be fixed within GCC itself. This thread describes how GCC might be patched to allow gdb to work correctly with distcc, but at this time no action was taken.
This is not a huge problem - distcc is still great for production builds.

Distcc has a problem with gdb.
It appears when a file is pre-processed on the host machine gcc does not record the directory to the pre-processed output file. When the pre-processed file is farmed out to a remote gcc build machine the remote gcc compiler (not knowing any better) compiles the file and records the remote machine's directory to the object file. Now when a user tries to debug the program gdb cannot find the source directories.
Unfortunately, this "debugging" bug has to be fixed within GCC itself. This thread describes how GCC might be patched to allow gdb to work correctly with distcc, but at this time no action was taken.
This is not a huge problem - distcc is still great for production builds.

slightly off topic, but i've found that ccache to be amazing at speeding up compiles when developing code.

it basically hashes (after a cpp pass) and caches. alot of times one has to make clean, tweak a Makefile.am, change a preprocessor variable, or work with multiple different branches, such that most source files are still the exactly the same. in that case, huge speedups.

-- p

ccache is a compiler cache.
It acts as a caching pre-processor to C/C++ compilers, using the -E compiler switch and a hash to detect when a compilation can be satisfied from cache.
This often results in a 5 to 10 times speedup in common compilations.

The binary incompatibility issue only exists for C++, but it is still very important. I took a brief look at the distcc manual, and it did mention (see this section) that you may want to use the same version of g++ on all machines if you are compiling C++. I would change "may" to "must"; you really don't want to take the risk of having incompatibility bugs.

Regardless of the language, if some of the machines are running a different OS, or have a different architecture, they will have to do cross-compilation and things will get more complicated.

Colin Howell

In that case you might like to look at ccache which is a compiler cache for a single machine.

It will cache the compiler options for each source file and the resultant object file generated. I use a lot when I'm building packages for software - which require multiple compilations. It works very nicely - I'd love to see how well it would integrate with distcc....

Does it matter whether the license is dual or not? If it's GPL, then the GPL rules apply, and I still stand by my statement.

My statement mostly applied to newer projects.. Samba, for instance, is GPLed

I mentioned the Wine project. While it's not a server application, it's a very popular piece of software and will probably play a big role in creating an alternative to our current dominant platform. In fact, the Wine project switched their license to LGPL (which still protects users rights) because they saw the potential for everyone to get screwed over.

My point was that I think the success and momentum behind Linux is due to it's licensing. KDE, Samba, Wine, Linux, etc are all what end users see, and it is those people who the GPL seeks to protect.

Go here

After you've got your answer, why don't you buy a tshirt to say thanks? I've done it before too and its a nice gesture.

Go here

After you've got your answer, why don't you buy a tshirt to say thanks? I've done it before too and its a nice gesture.

I'll treat this message as if it's not a troll.

If you're dealing with a free Unix (Linux, BSD etc), the most 'standard' way for mounting network partitions is using NFS (the Network Filesystem.)

Several companies will sell you NFS utilities for Windows. nfsAxe is by the people who make WinaXe, a Win32 X server. A quick search doesn't turn up a standard Windows open-source solution for this.

SMB has been rebranded by Microsoft as CIFS, the Common Internet File System. Microsoft have all the official docs, but of course samba.org have more information about it than they do.

Samba is supported by all Windows machines. It doesn't even work too badly for sharing filesystems between unices (I have a public SMB share on a FreeBSD file server machine mounted on my Linux gateway: you wouldn't know it wasn't a local FS.) The permissions model isn't perfect, of course, but for a shared FS, it works good.

Your question asked "is there a better way." Well, without getting into what's wrong with Samba, it's hard to answer. If you want Windows interoperability (and it's hard to find a situation where it's not a plus), you can't go wrong with Samba. It's a very mature, stable, complete solution.

Check out why samba doesn't use threads: here.

judge-for-yourself is amusingly lame. If you feel like you're not getting enough Weight Loss, Make Money Fast, and Net Detective offers in email, jfy is the perfect place to find them.

There should be a smbmount command to do such a thing, though I do not remember, let me look...

findsmb seems to do the trick, though iirc there's a way to do so manually with smbclient or nmblookup.

I wonder if this will be available outside the US, specifically in Australia...Tridge's brilliant hacks for addinf Ethernet and PAL support aside, I'd really love one of these things.

Of course, I suppose it's all worth nothing without a good supply of TV Guide type information. Sofcom is probably the best we've got down here, but I imagine a real service would include a lot more meta information connecting the episodes in a series etc.