Getting Software Added to Unix Distributions?

suso asks: "I've been working on a set of programs called num-utils that I would eventually like to be considered for inclusion in some of the many free Un*x distributions (on the install CDs, etc). So my question is, how does one put their applications on the track to be included in the main distribution of Red Hat, Debian, SuSE, *BSD, and so on? Is this just something that is up to the maintainers or are there submission forms of some kind?"

Comments

Usefulness and Popularity

[turgid]

If your programs are genuinely useful and well-written, they will build up a user base over time. Eventually they will become viewed as worth putting in a distribution.

Simple

[sporty]

1. port it to as many systems as possible, even non-targert systems. possibly AIX, old Digital Unix.. you name it.

2. get the werd out. If people know about your package, it could solve a problem somewhere that would get it installed.

3. support it. if you support it, people will keep using it. even if it is initially crappy, you'll get bug fixes and advice.

4. package it. no one more than me.. 'cept for those that hate it more than me, hate doing custom compiles on a system that doesn't have /usr/ports or emerge.

Then you live on w/ your life. If your software is good and fulfills a need, you'll see it get put in.

Then you can go onto 5. Profit. or ????. YMMV

Make noiseb about it

[makapuf]

Not to sound flamebait, but you're quite right in doing it : giving it the maximum visibility (for example by posting a link to it on a popular news discussion site) will make a few people notice it exists.

Now, the main question is does it do ogg ?

Re:Make noiseb about it

[sukottoX]

i wouldn't be surprised to find your utilities included in the next version of some big distro after this nice self-slashdotting. this probably was the most effective way to get your programs noticed!

Step-wise procedure...

[jkrise]

1. Write software.
2. Take out a coyright in your name.
3. Apply GPL notices to code.
4. Publish code via ftp.
5. Send code to Source Forge and Freshmeat.

Very difficult?

-

Re:Step-wise procedure...

[acidtripp101]

Well, I don't see your point here. The GPL liscence only contradicts the BSD liscence if they are in the same binary (ie kernel code). The BSDs don't have to steer clear of the GNU Compiler Collection because gcc isn't part of the kernel.
I honestly don't think anyone would be THAT much of a zealot about their liscences, but then again..

Reputation + Packaging

[rf0]

Half of it is about reputation and having a good following. The other thing that helps is to supply packages for each distro. i.e rpm for redhat, deb for debian. For others like FreeBSD which have the ports tree see if you can get your files commited

HTh

Rus

Re:do some serious work

[Anonymous Coward]

Sorry to say this, but from what I have seen this set of tools can be written in less than a day.

But if he's put the work in, and got it dead on right, then he might as well save *you* that day, huh? If it's small and useful who cares how complex it really is?

Re:On Perl and command-line utilities

[BJH]

The other thing is that several of the utilities (or at least 90% of their functionality) can be implemented in 1-2 lines of sh or awk.

Really, if you're shooting to be included in a distribution, make it for something worthwhile.

Version?

[pjdepasq]

Dude, you're only at version 0.3. Don't you agree that it's a little early to be talking about adoption by the distributions?

unique?

[battjt]

Your commands are already installed on my system, or aren't needed.

average
awk '{sum+=$1}END{print sum/NR}'
bound
awk 'NR==1{min=$1}{max=$1>max?$1:max;min=$1 < min?$1:min}END{print min,max}'
interval
awk '{print $1-last; last=$1}'
numgrep ( 500 < x < 1000 or x is a multiple of 3)
awk '$1 < 1000 && $1>500 || $1%3 == 0 {print $1}'
numproc
awk '{print $1 [[your math expression expressed in standard infix notation]]}'
numsum
awk '{sum += $1}END{print sum}'
random
awk supports rand(), bash has $RANDOM, Linux has /proc/random for a stream of random data. Any range can be chosen using 'numproc', for instance /1..10/ is 'rand() * 10 + 1'
range
Why are you ever instantiatin ranges? It wastes space. Ranges should be abstractly manipulated.
round(floor)
awk '{print int($1)}'
round(nearest n)
awk '{print int($1/n)*n}'

These commands are easy to use and have a transparency that makes it very clear what the bugs would be, where as num-utils has warnings like

round will drop off the decimal places in decimal numbers. This may cause some calculations to be in error, depending on how you are using the data.

that make me wonder what round does if it has problems with decimal numbers.

Joe

Re:unique?

[Anonymous Coward]

The snide parent post makes a decent point: these utilities need more features to be worth using. Here are some things that I would have found useful on many occasions (and yes, they could be done in awk, but so could split, cut, and csplit):

The ability to pick a column (and the column separator, of course) to work on, rather than the first.

More statistical functions: what's the standard deviation of a column of numbers, etc...?

A suite of programs to work on multiple columns. For instance, given a series of numbers, output an n-period moving average of those numbers. Or: add/multiply two columns against each other and output the result.

Rocket science - no. Able to be accomplished w/ awk and sed? Yes. But still, useful and worthwhile.

Re:On Perl and command-line utilities

[groomed]

The benefits of scripting over languages such as C are grossly overrated.

C requires painstaking attention to detail, but there is a large number of tools to check C programs for common mistakes such as memory allocation errors and pointer bugs. Scripting languages generally do not have an equally powerful toolset to check for runtime problems such as the typing and dependency errors which may occur when (components of) the scripting runtime are upgraded.

Scripts are fragile. Truly the last thing operating systems need is more fragility.

By adhering strictly to a formalized development process which includes unit tests as well as rigorous system tests it is possible to overcome some of this fragility, but then you also lose many of the benefits that scripting is supposed to provide, viz. rapid development and turnaround.

God only knows why colleges keep churning out students who insist that mashed potatoes are the most flexible material to build a bookshelf...

Oh My God.

[pongo000]

I'm sorry, but I think we've all just been trolled. I don't believe there's really an attempt to ask a valid question here. This individual has written a couple of perl scripts, and truly believes they will change the world. He hasn't done any research (no mention of CPAN, thinks that FreeBSD does Perl, etc.), and truly believes that a few Perl math routines will change the world. Can you spell "ego trip"?

But just in case I'm wrong, here's what you do: Point your browser to CPAN. Carefully read the instructions. Submit your scripts. If they're good, they'll get used, you'll make a name for yourself, and will be on the way to The Big Time.

I really can't believe this made /.

Re:On Perl and command-line utilities

[smallpaul]

C requires painstaking attention to detail, but there is a large number of tools to check C programs for common mistakes such as memory allocation errors and pointer bugs.

First, these programs cannot find all or even most such errors. Second, these programs cannot help you with the fundamental problem that you have written more code and on average the number of bugs is proportional to the amount of code.

Scripting languages generally do not have an equally powerful toolset to check for runtime problems such as the typing and dependency errors which may occur when (components of) the scripting runtime are upgraded.

C programs have also been known to break with components of the underlying runtime are upgraded. Type checking tools only catch the most obvious and blatant problems that would have been caught with unit testing anyhow. Subtle problems are as hard to find in either case except that when the script fails it typically gives a very clear traceback whereas the C may merely corrupt memory and then crash at an arbitrary time later. The behaviour of the script is determinstic but the behaviour of the C program could depends on what random crap happened to be in a memory location that you accidentally dereferenced.

Scripts are fragile.

I'm sorry, this is just false. Slashdot is widely acknowledged to be written in piss-poor Perl and is no more fragile than sites written in Java and is probably more reliable than those few sites written in C. Yahoo and Amazon are also substantially written in script (PHP and Perl respectively). If scripts are robust enough for them, they are robust enough for me. But then again, some people still claim assembly and FORTRAN are the one try way of writing large-scale software. Every layer will have adherents long past the time when they should have moved on to take advantage of the fruits of Moore's law.

Truly the last thing operating systems need is more fragility.

He's adding a few tools for doing numerical computations. This is hardly a core part of the operating system kernel.

By adhering strictly to a formalized development process which includes unit tests as well as rigorous system tests it is possible to overcome some of this fragility, but then you also lose many of the benefits that scripting is supposed to provide, viz. rapid development and turnaround.

Let me see if I've got this right. I can spend five days scripting plus five days testing or ten days coding in C and zero days testing. Which code will be more fragile? Or lets try this this way: five days scripting plus five days testing versus ten days coding and five days testing. Seems like I'm still geting "rapid development and turnaround". It is totally, empirically false to say that C-coded programs require less testing than scripts. In general they require more because there is just more code to test.

I am not going to claim that scirpts are always better than C code (and we've hardly touched on middle grounds like Java, C#, Eiffel etc.). But I strongly dispute the assertion that scripted code is in general more fragile than C code or requires more testing. The secret to reliable software is testing, testing and testing. Programming language is a distant second place but even there, C tends towards the less reliable side of the spectrum. Of course with proper testing you can make rock-solid C code just as you can make rock-solid code in any language.

Re:On Perl and command-line utilities

[Arandir]

I think Perl is a pretty awful language, but not including it in FreeBSD strikes me as a stupid decision.

It is included in FreeBSD, and it is installed by default, it's just not a part of the base system. The same situation occurs with XFree86. The base system is the operating system and environment. Stuff that isn't the OS, or isn't expected to be included with the OS, shouldn't be there. Bash isn't there. Emacs isn't there. Why should Perl be there? If you've ever used Slackware, you could consider the base system to be the "A" distribution, and Slackware doesn't have Perl in the "A" distribution. So far I've seen no one argue that it should be there.

What is BSD doing instead? Implementing all utilities in C? Gee, that's bright: let's create lots of unnecessary work for ourselves

Actually, it's making things a heck of a lot easier for the user. In the olden days (a few months ago) you used the Perl version that was in the base system. It was very difficult to use a new version. You were stuck with the default. There were several pieces of software that I was never able to use simply because I was unable to update my Perl. But the situation now is that I can use any version of Perl I want. Upgrading is simple and hassle free.

Rewriting a *few* Perl scripts in C and bourne was a good thing. These are lowest-common-denominator languages, and should be used whenever possible for low level system software (which is not to say that they are necessarily appropriate for higher level software).

In short, you will see absolutely zero drawbacks in FreeBSD if you are a Perl developers. Zero. But you will see several advantages to it not being in the base system.

Linux is not Posix

[Anonymous Coward]

And NT's Posix layer wouldn't really count as a *nix either. GNU/Linux might be Posix compliant, but that's more an issue of GNU than Linux. In general, Un*x refers to the varients of Unix based on Unix code (AIX, Unixware, FreeBSD, etc). *nix means simply Un*x work-alikes which behave reasonably well like a Un*x system. NT really doesnt quality considering its limited Posix layer, but cygwin might. So, in some ways, we're probably more talking about GNU and the like more than the actual OS. But, there's no way to include Unix and GNU is not Unix in a single acronym nicely. If you can think of a better one, I'm all ears.

Re:On Perl and command-line utilities

[73939133]

It's 100 megs of compiled code, not C.

That's obviously what I'm referring to: it's a 100M of binary code corresponding probably to millions of lines of messy C source code. And the notion that a 100M binary install is "lean" is itself absurd.

Also, the overhead of an interpreted language is inappropriate for the kind of command that's expected to be ran often and not take up too much time (like most of the applications found in various toolchains on unix)

That's silly: most of the stuff in a base OS install either is not performance critical at all, or it is I/O bound. You could rewrite init, cron, ps, sendmail, ftpd, apcid, lpd, rpc.mountd, xinit, and almost all other of the programs in a base install in an interpreted language, and system performance would probably improve (because of much better VM sharing) and install size would decrease.

C is used extensively throughout the base operating system, in nearly all applications it's faster, it's proven over the course of decades, and it's very widely understood. Why switch to perl?

The notion that "C [...] is faster" is ridiculous; most C programs are probably a lot slower than they would be if an equivalent amount of time had been spent writing them in a decent high-level language because C makes it so much work to build complex data structures or try different implementations and because C debugging and testing takes so much time compared to high-level languages. The BSD and Linux kernels, as well as many of those "efficient" C programs you area thinking of are full of linear search over potentially large data structures because it's so much more work to do anything better in C. And mostly what has been proven about C is that just about any significant C program has serious pointer and memory management bugs.

C has shown to be particularly unsuitable to building component based software: software built from a framework with dynamically loadable components really needs to be robust against buggy components. What are examples of component based software? The Linux kernel, MS Office, the Gimp, OpenOffice, Mozilla, and lots more. That's why Microsoft is betting on C#--the only way to build large, reliable, component-based systems is to use a language with fault isolation, reliable error detection, and reflection.

My general point is: much more stuff should be in interpreted languages in UNIX/Linux/BSD systems, in a decent, high-level interpreted language. An all-C/shell base install is just the wrong direction. Whether that language is Perl or something else, I don't really care. Perl is probably technically the worst language you could pick, but it is also the most commonly used one.

And, in fact, large parts of systems like the Gnome desktop are written in interpreted high-level languages, so it's not even a question of whether it's happening. It's only some misguided notion of "minimalism" that causes the "base installs" to buck the trend. Of course, again, the notion that there is anything "lean" about a 100M binary install anyway is just laughable.