Firstly, I do think it's great that the guy released it under the GPL. But I still think the open source community can do a better job of commenting its code.
And I don't agree that I should "shut up" just because I'm not willing to comment this particular piece of code. I was making a point about open source code in general.
You can view some of my code in the following projects on SourceForge:
Both those links take you to web interfaces to the CVS repositories for the projects. POPFile is written mostly in Perl for all platforms, PrivateSearchBar is written in C++ for Windows.
> Having uncommented code is a hell of a lot better than having no code at all.
Agreed.
And the open source model means that someone can come along and fix that problem and many others. And yes I sympathize with doing reverse engineering. In my first job I reverse engineered Novell's DOS TCP/IP API and it's ugly work. Of course, having the code would have made my life easier.
But... that's not a good reason to object to the idea that open source software can do better. It can set a high standard.
> That's what I do with the commentless code I get from my co-workers. Do I bitch at them to add comments?
I truly believe that this is a mistake. I believe that either you or they should be fixing that situation so that the third person to look at the code understands it quickly and easily.
> Code is what runs ! Code makes me money ! I don't want these guys oppressed by my bitching of some bureaucratic requirements doc or checkin procedure.
I was *not* suggesting bureaucracy, I was suggesting that code can be made clearer by comments and that in the long run makes it more maintainable and that will make you and me more money (in the closed source world).
So I actually downloaded the code to see what's up and I'd say that the chances that someone is going to rewrite this in Java are pretty damn low. Think whatever you want to think about Visual Basic but this code is almost entirely uncommented, and whatever language you are writing in that's a crime.
In fact a lot of the comments are... commented out code.
Here's a quick sample which actually has comments in it, one of which is a totally useless explanation of what the "Randomize" function does in Visual Basic, the second is just about is pointless.
(If you are not initiated into the VB way of doing things: a ' starts a comment)
'Randomize Random Number Function Randomize
'Set Vis Team Home Bonus VisTeam.HomeBonus = 0
If GstrMatchType = "Neutral" Or _ GstrMatchType = "SONeutral" Or _ GstrMatchType = "C1Neutral" Or _ GstrMatchType = "C1SONeutral" Or _ GstrMatchType = "C2Neutral" Or _ GstrMatchType = "C2SONeutral" Then
HomeTeam.HomeBonus = 0
Else If GstrHB = "S" Then If HomeTeam.HomeBonus <= 8 Then HomeTeam.HomeBonus = 8 ElseIf HomeTeam.HomeBonus > 8 Then If HomeTeam.HomeBonus > 13 Then HomeTeam.HomeBonus = 13 End If Call CheckHomeBonus(HomeTeam) End If ElseIf GstrHB = "A" Then If HomeTeam.HomeBonus <= 3 Then HomeTeam.HomeBonus = 3 ElseIf HomeTeam.HomeBonus > 3 Then If HomeTeam.HomeBonus > 8 Then HomeTeam.HomeBonus = 8 End If Call CheckHomeBonus(HomeTeam) End If End If End If
Call MatchRevenue(VisTeam, HomeTeam)
This sort of code is a pet peeve of mine. Just because it's been released under the GPL means nothing if others can't actually work on the code and make sense of it. The story says that the extensive manual will help (which it will a little), but some explanation of the actual code is vital for any open source project to survive beyond a few programmers initiated into the dark secrets of the code.
If we want open source to thrive we need to start having some standards of coding so that others can use the code. Companies do this internally for a reason: they know that they are going to hire new people who are going to need to look at the existing code.
Why do so few project bother with comments? It's almost as if the Open Source Initiative should have a quality stamp that gets given to projects that actually care about their longevity.
I avoided mentioning which project because I'd get accused by the Slashdot crowd of hyping it... but since you ask: POPFile.
Here's what I wrote in response to someone's question about what's happening with POPFile:
[snip]
I've actually reached a point with POPFile where 100% of my effort is going into the test suite. I've become sick of seeing silly little bugs get introduced that could easily be caught by an automated suite and I've been writing code to test the complete program. At this point I have all the infrastructure in place... the test suite even counts the lines of code executed to determine the code coverage and produces colored HTML reports showing which lines have not been tested.
Currently the test suite gives the following output:
Running test suite with code coverage
Running TestBayes.tst... ok (1172 ok) Running TestConfiguration.tst... ok (69 ok) Running TestHTML.tst... ok (4 ok) Running TestLogger.tst... ok (30 ok) Running TestMailParse.tst... ok (2185 ok) Running TestModule.tst... ok (25 ok) Running TestMQ.tst... ok (27 ok) Running TestProxy.tst... ok (77 ok) Running TestWordMangle.tst... ok (24 ok)
3613 tests, 3613 ok, 0 failed
Coverage of POPFile/Configuration... 100% Coverage of POPFile/Module... 100% Coverage of POPFile/Logger... 100% Coverage of Proxy/Proxy... 100% Coverage of Classifier/WordMangle... 100% Coverage of POPFile/MQ... 100% Coverage of Classifier/MailParse... 100% Coverage of Classifier/Bayes... 53% Coverage of UI/HTTP... 15% Coverage of UI/HTML... 1%
I'm aiming for as close to possible to 100% line coverage for the complete program. I estimate that bug August 1 I will have the complete suite in place.
[snip]
Essentially I'm aiming for a professional suite of tests that cover the code. I do this in the day time for my paying job so I know it can be done...
You might be interested in this view of the code coverage system I wrote. Here's one module after running through the test suite. Red lines haven't been tested.
I'd start with not setting aside an entire day for work, that's just overwhelming for anyone. When you start by telling yourself "I'm going to work all day" you are probably going to fail because just the sheer length of the day and magnitude of what you have to get done can become overwhelming.
The key to fixing your problem is probably to make the tasks in front of you not seem so overwhelming through a number of techniques. I sympathize with your plight because as a student myself I had a hard time initially, but it's worth knowing that over time your ability to work hard for longer will improve... like so many other things it's a question of practice.
Music may or may not help you, that seems to be a very personal thing. I can't stand to have music while I work (because I want to listen to it and not work) but have a colleague who has music on (low volume) all the time. Personally I have found that the quiet droning voices on NPR help keep my mind on the job and if something I really do care about comes on it's a little welcome break from what I am doing.
You might also find that some other non-work activities actually bring more focus when you are working. If you go to a gym, run or do some other physical exercise I've found that it has a great effect on concentrating the mind. If you are drinking a lot of caffeine laden drinks while working you might find that cutting back enables you to concentrate more because you are not overstimulated by caffeine.
But specifically...
1. Prioritize the work
Sit down and make a list of all the tasks that you have to get done. I use a real paper notebook for that sort of thing because it's satisfying to cross them off as you go.
Once you've made the list order them (1, 2, 3,...) in terms of how much of the job you'll get done, or how hard they are to do. If you knock off a few hard tasks at the start when you are more focused you'll start to feel better and the smaller tasks coming later will seem less overwhelming. (I think in the Seven Habits book this is "Put first things first"---but really it's commonsense, if you get out of the way the stuff you are dreading doing you'll feel better and get more done).
For example, right now I am working on the test suite for my open source project and it's *boring*, *long* and *hard*. But I've got a list and slowly by slowly I'm seeing progress.
One reason that lists can be problematic is if you write down all the tasks and realize that you haven't got enough time... hence the next topic...
2. Set yourself some goals
It's important to take your list and set some goals. "I'm going to finish task X by lunch". Then try to stick to them. If you find yourself unable to stick to the goals and timings then go back and replan. You'll have a better idea of how long the task is going to take and that will motivate you more... Thinks "If I finish Y tonight, then tomorrow I'll just need to do A, B and C"
3. Reward yourself
I've found that stopping my main tasks and doing a little other task that I find interesting is a good way to keep the motivation up. For example, I'll have a goal "finish X" and when I've done it I'll stop and do something unrelated which I enjoy.
For example on my open source project I have this long boring test suite to write, each time I complete a task I work on a fun task associated with the performance of the project. You can do something similar which means you actually praise yourself through a reward for going something done.
4. Eat well
Nothing like being hungry to screw things up. Eat good food, stop for meals and eat them.
At any rate, thanks for the comments. It's heartening to hear from people that what you are doing is professional, especially when faced with the challenges of the demands of people worldwide on your "little" project.
I assume from what you've written that the problem is that the person who wants to employ you does not want the source code to become open and you'd like to see an open source version of the code.
The key thing to clear up is who owns the copyright on the code. If you own the copyright then you can choose how and when you release the code (open or closed). But it's vital that you keep control of the copyright since it gives you the maximum flexibility. (This is achieved in my project, POPFile, through the POPFile License Agreement).
Specifically,
1. You should make clear to the employer that you hold the copyright and that the code is valuable property which you are willing to license to them in exchange for X.
X could be a job with them, or it could be $$$ or royalties. Exactly what depends on what you want out of the agreement.
2. The license to the company needs to be non-exclusive (giving you the freedom to license to someone else), or exclusive with an exception for an open source version of the code.
3. Once the agreement is in place release the code under the GPL. This will help protect the company's investment because anyone else using the code will be forced to release their code lowering the likelihood that someone else will try to make money off it.
4. When you get contributions from the community who are using the GPL code make sure that you get signed agreements from the contributors transferring copyright to you so that your source base is not contaminated and you maintain control of the copyright. (I've included the text of the agreement we use for POPFile below for reference).
5. Make clear in your contract with the company who owns copyright on the changes that they make or that you make while employed by them. The best solution is that you keep the copyright for yourself.
6. You should expect that the open source version of the code will make the company lower what they are willing to pay (they are after all sharing the code with someone else). You need to argue back that in fact you will be leveraging the open source community to improve the product free of charge to them.
The FSF has a page covering copyright issues here: http://www.fsf.org/licenses/gpl-faq.html and here: http://www.fsf.org/licenses/why-assign.html
John.
Here's what we use for POPFile...
[snip]
POPFILE LICENSE AGREEMENT
CONTRIBUTION DESCRIPTION:
John Graham-Cumming ("jgc") acknowledges, with many thanks, the receipt by jgc from Licensee of the above-described Contribution ("Contribution") to the POPFile software and its related documentation.
Licensee confirms to jgc that, to the best of Licensee's knowledge and belief, the Contribution is free of any claims of parties other than Licensee under copyright, patent or other rights or interests ("claims"). To the extent that Licensee has any such claims, Licensee hereby grants to jgc a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use the Contribution as part of the POPFile software and its related documentation, or any derivative versions thereof, at no cost to jgc or its licensed users and without any accounting obligation to Licensee of any kind, and to authorize others to do so.
Licensee hereby acknowledges that jgc may, at his sole discretion, decide whether or not to incorporate the Contribution in the POPFile software and its related documentation.
EXCEPT AS OTHERWISE PROVIDED HEREIN, LICENSEE MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, WITH RESPECT TO THE ABOVE-DESCRIBED CONTRIBUTION. BY WAY OF EXAMPLE, BUT NOT LIMITATION, LICENSEE MAKES NO AND DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT SHALL LICENSEE BE LIABLE TO USERS OF THE CONTRIBUTION FOR ANY INCIDENTAL, SPECIAL OR CO
Located between Washington, DC and Baltimore at the HQ of the NSA this is a tiny and fascinating museum. You can even try out a real Engima machine.
2. Computer History Museum in Silicon Valley
http://www.computerhistory.org/
This is a good place to get an overview of the history of computing and it's a good excuse to visit Silicon Valley (which is worth seeing but not worth staying in for long because it's essentially a big suburb).
3. Intel Museum
http://www.intel.com/intel/intelis/museum/
Also in Silicon Valley and worth seeing to get an overview of the history of the microprocessor. Of course it's skewed to Intel but worth it.
4. Microsoft Campus and Museum
http://www.microsoft.com/museum/
If you really want to understand the power of Microsoft go to the HQ. There's a great little Microsoft Museum that takes you through the company's history and the sheer size of the campus is worth contemplating.
5. MIT Museum
http://web.mit.edu/museum/
A lot of very important stuff has happened at MIT and the museum is a good way to get oriented to the place. Plus Boston has its own computer museum and is a cool city to visit.
6. HP garage in Palo Alto
Visit the place where HP started: 367 Addison Avenue, Palo Alto, CA. While in Palo Alto take a walk around Stanford University's campus and contemplate the Bills Gates center.
7. Air and Space Museum, Washington, DC
http://www.nasm.si.edu/
You can spend at least 2 days here looking at, sitting in and understanding more things than you ever wanted to know about earth and space flight.
8. Kitty Hawk, NC
http://www.kitty-hawk.com/
Visit the place where (4 miles south on Kill Devil Hill) the first human controlled power flight occurred).
The problem is that white on white text needs to be turned into spaces. Since the user doesn't see the text it shouldn't be used for making a decision about whether the email is spam or not. Typically a spammer will do something like this:
Buy Viagra! How are you?
Whatever renders the HTML needs to not include the words "How are you?" since they are not going to be displayed to the user and could poison the filter.
The spammers already know these tricks and so this does not increase the threat. From ActiveState's perspective it's good marketing because it gets news stories on Slashdot.
This is very close to what we do in POPFile. But we realized that it was not necessary to do the full render (which would be time consuming) because the presence of the tricks is enough to spot the spam.
In POPFile we use the presence of a trick as an indicator of the type of mail and feed that into the Bayesian engine.
All of the tricks outline on the ActiveState web site are covered in the latest version of POPFile with "pseudowords" such as trick:invisibleink. The presence of such tricks is a strong indicator of spamminess.
The irony is that the more the message is obscured the easier it is to filter.
It's worth noting that my research shows that 85% of all spam is in HTML format which I think is due to the fact that it's easy to include these tricks AND it's easy to make the spam message more compelling with colors, fonts, and pictures.
I personally agree that HTML email is evil and wrote an article on this very topic here: http://www.usethesource.com/cgi-bin/article.pl?sid =03/04/07/122224
The research shows the recognizing phrases instead of words has little additional value of the basic Bayesian system and it's computationally expensive and takes up extra storage.
Bayesian filtering can be made to work well with HTML tricks and in fact the tricks can be used to make the Bayesian system more accurate. In POPFile we use the tricks as another indicator of the type of message.
Specifically, the background/foreground contrast of text is calculated using Pythagoras' Theorem over the two color RGB values to determine readability of text. Use of unreadable text is covered by the Invisible Ink and Camouflage tricks and handled in POPFile by the Classifier::MailParse module.
That's right. Good spam filters need to perform some level of HTML "rendering" at the very least recognizing the presence of HTML comments and invalid HTML tags and removing them and treating them as not adding whitespace.
More problematic is the use of as a way to break up spam words because it does not add whitespace but is valid.
The spammer typically wants to hide text that a spam filter will read and you will not. The text is full of innocent or random words that are designed to overwhelm the spam filter and make it assume that the message is genuine.
My advice is to not bother trying to do this at work, what you need to do is separate taking exercise from your working environment. I was in the same position as you and took the following actions:
1. Stop drinking sugary soda. Yep, this pretty much sucks at the beginning since it's so addictive but you need to replace soda with water. A can of coke is 140 Calories, a glass of water 0. I still find this tough and get tempted by soda but the key to converting to water was two steps for me: firstly I drank very cold carbonated water instead of soda, then after months switched to plain water. I also told someone (in the company I was working for I had a personal assistant, I told her) what I was doing so that they could stop me if they saw me with a Coke.
2. Get a gym membership. Find a gym with good personal trainers and sign up for both. A personal trainer really keeps you focused and after 6 months you'll be strong, fit and able to train yourself. This is expensive but in my experience well worth it. I go to the gym before work, which is hard, but great because doing hard physical exercise totally focuses your mind afterwards.
3. Don't skip breakfast. This meal is really important to keeping your energy up all day. A really good start is oatmeal with a little sugar and/or a banana sliced up into it. Oatmeal will give you energy for a lot of the day. Skipping breakfast gets things off to a bad start.
4. Geek out on the stats. Use some charting software to track your progress... chart your weight, LDL/HDL cholesterol, max weight you can bench press, resting heart rate etc. and turn exercising into a geek activity.
Good luck, John.
Leverage the open source reputation
on
Funding Open Source?
·
· Score: 2, Interesting
Another way that you can make money off open source is by leveraging the reputation you get from being an open source developer. In my own case I created POPFile an open source email filtering/sorting tool which some people use for spam fighting.
Because the project is open source many people are using it and my knowledge of spam went up, because of that I got invited to the MIT Spam Conference. Because of that I'm now a paid consultant for ActiveState on their Anti-spam Task Force. I never would have been given that job or money if it weren't for the credibility of the open source tool I created.
In fact it's clear that ActiveState *likes* the open source part of the equation because having me there buys them credibility. (No they are not paying me to write this entry).
In addition to ActiveState I got other offers to consult for people on spam and email. This is a good way to make money without affecting the open source project. Nothing in POPFile has even been done for cash.
I think Open Source survives in the same way that volunteering at a kids school, your church, some community group, for Greenpeace etc. works. You make no money from that but many many people do it. The reason is personal satisfcation.
I have an open source project (POPFile) and a real (closed source) job. The latter pays the rent, the former makes me happy.
Ultimately, I'm working to make myself happy too by having enough money to eat etc. So it doesn't all come down to money but how you feel.
Slashdot Mirror
Firstly, I do think it's great that the guy released it under the GPL. But I still think the open source community can do a better job of commenting its code.
And I don't agree that I should "shut up" just because I'm not willing to comment this particular piece of code. I was making a point about open source code in general.
You can view some of my code in the following projects on SourceForge:
POPFile: CVS
PrivateSearchBar: CVS
Both those links take you to web interfaces to the CVS repositories for the projects. POPFile is written mostly in Perl for all platforms, PrivateSearchBar is written in C++ for Windows.
John.
They've set up an official web site for the revival here.
Unfortunately it uses Flash but there's more information about the plans and there's a cool new Blake's 7 logo.
John.
> Having uncommented code is a hell of a lot better than having no code at all.
Agreed.
And the open source model means that someone can come along and fix that problem and many others. And yes I sympathize with doing reverse engineering. In my first job I reverse engineered Novell's DOS TCP/IP API and it's ugly work. Of course, having the code would have made my life easier.
But... that's not a good reason to object to the idea that open source software can do better. It can set a high standard.
> That's what I do with the commentless code I get from my co-workers. Do I bitch at them to add comments?
I truly believe that this is a mistake. I believe that either you or they should be fixing that situation so that the third person to look at the code understands it quickly and easily.
> Code is what runs ! Code makes me money ! I don't want these guys oppressed by my bitching of some bureaucratic requirements doc or checkin procedure.
I was *not* suggesting bureaucracy, I was suggesting that code can be made clearer by comments and that in the long run makes it more maintainable and that will make you and me more money (in the closed source world).
John.
So I actually downloaded the code to see what's up and I'd say that the chances
that someone is going to rewrite this in Java are pretty damn low. Think whatever
you want to think about Visual Basic but this code is almost entirely uncommented,
and whatever language you are writing in that's a crime.
In fact a lot of the comments are... commented out code.
Here's a quick sample which actually has comments in it, one of which is a totally
useless explanation of what the "Randomize" function does in Visual Basic, the second
is just about is pointless.
(If you are not initiated into the VB way of doing things: a ' starts a comment)
'Randomize Random Number Function
Randomize
'Set Vis Team Home Bonus
VisTeam.HomeBonus = 0
If GstrMatchType = "Neutral" Or _
GstrMatchType = "SONeutral" Or _
GstrMatchType = "C1Neutral" Or _
GstrMatchType = "C1SONeutral" Or _
GstrMatchType = "C2Neutral" Or _
GstrMatchType = "C2SONeutral" Then
HomeTeam.HomeBonus = 0
Else
If GstrHB = "S" Then
If HomeTeam.HomeBonus <= 8 Then
HomeTeam.HomeBonus = 8
ElseIf HomeTeam.HomeBonus > 8 Then
If HomeTeam.HomeBonus > 13 Then
HomeTeam.HomeBonus = 13
End If
Call CheckHomeBonus(HomeTeam)
End If
ElseIf GstrHB = "A" Then
If HomeTeam.HomeBonus <= 3 Then
HomeTeam.HomeBonus = 3
ElseIf HomeTeam.HomeBonus > 3 Then
If HomeTeam.HomeBonus > 8 Then
HomeTeam.HomeBonus = 8
End If
Call CheckHomeBonus(HomeTeam)
End If
End If
End If
Call MatchRevenue(VisTeam, HomeTeam)
This sort of code is a pet peeve of mine. Just because it's been released under
the GPL means nothing if others can't actually work on the code and make sense of it.
The story says that the extensive manual will help (which it will a little), but some
explanation of the actual code is vital for any open source project to survive beyond
a few programmers initiated into the dark secrets of the code.
If we want open source to thrive we need to start having some standards of coding
so that others can use the code. Companies do this internally for a reason: they know
that they are going to hire new people who are going to need to look at the existing code.
Why do so few project bother with comments? It's almost as if the Open Source Initiative
should have a quality stamp that gets given to projects that actually care about their
longevity.
John.
I avoided mentioning which project because I'd get accused by the Slashdot crowd of hyping it... but since you ask: POPFile.
Here's what I wrote in response to someone's question about what's happening with POPFile:
[snip]
I've actually reached a point with POPFile where 100% of my effort is going
into the test suite. I've become sick of seeing silly little bugs get
introduced that could easily be caught by an automated suite and I've been
writing code to test the complete program. At this point I have all the
infrastructure in place... the test suite even counts the lines of code
executed to determine the code coverage and produces colored HTML reports
showing which lines have not been tested.
Currently the test suite gives the following output:
Running test suite with code coverage
Running TestBayes.tst... ok (1172 ok)
Running TestConfiguration.tst... ok (69 ok)
Running TestHTML.tst... ok (4 ok)
Running TestLogger.tst... ok (30 ok)
Running TestMailParse.tst... ok (2185 ok)
Running TestModule.tst... ok (25 ok)
Running TestMQ.tst... ok (27 ok)
Running TestProxy.tst... ok (77 ok)
Running TestWordMangle.tst... ok (24 ok)
3613 tests, 3613 ok, 0 failed
Coverage of POPFile/Configuration... 100%
Coverage of POPFile/Module... 100%
Coverage of POPFile/Logger... 100%
Coverage of Proxy/Proxy... 100%
Coverage of Classifier/WordMangle... 100%
Coverage of POPFile/MQ... 100%
Coverage of Classifier/MailParse... 100%
Coverage of Classifier/Bayes... 53%
Coverage of UI/HTTP... 15%
Coverage of UI/HTML... 1%
I'm aiming for as close to possible to 100% line coverage for the complete
program. I estimate that bug August 1 I will have the complete suite in
place.
[snip]
Essentially I'm aiming for a professional suite of tests that cover the code. I do this in the day time for my paying job so I know it can be done...
You might be interested in this view of the code coverage system I wrote. Here's one module after running through the test suite. Red lines haven't been tested.
John.
I'd start with not setting aside an entire day for work, that's just overwhelming
...) in terms of how much of the
for anyone. When you start by telling yourself "I'm going to work all day" you
are probably going to fail because just the sheer length of the day and magnitude
of what you have to get done can become overwhelming.
The key to fixing your problem is probably to make the tasks in front of you not
seem so overwhelming through a number of techniques. I sympathize with your plight
because as a student myself I had a hard time initially, but it's worth knowing
that over time your ability to work hard for longer will improve... like so many
other things it's a question of practice.
Music may or may not help you, that seems to be a very personal thing. I can't
stand to have music while I work (because I want to listen to it and not work)
but have a colleague who has music on (low volume) all the time. Personally I
have found that the quiet droning voices on NPR help keep my mind on the job and
if something I really do care about comes on it's a little welcome break from
what I am doing.
You might also find that some other non-work activities actually bring more focus
when you are working. If you go to a gym, run or do some other physical exercise
I've found that it has a great effect on concentrating the mind. If you are
drinking a lot of caffeine laden drinks while working you might find that cutting
back enables you to concentrate more because you are not overstimulated by caffeine.
But specifically...
1. Prioritize the work
Sit down and make a list of all the tasks that you have to get done. I use a
real paper notebook for that sort of thing because it's satisfying to cross them
off as you go.
Once you've made the list order them (1, 2, 3,
job you'll get done, or how hard they are to do. If you knock off a few hard
tasks at the start when you are more focused you'll start to feel better and the
smaller tasks coming later will seem less overwhelming. (I think in the Seven
Habits book this is "Put first things first"---but really it's commonsense, if
you get out of the way the stuff you are dreading doing you'll feel better and
get more done).
For example, right now I am working on the test suite for my open source project
and it's *boring*, *long* and *hard*. But I've got a list and slowly by slowly
I'm seeing progress.
One reason that lists can be problematic is if you write down all the tasks and
realize that you haven't got enough time... hence the next topic...
2. Set yourself some goals
It's important to take your list and set some goals. "I'm going to finish
task X by lunch". Then try to stick to them. If you find yourself unable to
stick to the goals and timings then go back and replan. You'll have a better
idea of how long the task is going to take and that will motivate you more...
Thinks "If I finish Y tonight, then tomorrow I'll just need to do A, B and C"
3. Reward yourself
I've found that stopping my main tasks and doing a little other task that I
find interesting is a good way to keep the motivation up. For example, I'll
have a goal "finish X" and when I've done it I'll stop and do something unrelated
which I enjoy.
For example on my open source project I have this long boring test suite to write,
each time I complete a task I work on a fun task associated with the performance
of the project. You can do something similar which means you actually praise
yourself through a reward for going something done.
4. Eat well
Nothing like being hungry to screw things up. Eat good food, stop for meals and
eat them.
Good luck,
John.
At any rate, thanks for the comments. It's heartening to hear from people that what you are doing is professional, especially when faced with the challenges of the demands of people worldwide on your "little" project.
Glad to hear that POPFile is working out for you.
John.
The link for POPFile is
http://popfile.sourceforge.net
John.
I assume from what you've written that the problem is that the person who wants to employ you does not want the source code to become open and you'd like to see an open source version of the code.
The key thing to clear up is who owns the copyright on the code. If you own the copyright then you can choose how and when you release the code (open or closed). But it's vital that you keep control of the copyright since it gives you the maximum flexibility. (This is achieved in my project, POPFile, through the POPFile License Agreement).
Specifically,
1. You should make clear to the employer that you hold the copyright and that the code is valuable property which you are willing to license to them in exchange for X.
X could be a job with them, or it could be $$$ or royalties. Exactly what depends on what you want out of the agreement.
2. The license to the company needs to be non-exclusive (giving you the freedom to license to someone else), or exclusive with an exception for an open source version of the code.
3. Once the agreement is in place release the code under the GPL. This will help protect the company's investment because anyone else using the code will be forced to release their code lowering the likelihood that someone else will try to make money off it.
4. When you get contributions from the community who are using the GPL code make sure that you get signed agreements from the contributors transferring copyright to you so that your source base is not contaminated and you maintain control of the copyright. (I've included the text of the agreement we use for POPFile below for reference).
5. Make clear in your contract with the company who owns copyright on the changes that they make or that you make while employed by them. The best solution is that you keep the copyright for yourself.
6. You should expect that the open source version of the code will make the company lower what they are willing to pay (they are after all sharing the code with someone else). You need to argue back that in fact you will be leveraging the open source community to improve the product free of charge to them.
The FSF has a page covering copyright issues here: http://www.fsf.org/licenses/gpl-faq.html
and here: http://www.fsf.org/licenses/why-assign.html
John.
Here's what we use for POPFile...
[snip]
POPFILE LICENSE AGREEMENT
CONTRIBUTION DESCRIPTION:
John Graham-Cumming ("jgc") acknowledges, with many thanks, the receipt by jgc
from Licensee of the above-described Contribution ("Contribution") to the
POPFile software and its related documentation.
Licensee confirms to jgc that, to the best of Licensee's knowledge and belief,
the Contribution is free of any claims of parties other than Licensee under
copyright, patent or other rights or interests ("claims"). To the extent that
Licensee has any such claims, Licensee hereby grants to jgc a nonexclusive,
irrevocable, royalty-free, worldwide license to reproduce, distribute, perform
and/or display publicly, prepare derivative versions, and otherwise use the
Contribution as part of the POPFile software and its related documentation, or
any derivative versions thereof, at no cost to jgc or its licensed users and
without any accounting obligation to Licensee of any kind, and to authorize
others to do so.
Licensee hereby acknowledges that jgc may, at his sole discretion, decide
whether or not to incorporate the Contribution in the POPFile software and its
related documentation.
EXCEPT AS OTHERWISE PROVIDED HEREIN, LICENSEE MAKES NO REPRESENTATIONS OR
WARRANTIES, EXPRESS OR IMPLIED, WITH RESPECT TO THE ABOVE-DESCRIBED
CONTRIBUTION. BY WAY OF EXAMPLE, BUT NOT LIMITATION, LICENSEE MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS FOR ANY
PARTICULAR PURPOSE. IN NO EVENT SHALL LICENSEE BE LIABLE TO USERS OF THE
CONTRIBUTION FOR ANY INCIDENTAL, SPECIAL OR CO
1. National Cryptologic Museum
o v/
http://www.nsa.gov/museum/
Located between Washington, DC and Baltimore at the HQ of the NSA this is a tiny and fascinating museum. You can even try out a real Engima machine.
2. Computer History Museum in Silicon Valley
http://www.computerhistory.org/
This is a good place to get an overview of the history of computing and it's a good excuse to visit Silicon Valley (which is worth seeing but not worth staying in for long because it's essentially a big suburb).
3. Intel Museum
http://www.intel.com/intel/intelis/museum/
Also in Silicon Valley and worth seeing to get an overview of the history of the microprocessor. Of course it's skewed to Intel but worth it.
4. Microsoft Campus and Museum
http://www.microsoft.com/museum/
If you really want to understand the power of Microsoft go to the HQ. There's a great little Microsoft Museum that takes you through the company's history and the sheer size of the campus is worth contemplating.
5. MIT Museum
http://web.mit.edu/museum/
A lot of very important stuff has happened at MIT and the museum is a good way to get oriented to the place. Plus Boston has its own computer museum and is a cool city to visit.
6. HP garage in Palo Alto
Visit the place where HP started: 367 Addison Avenue, Palo Alto, CA. While in Palo Alto take a walk around Stanford University's campus and contemplate the Bills Gates center.
7. Air and Space Museum, Washington, DC
http://www.nasm.si.edu/
You can spend at least 2 days here looking at, sitting in and understanding more things than you ever wanted to know about earth and space flight.
8. Kitty Hawk, NC
http://www.kitty-hawk.com/
Visit the place where (4 miles south on Kill Devil Hill) the first human controlled power flight occurred).
9. Houston, TX or Cape Kennedy, FL
http://www.jsc.nasa.gov/
http://www.ksc.nasa.g
Either will give you a good idea of the history of NASA and the exhibits are great.
10. Boeing Field/Museum of Flight
http://museumofflight.org/
Another great place to visit in Seattle, WA after you've had your fill of Microsoft.
11. FBI HQ Tour
http://www.fbi.gov/aboutus/tour/tour.htm
Worth it for the tour of the forensic labs and (currently closed) the live fire demonstration.
John.
And here's a concrete example...
;
; ; ; ; ;
Run the following:
require HTML::TreeBuilder;
$tree = HTML::TreeBuilder->new->parse_file("test.html")
require HTML::FormatText;
$formatter = HTML::FormatText->new(leftmargin => 0, rightmargin => 50);
print $formatter->format($tree);
with the following input (which illustrates two tricks: invisible ink and the slice and dice table trick):
Buy Viagra!
<p>
<font color="white">It was nice seeing you last night</font>
<p>
<table border=0 cellpadding=0 cellspacing=0>
<tr valign=top>
<td><font face=Courier>V<br>s<br>F</font></td>
<td><font face=Courier>i<br>a<br>R</font></td>
<td><font face=Courier>a<br>m<br>E</font></td>
<td><font face=Courier>g<br>p<br>E</font></td>
<td><font face=Courier>r<br>l</font></td&g t;
<td><font face=Courier>a<br>e</font></td&g t;
<td><font face=Courier> <br>s</font></td>
</tr>
</table>
and you get the output:
Buy Viagra!
It was nice seeing you last night
V
s
F
i
a
R
a
m
E
g
p
E
r
l
a
e
s
So the invisible text is now visible and ready to fool the spam filter and the table has essentially been ignored.
John.
Yes, I read the POD.
The problem is that white on white text needs to be turned into spaces. Since the user doesn't see the text it shouldn't be used for making a decision about whether the email is spam or not. Typically a spammer will do something like this:
Buy Viagra!
How are you?
Whatever renders the HTML needs to not include the words "How are you?" since they are not going to be displayed to the user and could poison the filter.
Yes, I read the POD, it's not very informative.
John.
I don't buy that at all.
The spammers already know these tricks and so this does not increase the threat. From ActiveState's perspective it's good marketing because it gets news stories on Slashdot.
John.
Very funny. Not.
You are lucky though. There are a number of web sites that refuse to sign me up under my real name because it's "offensive".
John.
This would not work well because HTML::FormatText removes all the color and fonts that spammers are using to hide text and doesn't render tables.
John.
This is very close to what we do in POPFile. But we realized that it was not necessary to do the full render (which would be time consuming) because the presence of the tricks is enough to spot the spam.
John.
In POPFile we use the presence of a trick as an indicator of the type of mail and feed that into the Bayesian engine.
d =03/04/07/122224
All of the tricks outline on the ActiveState web site are covered in the latest version of POPFile with "pseudowords" such as trick:invisibleink. The presence of such tricks is a strong indicator of spamminess.
The irony is that the more the message is obscured the easier it is to filter.
It's worth noting that my research shows that 85% of all spam is in HTML format which I think is due to the fact that it's easy to include these tricks AND it's easy to make the spam message more compelling with colors, fonts, and pictures.
I personally agree that HTML email is evil and wrote an article on this very topic here: http://www.usethesource.com/cgi-bin/article.pl?si
John.
The research shows the recognizing phrases instead of words has little additional value of the basic Bayesian system and it's computationally expensive and takes up extra storage.
Bayesian filtering can be made to work well with HTML tricks and in fact the tricks can be used to make the Bayesian system more accurate. In POPFile we use the tricks as another indicator of the type of message.
Specifically, the background/foreground contrast of text is calculated using Pythagoras' Theorem over the two color RGB values to determine readability of text. Use of unreadable text is covered by the Invisible Ink and Camouflage tricks and handled in POPFile by the Classifier::MailParse module.
John.
That's right. Good spam filters need to perform some level of HTML "rendering" at the very least recognizing the presence of HTML comments and invalid HTML tags and removing them and treating them as not adding whitespace.
More problematic is the use of as a way to break up spam words because it does not add whitespace but is valid.
John.
The spammer typically wants to hide text that a spam filter will read and you will not. The text is full of innocent or random words that are designed to overwhelm the spam filter and make it assume that the message is genuine.
John.
Thanks for the kind words.
John.
The field guide only points out the tricks and does not point out how we get around them.
John.
My advice is to not bother trying to do this at work, what you need to do is separate taking exercise from your working environment. I was in the same position as you and took the following actions:
1. Stop drinking sugary soda. Yep, this pretty much sucks at the beginning since it's so addictive but you need to replace soda with water. A can of coke is 140 Calories, a glass of water 0. I still find this tough and get tempted by soda but the key to converting to water was two steps for me: firstly I drank very cold carbonated water instead of soda, then after months switched to plain water. I also told someone (in the company I was working for I had a personal assistant, I told her) what I was doing so that they could stop me if they saw me with a Coke.
2. Get a gym membership. Find a gym with good personal trainers and sign up for both. A personal trainer really keeps you focused and after 6 months you'll be strong, fit and able to train yourself. This is expensive but in my experience well worth it. I go to the gym before work, which is hard, but great because doing hard physical exercise totally focuses your mind afterwards.
3. Don't skip breakfast. This meal is really important to keeping your energy up all day. A really good start is oatmeal with a little sugar and/or a banana sliced up into it. Oatmeal will give you energy for a lot of the day. Skipping breakfast gets things off to a bad start.
4. Geek out on the stats. Use some charting software to track your progress... chart your weight, LDL/HDL cholesterol, max weight you can bench press, resting heart rate etc. and turn exercising into a geek activity.
Good luck,
John.
Another way that you can make money off open source is by leveraging the reputation you get from being an open source developer. In my own case I created POPFile an open source email filtering/sorting tool which some people use for spam fighting.
Because the project is open source many people are using it and my knowledge of spam went up, because of that I got invited to the MIT Spam Conference. Because of that I'm now a paid consultant for ActiveState on their Anti-spam Task Force. I never would have been given that job or money if it weren't for the credibility of the open source tool I created.
In fact it's clear that ActiveState *likes* the open source part of the equation because having me there buys them credibility. (No they are not paying me to write this entry).
In addition to ActiveState I got other offers to consult for people on spam and email. This is a good way to make money without affecting the open source project. Nothing in POPFile has even been done for cash.
John.
I think Open Source survives in the same way that volunteering at a kids school, your church, some community group, for Greenpeace etc. works. You make no money from that but many many people do it. The reason is personal satisfcation.
I have an open source project (POPFile) and a real (closed source) job. The latter pays the rent, the former makes me happy.
Ultimately, I'm working to make myself happy too by having enough money to eat etc. So it doesn't all come down to money but how you feel.
John.