Slashdot Mirror

Your computer doesn't necessarily use 400 watts all the time. It might use mebbe 40-50w in idle, AKA, no 3D apps running, nothing using too much power.

Here are a few sites with some more info:
http://windows.uwaterloo.ca/Hardware/PC_Power_Cons umption.asp
http://www.codinghorror.com/blog/archives/000662.h tml

Indeed, straight from the Chief Justice Herself, Right Honourable Beverley McLachlin http://www.scc-csc.gc.ca/aboutcourt/judges/speeche s/ComparativeView_e.asp That said, the debate isn't new in fact, http://www.cs.uwaterloo.ca/~shallit/libel3.html decribes how lawsuit can be used to shout down criticism through intimidation. Jeffrey Shallit http://www.cs.uwaterloo.ca/~shallit/ is vice-president of Electronic Frontier Canada http://www.efc.ca/. One of us!

Indeed, straight from the Chief Justice Herself, Right Honourable Beverley McLachlin http://www.scc-csc.gc.ca/aboutcourt/judges/speeche s/ComparativeView_e.asp That said, the debate isn't new in fact, http://www.cs.uwaterloo.ca/~shallit/libel3.html decribes how lawsuit can be used to shout down criticism through intimidation. Jeffrey Shallit http://www.cs.uwaterloo.ca/~shallit/ is vice-president of Electronic Frontier Canada http://www.efc.ca/. One of us!

a low false positive average over an entire set (like <1%) might seem okay, but that doesn't take into account what's important to users.

Microsoft's own Hotmail, of course, is notorious for spamboxing messages like that. And yet the conference is being held at Microsoft, and Microsoft's own spam researchers proudly touted their system in the February 2007 Communications of the ACM.

Something tells me the leaders in the field are sort of missing the point. Simply bringing down the aggregate false positive rate is *not* enough. The measure needs to take into account how often the user actually misses information that's important to them.

The data will be available to participants to do whatever post-hoc analysis they like. They may, for example, wish to classify misclassified mail into genres, as discussed here.:

Caution should be exercised in treating ham misclassification as a simple propor- tion. Extremely large samples would be needed to estimate it with any degree of statistical confidence, and even so, it is not clear what effect differences in proportion would have on the overall probability of catastrophic loss.

But if you're going to rank systems you need some sort of simple summary measure and the logistic mean of false positive and false negative rates works pretty well. Have a look at TREC 2005 or TREC 2006 summary results, for example.

a low false positive average over an entire set (like <1%) might seem okay, but that doesn't take into account what's important to users.

Microsoft's own Hotmail, of course, is notorious for spamboxing messages like that. And yet the conference is being held at Microsoft, and Microsoft's own spam researchers proudly touted their system in the February 2007 Communications of the ACM.

Something tells me the leaders in the field are sort of missing the point. Simply bringing down the aggregate false positive rate is *not* enough. The measure needs to take into account how often the user actually misses information that's important to them.

The data will be available to participants to do whatever post-hoc analysis they like. They may, for example, wish to classify misclassified mail into genres, as discussed here.:

Caution should be exercised in treating ham misclassification as a simple propor- tion. Extremely large samples would be needed to estimate it with any degree of statistical confidence, and even so, it is not clear what effect differences in proportion would have on the overall probability of catastrophic loss.

But if you're going to rank systems you need some sort of simple summary measure and the logistic mean of false positive and false negative rates works pretty well. Have a look at TREC 2005 or TREC 2006 summary results, for example.

a low false positive average over an entire set (like <1%) might seem okay, but that doesn't take into account what's important to users.

Microsoft's own Hotmail, of course, is notorious for spamboxing messages like that. And yet the conference is being held at Microsoft, and Microsoft's own spam researchers proudly touted their system in the February 2007 Communications of the ACM.

Something tells me the leaders in the field are sort of missing the point. Simply bringing down the aggregate false positive rate is *not* enough. The measure needs to take into account how often the user actually misses information that's important to them.

The data will be available to participants to do whatever post-hoc analysis they like. They may, for example, wish to classify misclassified mail into genres, as discussed here.:

Caution should be exercised in treating ham misclassification as a simple propor- tion. Extremely large samples would be needed to estimate it with any degree of statistical confidence, and even so, it is not clear what effect differences in proportion would have on the overall probability of catastrophic loss.

But if you're going to rank systems you need some sort of simple summary measure and the logistic mean of false positive and false negative rates works pretty well. Have a look at TREC 2005 or TREC 2006 summary results, for example.

...I'd stick with uC++: http://plg.uwaterloo.ca/~usystem/uC++.html

Use low-level constructs in low-level languages. If you're writing in a high-level language (like C++) you're doing so specifically to avoid things like pthreads. ...or because it's mandated by work/school.

Figure 17 is particularly interesting.

Yes, that's another thing I didn't consider. CS is such a broad field that it's hard to narrow it down to a single definition. Even on the wikipedia page of computer science there have been numerous debates about what computer science is. Computer science can be anywhere from being a branch of mathematics to a purely technical trade, with all kinds of variations in the middle, depending on who you ask.

And then there's all the specializations you can do with that degree as well. Take for instance the course calendar section for Computer Science at my university (here). I don't think having those options gives is a bad thing (personally, I feel the contrary), but it could probably making having a clear definition everyone can agree on being more difficult.

The volume of spam is definitely up, and most of it is pump and dumps from a very few distinct sources. In December, about 20% of the 30,000 spams I received were for one particular stock.

http://it.slashdot.org/article.pl?sid=06/12/21/231 4241

But it is wrong to say that this new spam requires radical new filtering techniques. That's what the spam solution vendors (whose press releases drive these /. articles) want you to believe so you'll buy their products. In general, word salads, obfuscated words and image spam do not defeat state-of-the-art statistical filters.

See, for example, the recent TREC tests: http://plg.uwaterloo.ca/~gvcormac/trecspamtrack06

These results show that filters achieve about the same results on 2006 spam as on 2004 spam, and those results are pretty good. Ongoing tests show that the effectiveness of filters is unchanged for 2007. In general, the volume of spam has increased, and spammers have tried various methods of defeating spam filters. But their efforts have not been particularly successful against statistical filters.

The article title had me excited for a second - as a (ex-)member of the University of Waterloo's Midnight Sun Solar Race Car Team that broke the world record for the longest distance traveled on a solar car, I thought they were trying to erase us from the record books. I guess the distance record will be safe for next little while longer. :)

I wish you guys the best in your journey ahead! UNSW, for those that don't know, has one of the most advanced photovoltaic research labs in the world and probably still holds all the records for getting the highest efficiency out of Si-cells.

The problem with content based filtering is it either increases the amount of wading due to quality control needs or decreases the amount of wading at the expense of lost messages.

There's no evidence that the statement above is true. A user who has to wade through a mixture of spam and non-spam will overlook some of the non-spam. The question is whether the human or the machine will overlook more. A subsidiary question is, once overlooked, how likely is the message to be retrieved using some subsidiary mechanism (second look, scanning the quarantine, whatever). There *is* evidence that content filters are better than humans at the initial separation of good email and spam, *and* that separate good and quarantine folders improve performance on the second task.

Here are two content-based filters that work very well: OSBF-Lua and Bogofilter. SpamAssassin's "Bayes filter" works well, too, but you have to configure it a bit differently: http://plg.uwaterloo.ca/~gvcormac/spamassassin

I wouldn't bash content filtering if I thought it worked

The Bayesian analysis in spam filters only works on text. Spammers realized that they could get around it by filling the text portion of the message with some random passage from a Project Gutenberg file, thus making it seem innocuous, and then putting the real advertisement in a GIF or PNG file that would be displayed by HTML-capable mail readers. Bayesian analysis can still work, but only in combination with OCR software.

Bayesian filters (and other statistical filters colloqually known as Bayesian) can work on any features at all; not necessarily text. In particular they can use the markup in the header of the message, the message encoding, and so on. Some of the best-performing filters don't use 'text' at all and simply treat the entire message, images and all, as a bit string; for example, compression-based filters. Another well performing filter, OSBF-Lua, uses orthogonal sparse binomial bigrams rather than individual tokens.

Recent standardized testing shows that these methods work just fine on image spam, without any OCR component.

Here's my spam filter setup. Some assembly required.

    http://plg.uwaterloo.ca/~gvcormac/spamassassin.htm l

I think you wanted it like this mailto:gvcormac@uwaterloo.ca for maximum affect.

http://www.uwaterloo.ca/ - swathes of the 1-800 space are blocked for most calls from staff phones.

It's even worse if your cipher of permutations forms a group. In that case, for any two ENC keys k1 and k2, there exists a third ENC k3 such that ENC_k2 (ENC_k1 (x)) = ENC_k3 (x). In other words you can find a third key that produces the same permutation as the composition of keys 1 and 2. This means that breaking a double-encryption (or triple, quadruple, etc) is no harder than breaking a single encryption: the resulting permutation can always be described by a single key.

That's why 3DES uses EDE instead of EEE. While DES doesn't form a group, it does have some group-like structures which reduce the workload quite a bit. This doesn't apply to all ciphers btw; there are many more possible 64-bit permutations than 64-bit keys, so compositions can fall well outside those covered by keyspace.

I think this is covered in chapter 7 of the Handbook of Applied Cryptography.

The Handbook of Applied Cryptography is an excellent book and is now available online for free. If you passed a couple college calculus courses, the material should be very accessible to you. In general, it's an excellent source of facts and analysis to debunk incorrect statements about cryptography.

In particular, the part that answers you question is found in Chapter 7 (block ciphers). The justification for fact 7.33 explains why, if you can store 2^64 blocks of ciphertext, you can break double 64-bit encryption in 2^64 operations.

If your coworker is not capable of understanding the math behind this reasoning, he really should not be making decisions about encryption :-).

(Note: I'm not witholding the justification in this post simply to be obtuse. It's too much of a pain to format it reasonably in this input box... you'll just have to refer to the PDF.)

The Handbook of Applied Cryptography is an excellent book and is now available online for free. If you passed a couple college calculus courses, the material should be very accessible to you. In general, it's an excellent source of facts and analysis to debunk incorrect statements about cryptography.

In particular, the part that answers you question is found in Chapter 7 (block ciphers). The justification for fact 7.33 explains why, if you can store 2^64 blocks of ciphertext, you can break double 64-bit encryption in 2^64 operations.

If your coworker is not capable of understanding the math behind this reasoning, he really should not be making decisions about encryption :-).

(Note: I'm not witholding the justification in this post simply to be obtuse. It's too much of a pain to format it reasonably in this input box... you'll just have to refer to the PDF.)

You're half right. I had forgotten about the quantum properties of transistors.

Though a transister does use Quantum Mechanics to function, it is a discrete unit (a "black box" if you will) with a preidctable outcome. A quantum computer, on the other hand, uses a property of QM known as "superposition of states". A qbit in a quantum computer isn't 0 or 1, but some combination of 0 and 1 at the same time. It's only when the qbit is "observed" (read) that it becomes a 0 or 1.

If we can get these qbits to interact with each other without reading them (or "collapsing the wavefunction", in quantum mechanics lingo) then we can have various superpositions of 0s and 1s interacting with each other within an algoritm. Essentially the algorithm run by the quantum computer is acting in parallel with itself. When we observe the qbits when the algoritm is finished, we see the desired result. I know that sounds like magic, but I've probably explained it poorly. I've explained it better in the past.

Incidentially, someone who is uneducated (not stupid, mind you, just uneducated) may have difficulty distinguishing between the BS in the original article and the more scientifically accepted BS I've spouted. See? That's how these con artists are allowed to succeed!

Hey, we can't help it if people decide to post our videos to ./ and Digg!
[/innocence]

Here are UW's traffic stats, in case anyone's interested:
http://noc.uwaterloo.ca/cgi-bin/14all.cgi?log=cn-r text_gi2&cfg=cn-rtext.cfg

Also note the spikes on Monday and Tuesday from when we posted our last two talks.

Here are the slides from the 400MB video presentation.

Fidelis Assis (who has now gone solo after having participated in the CRM114 project) shows great results for his recent solo effort: OSBF-lua Bratko's PPM spam filter -- the one that did great at TREC -- is not yet packaged as a drop-in filter. Same for my DMC spam filter.

The actual TREC 2005 tests referred to in TFA are here.

Fidelis Assis (who has now gone solo after having participated in the CRM114 project) shows great results for his recent solo effort: OSBF-lua Bratko's PPM spam filter -- the one that did great at TREC -- is not yet packaged as a drop-in filter. Same for my DMC spam filter.

The actual TREC 2005 tests referred to in TFA are here.

TREC 2006 evaluations are now underway.

While it is reasonable to conjecture that spam has changed so as to defeat spam filtering techniques, or will change so as to defeat the PPM technique that did well at TREC, the historical evidence does not support this conjecture. In particular:

• The spam filters tested in 2004 give pretty well exactly the same performance on 2005 and 2006 data.
• New versions of the filters are a little bit better, but not by leaps and bounds, and also get about the same results over the last 2.5 years of data.
• There is no evidence that "Bayesian poisining" is a viable technique for defeating statistical spam filters in anything but a very artifical laboratory environment where the poisoner has access to the recipient's inbox

Andrej Bratko used PPM -- a well-known data compression technique to compress ham and spam separately. Well actually he didn't compress them but just build the statistical model necessary to compress them. Then he simply (tentatively) added the unknown message to each model and chose the one that compressed it best. The general technique of using compression has been mentioned here and elsewhere but Bratko used a much stronger compression scheme and was somewhat clever about it.

I later reproduced Bratko's results using DMC -- a compression schem that I invented 20 years ago -- and got some interesting results. We have a journal article in press describing it and also an evaluation paper at CEAS 2006.

Bratko A., Cormack G. V., Filipic B., Lynam T. R. and Zupan B., Spam Filtering Using Statistical Data Compression Models

TREC 2006 evaluations are now underway.

While it is reasonable to conjecture that spam has changed so as to defeat spam filtering techniques, or will change so as to defeat the PPM technique that did well at TREC, the historical evidence does not support this conjecture. In particular:

• The spam filters tested in 2004 give pretty well exactly the same performance on 2005 and 2006 data.
• New versions of the filters are a little bit better, but not by leaps and bounds, and also get about the same results over the last 2.5 years of data.
• There is no evidence that "Bayesian poisining" is a viable technique for defeating statistical spam filters in anything but a very artifical laboratory environment where the poisoner has access to the recipient's inbox

Andrej Bratko used PPM -- a well-known data compression technique to compress ham and spam separately. Well actually he didn't compress them but just build the statistical model necessary to compress them. Then he simply (tentatively) added the unknown message to each model and chose the one that compressed it best. The general technique of using compression has been mentioned here and elsewhere but Bratko used a much stronger compression scheme and was somewhat clever about it.

I later reproduced Bratko's results using DMC -- a compression schem that I invented 20 years ago -- and got some interesting results. We have a journal article in press describing it and also an evaluation paper at CEAS 2006.

Bratko A., Cormack G. V., Filipic B., Lynam T. R. and Zupan B., Spam Filtering Using Statistical Data Compression Models

TREC 2006 evaluations are now underway.

While it is reasonable to conjecture that spam has changed so as to defeat spam filtering techniques, or will change so as to defeat the PPM technique that did well at TREC, the historical evidence does not support this conjecture. In particular:

• The spam filters tested in 2004 give pretty well exactly the same performance on 2005 and 2006 data.
• New versions of the filters are a little bit better, but not by leaps and bounds, and also get about the same results over the last 2.5 years of data.
• There is no evidence that "Bayesian poisining" is a viable technique for defeating statistical spam filters in anything but a very artifical laboratory environment where the poisoner has access to the recipient's inbox

Andrej Bratko used PPM -- a well-known data compression technique to compress ham and spam separately. Well actually he didn't compress them but just build the statistical model necessary to compress them. Then he simply (tentatively) added the unknown message to each model and chose the one that compressed it best. The general technique of using compression has been mentioned here and elsewhere but Bratko used a much stronger compression scheme and was somewhat clever about it.

I later reproduced Bratko's results using DMC -- a compression schem that I invented 20 years ago -- and got some interesting results. We have a journal article in press describing it and also an evaluation paper at CEAS 2006.

Bratko A., Cormack G. V., Filipic B., Lynam T. R. and Zupan B., Spam Filtering Using Statistical Data Compression Models

TREC 2006 evaluations are now underway.

While it is reasonable to conjecture that spam has changed so as to defeat spam filtering techniques, or will change so as to defeat the PPM technique that did well at TREC, the historical evidence does not support this conjecture. In particular:

• The spam filters tested in 2004 give pretty well exactly the same performance on 2005 and 2006 data.
• New versions of the filters are a little bit better, but not by leaps and bounds, and also get about the same results over the last 2.5 years of data.
• There is no evidence that "Bayesian poisining" is a viable technique for defeating statistical spam filters in anything but a very artifical laboratory environment where the poisoner has access to the recipient's inbox

Andrej Bratko used PPM -- a well-known data compression technique to compress ham and spam separately. Well actually he didn't compress them but just build the statistical model necessary to compress them. Then he simply (tentatively) added the unknown message to each model and chose the one that compressed it best. The general technique of using compression has been mentioned here and elsewhere but Bratko used a much stronger compression scheme and was somewhat clever about it.

I later reproduced Bratko's results using DMC -- a compression schem that I invented 20 years ago -- and got some interesting results. We have a journal article in press describing it and also an evaluation paper at CEAS 2006.

Bratko A., Cormack G. V., Filipic B., Lynam T. R. and Zupan B., Spam Filtering Using Statistical Data Compression Models

Go get VideoLAN client and you can stream download the OGG version. Just open the URL as a Network Stream:

http://www.csclub.uwaterloo.ca/media/files/cormack -spam.ogg

Very handy use of VLC! :)

http://plg.uwaterloo.ca/~gvcormac/spamcormack.html /

Dear Slashdot,
At the university where I work, they have recently adopted a pesky policy banning the use of bitTorrent.
What can I do to fix this ?
Yours faithfully,
Dr. Gord Cormack

The official tests of spamfilters were done in last year's TREC conference, you can read the writeup here (or pdf overview).

You can duplicate those tests yourself if you download the evaluation toolkit (GPL). It's a modular system where you can add a mail corpus (either one of the public TREC ones, or you can make your own trivially), and add a spamfilter package (there are 10 or so to download from the web, or create your own as per documentation).

There's also a video talk given at Microsoft research which should cover pretty much the same ground, if text mode is slashdotted :).

There's a new scheduled test towards the end of the year at TREC 2006.

The official tests of spamfilters were done in last year's TREC conference, you can read the writeup here (or pdf overview).

You can duplicate those tests yourself if you download the evaluation toolkit (GPL). It's a modular system where you can add a mail corpus (either one of the public TREC ones, or you can make your own trivially), and add a spamfilter package (there are 10 or so to download from the web, or create your own as per documentation).

There's also a video talk given at Microsoft research which should cover pretty much the same ground, if text mode is slashdotted :).

There's a new scheduled test towards the end of the year at TREC 2006.

The official tests of spamfilters were done in last year's TREC conference, you can read the writeup here (or pdf overview).

You can duplicate those tests yourself if you download the evaluation toolkit (GPL). It's a modular system where you can add a mail corpus (either one of the public TREC ones, or you can make your own trivially), and add a spamfilter package (there are 10 or so to download from the web, or create your own as per documentation).

There's also a video talk given at Microsoft research which should cover pretty much the same ground, if text mode is slashdotted :).

There's a new scheduled test towards the end of the year at TREC 2006.

I think this is the second time the csclub has been slashdotted with large video files. Last time, it went up to the top of IST and then nothing changed. This time, perhaps things will change, but the general internet connection is still not being maxed out. IST has there mrtg stats online, and you can clearly see when the slashdot article went up :-)

There's also more talks at http://www.csclub.uwaterloo.ca/media/

I agree with Scott Adams about the whole thing.

You might want to check out ONA - Open Network Administrator from Bruce Campbell at U of Waterloo. And his paper from the LISA 2005 conference.
http://ona.uwaterloo.ca/

Warren has made his own problems. I tried Mepis in 2004 and quite liked it. I used it for more than a year and installed it on several people's machines. However, I will not use it any more.

My reasons are several, but one of the top ones is murky licensing.

No doubt somebody from the MEPIS community will loudly declare that licensing is not a problem. If this is the case, exactly how can I get the source to build myself a MEPIS distro?

There has been considerable bad blood in the MEPIS community and former community. I am not a member of any faction. I have done my share to contribute. I simply tried to get my questions answered and MEPIS and Warren came up short. His many rants -- the one cited in the story is one of many over the last three years -- further convince me that I was right to walk away.

MEPIS is because is non-standard. Warren repeatedly warns against upgrading packages from the standard Debian repositories. There is no upgrade path from one version of MEPIS to the next. There appears to be a very weak mechanism for collecting community know-how as to how to configure the system to "just work" on a particular platform.

I can't find the link right now, but I beleive my university (Univeristy of Waterloo) and Georgia have signed a nice little deal. Both are investing huge amounts of money into nanotechnology (our Centre for Nanotechnology and Quantum computing is supposed to be completed in 2010). I'm actually lucky enough to be in that first batch of graduating Nanotechnology engineers. Cost of ours is around the same, we're getting tons of money from RIM. Here's the link :)
http://www.nanotech.uwaterloo.ca/research/qnc.html and for mearly bragging rights, our nano building's bigger than yours...wait...is that a good thing?

You can clearly see from the webcam (hosted on a different machine) clearly shows the server is not yet on fire.

Someone else provided links back to that original article. He's updated his design and even written a paper for his ME. check it out.

Someone at the University of Waterloo made their own "air conditioner" last year.

Granted, the CoP is probably miles worse than that of a normal heat-engine based A/C, especially when you consider the energy costs of freezing the ice, but . . .

In case of slashdotting, this is what he did:

Fill a bucket with icewater
Put a coil of copper tubing on the back of a fan
Use rubber hosing to connect the copper tubing to the bucket of icewater
Use a 2nd piece of hose between the copper tubing & the window
Siphon action pulls the cool water through the copper tube & drains it out the window
Fan moves air over the cold copper tubing

If you're not paying utilities, you could even do this with just cold tapwater, in your own home. Or, harvest the waste water for your garden, filling toilet tanks, or whatever!

But if they exclude Pluto from the planets, what does that mean for Sailor Pluto?

Will she become Sailor Kuiper? Would she be the first of the Outer Outer Senshi, to be joined by Sailor Oort at a later date?

Another great online crypto resource is the Handbook of Applied Cryptography. The full text is available for download.

Speak for yourself.

In Canadian engineering schools at least, there is approximately a 1:1 ratio of "Oh Shit" posters to Iron Rings.

Oh, and I think you've violated the DMCA or something by even looking at the site.

I go here.

First off, not every residence everywhere is two people to a room, so not everyone has to deal with it.

Second, there's usually a few people who don't show up to university, for one reason or another (last minute change of heart, couldn't get a visa, etc.) When I was in first year, a friend of mine had that happen on her floor. The double-room-with-one-person got dubbed "the sex room", as frustrated students got forced to sleep in the spare bed there when their roommates were having sex. This happened to my friend quite a few times.

Universal constant: people want to fuck. Who lets challenges get in the way?

You make me sad, I never got assigned to do any of these cool projects. Do programs like this still exist?