Slashdot Mirror

← Back to Stories (view on slashdot.org)

Poison Attacks Against Machine Learning

Posted by timothy on from the think-zippy-the-pinhead dept.

mikejuk writes "Support Vector Machines (SVMs) are fairly simple but powerful machine learning systems. They learn from data and are usually trained before being deployed. SVMs are used in security to detect abnormal behavior such as fraud, credit card use anomalies and even to weed out spam. In many cases they need to continue to learn as they do the job and this raised the possibility of feeding it with data that causes it to make bad decisions. Three researchers have recently demonstrated how to do this with the minimum poisoned data to maximum effect. What they discovered is that their method was capable of having a surprisingly large impact on the performance of the SVMs tested. They also point out that it could be possible to direct the induced errors so as to produce particular types of error. For example, a spammer could send some poisoned data so as to evade detection for a while. AI based systems may be no more secure than dumb ones."

4 of 82 comments (clear)

  1. Why solely the link to "i-programmer.info"? by Anonymous Coward · · Score: 5, Informative

    Why the hell is the only link in the summary to that rather useless "I Programmer" website? The summary here at Slashdot is basically the content of the entire linked "article"!

    Here is a much more useful link for anyone interested in reading the actual paper: http://arxiv.org/abs/1206.6389v1

  2. SVM != AI by SpinyNorman · · Score: 3, Informative

    Support Vector Machines are just a way of performing unsupervised data partitioning/clustering. i.e. you feed a bunch of data vectors into the algorithm and it determines how to split the data into a number of clusters where the members of each cluster are similar to each other and less similar to members of other clusters.

    e.g. you feed it (number of wheels, weight) pairs of a lot of vehicles and it might automatically split the data into 3 clusters - light 2-wheeled vehicles, heavy 4-wheeled ones, and very heavy 4-wheeled ones. If you then labelled these clusters as "bikes", "cars" and "trucks" you could in the future use the clustering rules to determine the category a new data point falls into.

    This isn't Artificial Intelligence - it's just a data mining/classification technique.

    1. Re:SVM != AI by tommeke100 · · Score: 5, Informative

      Wrong. SVM is a supervised learning technique. It looks like you're talking about K-means clustering which is unsupervised.
      The difference between supervised and unsupervised is that in the first you use both features and outcome in your training of the system, where the unsupervised will just use the features. So supervised uses both X and Y to learn (if X are the features and Y is the class/cluster), whereas unsupervised will just use X.

  3. Comment removed by account_deleted · · Score: 4, Informative

    Comment removed based on user account deletion