Slashdot Mirror


Microsoft Seeks Patent For "Search By Sketch"

theodp writes "So, how does one search for images that aren't tagged with keywords? Google does offer its sometimes-spotty search by image, but what if you don't have an image handy that looks like what you're searching for? Microsoft, reports GeekWire, offers a solution that's 'a little like playing Pictionary with a search engine — drawing a sketch and seeing if the algorithm can return pictures that match it.' That's the concept behind Microsoft Research's patent-pending 'MindFinder' project, which has already been incorporated into a Windows Phone app called Sketch Match. A patent application made public Thursday notes that touch computing makes sketching easier than ever, making one wonder if we'll be 'giving Bing the finger' with Windows 8!"

22 of 104 comments (clear)

  1. Testing would be interesting by Rogerborg · · Score: 5, Funny

    Could they make it work for the 5% of sketches that aren't boobs or dongs?

    --
    If you were blocking sigs, you wouldn't have to read this.
    1. Re:Testing would be interesting by mwvdlee · · Score: 4, Funny

      They'll just return the same nude picture on every search and claim 95% accuracy.

      --
      Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
  2. Already Done by Anonymous Coward · · Score: 5, Informative

    Already done here for LaTex symbols: http://detexify.kirelabs.org/classify.html

    Please please please don't approve the patent. I haven't read it, but according to the summary there's nothing new about it.

    1. Re:Already Done by JasterBobaMereel · · Score: 2

      3 minutes and we are up to 4 systems that show prior art already, do the Patent office actually do any searches ...?

      oh sorry this is "...on a mobile device" so they don't count ..?

      --
      Puteulanus fenestra mortis
    2. Re:Already Done by Miros · · Score: 3, Insightful

      You're citing prior art for the concept, which is not the same thing as prior art for what is actually being patented. There are 1000s of patents for "mousetraps." The fact that a mousetrap had been invented by someone first did not mean that other people could not invent other kinds of mousetraps. Just because people have built sketch based search technology before does not mean that Microsoft (or anyone else for that matter) can't invent an improved version of sketch based image search and then patent that method. That appears to be what they have done here.

    3. Re:Already Done by Miros · · Score: 2

      Also, as a note, there are already dozens if not hundreds of patents for this concept as well: https://www.google.com/search?q=sketch+image+search&btnG=Search+Patents&tbm=pts&tbo=1&hl=en

  3. Re:Overaged business model by Robert+Zenz · · Score: 2

    As long as it works: Never.

  4. My first thought by subreality · · Score: 2, Funny

    Oh please [sketch sketch] let there be porn like this [scribble]

    1. Re:My first thought by Guppy · · Score: 3, Funny

      Oh please [sketch sketch] let there be porn like this [scribble]

      Now pipe 4chan to it, and watch Microsoft's server commit suicide by setting itself on fire.

  5. Re:Interesting concept, but... by zill · · Score: 5, Informative

    Sounds like you're talking about Google Goggles.

  6. Patent the concept or specific algorithm? by rollingcalf · · Score: 5, Insightful

    I oppose software patents in general, but if this patent covers their specific algorithm I don't really have a problem with that.

    The trouble is that regardless of whatever algorithm they describe in the patent, the patent can be used to sue others who use a different algorithm to implement the same concept.

    So they can get a patent on an ineffective algorithm for image search, wait for somebody else to create a better algorithm that is actually effective, then sue the implementor of the better algorithm. The patent effectively covers the concept, not the algorithm, hindering innovation by preventing others from implementing their own different algorithms for the same concept.

    --
    ---------
    There is inferior bacteria on the interior of your posterior.
    1. Re:Patent the concept or specific algorithm? by rveldpau · · Score: 2

      The problem is that you can't actually patent an algorithm, there are laws that prevent that; but that does stop sleazy lawyers from working around that problem, and what that gives us is overly broad patents. If we let people patent algorithms, we might actually be better off.

  7. Re:Already exists by __aaltlg1547 · · Score: 2

    That functionality was integrated a few years back into digikam.
    How can those corporations always get away with patenting stuff under prior art?

    There are two ways.

    1. Simply ignore prior art and claim everything as your own invention.

    2. Make a trivial change and then claim that and bamboozle the patent office into believing you have therefore invented the wheel.

  8. Rtriever by cyrano.mac · · Score: 5, Informative

    Rtriever has been doing exactly the same thing since 2006. http://labs.systemone.at/retrievr/

  9. Re:What about false positives? by Anonymous Coward · · Score: 3, Insightful

    Is this serious or a really stupid joke? Obviously "Safe Search" options already exist. Also obvious, it's never a good idea to search for anything of visit webpages you've never been to before in front of 400 other people. Neither of these things are new, and "Search by Sketch" changes neither of them. For the record, you hit porn on page 2 on Google Image Search for "Wrench or bracket" if Safe Search is off. So far, I've yet to find a search term that doesn't produce porn on page 2 or earlier if Safe Search is off.

  10. Prior art by IBM? by Tal+Cohen · · Score: 3, Informative

    I remember IBM DB2 ads from the 90s about, well, sketch-based DB searches. IBM called it "Query by Image Content", or QBIC. It was easy to find one using Google Books -- in this case, CIO Magazine, Sept. 1, 1995: http://books.google.com/books?id=AAcAAAAAMBAJ&lpg=PA25&ots=GGDbllo74W&dq=ibm%20db2%20ad%20bottle%20%22perfect%22&pg=PA24#v=onepage&q&f=false

    --
    - Tal Cohen
  11. Patent will be invalid - It's been done before by Etylowy · · Score: 4, Informative

    Patent is invalid if there is prior art. Well, imgSeek has had that function since at least 2008 (can't find changelog for desktop version - it might have been there already in 2006).

  12. Probably not as simple as it seems by Miros · · Score: 3, Informative

    This is probably not as simple as Microsoft trying to patent something which has already been done before. You may want to check out some of the actual research they have done here: http://research.microsoft.com/en-us/projects/mindfinder/ I seriously doubt that they are not familiar with all of the prior art examples that have been brought up here, and they would not have spent the money on a patent if they did not think they had improved on the existing methods sufficiently for it to stand up to even basic scrutiny.

  13. Common misunderstanding by Theaetetus · · Score: 2

    it used to be that they added "on the internet", now they add "on a mobile phone", looks like this one adds "with fingers on a mobile phone"

    Common misunderstanding, but no. This came up because many patent applications have dependent claims that may say things like "2. The method of claim 1, wherein the network is the Internet" but that's not independently claiming the Internet... Or even claiming that novelty is due to 'the internet'. Rather, it's a doctrine called claim differentiation: dependent claims are subsets of the independent claims from which they depend from. Like, if Claim 1 is a large box on a Venn diagram, then Claim 2 would be a circle within that box. So, when Claim 2 says "wherein the network is the Internet," all that means is that the 'network' of Claim 1 must be broader, and include both the Internet and any other type of network.

    But, for claim 2 to be patentable, claim 1 has to be patentable. And adding "on the internet" doesn't actually make claim 1 patentable... rather, claim 1 must already have enough new and nonobvious elements to be patentable on its own.

    So, this misunderstanding about claim differentiation also meshed with a misunderstanding of patent titles and abstracts... They have no legal weight, the only important part is the claims, but people look at the title and say, for example, "'a network operating system'? Networks are known! Operating Systems are known! This is a bullshiat patent!" Add that to a claim saying "wherein the network is the Internet" and suddenly it looks as if someone was trying to patent an operating system... but on the internet! But no, that's simply not true at all.

    In summary, there are no patents that claim "A method, comprising [X, Y, Z] on the Internet" where X, Y, and Z are all known and the only claimed novelty is 'on the internet'.

  14. Re:Already exists by Stewie241 · · Score: 4, Insightful

    or
    3. Recognize that there is legitimate room within patents for doing something that has already been done but doing it in a non obvious, novel way.

    I'm not saying this is the case for this particular patent, but just pointing out that in my understanding patents aren't always about doing a certain thing but rather doing a certain thing in a certain way. Obviously this limits the scope of the patent to that one certain way.

    So, to put it in slashdot terms (i.e. a car analogy), pretty much every car sold today has shock absorbers, and the standard way of doing this is to use metal coil springs. If somebody were to put time and research into improving the design of shock absorbers and came up with a design that used marshmallows instead of metal coil springs, I would consider that novel and I think you would agree that it is non-obvious. To me, if we have the patent system that we have, that is at least a legitimate use of it. You aren't patenting shock absorbers, you are patenting shock absorbers that use marshmallows to absorb the shock of the impact.

  15. Re:What about false positives? by mjjochen · · Score: 2
    Um, two things:

    1. Lesson Plan
    2. Prepare before going into the classroom

    Seriously, if you are planning your lecture while doing it, you're doing a huge disservice to your students. That's not to say that open discourse and exploratory learning aren't good in the classroom, this can be great - let the discussion go where the students take it. But on the technology/course material side, I would be very concerned with adding material to my class at the last minute. What happens if that image/movie/website that you were counting on weren't there? What if your connectivity tanks when you wanted it? What if you get that huge phallus instead of the ICBM that you were looking for? I download anything that I want to use ahead of time.

    If you are doing this last minute, to me that demonstrates that you are not taking your planning/prep seriously (or that you are lazy), and that you are putting your laziness ahead of the learning of your students. They (the students) deserve better than that.

    Now that I've nibbled at the troll-bait, I think this is seriously off course - search by sketch, search by term, they all have the same possibilities to return content that someone out there may find offensive. So I fail to see how this is any different. That's the glory & risk of the Internet - use at your own risk.

  16. Re:Interesting concept, but... by leptogenesis · · Score: 2

    No, Google Goggles does nothing like this. Google Goggles (and Google search-by-image) is, from the experiments I've done, instance-based image retrieval. That is, it can match objects with exactly the same shape (given a picture of the Eiffel Tower, it will return other images of the Eiffel Tower). However, given a drawing, even a good one, the contour shapes won't match quite well enough, and the algorithm will return garbage. The same can be said for 'deformable objects' like dogs and people.

    In fact, I'm quite sure that nothing like this exists. I'm not sure about the actual search engine part of all this, but I did see a talk last fall by one of the researchers who worked on ShadowDraw, which I'm reasonably sure is going to be a component of the final system. The real problem that *they* had to solve was the simple fact that the average person is a horrible, HORRIBLE artist. Ask them to draw a rabbit and for 90% of people, it will come out as a blob that might be an animal, but that's about all you can tell. The algorithms they talked about that actually make the system work as well as it does were actually quite impressive--extremely fast contour indexing, contour combination, converting real photos into convincing sketches--it all sounds easy, but I dare you to actually try implementing it.

    Now--and let's see what happens to my karma for saying this--I actually kinda think they deserve a patent for this. Not for coming up with the idea of drawing-based search; that idea is obvious. However, making a system that works as well as ShadowDraw is quite an achievement, and more importantly, Microsoft Research would never have released the algorithm to the public unless it could be patent-protected. Patents in this case aren't about protecting Microsoft's innovation; it's about motivating Microsoft to publish for the sake of other innovators.