Slashdot Mirror
Incorporating Machine Learning into Firefox 2.0?
blakeross asks: "I will be doing research this summer at Stanford with Professor Andrew Ng about how we can incorporate machine learning into Firefox. As we work to finish up Firefox 1.0, we're also seeking ideas that will make Firefox 2.0 blow every other browser out of the water. People who come up with the best 3-5 ideas that involve the use of machine learning will win Gmail accounts, and if we implement your idea you'll be acknowledged in both our paper and in Firefox credits. Your idea will also be appreciated by the millions of people who use Firefox. We'll also entertain Thunderbird proposals. See my weblog post for more details; I'll read all comments posted in response to this story or to my weblog."
Here are the best five ideas incorporating machine learning:
... "this is a picture of a sunflower") and let them group and search for items. Eg. "Pictures like this" or "Documents about cats."
1. Based on the user's browsing habits, automatically bookmark the most frequently visited sites, and automatically put them into *multiple* categories (not just one category) to make them easy to find.
2. Create a full-text index in real-time of every page that has been browsed. When the user visits any web page, display a sidebar of "Related previously-viewed pages."
3. A Google-News-like consolidation feature for the user's most-frequently visited news site, automatically highlighting stories of interest based on ones they've previously viewed.
4. Allow user to select "Fewer images like this" or "More images like this" or "Less text like this" and "More text like this" and using Bayesian or other similar filters, automatically block or highlight content. For blocking advertisements, or highlighting certain key passages.
5. Allow the user to browse their own hard drive, and categorize content automatically ("this is a document about lambs"
Please give my Gmail accounts to Gmail for the troops.
.... considering how much Google gave out to drop the prices on eBay.
:p
I suggest better prizes. Y'know, like a girlfriend? I'm sure lots of us Slashdotters would like to have one over a Gmail account
Founder of Mirror Moon - Tsukihime Game Trans
- The pop-up management in modern browsers who provide this feature although more efficient than in the past is still not perfect. Adapt to what pop-ups a person normally uses
- Content highlighting (especially in news sites). Learn what types of news articles / subjects a user is interested in, and highlight titles in news pages that suit the user.
- Accelerator for narrowband connections. Predict which pages the user is more likely to visit next, and start loading them as the user still reads the previous page.
- Recognise efficiently scam sites? Protect users from fraudsters?
PS: Not machine learning, but the sole requirement by me for a browser (dunno if its done in firefox now as hvent used it for a long time): Open new tab as a default rather than a new window, or at least provide the option.
Make it so you can open all links on a page in new tabs, and the browser will sort them by content.
Also, it would be awesome if using the internet were more like playing Fallout. That was a great game.
Since when has this country used intellectual elite as a pejorative term?
Make it so when the user hits the Page Down key, a horizontal line appears for a few seconds where the old bottom of the page was, then fades away. So when you're reading long sections of text and hit Page Down, your eye can quickly scan to where you left off.
Sick of people knocking on Gentoo's greatness in completely unrelated
(Undisclaimer: I do machine learning research at BYU.)
:)
:)
Machine learning, in general, is getting computers to generalize based on data instances. The two main flavors are classification (inferring classifications of data instances based on previous instances) and regression (inferring a function based on input/output pairs).
A lot of people incorporate artificial intelligence into the category "machine learning," though it's not strictly correct. Machine learning is more a branch of AI than anything. One way to keep them straight is to think AI = deduction, ML = induction. (That's vastly simplifying, but it helps to classify them roughly.)
I wonder which way the author leans? Could he possibly post to clarify his meaning?
You can do an awful lot with machine learning that you can't do with conventional techniques. You can often get great results for otherwise NP-hard problems. Slashdot had a story a while back about using machine learning to do mesh compression, in which their algorithm comes up with a close approximation to the real answer to an NP-hard problem in polynomial time.
I'm currently using it to interpolate 2D images, and kicking bicubic B-spline interpolation all to heck. (Paper pending...) The machine learning algorithm infers shapes from the pixels, and keeps edges sharp.
If I come up with an idea, I'll post it later. In the meantime: isn't Firefox supposed to be lean and mean?
I got my Linux laptop at System76.
...a browser that doesn't have machine learning in it. Seriously, Firefox is slow enough for me. What on earth would you possibly need "machine learning" for in a web page browser? I'd immediately switch back to Opera (I don't use it simply because input forms lag during page-loading, some sort of multithreading issue).
That kind of automatic crap is the same sort of stuff people would bitch about if Microsoft put it into IE. I mean, do you really want your browser actually learning anything about you? Imagine the havoc it could wreak, especially if trojans started fucking around with it.
Just give me the leanest, meanest browser out there. That's all Firefox 2.0 needs to be. Not a damn learning machine. Sheesh.
There. Your most important feature that browsers never had. Searchable bookmarks. Doesn't get much simpler than that. Am I the only one who thinks it's something every browser should have had long time aog?
Your pizza just the way you ought to have it.
How about this: How about browsing the filesystem using tabs?
So for example, in one Firefox window you see the contents of your hard drive (or network folders) pretty much the same way as the Windows Explorer or Gnome/KDE/MacOSX show it to you today BUT if you click the middle mouse button on a directory (or select "open in new tab") you end up with the new directory being open in a new tab.
Think about it, how many windows do you usually have open browsing your filesystem? with this thing you have ALL those windows in the same window organized by tabs, PLUS you also have all you websites as well on tabs right along the filesystem tabs!
And here's another kicker: You can bookmark a group of filesystem browser tabs and later go back to them. You can even drag the group of bookmark tabs to the desktop so that when you double-click on it Firefox opens up all of them at once.
This should all be done with host filesystem integration so that you can drag-and-drop files between the firebox filesystem view and the normal host OS desktop.
>> we're also seeking ideas that will make Firefox
>> 2.0 blow every other browser out of the water.
>The competition: Internet Explorer, Netscape, Lynx,
>and Safari.
>I'd say it's already pretty much covered...
I would have to agree that FireFox is pretty much the best browser today. But the performance on older hardware isn't much better than that of a new build of Mozilla. Also, the move to GTK2 and feature-creep are slowing it down futher.
Safari on the other hand is very basic and very fast. Try it on a 300 MHz G3 to see what I mean. Apple did a great job of making a basic browser out of Konq/KHTML. I would **LOVE** to see a similar app for X11 (so I can run it on Linux, Solaris, and IRIX). Konq Embedded is close, but it's lacking some important features and isn't as fast as it could be.
Long live FireFox--but keep it slim!
Does anyone else get the feeling that they are adding this just for the sake of it or so they can say they have it? I mean when you have the technology before any useful uses for it then clearly there is something wrong.
I think that creating a good browser though gimmicks is a poor long term strategy and seriously doubt this route will turn up anything useful. Ideas should be so simple and obvious and inspire us to say 'who dont we have that already?!' not something we search for!
Whoa! Good call! That's an awesome, basic feature that could easily be added to FireFox without bogging it down.
(So many of the other suggestions so far would make FireFox slow to a crawl. Lets keep it lean and mean, please!)
Isn't this going about things backwards a little?
To me this sounds like a clear case of "technology X is really cool. Let's find some reason to include it in product Y." Which often means that product Y becomes much more complicated than it needs to be.
How about first looking for a list of browser "needs" so to speak. What would make the best browser? What current deficiencies to browsers have? And so on. Then, if you really want to, try to figure out if any of these problems could be solved with machine learning.
Don't just inject a technology into a product because it's cool. Make sure there's a real need for it.
Who said Freedom was Fair?
I don't know if this has been posted, but I'll give it a shot...
I've accumulated well over a thousand bookmarks and have been much too lazy to organize them into folders. If you could automatically cluster bookmarks (http://vivisimo.com/ does this with web results) I would be eternally grateful.
One more suggestion is to learn usage patterns in a particular website. For example, when I go to http://www.nytimes.com, I generally click on the opinions sections. If the browser could anticipate that I typically go to the opinion section, it could start to preload it before I click on it.
I realize the later suggestion is much easier to implement than the former, but the clustering would be very useful for lazy surfers like me.
If you're a fan of women, add me to your friends list.
Make it an extension only!
Seriously, it would be a really neat feature if some of the suggestions posted here were realized... but this whole idea screams of bloat bloat bloat. What makes FireFox so appealing for some (including me) is it's compactness and lack of bells and whistles. The FireFox project FAQ echos these sentiments: It's small, fast, simplified, nothing other than what you need. "Just a browser"
Don't let feature creep ruin it!
=Smidge=
I hate it when anything software tries to "predict". I don't want it. Please make sure it has an OFF button. Seriously. Thank you.
Do NOT bloat the browser.
Want to add crap? PLUGINS!
Hate me!
...Is to make it very easy to turn whatever machine learning features incorperated into 2.0 off totally, with minimum fuss and searching.
It is my firm belief that then #1 rule of UI design is that the program should should look and act consistant. And the number two rule is that the program should never assume anything, or perform any action without the user explicitly telling it to (barring sane default behaviors that will fit > 85% of the users). Every ML feature I have ever seen breaks #1 and #2 with reckless abandon by changing something to make it more 'friendly', which in turns makes it less friendly because I don't know _exactly_ what to expect from my program.
Looking at the comments on that weblog, I can not find a single idea that does not either violate my top two rules, or would otherwise annoy me to no end. If they have to add that to Firefox then please, let me turn that crap off in three mouse clicks or less.
The firefox download manager should scan downloads for malicious spyware, stop the bad download(s) and warn the user of the danger posed by the file(s).
Now, because this has a lot of discussions, when I start typing basebal... I get a lot of urls in the autocompletion field like http://www.baseballthinkfactory.org/files/primer/o racle/
or even unrelated baseball sites. So it's not uncommon for me to have to press downarrow several times. A very useful application of machine learning would be to order the autocompletion possibilities so that my average number of downarrow presses is minimized.
Here's an example:
Bayesian filtering
Thunderbird wouldn't be the same without it. Does it drag your system to a halt? Nope.
I'd be awfully surprised if anything real CPU intensive would ever be installed into Firefox by default. Give these guys some credit.
Ironically, the word ironically is often used incorrectly.
Adding what passes for "machine learning" to a user interface usually results in something that does the right thing some of the time, the wrong thing some of the time, and you can't figure out why.
Bayesian spam filtering is becoming like that. At first it worked, but it's breaking down under the rising percentage spam.
Have it learn where I am saving which files and offer up that directory as default. If I am saving all pictures into one directory and all movies into another it should know that.
I want virtual folders in my mail. These are "live" queries like "all mail today" or "all mail marked urgent". As I mark metadate on the email they will show up in the proper virtual folder.
Full text search of all email.
Choice of multiple home pages. It learns when I want my home pageX and homepage Y.
Roaming bookmarks!!!. While I am at it roaming everything including profiles and preferences. The ability to carry my email filters from location to location would be awsome.
A network install where the administrator can set global prefs and install global plugins. I also want the option to override the users preferences and lock them out of certain setting.
It should learn to adjust my font size (and other settings?) based on site. If a web site always puts tiny print then I want the fonts larger only for that site. Perhaps have it learn "ugly" sites and put my default styles instead.
Auto proxy. I want to feed a list of proxy servers and have it switch randomly (even from one site to another). Think of this as super privacy.
Ability to arbitrarily morph the the incoming text stream using regexp or javascript. This would allow me to roll my own weird crap.
Make XUL 50 times better. Make it so it's trivial to use XUL to make database front ends. Give me a great GUI builder for it.
I have lots more ideas but that's enough for now.
evil is as evil does
Currently, if I start typing URL in the address bar, it matches URLs alphabetically. This gets very annoying at times, especially if you accidentally type giigle.com instead of google.com and then it keeps on matching giggle.com for weeks when I type "g".
This problem can be fixed by using frequency count with some time decay. For example, if I went to google.com 100 times within last week and once to giggle.com, then match to google.com on "g". If, however, I went to giigle.com 5 times recently, then match to giigle.com
While one might argue that this makes the algorithm unpredictable from user's standpoint, in my experience people keep on typing until they see the correct match. So, this way they'll see the right match sooner on average.
"You mortals are so obtuse." -Q
(Clippo, from Office, featured in Firefox...) Clippo: It looks like you're browsing pornography. You also appear to be typing with your left hand. Would you like to enable the spellchecker?
Wow, he followed that suggestion fast:
From Mozilla Firefox 0.9 (One Tree Hill) Release Notes
I don't know if he's a "programming god," but I seriously doubt he's "some highschool kid with all summer to screw around."
Extension. It's why that framework exists.
Speaking of Bayesian filtering, some form of clever-er guessing as to where my next bookmark in my ecclectic collection of bookmarks goes. Sample relatively unique keywords in pages as bookmarked, weight towards bookmark folder baskets, bingo.
Avoid more sophisticated algorhythms that infer a sorting methodology the same as the developer, however. Maybe I have a Programming folder which has C in it, and so you'd infer that all characteristics of matches to Programming inherit to C, if that's the sort of sorter you are, and that fits with you, me, and program-think, so that's right? Right? Except perhaps I'm a university student who has a University folder, and I'm studying Java, whose extrinsic attribute prioritizes sorting it into that group... so you'd end up with a word weighting argument between superclass Programming, which is wrong, and Java, which is right.
Let me be clear. This suggests nothing at all about helping the user organize their bookmarks - everyone has their own system (although perhaps a Bayesian category guesser would be a separate fun feature). This suggestion is merely better guessing of first suggested folder when I CTRL-D.
Often masses of information are broken into multi page presentations.
Somewhere on the page you have buttons named things like Next, Previous, or Page: 1 2 3 4 5 6.
There may be good design rules for positioning these elements but often they are not followed.
I've found many instances where I have to scroll up or down just to find the Next button so that I can click it.
It should be possible to learn for a given site (or sub-tree of a site) what the Next and Previous buttons are just from user behavior and the nearly identical layout of say page 2 to page 3. I think this could be done without parsing any of the html or gifs associated with the buttons.
If Firefox could learn and extract multi-page navigation then these functions could be bound to buttons up on the menu bar, or assigned to keys, and the whole problem of scrolling to find a Next would go away.
There are three types of sites in the world:
Those that use flash for ads
Those that use flash for content
Those that stay the hell away from flash
Rightnow, Firefox doesn't have any way to tell the difference between 1 and 2. But I do, I can clearly see if it's an ad or not. On every flash ad give me the option to tell the browser it's good flash or bad flash and intelligently learn what sites ("sites" also being defined by study of the urls, if I say www.bob.com/~jimbo/whatever.htm and www.john.com/~jimbo/howie.htm and www.curly.com/~jimbo/marthastewart.html are bad it should figure out there is a commonality in the ~jimbo part and apply my preference) have bad flash and block flash content on those sites, instead presenting me with a button to load to allow that content to load.
It should use a number of pieces of information, the url of the page, the url of the flash animation, the size of the animation, the name of the animation, the server the page is being served off of, etc.
Smater Front Page: Making use of the first thing that the user sees when starting up.
How about creating an interface for the default page for Firebird. Instead of pointing to the Mozilla.org homepage creating a default Front page designed to evolve to the habits of the user. Whatever way you want to utilize machine learning, you will need a centralized location to acess the results, why not use "Home". That being, creating a simple interface (XUL, not html) that points the user to their most visted bookmarks, or a catagorised and searchable list of their bookmarks(or internet), or updating the user if their most visted sites are updated, aggregating information from sites from their own browsing habits in a single interface when the browser starts up.
Also, if the user uses Thunderbird or Sunbird updating the user of new E-mails and new appointments on the front page. A front page that is customizable to the needs of the user, and avoiding the clutter and ads of commerical sites, and that is local on the users computer and not centralized on a website. And most importantly makes the individual users own data most intuitively accesible to themselves, and evolves to fit the individual user.
"1. Keep track of how users enlarge/reduce the font size: if sites that use a 10 point font are repeatedly enlarged to 14 or 16 point then it is fairly safe to assume that the user has poor eyesight and all sites with tiny text should automatically be sized up."
This is a good concept in several ways.
First, what most people with eyesight limitations do is adjust the really severe problem text and put up with the less severe sorts, so if they enlarge 10 point to 16 consistently, they enlarge 12 to 16 only late in a browsing session, and just put up with 14 point type even though it's a bit smaller than optimum for them. People will go to an effort only when the threshold of discomfort is crossed and the problem gets their consious attention, and many people will put up with a problem beyond that.
Second, it's a clearly quantifiable area, making it the sort of thing machines can excel at. If it turns out to have unexpected complexities, we will get a warning about how much worse other tasks, such as adjusting web sites based on the user's color preference or aestetic criteria, will be (no plaid backgrounds)
Who is John Cabal?
Don't ask me if I want to remember a username/password combo until AFTER the login has been successful.
Spoon not. Fork, or fork not. There is no spoon.
According to ebay, that prize is worth about 95 cents!
Firefox needs an option to make the browser detect, and work around, user-interface abnormalities in poorly-designed websites.
It's fairly well-established that the best user interfaces are the ones where there is no discrepancy between what the user thinks is going to happen, and what actually happens.
When a user single-clicks a link, the link should open in the current window. Always. Any other behaviour (such as opening a new window) causes the user to be frustrated (or at least slowed down).
Similarly, when the user middle-clicks a link (or shift-clicks or whatever), the link should always open in a new window/tab. No oddities like "javascript:gotosite()" or "http://path/to/exact-same-page.html#" should happen.
Unfortunately, there are a lot of misguided website authors that think they're being helpful by doing non-standard things in an attempt to anticipate users' needs. This means that you'd need some type of machine-learning in order to work around these problems at the browser level.
I imagine this would be done in a way similar to how SpamAssassin works.
I frequently adjust different aspects of my browser for different sites. Adjusting the window size/position, bump up font size by 10%, allow/block images, whatever.
I'd like a system that remembers those adjustments, and not only reuses them when I return to the same site, but applies them again where appropriate. 'Where appropriate' is where machine learning comes in.
plus-good, double-plus-good
I would want a credibility rating on web pages.
There is a lot of information on the web but almost no way to verify the data. I would like a way for people to report the credibility of the information contained on a web page.
This is especially important with news reporting and double-extra-especially important in times of war.
It is also all too simple for politicians, journalists and other people of power to repeat the same old lies over and over.
The memory of media is short. A Truthalizer would help make it a bit longer.
Yes, I know that there are plugins for this one, but keeping bookmarks in sync on multiple machines via server storage. Since I got my laptop, it's not a big issue, but I hate having to go back to my other machine cause that's where the bookmark is.
Enabling persistent storage of passwords. I honestly don't know how much or little of a security hole this would be, but I am constantly using the "remember my password" feature in Moz/FF, and it KILLS me when I have to reinstall and start adding them all again. Have it store that file to a spot on the hard drive (or better, server sync if that option is turned on), and allow me to keep that data even if I have to uninstall/reinstall the application.
-9mm-
thats in reference to modifying currently executing code in tight loops. that used to be a common way to avoid branching back in the day.. nowadays if you have to use a conditional jump in a tight loop its usually not faster to try and work around it.
lately the term "self modifying code" is commonly attributed to dynamic code generation (it does sound cooler), but dynamic code generation is still the best way to accomplish many things and nothing intel says about "self modifying code" applies to dynamic code generation techniques.
anyways, what your parent is speaking of is neither of these things. loading only the code that is required is a technique known as "late binding" and is a great way to modularize an otherwise bloated application.. I think firefox is already on this path with its extensions. hopefully they remove more extension-like features from the main app and implement those features in extensions, perhaps ones that are installed by default.
bite my glorious golden ass.
People regard their privacy highly and are reluctant to provide their personal information to websites. Website which need personal details from its users have to convince a new user that it won't misuse the data that it has. This is usually made with a statement, known as Privacy Policy.
Privacy Policies have problems. First, they are usually written in a legal language incomprehensible to a lay person. Second, people have no way of knowing that website in fact follows the policy it has.
One way of assuring people might be machine readable privacy policies. P3P (Platform for Privacy Preferences) is W3C framework for such privacy protocol. It allows user to store his personal data to a P3P agent, which will then follow the user given rules to share private information.
This agent should be implemented to firefox and it could use a machine learning to automate further the interactions with websites. The agent could for example learn, that if website's privacy policy promises to use user's e-mail address only for initial consistency check and to send a forgetten password if user explicitly asks for it, the agent can give it to the website without prompting the user.
Of course, this won't solve the problem of malicious websites which don't follow their privacy policies, but is a step into right direction where privacy policies are certified and their enforcing is auditted.
The following is the best e-mail idea of the last two years, and should be a feature of every e-mail program.
As a person who writes a lot of e-mail, or as a manger, one major organizational problem is simply not having your messages replied to. It currently takes a ton of manual effort to decide which of your e-mails need a followup by you. So much effort that nobody does it. Some questions fall through the cracks when you write a hundred e-mails a day. Closing up these cracks would measurably, demonstrably improve the effectiveness of managers, project coordinators, and any other heavy communicators, since e-mail has become the most important and most used mode of communication for a large number of people. (It is very rare to see a manager actually send out an e-mail asking "Was this resolved?" Generally, un-followed-up e-mail is simply forgotten.)
A system for followup could be partially automated.
1. A checkbox exists when writing an e-mail. If turned on, it tells the e-mail client, locally, that you want to make sure this e-mail has been followed up. I'll call this a "follow-this-up", or FTU, e-mail.
2. After an FTU e-mail has been sent, the e-mail client remembers it in a list. Once an FTU e-mail is sent, a copy of it is placed in a mail folder which I'll call the FTU folder. The user of the e-mail client can open this mail folder at any time to see the FTU e-mails that still have to be followed up.
3. When the client detects that a recipient of an FTU e-mail has replied to that e-mail, then it provisionally removes the FTU e-mail from the FTU folder. (Probably the e-mail is grayed out in the list but not actually removed.) Making this detection complete and thorough is an interesting problem. The starting point would probably be based on receiving an e-mail from the recipient with an appropriate subject line (e.g. the same subject line prefaced by FW: or Re: or Re[5]:). And in order to increase the effectiveness of this technique, the client might actually maintain a database of previously-used subject lines that are already in the FTU folder, and nag the user if he sends a second e-mail with the same subject, asking him to write a more elaborate Subject line.
Other starting techniques could include parsing the e-mail's content to see if part of the content matches an FTU e-mail that has been received. Or by utilizing e-mail fields or even implementing a new e-mail field which hopefully doesn't get stripped when the recipient replies.
This system, then, tries to ensure that after an FTU e-mail is sent, there is either a copy of the e-mail in the FTU folder so the user can see that the recipient hasn't followed up, and the user can follow up with a question; or there is a response from the recipient in the user's Inbox.
User interface is critical to making this system useful for the user:
4. When the client sees that an FTU e-mail has been replied to, it presumes that a followup has actually occurred. This obviously may not be true; the recipient may have responded with a joke, or with a followup question, or with "I'll get back to you Thursday". Presumably when you read a followup to an FTU e-mail, a new bar of UI should appear in the client saying to the user "This looks like a followup to an FTU e-mail you sent, which you can view by clicking here." Buttons would let the user choose things like "Yes, this resolves my FTU e-mail completely" or "No, I still need a followup", or "I now want my reply to this e-mail to be an FTU e-mail, and not the parent."
5. I imagine that the FTU folder displays its FTU e-mails in date order, showing the oldest non-followed-up e-mail at the top (colored red after 2 to 4 days or so). The user can mark these e-mail copies as already-followed-up (i.e. it's resolved, no more followup needed, because the recipient saw me in person and resolved it).