Slashdot Mirror

Can you show some examples of Microsoft research?

Pick any top-tier CS conference. They'll probably have something there.

For example, OSDI '12 (MSR personnel on 5 papers, 2 of which all coauthors worked at MSR), PLDI 2012 (MSR personnel on 6 papers), SIGGRAPH 2013 (harder to sort through, but I count 16 papers with at least one MSR co-author), VLDB 2011 (8 research papers as well as several other things like demos, a keynote, an industrial paper, and a 10-year-retrospective best paper award), STOC 2013 (16 papers if I counted right!), etc.

Seriously, I was not being choosy with those conferences -- the only choosy things I did was pick years for which there was an obvious page that listed the institutions with the authors instead of just the authors (e.g. VLDB 2013) because I'm lazy. If you pick a conference that covers a topic of interest, MSR has had something there. :-)

Since the release of PostgeSQL version 9.1, PostgreSQL has an optimistic technique for implementing SERIALIZABLE transactions which doesn't require traditional read locks: Serializable Snapshot Isolation. (Full disclosure: this was developed by myself and Dan Ports of M.I.T.) For some performance graphs using several standard benchmarks, see Proceedings of the VLDB Endowment, Volume 5:

http://vldb.org/pvldb/vol5/p1850_danrkports_vldb2012.pdf

This lets you set a default transaction isolation level and just do your work without worrying much about interactions with other transactions, as long as you are set up to retry your transaction from the start on a serailization failure. There is no need for SELECT FOR UPDATE, and since there is a guarantee that any group of serializable transactions will behave the same as some serial (one-at-a-time) execution of those transactions, you can be sure that if the transaction does what you want when run by itself, it will do what you want when run in any mix of serializable transactions -- without the blocking described for SQL Server.

So you can have your transactional consistency without the blocking.

I would like to see a search capability like that in the defunct Bloomba e-mail client (now the basis of WordPerfect Mail). The entire text of every piece of mail was indexed which made searches very fast. It was also easy to set up virtual folders (based on search criteria) to associate your e-mail according to several criteria. A given mail could appear in several folders, not just one. The company called it a Personal Content Database. The Bloomba client also incorporated a calendar and an anti-spam proxy.

The company producing the software, Stata Labs, sold the technology to Yahoo in 2004. It has since been resold to Corel for use in their WordPerfect Mail.


Never let reality temper imagination

XML Tagging is tedious and stupidly top heavy in overhead. Contrary to being human friendly it isn't. XML Tagging should be shortened to a simple set of defined tag names and then type definitions. After that each name would be addressed by an index. Typing of data should be contained in a process to extract that is associated with either the tagging index or an over the top wrapper which is similar in function to the DTD. But frankly the whole process is currently a mess.

Once the file is retreived it should be crunched into something like MySQL or such if any real processing is going to happen.

Nothing really is gained by such a markup system over just a series of hashed tags that are indexed. Such tagging and indexing is a lot less of a tax on band width.

This having been said, XML works and is OK for many uses. I am not sure it really has any advantage over flat files or such. It drinks band width and program operations time. I think in time it will turn out to be a fun toy but not much else. Of course someone else might find a good way to tell me why I should use 40 characters to transmit what should have taken 10 characters and how it should have been faster or more efficient some way to use it. The whole concept was definitely good for a lot of programmer payroll time.

There was a very interesting paper at the VLDB 2000 conference (the full PDF is available from that page). Quotable points from the paper (the first lot are section titles):

"Observation 1: Featurism drives products beyond manageability.
Observation 2: SQL is painful.
Observation 3: Performance is unpredictable.
Observation 4: Tuning is a nightmare and auto-tuning is wishful thinking at this stage.
Observation 5: [...] Database systems are not (or no longer) at the center of the IT universe.
Observation 6: System footprint considered harmful.
Observation 7: System-oriented database research is frustrating.

All these observations together strongly indicate that database systems are in crisis: they have reached or even crossed a complexity barrier beyond which their lack of manageability and predictability will become so painful that information technology is likely to abandon database systems as a cornerstone of data-intensive applications."

The paper goes on to argue that for a large number of applications we'd do better to abandon 'universal' databases (UDB/Oracle 9 and their ilk) because it should be possible to ship 'db-in-a-box' style units which support a smaller but sufficient featureset, which can self tune (no DBA required), because of their simpler theoretical basis.

Where is the relevance? Well, MS and IBM complain that MySQL is missing 'feature X'; and most of the arguments here are "yes MySQL is crap", or "yes but it has feature Y", or "yes but product Z is overkill", or "yes but its cheap" - they accept the premise of the MS/IBM argument. I'm pointing out that the premise is actually wrong - its more than reasonable to argue that MySQL has way too many features already.

-Baz

Having so few transactions for a database of this size probably helps them run without needing large expensive machines. Many VLDBs support thousands of transactions per second. I found a list here of top ten winners of a very large database scalability contest. The winner for peak performance was something like 20,000+ TPS.

Anyone can explain to me what is suddenly so wrong about relational database with hierarchical indexing?

Maybe its just me, but the goal today is integration and having a special database for XML and special database for this and that just because its faster for this particular problem creates such a level of complexity, which prevents accomplishing even of the most trivial tasks.

Forgive me for tooting my own horn on this one, but I believe that (for once on /.) there is a correct answer.

I summarize the answer in a paper written for VLDB 2001 (www.vldb.org). The paper presents joint work between Stanford, Berkeley, and RightOrder, Inc. It can be found online here (in PDF).

What we found is that relational systems, with appropriate indexes for XML data, give the advantages of both worlds. XML is a hierarchical representation in only the loosest sense. It's written linearly in a flat text document, just as a child learns to write things down on a piece of paper. However, you wouldn't convince anyone but that same child that something written on paper can only represent two-dimensional objects just because the paper itself is flat. XML in many variants is plainly richer in concept than its simple hierarchical representation and thus quite suited to ER. I believe a previous poster mention RDF... a perfect example.

Punchline: XML is neat, XML is tasty, but XML is not inherently more or less expressive than ER; it just requires a little critical thinking (and index tweaking) to tune ER engines to deal with it. (Once tuned, the ER engines dominate all others in performance.)

Anyone can explain to me what is suddenly so wrong about relational database with hierarchical indexing?

Maybe its just me, but the goal today is integration and having a special database for XML and special database for this and that just because its faster for this particular problem creates such a level of complexity, which prevents accomplishing even of the most trivial tasks.

Forgive me for tooting my own horn on this one, but I believe that (for once on /.) there is a correct answer.

I summarize the answer in a paper written for VLDB 2001 (www.vldb.org). The paper presents joint work between Stanford, Berkeley, and RightOrder, Inc. It can be found online here (in PDF).

What we found is that relational systems, with appropriate indexes for XML data, give the advantages of both worlds. XML is a hierarchical representation in only the loosest sense. It's written linearly in a flat text document, just as a child learns to write things down on a piece of paper. However, you wouldn't convince anyone but that same child that something written on paper can only represent two-dimensional objects just because the paper itself is flat. XML in many variants is plainly richer in concept than its simple hierarchical representation and thus quite suited to ER. I believe a previous poster mention RDF... a perfect example.

Punchline: XML is neat, XML is tasty, but XML is not inherently more or less expressive than ER; it just requires a little critical thinking (and index tweaking) to tune ER engines to deal with it. (Once tuned, the ER engines dominate all others in performance.)