Here's a crazy idea: how about suggesting that what websites you see in a browser window are kind of like "the commons" (somehow public) and therefore, to ensure that the commons can be consumed by everyone, that all web browsers must, BY LAW, adhere to an internationally maintained standard for web browser rendering.
Although we know that might never happen for political reasons, at least that way, which browser you use would no longer be an issue - if they all render the same! Then it would be about other features that make one browser better than the other, like, integrated spell check, who knows.
Microsoft has a right to include a browser. A browser is a common tool... and Windows is their own product. However, where they would go wrong if it can be said that they damage the public good by not following "accessibility of websites" standards.
Welll... except for Microsoft's huge investment in web services and service oriented architectures (SOA). I don't see the problem you describe so long as people follow, say, the SOAP protocol over some TCP port, and make use of the WS-* frameworks, or even common sense, for securing their web services. The format of the actual messages exchanged - well - that's a different story.. in the healthcare industry we have the X12 (still not XML) and HL7 (some XML, some not XML) data standards which are not rigid enough.
As for old software not being designed to work in this environment, I agree that there still are some legacy environments that aren't up to this level yet, but many have written and/or are writing.NET data providers that can be used to expose their vast functionality and data.
I for one have used PHP, for example, to work with web services written in.NET, and it worked quite well. If that.NET webservice had been exposing data from an old legacy COBOL system, I'd have an open- and closed-source, web services enabled COBOL legacy application. Phew, what a mouthful. But doable. And not just with.NET - I just think.NET's implementation is pretty good.
Phew, that's a good question. I'm not sure there is a direct relationship between the column-oriented and non-relational features - it depends on how the non-relational features such as the XML datatype in SQL Server 2005 are actually implemented. I think the XML data type of SQL 2005 can serialize the XML and also index it into something resembling a row-oriented table hidden behind the scenes.
However, taking into account that the non-relational features seem to be leaning towards storing specific data types which may be more complicated, like XML, binary objects, other tables, who knows, it looks like we are perhaps bordering on object-relational databases.
So if you apply the strengths of a column oriented data store (fast retrieval speed, strongly typed columns, possibly very efficient compression due to data type/structure similarity), what it seems like is you get lists of objects, one object type per file, in a column store.
In a row-oriented DB, you would get a tuple object which consists of other objects, but all related objects are stored in the same tuple, thus increasing write speed but possibly at the cost of read speed (you have to read the whole tuple to find a specific column). Basically, here you have lists of tuples, and tuples are objects themselves - very close to a column oriented data store with only one column of tuples - not necessarily as efficient for reading as partitioning the data into separate files by column/data type of each column in the tuple.
So, maybe my answer is not worded well, but maybe the non-relational features are an admission to the fact that people want to store more complicated data types in new and different ways. Whether or not the storage is column or row based has to do with how one partitions and separates the data. To me these concepts are loosely related.
I know, I know, it's Microsoft.. BUT - check out the upcoming LINQ (Language Integrated Query) and DLINQ (LINQ for databases) in the.NET languages - I think this might be a step in the direction you desire.
Just to add to that - PICK/D3 is called a "multivalue" database, and there are a lot of interesting kinds of databases out their that use this "multidimensional"/sparse array storage concept, such as the ancient MUMPS legacy system/database (now implemented as Intersystem's Cache product) found in old Veteran's Affairs (VA) systems in the US.
I actually wonder if some of the current databases such as Microsoft SQL Server, etc. aren't going to actually start morphing into these older styles of databases due to the increase in use of XML files/data, which by their very nature are hierarchical and kind of "multidimensional". Actually, the company that now maintains PICK/D3 (Raining Data) has an interesting XML database (Tigerlogic) that uses some of the old technology and new technology combined.
This could be a great and possibly painful experience of history repeating itself!
Using compactflash to IDE adapters is inexpensive and readily available, but the real solid state drives perform much better, last much longer, and are directly comparable to their mechanical counterparts.
Actually, solid-state hard drives with SCSI and IDE interfaces already exist (BitMicro and Sandisk both have them, Sandisk having bought one of the original SSD vendors M-Systems). There are also some drives with SATA 1.5 GB/s interfaces, but I'm personally waiting for the next generation of drives (which is currently being developed) that has SATA 3.0Gb/s interfaces, and are standard sizes (3.5"). Those are going to be expensive at first, but they'll be amazing!
You can already swap out a 2.5" SATA laptop mechanical drive with a 2.5" SATA solid-state one, for example. They just cost too much (small drives cost a few thousand!).
I have been researching some of the more current SSD drives lately, and I know that they greatly improved the technology/algorithms behind how they write data to the physical memory. Most companies use some kind of wear-leveling techniques that evenly distribute the writes over the entire surface of the disk, maximizing the disk's life span. I have also read that the different-sized memory modules have different physical characteristics such that smaller modules are actually outlived by larger ones.
I can't give exact figures, but I've seen comparisons showing a reasonable life span (>20 years @ 100GB of writes/day) - some of the numbers are even comparable to those of spinning/mechanical hard drives. Considering how often mechanical hard drives seem to fail, it doesn't seem that there will be any major roadblocks in terms of reliability.
I know what I've written is mostly qualitative (apologies on that), but I know the research into how to mitigate the problem of life span has truly advanced in the last few years as interest in SSD has increased. Jim Gray of Microsoft Research fame, predicted that SSD would replace mechanical drives not far off from now. Check out his paper "Flash Disk Opportunity for Server-Applications" for more on that.
I am pretty sure they mean something like the W3C's message transmission optimization mechanism (MTOM) which is the optimized way to send binary data/large SOAP messages over HTTP protocols ("web services"). I didn't RTFA, but if any of the claims made involve the specific method of transmission for said data files, MTOM is probably prior art.
Other prior art may include the Microsoft research done with web services and data storage used in the Microsoft TerraServer project. Dr. Jim Gray and Microsoft had basically proposed an idea pretty similar to this a while ago (see his presentation of grid-computing based web services - careful that's a powerpoint presentation!).
Obligatory "The Far Side" cartoon reference (slightly embellished): Upon investigating further, scientists discovered a broken piano in the large black spot (as well as thousands if not millions of missing aircraft, broken computers, loose change, small animals and socks)... it turns out that the spot the sun never shines wasn't in Iowa! It was on Mars!
Well, now we know where to stick it where the Sun don't shine.
I think the first thing that must be changed is the concept that you are "buying a naked pc". That implies that without the operating system, the PC is naked! Why isn't a PC without it's case screwed on considered a naked PC?
Basically, vendors don't seem willing to believe that people have the know-how to buy some hardware and then somehow make it work, which kind of makes sense. For example, I'm sure few people would buy a PC without the BIOS installed, and Dell and the like aren't going to cater their huge business to the hobbyists who would flash a PC with their own BIOS, for example.
On the other hand, why they can't make a small stipulation to sell X% of units raw to folks that are DIY'ers, is beyond me.. they could even sell it with a disclosure that they don't support ANY operating system in their contract, however their hardware has been tested with XYZ operating systems.
I admit to not knowing what they are either, but here it is - radix tree. Sounds like a pretty nice tree data structure, an alternative to your hashtables, balanced trees, etc. designed for sets of strings.
Since I love the PHP and PERL implementations of associate arrays, I guess it's neat to have these data structures implemented directly in the kernel! Sounds like they might be using them as a sort of in-memory data store.
I don't disagree with your comment, as I have spent many hours in usability testing, however, I guess I have even had the experience that a group of users has confirmed through usability testing that a certain UI was what they wanted, the final product was produced as agreed upon, and then the user decided that the UI was not really what they wanted. They "flip-flopped".
This is one reason I admire the simple UI of Google, for example. One simple search box does nearly everything, or at least, so it seems to the end-user. The dynamic aspect of the UI seems to appeal to many people.
I had to reply to this because hearing someone else say it just felt so good.
It begs a certain question - users often REALLY want a dynamic and flexible system, yet most software is rigid and structured. Users often say they want rigid, structured software (they want the structure to help them drive their process) but then realize that the structure is too rigid and give up using the software.
It's like some kind of a chicken and egg problem - do you give the user a free-text box and let them type whatever they want, however they want, or do you give them 3 entry boxes and force them to input a first name, middle initial and last name? In the end, does it matter which you choose if they just change their mind after you designed, wrote, tested and implemented their software?
This is a bit rhetorical/satirical, but I guess as long as they pay well enough, who cares?
Seriously though, it is an art and a science to accurately project what a user is actually going to need versus what they say they want. It is also a little bit of luck and experience that makes it so you get it right.
I'm not sure I'd be happy answering these questions at an interview either, but it sounded an awful lot like the Keirsey Temperment sorter (see http://www.advisorteam.org/instruments/KTS-II_orig inal.html) or a Meyers-Briggs (see http://en.wikipedia.org/wiki/Myers-Briggs_Type_Ind icator) personality test. HR departments for large organizations often choose these because they are considered useful for organizational development. I personally am not sure if they really work!
You make an interesting point. Maybe none of these darn "value exchange"/currency systems really make any sense, including dollars, euros, yen, yootles, Beanz, etc.. So here's a question - would you rather have the Star Trek economy instead?
Oh, please don't misunderstand my mostly light-hearted comment - I recognize the economic "utils" hidden in there. But is it not a strange thought to think that anyone other than Yahoo would use a "yahoo branded" kind of value system? I don't know why, but that just sounds funny.
For the record (my economics is a bit rusty), AFAIK a util represents the value or utility of the item to me, and that is not tied to money. However, the saying "put your money where your mouth is" is something that stands out to me as putting something of value on the line to state how important something is to a person. A "yootle" is a tool that only has value if it has someone enforcing its value or if it is something everyone values. Since everyone already uses "currency", and currency exchange, etc. is already figured out, why not use that?
This is actually a problem I've seen crop up in knowledge economies/markets and compensation for cooperating/contributing to them. From what I've seen in past experiments, anything less than something that makes you happy when you have it or hurt when you don't (like money) doesn't seem to work. Of course, that's a pessimistic view - that everything revolves around money. I hope that really isn't the case, but sometimes I fear it is!
I'm sorry, but I fail to see the wisdom in a name as gawd-awful as "yootles"! I mean, you wouldn't expect to see some guru walking around and complaining that his/her existential theory of quantum physics and intracellular electromechanical equilibrium & interstellar space travel (IANAG - I am not a guru in those regards) was upset by not having enough "yootles" to buy a cup of coffee. Yootles are yet another substitute for good old hard cash?
Ok, maybe it's a little bit interesting, but seriously folks..
Actually, the military used Bayesian subjective probability (a variation on the standard Bayesian probability theory) to find lost "toys" several times. The book "Blind Man's Bluff" (ISBN: 006103004X) actually describes how they used it to find sunken Russian test nuclear missles and some lost American weapons during the Cold War. It's a great read!
I have been fascinated by Bayesian subjective probability because, in layman's terms, the premise of it is that it uses the subjective opinion of an expert to extrapolate statistically what the most likely possible outcome is from a set of given scenarios.
In the case of these events in particular, BSP turned out to be quite accurate, thank goodness!
Slashdot Mirror
Don't get overloaded reading my FIRST POST!!! w00t!
Why push just Firefox?
Here's a crazy idea: how about suggesting that what websites you see in a browser window are kind of like "the commons" (somehow public) and therefore, to ensure that the commons can be consumed by everyone, that all web browsers must, BY LAW, adhere to an internationally maintained standard for web browser rendering.
Although we know that might never happen for political reasons, at least that way, which browser you use would no longer be an issue - if they all render the same! Then it would be about other features that make one browser better than the other, like, integrated spell check, who knows.
Microsoft has a right to include a browser. A browser is a common tool... and Windows is their own product. However, where they would go wrong if it can be said that they damage the public good by not following "accessibility of websites" standards.
Dimension Six --- full of hot air
Welll... except for Microsoft's huge investment in web services and service oriented architectures (SOA). I don't see the problem you describe so long as people follow, say, the SOAP protocol over some TCP port, and make use of the WS-* frameworks, or even common sense, for securing their web services. The format of the actual messages exchanged - well - that's a different story.. in the healthcare industry we have the X12 (still not XML) and HL7 (some XML, some not XML) data standards which are not rigid enough.
.NET data providers that can be used to expose their vast functionality and data.
.NET, and it worked quite well. If that .NET webservice had been exposing data from an old legacy COBOL system, I'd have an open- and closed-source, web services enabled COBOL .NET - I just think .NET's implementation is pretty good.
As for old software not being designed to work in this environment, I agree that there still are some legacy environments that aren't up to this level yet, but many have written and/or are writing
I for one have used PHP, for example, to work with web services written in
legacy application. Phew, what a mouthful. But doable. And not just with
SixD
Phew, that's a good question. I'm not sure there is a direct relationship between the column-oriented and non-relational features - it depends on how the non-relational features such as the XML datatype in SQL Server 2005 are actually implemented. I think the XML data type of SQL 2005 can serialize the XML and also index it into something resembling a row-oriented table hidden behind the scenes.
;)
However, taking into account that the non-relational features seem to be leaning towards storing specific data types which may be more complicated, like XML, binary objects, other tables, who knows, it looks like we are perhaps bordering on object-relational databases.
So if you apply the strengths of a column oriented data store (fast retrieval speed, strongly typed columns, possibly very efficient compression due to data type/structure similarity), what it seems like is you get lists of objects, one object type per file, in a column store.
In a row-oriented DB, you would get a tuple object which consists of other objects, but all related objects are stored in the same tuple, thus increasing write speed but possibly at the cost of read speed (you have to read the whole tuple to find a specific column). Basically, here you have lists of tuples, and tuples are objects themselves - very close to a column oriented data store with only one column of tuples - not necessarily as efficient for reading as partitioning the data into separate files by column/data type of each column in the tuple.
So, maybe my answer is not worded well, but maybe the non-relational features are an admission to the fact that people want to store more complicated data types in new and different ways. Whether or not the storage is column or row based has to do with how one partitions and separates the data. To me these concepts are loosely related.
Or I'm full of hot air. LOL
SixD
I know, I know, it's Microsoft.. BUT - check out the upcoming LINQ (Language Integrated Query) and DLINQ (LINQ for databases) in the .NET languages - I think this might be a step in the direction you desire.
SixD
Just to add to that - PICK/D3 is called a "multivalue" database, and there are a lot of interesting kinds of databases out their that use this "multidimensional"/sparse array storage concept, such as the ancient MUMPS legacy system/database (now implemented as Intersystem's Cache product) found in old Veteran's Affairs (VA) systems in the US.
I actually wonder if some of the current databases such as Microsoft SQL Server, etc. aren't going to actually start morphing into these older styles of databases due to the increase in use of XML files/data, which by their very nature are hierarchical and kind of "multidimensional". Actually, the company that now maintains PICK/D3 (Raining Data) has an interesting XML database (Tigerlogic) that uses some of the old technology and new technology combined.
This could be a great and possibly painful experience of history repeating itself!
SixD
Using compactflash to IDE adapters is inexpensive and readily available, but the real solid state drives perform much better, last much longer, and are directly comparable to their mechanical counterparts.
Actually, solid-state hard drives with SCSI and IDE interfaces already exist (BitMicro and Sandisk both have them, Sandisk having bought one of the original SSD vendors M-Systems). There are also some drives with SATA 1.5 GB/s interfaces, but I'm personally waiting for the next generation of drives (which is currently being developed) that has SATA 3.0Gb/s interfaces, and are standard sizes (3.5"). Those are going to be expensive at first, but they'll be amazing!
You can already swap out a 2.5" SATA laptop mechanical drive with a 2.5" SATA solid-state one, for example. They just cost too much (small drives cost a few thousand!).
SixD
I have been researching some of the more current SSD drives lately, and I know that they greatly improved the technology/algorithms behind how they write data to the physical memory. Most companies use some kind of wear-leveling techniques that evenly distribute the writes over the entire surface of the disk, maximizing the disk's life span. I have also read that the different-sized memory modules have different physical characteristics such that smaller modules are actually outlived by larger ones.
I can't give exact figures, but I've seen comparisons showing a reasonable life span (>20 years @ 100GB of writes/day) - some of the numbers are even comparable to those of spinning/mechanical hard drives. Considering how often mechanical hard drives seem to fail, it doesn't seem that there will be any major roadblocks in terms of reliability.
I know what I've written is mostly qualitative (apologies on that), but I know the research into how to mitigate the problem of life span has truly advanced in the last few years as interest in SSD has increased. Jim Gray of Microsoft Research fame, predicted that SSD would replace mechanical drives not far off from now. Check out his paper "Flash Disk Opportunity for Server-Applications" for more on that.
SixD
It's spelled HIPAA!!!!!! Sorry, that drives me crazy!
-6D
I am pretty sure they mean something like the W3C's message transmission optimization mechanism (MTOM) which is the optimized way to send binary data/large SOAP messages over HTTP protocols ("web services"). I didn't RTFA, but if any of the claims made involve the specific method of transmission for said data files, MTOM is probably prior art.
Other prior art may include the Microsoft research done with web services and data storage used in the Microsoft TerraServer project. Dr. Jim Gray and Microsoft had basically proposed an idea pretty similar to this a while ago (see his presentation of grid-computing based web services - careful that's a powerpoint presentation!).
-SixD
Glad to lay this ages old argument to rest.. women are actually smarter than men.. but because men's pheromones make them that way?!?!
I sense a disturbance in the force! It's as if a million feminists shouted out at once and were suddenly silenced, if not utterly confused!
SixD
.. they found it!
Obligatory "The Far Side" cartoon reference (slightly embellished):
Upon investigating further, scientists discovered a broken piano in the large black spot (as well as thousands if not millions of missing aircraft, broken computers, loose change, small animals and socks)... it turns out that the spot the sun never shines wasn't in Iowa! It was on Mars!
Well, now we know where to stick it where the Sun don't shine.
-6d
Just look at my nickname - I've thought there were 6 all along!
*ducks from rotten tomatoes*
-6d
I think the first thing that must be changed is the concept that you are "buying a naked pc". That implies that without the operating system, the PC is naked! Why isn't a PC without it's case screwed on considered a naked PC?
Basically, vendors don't seem willing to believe that people have the know-how to buy some hardware and then somehow make it work, which kind of makes sense. For example, I'm sure few people would buy a PC without the BIOS installed, and Dell and the like aren't going to cater their huge business to the hobbyists who would flash a PC with their own BIOS, for example.
On the other hand, why they can't make a small stipulation to sell X% of units raw to folks that are DIY'ers, is beyond me.. they could even sell it with a disclosure that they don't support ANY operating system in their contract, however their hardware has been tested with XYZ operating systems.
-6d
I admit to not knowing what they are either, but here it is - radix tree. Sounds like a pretty nice tree data structure, an alternative to your hashtables, balanced trees, etc. designed for sets of strings.
Since I love the PHP and PERL implementations of associate arrays, I guess it's neat to have these data structures implemented directly in the kernel! Sounds like they might be using them as a sort of in-memory data store.
SixD
SixD
Ahhh yes, the sweet sound of "managing user expectations"! :)
I don't disagree with your comment, as I have spent many hours in usability testing, however, I guess I have even had the experience that a group of users has confirmed through usability testing that a certain UI was what they wanted, the final product was produced as agreed upon, and then the user decided that the UI was not really what they wanted. They "flip-flopped".
This is one reason I admire the simple UI of Google, for example. One simple search box does nearly everything, or at least, so it seems to the end-user. The dynamic aspect of the UI seems to appeal to many people.
SixD
I had to reply to this because hearing someone else say it just felt so good.
It begs a certain question - users often REALLY want a dynamic and flexible system, yet most software is rigid and structured. Users often say they want rigid, structured software (they want the structure to help them drive their process) but then realize that the structure is too rigid and give up using the software.
It's like some kind of a chicken and egg problem - do you give the user a free-text box and let them type whatever they want, however they want, or do you give them 3 entry boxes and force them to input a first name, middle initial and last name? In the end, does it matter which you choose if they just change their mind after you designed, wrote, tested and implemented their software?
This is a bit rhetorical/satirical, but I guess as long as they pay well enough, who cares?
Seriously though, it is an art and a science to accurately project what a user is actually going to need versus what they say they want. It is also a little bit of luck and experience that makes it so you get it right.
SixD
I'm not sure I'd be happy answering these questions at an interview either, but it sounded an awful lot like the Keirsey Temperment sorter (see http://www.advisorteam.org/instruments/KTS-II_orig inal.html) or a Meyers-Briggs (see http://en.wikipedia.org/wiki/Myers-Briggs_Type_Ind icator) personality test. HR departments for large organizations often choose these because they are considered useful for organizational development. I personally am not sure if they really work!
SixD
You make an interesting point. Maybe none of these darn "value exchange"/currency systems really make any sense, including dollars, euros, yen, yootles, Beanz, etc.. So here's a question - would you rather have the Star Trek economy instead?
SixD
Oh, please don't misunderstand my mostly light-hearted comment - I recognize the economic "utils" hidden in there. But is it not a strange thought to think that anyone other than Yahoo would use a "yahoo branded" kind of value system? I don't know why, but that just sounds funny.
For the record (my economics is a bit rusty), AFAIK a util represents the value or utility of the item to me, and that is not tied to money. However, the saying "put your money where your mouth is" is something that stands out to me as putting something of value on the line to state how important something is to a person. A "yootle" is a tool that only has value if it has someone enforcing its value or if it is something everyone values. Since everyone already uses "currency", and currency exchange, etc. is already figured out, why not use that?
This is actually a problem I've seen crop up in knowledge economies/markets and compensation for cooperating/contributing to them. From what I've seen in past experiments, anything less than something that makes you happy when you have it or hurt when you don't (like money) doesn't seem to work. Of course, that's a pessimistic view - that everything revolves around money. I hope that really isn't the case, but sometimes I fear it is!
SixD
Mod parent up - good point on "util"!
I'm sorry, but I fail to see the wisdom in a name as gawd-awful as "yootles"! I mean, you wouldn't expect to see some guru walking around and complaining that his/her existential theory of quantum physics and intracellular electromechanical equilibrium & interstellar space travel (IANAG - I am not a guru in those regards) was upset by not having enough "yootles" to buy a cup of coffee. Yootles are yet another substitute for good old hard cash?
Ok, maybe it's a little bit interesting, but seriously folks..
-6d
Actually, the military used Bayesian subjective probability (a variation on the standard Bayesian probability theory) to find lost "toys" several times. The book "Blind Man's Bluff" (ISBN: 006103004X) actually describes how they used it to find sunken Russian test nuclear missles and some lost American weapons during the Cold War. It's a great read!
I have been fascinated by Bayesian subjective probability because, in layman's terms, the premise of it is that it uses the subjective opinion of an expert to extrapolate statistically what the most likely possible outcome is from a set of given scenarios.
In the case of these events in particular, BSP turned out to be quite accurate, thank goodness!
-6d