Not directly, anyway. Fair enough. But there are smartpeopleatmicrosoft working directly on their retail products too.
Re:Dynamic Relational: change it, DON'T toss it
on
The NoSQL Ecosystem
·
· Score: 1
UI designer doesn't want to consider that Mary might have 5 addresses or 6 mothers or work 7 jobs simultaneously
As an aside, it's a good rule of thumb that the only limits that make sense are 0, 1, and infinity. If you find yourself hard-coding a limit of 7, 42, or 6175, there's probably something wrong with your design.
Reading your post, I was half-expecting to see "educated stupid" crop up. You're claiming that 40 years of accumulated wisdom managing petabytes worth of data is worthless, and that your "advanced programming techniques" can do a better job? Put a sock in it. Maybe it's you who has the problem here. After all, you can't grasp that people actually do model data relationally, do produce object-relational layers that work just fine, and do produce systems that process vast amounts of information relationally.
What's next? Will you advocate that people who advocate antiobiotics don't your "advanced medicinal techniques" based on the four humors?
Oh, and by the way: "data" is already plural, you numbskull.
Yep. You don't need SOAP per se, though. The important thing is having the merchant talk directly to Microsoft. Some people are oddly resistant to that notion though, and if you're going to use the tracking pixel approach, you need a MAC.
That indicates there is absolutely no auditing or rationalization between what the e-tailer thinks should be paid out and what MS thinks should be paid out. Even something as stupid as end-of-month totals should flag that there are bogus transactions.
Agreed. The sad thing is that from a certain point of view, it can make more sense to limit the damage through audits than to try to make the system secure in the first place. Just look at the state of the credit card system.
Merchants must at least have some ability to program, otherwise they wouldn't be able to create sites at all. Creating a MAC authenticator isn't hard: all you need to do is call a hash function a few times. But as another poster mentioned, the better thing to do is to just have the merchant talk directly to Microsoft and sidestep the whole problem.
I gave concrete reasons why DNS wouldn't work well implemented as if it were a single global relational database. You reply with hysterical rhetoric and conspiratorial allegations. I'm done with you.
By the way: it's perfectly possible (and in some cases, even reasonable) for a DNS server to use a relational database to store its records.
First, we're comparing local database access via SQL with local database access using lower-level APIs. Networks don't factor into this discussion.
Second, you're letting your prejudices get in the way. You feel like the SQL parsing must be incurring some significant overhead, so you feel like the best way to avoid it is to "just" seek and read. (The word "just" in a rationale is a giant red flag.)
Why don't you actually profile your code? The time spend dealing with the SQL will be minuscule compared to the time you spend accessing your actual data.
In MSSQL a FUNCTION doesn't perform well as a named sub-query - because it merely evaluates the function over and over for each row it encounters.
I'm surprised the query optimizer doesn't try to inline the function. That's a shame. Views are nice too, but aren't quite as general-purpose.
Re:Everything old is new again
on
The NoSQL Ecosystem
·
· Score: 3, Interesting
Your question reminds me of the people who say, "if flight records are so strong, why don't we just build the whole plane out of the stuff they use to make them?" You might as well ask, "if DNS is so great, why don't we implement filesystems in terms of it?" Your post demonstrates that you you haven't considered context and purpose.
Relational databases are models. You can certainly describe DNS in terms of a relational schema. In principle, you could construct a wrapper and query it with SQL. But there's no reason to do that, because with someone as simple as DNS, the full power of a relational query engine doesn't buy you much.
Most datasets aren't that simple.
Furthermore, DNS is an open standard that needs to be accessible in as simple a way as possible. Complicating it with relational semantics wouldn't have been worthwhile (because of DNS's relative simplicity), and would have significantly hampered DNS's interoperability.
That is, if relational databases had existed when DNS was implemented, which they didn't.
Furthermore, DNS is a distributed, decentralized database. You couldn't use a RDBMS (the software that realized the abstract model of a relational database) to manage it even if you wanted to. That doesn't apply to most datasets, which however large, are still managed by a single organization, and which are accessed by software under the control of that organization.
Your comparison really makes no sense whatsoever. The vast majority of databases aren't put under the same constraints DNS, and so can take advantage of the much greater flexibility an RDBMS affords.
You're basically arguing that we can't have efficient engines in automobiles because of a few of them might need to tow 18 ton trailers and withstand mortar rounds. It's ridiculous.
Sure. A MAC actually can mean two things, depending on context: an algorithm or a value. I'm going to use "MAC" to mean the algorithm, and "authenticator" to refer to the output of the algorithm. YMMV.
The MAC takes as input the message to be authenticated, M, and a key S. Let's say that M is information about the item to be purchased, and S is a password the merchant set up with Microsoft. Running the MAC on M and S produces A. The sender of the message sends both A and M to the recipient. In more concrete terms, the tracking pixel's URL includes both information about the purchased item (like it does now) and the output of the MAC algorithm.
The recipient runs the MAC algorithm on the M' he receives (using the agreed-upon S), and compares its output, A' to the A it received along with the message. If A = A', M is authentic. If not, M is a forgery.
A malicious user could alter the tracking pixel URL, sure, but because she doesn't know S, she can't generate an authenticator A that the server will accept. She can choose to either send A and M unaltered, or not send anything at all.
One of the more popular MACs is called an HMAC, which stands for Hash MAC. Unsurprisingly, it's a MAC build out of a hash function. The Wikipedia article provides details about how an HMAC is actually constructed. It's important to use that construction, because simpler approaches to building MACs out of hash functions (like just concatenating S and M) are vulnerable to various attacks.
Say what you will about Microsoft's business practices, but incredibly smart people work there. The idea expressed by your comment and the ten million others just like it is a cop out: it's a lot easier to call Micro$oft stupid than to take a hard look at our society and thinking about why large software companies (and large companies in general) have strong economic incentives to produce shit, about why they're not accountable to society at large, and why they can accumulate so much power.
Microsoft is hardly stupid: on the contrary, its managers are quite savvy, and are the reason Microsoft is where it is today. Other large software companies would do exactly the same things in the same position.
The real reasons we're angry are political. Our antitrust enforcement is lax. Our politicians are corrupt. We don't hold our government responsible for passing laws that favor the very few over the very many, like the DMCA. Our income taxes aren't progressive enough. We're not willing to enforce open standards. We let anything under the sun be patented. We need to address the root causes of these problems.
But thinking about all that is hard. It's easier to just say Microsoft sucks, isn't it?
Re:And I am missing it greatly on Linux
on
The NoSQL Ecosystem
·
· Score: 2, Insightful
One of the big attractions of using a database to store your information is having a consistent API for accessing your data. I'm not convinced that what you want, having both SQL and non-SQL methods to access the same dataset, is ever actually useful. The overhead SQL imposes is actually minuscule compared to the cost of data access itself.
If you go the Berkeley DB route, you're going to need to build an application-level data access layer anyway. If you have a complex query to perform, just do it through that access layer.
On the other hand, if you use a SQL engine, you can go "small and light" simply by using "small and light" queries. There's no particular reason you can't simply run SELECT * FROM mytable WHERE id=? repeatedly, incrementing id each time.
You're absolutely right. SHA-1 is sturdy enough, and would still have been a responsible choice[1]. Nevertheless, moving to one of the SHA-2 algorithms (like SHA-256 or SHA-512) moves the mental confidence gauge from "damn sure" to "would bet my career on it".
One point worth mentioning is that if you're worried about the output size of one of the SHA-2 hash functions (64 bytes is a little heavy), you can just truncate the output. SHA-512 truncated to the size of SHA-1 (160 bits) shouldn't be any less secure than SHA-1 itself.
[1] Unlike MD5, which is as secure as a treehouse in a tornado. It still absolutely boggles the mind that people use raw, unsalted MD5 to store passwords, and use raw MD5s file authentication.
Parsing and building SQL is just overhead, and especially parsing SQL is no easy and light task.
No optimization without quantification. Parsing is very fast, especially compared to disk IO. Are you sure the SQL is slowing your program to any appreciate (or even measurable) degree? You should be able to measure any supposed effect with a profiler.
Nevertheless, if SQLite so offends your sensibilities, you can always use Berkeley DB. It gives you a similarly powerful storage engine without the necessity (or ability) to write SQL to access it.
Is writing SQL table definitions instead of equivalent assertions in your language of choice really the bottleneck of your development? Switching to an entirely different database paradigm solely to avoid writing "CREATE TABLE" sounds inefficient.
There exists some schema for every data set. The lack of a schema is (being generous) actually a lack of imagination and ingenuity on the part of the programmer.
you don't have to copy everything to a new table or ALTER the existing one
How is that any better than creating a second table with the new schema, and slowly (or lazily) migrating records from the old to the new table?
Maybe one rooted in truth, however. I can imagine a bill-reader using some simple image recognition against just one corner of the bill. You could get two $20 bills that way.
All Microsoft needed to do was include a Message Authentication Code (such as, say, HMAC-SHA1) in the tracking image URL. Microsoft and the merchant obviously already have a shared secret they can use for the purpose. Using a MAC would have been practically free.
Given what Microsoft pays its programmers, I'm just appalled that nobody thought to include basic precautions in a brand-new interface written in this day and age. Whoever wrote the Bing API specification really should have known better.
I think I'd rather see the opposite: That non-relation DBs become the mainstream, and they have SQL added for the odd occasion it is useful. Relational has some nice properties for ad-hoc querying, but for everything else they are a nuisance.
Berkeley DB is a very good non-relational database with multiple language bindings, several storage engines, and transaction support. It's been around for 24 years, and has seen some appreciable use.
But that use was nothing compared to the database explosion that SQLite brought about when it was released. SQLite is almost exactly like Berkeley DB, except that it has a SQL engine on top. Almost everyone is using SQLite, and many Berkeley DB users are moving over to it.
Why? Because SQLite is relational! That constitutes some serious evidence that relationship databases are more than "a nuisance".
Slashdot Mirror
Not directly, anyway. Fair enough. But there are smart people at microsoft working directly on their retail products too.
As an aside, it's a good rule of thumb that the only limits that make sense are 0, 1, and infinity. If you find yourself hard-coding a limit of 7, 42, or 6175, there's probably something wrong with your design.
What makes you think that relational calculus can't be extended to support spatial information? After all, it's just another kind of index.
Reading your post, I was half-expecting to see "educated stupid" crop up. You're claiming that 40 years of accumulated wisdom managing petabytes worth of data is worthless, and that your "advanced programming techniques" can do a better job? Put a sock in it. Maybe it's you who has the problem here. After all, you can't grasp that people actually do model data relationally, do produce object-relational layers that work just fine, and do produce systems that process vast amounts of information relationally.
What's next? Will you advocate that people who advocate antiobiotics don't your "advanced medicinal techniques" based on the four humors?
Oh, and by the way: "data" is already plural, you numbskull.
And so we come to the core of the issue: people aren't really opposed to relational databases, but instead to relational database administrators.
Yep. You don't need SOAP per se, though. The important thing is having the merchant talk directly to Microsoft. Some people are oddly resistant to that notion though, and if you're going to use the tracking pixel approach, you need a MAC.
Agreed. The sad thing is that from a certain point of view, it can make more sense to limit the damage through audits than to try to make the system secure in the first place. Just look at the state of the credit card system.
Merchants must at least have some ability to program, otherwise they wouldn't be able to create sites at all. Creating a MAC authenticator isn't hard: all you need to do is call a hash function a few times. But as another poster mentioned, the better thing to do is to just have the merchant talk directly to Microsoft and sidestep the whole problem.
I gave concrete reasons why DNS wouldn't work well implemented as if it were a single global relational database. You reply with hysterical rhetoric and conspiratorial allegations. I'm done with you.
By the way: it's perfectly possible (and in some cases, even reasonable) for a DNS server to use a relational database to store its records.
First, we're comparing local database access via SQL with local database access using lower-level APIs. Networks don't factor into this discussion.
Second, you're letting your prejudices get in the way. You feel like the SQL parsing must be incurring some significant overhead, so you feel like the best way to avoid it is to "just" seek and read. (The word "just" in a rationale is a giant red flag.)
Why don't you actually profile your code? The time spend dealing with the SQL will be minuscule compared to the time you spend accessing your actual data.
I'm surprised the query optimizer doesn't try to inline the function. That's a shame. Views are nice too, but aren't quite as general-purpose.
Your question reminds me of the people who say, "if flight records are so strong, why don't we just build the whole plane out of the stuff they use to make them?" You might as well ask, "if DNS is so great, why don't we implement filesystems in terms of it?" Your post demonstrates that you you haven't considered context and purpose.
Relational databases are models. You can certainly describe DNS in terms of a relational schema. In principle, you could construct a wrapper and query it with SQL. But there's no reason to do that, because with someone as simple as DNS, the full power of a relational query engine doesn't buy you much.
Most datasets aren't that simple.
Furthermore, DNS is an open standard that needs to be accessible in as simple a way as possible. Complicating it with relational semantics wouldn't have been worthwhile (because of DNS's relative simplicity), and would have significantly hampered DNS's interoperability.
That is, if relational databases had existed when DNS was implemented, which they didn't.
Furthermore, DNS is a distributed, decentralized database. You couldn't use a RDBMS (the software that realized the abstract model of a relational database) to manage it even if you wanted to. That doesn't apply to most datasets, which however large, are still managed by a single organization, and which are accessed by software under the control of that organization.
Your comparison really makes no sense whatsoever. The vast majority of databases aren't put under the same constraints DNS, and so can take advantage of the much greater flexibility an RDBMS affords.
You're basically arguing that we can't have efficient engines in automobiles because of a few of them might need to tow 18 ton trailers and withstand mortar rounds. It's ridiculous.
Sure. A MAC actually can mean two things, depending on context: an algorithm or a value. I'm going to use "MAC" to mean the algorithm, and "authenticator" to refer to the output of the algorithm. YMMV.
The MAC takes as input the message to be authenticated, M, and a key S. Let's say that M is information about the item to be purchased, and S is a password the merchant set up with Microsoft. Running the MAC on M and S produces A. The sender of the message sends both A and M to the recipient. In more concrete terms, the tracking pixel's URL includes both information about the purchased item (like it does now) and the output of the MAC algorithm.
The recipient runs the MAC algorithm on the M' he receives (using the agreed-upon S), and compares its output, A' to the A it received along with the message. If A = A', M is authentic. If not, M is a forgery.
A malicious user could alter the tracking pixel URL, sure, but because she doesn't know S, she can't generate an authenticator A that the server will accept. She can choose to either send A and M unaltered, or not send anything at all.
One of the more popular MACs is called an HMAC, which stands for Hash MAC. Unsurprisingly, it's a MAC build out of a hash function. The Wikipedia article provides details about how an HMAC is actually constructed. It's important to use that construction, because simpler approaches to building MACs out of hash functions (like just concatenating S and M) are vulnerable to various attacks.
Say what you will about Microsoft's business practices, but incredibly smart people work there. The idea expressed by your comment and the ten million others just like it is a cop out: it's a lot easier to call Micro$oft stupid than to take a hard look at our society and thinking about why large software companies (and large companies in general) have strong economic incentives to produce shit, about why they're not accountable to society at large, and why they can accumulate so much power.
Microsoft is hardly stupid: on the contrary, its managers are quite savvy, and are the reason Microsoft is where it is today. Other large software companies would do exactly the same things in the same position.
The real reasons we're angry are political. Our antitrust enforcement is lax. Our politicians are corrupt. We don't hold our government responsible for passing laws that favor the very few over the very many, like the DMCA. Our income taxes aren't progressive enough. We're not willing to enforce open standards. We let anything under the sun be patented. We need to address the root causes of these problems.
But thinking about all that is hard. It's easier to just say Microsoft sucks, isn't it?
One of the big attractions of using a database to store your information is having a consistent API for accessing your data. I'm not convinced that what you want, having both SQL and non-SQL methods to access the same dataset, is ever actually useful. The overhead SQL imposes is actually minuscule compared to the cost of data access itself.
If you go the Berkeley DB route, you're going to need to build an application-level data access layer anyway. If you have a complex query to perform, just do it through that access layer.
On the other hand, if you use a SQL engine, you can go "small and light" simply by using "small and light" queries. There's no particular reason you can't simply run SELECT * FROM mytable WHERE id=? repeatedly, incrementing id each time.
You're absolutely right. SHA-1 is sturdy enough, and would still have been a responsible choice[1]. Nevertheless, moving to one of the SHA-2 algorithms (like SHA-256 or SHA-512) moves the mental confidence gauge from "damn sure" to "would bet my career on it".
One point worth mentioning is that if you're worried about the output size of one of the SHA-2 hash functions (64 bytes is a little heavy), you can just truncate the output. SHA-512 truncated to the size of SHA-1 (160 bits) shouldn't be any less secure than SHA-1 itself.
[1] Unlike MD5, which is as secure as a treehouse in a tornado. It still absolutely boggles the mind that people use raw, unsalted MD5 to store passwords, and use raw MD5s file authentication.
No optimization without quantification. Parsing is very fast, especially compared to disk IO. Are you sure the SQL is slowing your program to any appreciate (or even measurable) degree? You should be able to measure any supposed effect with a profiler.
Nevertheless, if SQLite so offends your sensibilities, you can always use Berkeley DB. It gives you a similarly powerful storage engine without the necessity (or ability) to write SQL to access it.
Is writing SQL table definitions instead of equivalent assertions in your language of choice really the bottleneck of your development? Switching to an entirely different database paradigm solely to avoid writing "CREATE TABLE" sounds inefficient.
There exists some schema for every data set. The lack of a schema is (being generous) actually a lack of imagination and ingenuity on the part of the programmer.
How is that any better than creating a second table with the new schema, and slowly (or lazily) migrating records from the old to the new table?
How on earth is that better than using an existing ORM library? Even if you have to write your own, an ORM isn't particularly difficult to write.
Maybe one rooted in truth, however. I can imagine a bill-reader using some simple image recognition against just one corner of the bill. You could get two $20 bills that way.
A cleverer backdoor would have been a weak custom MAC (say, just the H(M) + secret). Then it'd still be exploitable, yet not obviously bad.
This article goes into the reasons why HMACs are constructed the way they are, and about how naive constructions can be exploited.
All Microsoft needed to do was include a Message Authentication Code (such as, say, HMAC-SHA1) in the tracking image URL. Microsoft and the merchant obviously already have a shared secret they can use for the purpose. Using a MAC would have been practically free.
Given what Microsoft pays its programmers, I'm just appalled that nobody thought to include basic precautions in a brand-new interface written in this day and age. Whoever wrote the Bing API specification really should have known better.
Fixed that for you.
What do you think stored functions and procedures are?
Berkeley DB is a very good non-relational database with multiple language bindings, several storage engines, and transaction support. It's been around for 24 years, and has seen some appreciable use.
But that use was nothing compared to the database explosion that SQLite brought about when it was released. SQLite is almost exactly like Berkeley DB, except that it has a SQL engine on top. Almost everyone is using SQLite, and many Berkeley DB users are moving over to it.
Why? Because SQLite is relational! That constitutes some serious evidence that relationship databases are more than "a nuisance".