Amusing. I go for a +1 or +2 funny and I end up with a 0 troll....
I was thinking in a Patrick Stewart voice when I typed this one. OK, I now predict my comment moderation to be -1 troll for attempting to lynch the techies.
So, each email segment (What you see as each email in your gmail web interface) would probably be assigned a docID which would then be put into bigtable which then went onto a gfs partition / shard.
Yup especially if it's something like an email fragment which you could well re-request.
If you have enough bandwidth though..... After you have done the request for the data and it's been returned to the client, you would then have to update the master that the data is now stored on another location, which you then have to manage that data in another location. Nope, that doesn't make any sense. The front end would only get the data and that's it. If the master sees that the front end was too far away, it might ask another master to replicate the data. Again, if you had lots and lots of fibre and network bandwidth which Google apparently does, you might just wait for an intelligent sync during the off hours...
Whether it works this way or not, I haven't seen or heard anything either way.
That one I don't know about, plausibly. I guess at that point of you have to think about your data retention laws and your requirements for keeping that data secure and whether putting it into the cloud is actually the right thing. Perhaps put it into the cloud encrypted?
While looking for another link earlier, the city of Los Angeles ran into similar problems with their police dept and they didn't move. http://www.msnbc.msn.com/id/31967328/
Whether Google is working with them on this one, or whether LAPD is keeping it internal is the question I guess.
But honestly, would it make sense for it to be encrypted? I would think it would be too computationally expensive and increase the response time of the request to encrypt this data.
Presumably, plain text and binary data (Video / docs etc) would be stored as just that.
OK, but he isn't a US citizen (Well doesn't indicate that he is). What are they going to do? Extradite him and charge him? If this was the case, why the hell haven't they done this to the rest of the world?
IANAL but if you then accessed / distibuted that in the US you could be in trouble. Given that your data wouldn't be re-assembled (And certainly not in your possession) till you accessed it in The Netherlands you should be fine. Aside from plausible deniability and all that.
If you could get into one of their data centres, find a master (Hope it's got the index you are after) break into it, then break into the boxes (Online or offline) then re-assemble the shards, sure.
Wouldn't it just be easier to threaten you or someone you care about and just say that something bad is going to happen unless you hand over the password?
Honestly, if they want access to the data that badly, installing a keylogger / screendumper when your not looking would be a shitload simpler.
That would be correct, if you look at their BGP advertisements it would figure that Google would have to transit it's own data.
So if your request for data (YouTube video etc) isn't located in the DC that you connected to, they would have to transit that data across their own links. It would then make sense that they would replicate their own data over those same links during the night on that side of the world when the link is quiet.
Shards can be located by different masters and different masters are located in different locations according to the data type.
So I think Japan (That's where they just dropped their Asia - US cable after all, so it makes sense) has a "complete" replication of all Google data. Some data is also replicated to containers (YouTube etc) for hosting at major ISPs. So all email data would be replicated in non-realtime. If you request something that isn't in that DC it's located in the US or wherever is closest (I guess).
There are multiple "complete" copies on the east and west coast as well as European hub sites or directly connected to European hub sites.
If you ask for a citation, I can dig something up for you....
Great, but who is going to look at them, you would have to get all the ISPs to make the change (They won't). Aside from this, what about all the corporates and everyone else, it's just not feasible.
The only way to do it would be to advertise the space of all the other root DNS servers into the routing table (That'll piss off a LOT of people, probably lead to a disconnection actually; potentially of all of Aus until each party stopped readvertising the routes).
The way that this works in China is that the government IS the largest ISP over there, China Telecom is owned by the government, (This is the reason why it works in the first place) and anybody else would be under heavy heavy presure to conform.
That infrastructure costs money, infrastructure that the ISPs don't have in place at the moment, and the government isn't going to pay for it.
They could redirect the IPs perhaps (Again costs money, were are they going to redirect it to? See point 1), but it won't stop people going to a Google site outside of Australia. Google host in multiple places in Australia with some of the larger ISPs having Google containers.
You have to remember that the guys who built Google maps and wave etc all work there. They hold a lot of IP there so that they can push money around as well.
They are not going down without a fight (I question whether the government would even have the money to keep an ongoing legal battle going against Google), they employ too many people there and have too much of a presence for this to happen. Plain and simple.
No company wants all the phone calls saying "I can't access Google" broadband margins are that bad on a per customer basis, the moment they phone rings from a customer they are losing money.
There was an article a while ago about the unmanned predators etc, basically, it is still a human flying it, just not in the drone itself. There is a ground crew that gets it off the ground then they pass control to someone else. The person flying it flies it by cameras and an instrument panel of information fed from the aircraft itself.
If you are at 12 - 20,000 feet, at that height, dropping bombs, taking photos or whatever else having meat soldiers in the cockpit of an aircraft is not going to mean anything. Quite simply you can't see a "wedding" or a "random car", shit, you can't see a lot of roads, 6 lane highways are just small lines. Your ability to pin point ANYTHING from that height is, well, crap. Get verification? Verification of what? The bomb going off?
From the height that these drones operate at, bombs are often guided in by GPS or laser from a ground crew.
One of the biggest reasons for these things is from an intel perspective over a war zone, you can have someone effectively followed, get photos of someone day or night if the coverage is good enough and determine whether that person is a threat or not. If he / she is, then you can find out who their associates are, where they are going and what they are up to. This is the reason why they have so many of them flying.
There was another comment about this, yes, it's not just Europe thinking this.
When people (Americans) come overseas and apologise for presidents and the stupid things their country is doing, that's gotta be embarrassing.
People who I know / have met from Australia and New Zealand... (Let's not talk about Canada)... and a number of people I know scattered across Asia share this belief as well.
MS probably could not be as big as they are with support contracts.
You REALLY think this?
Sure, nobody in the OEM market has support contracts with MS (That's the whole point of the OEM license, is that Microsoft DOESN'T support it), if you look at retail, you do get limited support, and if you phone Microsoft with a bug and it's determined that it is a bug, then you don't get charged for support.
ANY organisation that has a reasonable amount of installed computers ( >5000 in my experience, and certainly a lot smaller >1000 ) has a Premier support agreement with Microsoft.
I don't have exact numbers, but I know that MS make a killing out of support contracts for their software. They also make a decent amount out of their services groups.
IMO MS should have separated the system state/settings from the user & app settings in some meaningful way to make the OS more failure-resistant.
True, but the app Foo would be far more likely to corrupt %APPDATA%\Publisher Foo\App Foo\settings.xyz than %APPDATA%\Publisher Bar\App Bar\settings.abc as opposed to poorly handling error conditions while using the registry API and corrupting the registry or leaving some registry handle open, causing a leak on shutdown and other badness.
In theory, an application should only be accessing HKLM\Software or HKCU\Software (Or Services I guess...) (Citrix client modifying the Gina chain...) (Device software modifying device driver parameters...) (Security Configuration Wizard... that touches basically everything) Yeah, OK, user applications modify a great deal of system parameters.
I don't know if the situation you allude to would be THAT much better to be fair... At the moment, thinking about the limitations that Citrix has with the print subsystem and terminal services... To a degree, while you do get some shit applications that don't behave themselves, I think the limitations of ONLY being able to program to an API could be limiting for a lot of customers.
Thats the worst rationale ever. The opposite of the registry isn't overwriting INI files or whatever else you're thinking of. It is to call SHGetFolderPath with CSIDL_APPDATA and follow Microsoft's own guide lines.
Historically, everyone was writing settings to ini files located in \windows\system and this is where people were falling over each other. All Microsoft (And yourself by extension with your arguement) are specifying there is the *location* of application specific files, not what they contain, how they work and how they should be changed and modified.
From my perspective as an admin who has to deal with thousands of users, I think the idea of trying to have to understand the (potentially) 1000s of different methods that the different software vendors would use to store their configuration data to change one flag to change a checkbox for a group of users would be a complete waste of my time. Now potentially, a developer might come up with an easy method for me to interpret all the different formats that other developers use to store their data for a single true / false flag, but I can say that I am glad I am not in that position.
The primary problem I have with the registry is that it can't be locked during modification, and you couldn't set any access restrictions on any of the keys preventing other apps from messing (read corrupting) with it (this is on W95)
Locking I'll give you, but really, two applications shouldn't be writing to the same location at the same time. You can't do the access restrictions (By default) on any platform that I am aware of. Applications run as a user, an application that's poorly (or otherwise) written that runs as that user has the ability to modify (corrupt) any other program that the user is permissioned to access, registry or not it doesn't make any difference.
You've never actually read the Windows UI and programming design guides have you?
It was originally designed to host OS settings in a convenient central location
Actually from Wikipedia "Windows registry's primary purpose was to store configuration information for COM-based components". I've only been doing admin since Win NT 3.51, so I only know about it since then, I didn't realise it was in Win 3.1 as well.
The whole purpose of HKCU and HKLM in particular \Software was PURPOSEFULLY to have a central location for DIFFERENT software vendors to store settings in a single location as opposed to different vendors trying to use the same ini file names and treading on each other's toes, as well as problems with size limitations etc etc.
Thus its became this giant cluster-fuck that it is now, primarily because of backwards compatibility and previous strategic mistakes on the part of MS.
Citation required. I don't understand how you consider it a cluster fuck. Just because you don't understand it doesn't make it a cluster fuck.
They should have kept the registry API hidden and not allowed apps to write their shit all over the place
With the hundred of thousands of ISVs out there, you suggest to me a better way of managing on a per user and per machine basis of managing configuration settings.
Your rant at this point seems to change tangent based upon kernels and what was and wasn't possible. I'll let you answer.
Huh? Anti-virus / content filtering proxy servers, these ones don't handle zip very well, change around the way the web server works and they add in zip compression to the web server. They could have added zip to the returned data at any point, but doing at at this point, seems very plausible.
This explains a lot, a couple - few months ago, I started getting complaints about "potentially virus infected" / "unscanable" zip files when being served content from facebook.com and fbcdn.net etc.
They probably changed at this point how they were sending data out of the web server with zip compression and it all started falling over at this point....
Slashdot Mirror
Amusing. I go for a +1 or +2 funny and I end up with a 0 troll....
I was thinking in a Patrick Stewart voice when I typed this one. OK, I now predict my comment moderation to be -1 troll for attempting to lynch the techies.
In...
3...
2...
1..
What are these ads of which you speak?
He got into trouble for passing data across a US network?
I think he got into trouble for more than that.....
BigTable perhaps?
http://en.wikipedia.org/wiki/BigTable
http://static.googleusercontent.com/external_content/untrusted_dlcp/labs.google.com/en//papers/bigtable-osdi06.pdf
So, each email segment (What you see as each email in your gmail web interface) would probably be assigned a docID which would then be put into bigtable which then went onto a gfs partition / shard.
Yup especially if it's something like an email fragment which you could well re-request.
If you have enough bandwidth though..... After you have done the request for the data and it's been returned to the client, you would then have to update the master that the data is now stored on another location, which you then have to manage that data in another location. Nope, that doesn't make any sense. The front end would only get the data and that's it. If the master sees that the front end was too far away, it might ask another master to replicate the data. Again, if you had lots and lots of fibre and network bandwidth which Google apparently does, you might just wait for an intelligent sync during the off hours...
Whether it works this way or not, I haven't seen or heard anything either way.
That one I don't know about, plausibly. I guess at that point of you have to think about your data retention laws and your requirements for keeping that data secure and whether putting it into the cloud is actually the right thing. Perhaps put it into the cloud encrypted?
While looking for another link earlier, the city of Los Angeles ran into similar problems with their police dept and they didn't move. http://www.msnbc.msn.com/id/31967328/
Whether Google is working with them on this one, or whether LAPD is keeping it internal is the question I guess.
Umm, ask Google?
But honestly, would it make sense for it to be encrypted? I would think it would be too computationally expensive and increase the response time of the request to encrypt this data.
Presumably, plain text and binary data (Video / docs etc) would be stored as just that.
OK, but he isn't a US citizen (Well doesn't indicate that he is). What are they going to do? Extradite him and charge him? If this was the case, why the hell haven't they done this to the rest of the world?
IANAL but if you then accessed / distibuted that in the US you could be in trouble. Given that your data wouldn't be re-assembled (And certainly not in your possession) till you accessed it in The Netherlands you should be fine. Aside from plausible deniability and all that.
Honestly, I would be more worried about the UK:
http://www.theregister.co.uk/2009/01/24/extreme_pron_law_live/
If you could get into one of their data centres, find a master (Hope it's got the index you are after) break into it, then break into the boxes (Online or offline) then re-assemble the shards, sure.
Wouldn't it just be easier to threaten you or someone you care about and just say that something bad is going to happen unless you hand over the password?
Honestly, if they want access to the data that badly, installing a keylogger / screendumper when your not looking would be a shitload simpler.
That would be correct, if you look at their BGP advertisements it would figure that Google would have to transit it's own data.
So if your request for data (YouTube video etc) isn't located in the DC that you connected to, they would have to transit that data across their own links. It would then make sense that they would replicate their own data over those same links during the night on that side of the world when the link is quiet.
Sorta, Google call them shards.
http://news.zdnet.com/2100-9588_22-141569.html
Shards can be located by different masters and different masters are located in different locations according to the data type.
So I think Japan (That's where they just dropped their Asia - US cable after all, so it makes sense) has a "complete" replication of all Google data. Some data is also replicated to containers (YouTube etc) for hosting at major ISPs. So all email data would be replicated in non-realtime. If you request something that isn't in that DC it's located in the US or wherever is closest (I guess).
There are multiple "complete" copies on the east and west coast as well as European hub sites or directly connected to European hub sites.
If you ask for a citation, I can dig something up for you....
Great, but who is going to look at them, you would have to get all the ISPs to make the change (They won't). Aside from this, what about all the corporates and everyone else, it's just not feasible.
The only way to do it would be to advertise the space of all the other root DNS servers into the routing table (That'll piss off a LOT of people, probably lead to a disconnection actually; potentially of all of Aus until each party stopped readvertising the routes).
The way that this works in China is that the government IS the largest ISP over there, China Telecom is owned by the government, (This is the reason why it works in the first place) and anybody else would be under heavy heavy presure to conform.
That infrastructure costs money, infrastructure that the ISPs don't have in place at the moment, and the government isn't going to pay for it.
They could redirect the IPs perhaps (Again costs money, were are they going to redirect it to? See point 1), but it won't stop people going to a Google site outside of Australia. Google host in multiple places in Australia with some of the larger ISPs having Google containers.
You have to remember that the guys who built Google maps and wave etc all work there. They hold a lot of IP there so that they can push money around as well.
They are not going down without a fight (I question whether the government would even have the money to keep an ongoing legal battle going against Google), they employ too many people there and have too much of a presence for this to happen. Plain and simple.
No infrastructure
Nobody is going to enforce it
No company wants all the phone calls saying "I can't access Google" broadband margins are that bad on a per customer basis, the moment they phone rings from a customer they are losing money.
Not going to happen
There was an article a while ago about the unmanned predators etc, basically, it is still a human flying it, just not in the drone itself. There is a ground crew that gets it off the ground then they pass control to someone else. The person flying it flies it by cameras and an instrument panel of information fed from the aircraft itself.
If you are at 12 - 20,000 feet, at that height, dropping bombs, taking photos or whatever else having meat soldiers in the cockpit of an aircraft is not going to mean anything. Quite simply you can't see a "wedding" or a "random car", shit, you can't see a lot of roads, 6 lane highways are just small lines. Your ability to pin point ANYTHING from that height is, well, crap. Get verification? Verification of what? The bomb going off?
From the height that these drones operate at, bombs are often guided in by GPS or laser from a ground crew.
One of the biggest reasons for these things is from an intel perspective over a war zone, you can have someone effectively followed, get photos of someone day or night if the coverage is good enough and determine whether that person is a threat or not. If he / she is, then you can find out who their associates are, where they are going and what they are up to. This is the reason why they have so many of them flying.
There was another comment about this, yes, it's not just Europe thinking this.
When people (Americans) come overseas and apologise for presidents and the stupid things their country is doing, that's gotta be embarrassing.
People who I know / have met from Australia and New Zealand... (Let's not talk about Canada) ... and a number of people I know scattered across Asia share this belief as well.
MS probably could not be as big as they are with support contracts.
You REALLY think this?
Sure, nobody in the OEM market has support contracts with MS (That's the whole point of the OEM license, is that Microsoft DOESN'T support it), if you look at retail, you do get limited support, and if you phone Microsoft with a bug and it's determined that it is a bug, then you don't get charged for support.
ANY organisation that has a reasonable amount of installed computers ( >5000 in my experience, and certainly a lot smaller >1000 ) has a Premier support agreement with Microsoft.
I don't have exact numbers, but I know that MS make a killing out of support contracts for their software. They also make a decent amount out of their services groups.
Berny
Yeah like selling one time password solutions to IT bosses when someone gets ahold of their SAM.....
IMO MS should have separated the system state/settings from the user & app settings in some meaningful way to make the OS more failure-resistant.
True, but the app Foo would be far more likely to corrupt %APPDATA%\Publisher Foo\App Foo\settings.xyz than %APPDATA%\Publisher Bar\App Bar\settings.abc as opposed to poorly handling error conditions while using the registry API and corrupting the registry or leaving some registry handle open, causing a leak on shutdown and other badness.
In theory, an application should only be accessing HKLM\Software or HKCU\Software (Or Services I guess...) (Citrix client modifying the Gina chain...) (Device software modifying device driver parameters...) (Security Configuration Wizard... that touches basically everything) Yeah, OK, user applications modify a great deal of system parameters.
I don't know if the situation you allude to would be THAT much better to be fair... At the moment, thinking about the limitations that Citrix has with the print subsystem and terminal services... To a degree, while you do get some shit applications that don't behave themselves, I think the limitations of ONLY being able to program to an API could be limiting for a lot of customers.
Winlogon debugging, create a logfile and take a look. http://support.microsoft.com/kb/221833 Doesn't work with Win 7.
User Profile Service
Thats the worst rationale ever. The opposite of the registry isn't overwriting INI files or whatever else you're thinking of. It is to call SHGetFolderPath with CSIDL_APPDATA and follow Microsoft's own guide lines.
Historically, everyone was writing settings to ini files located in \windows\system and this is where people were falling over each other. All Microsoft (And yourself by extension with your arguement) are specifying there is the *location* of application specific files, not what they contain, how they work and how they should be changed and modified.
From my perspective as an admin who has to deal with thousands of users, I think the idea of trying to have to understand the (potentially) 1000s of different methods that the different software vendors would use to store their configuration data to change one flag to change a checkbox for a group of users would be a complete waste of my time. Now potentially, a developer might come up with an easy method for me to interpret all the different formats that other developers use to store their data for a single true / false flag, but I can say that I am glad I am not in that position.
The primary problem I have with the registry is that it can't be locked during modification, and you couldn't set any access restrictions on any of the keys preventing other apps from messing (read corrupting) with it (this is on W95)
Locking I'll give you, but really, two applications shouldn't be writing to the same location at the same time. You can't do the access restrictions (By default) on any platform that I am aware of. Applications run as a user, an application that's poorly (or otherwise) written that runs as that user has the ability to modify (corrupt) any other program that the user is permissioned to access, registry or not it doesn't make any difference.
You've never actually read the Windows UI and programming design guides have you?
It was originally designed to host OS settings in a convenient central location
Actually from Wikipedia "Windows registry's primary purpose was to store configuration information for COM-based components". I've only been doing admin since Win NT 3.51, so I only know about it since then, I didn't realise it was in Win 3.1 as well.
The whole purpose of HKCU and HKLM in particular \Software was PURPOSEFULLY to have a central location for DIFFERENT software vendors to store settings in a single location as opposed to different vendors trying to use the same ini file names and treading on each other's toes, as well as problems with size limitations etc etc.
Thus its became this giant cluster-fuck that it is now, primarily because of backwards compatibility and previous strategic mistakes on the part of MS.
Citation required. I don't understand how you consider it a cluster fuck. Just because you don't understand it doesn't make it a cluster fuck.
They should have kept the registry API hidden and not allowed apps to write their shit all over the place
With the hundred of thousands of ISVs out there, you suggest to me a better way of managing on a per user and per machine basis of managing configuration settings.
Your rant at this point seems to change tangent based upon kernels and what was and wasn't possible. I'll let you answer.
What's your favourite datacentre where this is their attitude?
Huh? Anti-virus / content filtering proxy servers, these ones don't handle zip very well, change around the way the web server works and they add in zip compression to the web server. They could have added zip to the returned data at any point, but doing at at this point, seems very plausible.
This explains a lot, a couple - few months ago, I started getting complaints about "potentially virus infected" / "unscanable" zip files when being served content from facebook.com and fbcdn.net etc.
They probably changed at this point how they were sending data out of the web server with zip compression and it all started falling over at this point....
I was wondering what the change was....