See above. Some devices need a phone signal to get their maps, but that's an implementation choice. Google's implementation caches the image data for the route.
If it does it sucks. When I was in San Jose, my cell signal cut out in a few places, and every time it did, Google would quit updating the map, causing me to miss my turn, because Google didn't know where I was. It was quite annoying.
Uh, don't most automobile radios have the antenna located OUTSIDE the passenger compartment?
Only if you have an antenna mast outside your car. All my cars have integrated antennas. They are on the INSIDE surface of the rear glass, mounted just like the rear-window defogger.
The car's radio has an antenna OUTSIDE THE CAR so its reception won't be affected
Actually many cars these days have integrated antennas. They are on the INSIDE surface of the rear glass, mounted in the same fashion as your defoggers.
By the way, I'm not trying to be an ass, I'm just going by experience. Using XML as an example, I've gone to a plugfest event that relied on XML where there were > 20 companies in attendance, and almost every company there had errors in their xml handling on their first showing at the event.
Not necessarily because the parser was flawed, but also because certain assumptions/shortcuts made about the document being parsed were invalid, resulting in interop failures.
For example, one company was looking at the element name, and wasn't resolving the namespace. They took the literal element name as being (for example) x:foo, and rejected foo and y:foo, even if the default namespace of foo or the defined namespace of y were equivalent to the defined namespace given by x.
You'd be surprised at the types of shortcuts I've seen companies employ. One company didn't even bother parsing the element names at all, they just did a string search for the element name they were looking for. Other companies assumed the order of the elements were static. etc etc...
My point is that a developer who thinks making an XML parsing library is difficult isn't a very good developer. Bugs always happen, and it is usually a good idea to use libraries that have already gone though the bug fixing process. I do agree about that much. But XML is about as simple a nesting grammar as you can get, and still (almost) generate a Turing complete language (XML lacks a lambda.). Indeed, XML's grammar is formally equivalent to S-expressions, via a trivial syntactic transformation.
I'm not saying it's difficult, I'm saying that if you think that you can get it done correctly in 2 days, you're being arrogant. Haxml started development in 1998. That was 11 years ago. If you look at the change logs, there were still bugs found much later than 1998, that were indeed parsing bugs. I'm not saying that's a bad thing, I'm just saying if it is really so "easy" to develop a compliant XML parser, how come the haxml guys couldn't do it in 2 days? Are you just more brilliant than all the guys that worked/contributed to haxml?
Besides, the original point of this/. article was talking about windows. I really doubt someone that is writing win32 apps will necessarily be using or be familiar with Haskell.
and yes, I have seen Microsoft at many plugfest events, so at least they are doing their learning as well... I know of a few companies that release SDKs for such specs, that have never ever sent a rep to a plugfest. Than one day another company brings a device to a plugfest based on said SDK, and on day one, the engineer is trying to figure out why their device doesn't play well with others... Maybe that's why I'm jaded with blind recommendations for 3rd party libraries instead of MSFT libraries, when I've seen instances where such third party libraries are crap. Not saying MSFT libraries are necessarily better, just saying that at least they participate in many plugfest events with their toolkits. (Whether or not they apply patches based on their learnings is another thread)
I'm not trying to argue that writing an XML parser is going to take thousands of lines of code. I know, that XML parser I mentioned that I wrote, which was tested/certified for compliance, was only 8k in size, and that was written in C.
I'm just saying such tasks are almost never as trivial as originally thought. Sure, it might take a day or two, or even a few hours to write the initial version, but that initial version is almost always going to have bugs in it, that aren't going to be readily visible.
You may find that during testing a few months later, that some edge case scenario botches the parser. For example, even with haxml, an infinite loop condition was found in a later revision... Mishandling of certain characters were found later on as well.
You can try to test your module as well as you think you can, but some bugs just don't rear their ugly heads until you have a developer from another company using your module passing in data that they think is compliant to their interpretation of the spec, which is contradictory or unintended from you the creator of said module. I see it all the time.
This is why, generally it is usually better to reuse existing libraries than trying to roll your own... For example, I wrote an SDK for a particular spec, and it has since spent 5 years or so proliferating in the community participating in every international plugfest event. Fast forward a few years, and a newby coworker decides to roll their own implementation for a project they are working on rather than to even bothering to look at what is available, and the first plugfest they go to, their device suffers from basic conceptual bugs that I've known about for years. It's like they were starting over again in the learning process when they didn't need to.
In fact I see it with almost every company that sends a device to a plugfest for the first time. They all suffer from the same kinds of bugs, and go through the same evolutionary path. And the tools/SDKs our company releases, we give away for free and/or give to the opensource community, so I know all about OSS alternatives and such to things as well. So if you're going to recommend using an OSS alternative for something that's one thing, but to encourage a developer to home-brew their own implementation is quite another.
Your point? Look at the change log for haxml. There were many bug fixes along the way in each version. The whole project took more than "an hour" to get right. THAT was my point. You can't just sit down and roll your own XML parser thinking it's any easy one hour task to get right...
Maybe they also do what nintendo does, and include the firmware update on the Game disc itself... My Wii doesn't have a network connection, yet it installs updates all the time when I get new games.
No, sony broke it, because yeah, you have to agree to the license, but to not agree to the license, means you can't use your PS3 anymore, as the games won't run without the update being installed. You can call it user choice, but in reality it's a forced decision.
That's the main point of well-formedness. XML is *designed* to be easy to parse.
XML was designed to by easily HUMAN READABLE, not to be easily parsed. For example, a binary encoded data blob is much easier to parse than XML. Parsing XML tokens is easy. Assembling the tokens into their proper form not so much, especially when you have to deal with custom schema defined data types and such.
Working on international standards, I find many companies have a hard time just agreeing on the correct interpretation of a particular spec. All it takes is two implementations to interpret the spec differently, and you can have two incompatible solutions.
Like I said, I know it can be done (I actually wrote an XML parser to be used on an embedded platform, that was tested/certified for compliance), I'm just saying it is NOT a trivial task, and cannot be done correctly in only an hour's time....
I could write an XML parser in Perl in an hour, if I didn't mind re-inventing something that already exists on the CPAN.
You can implement the entire W3C XML spec, in an hour, and have no mistakes in your logic? BULLSHIT. I've seen companies fack up an XML implementation that they spent weeks/months working on. I have real world experience, as I worked with these said companies doing interop testing of their products. Almost all of them screwed up basic XML parsing.
What if the CDATA element did not cause you to prematurely close a tag, but instead had valid XML elements, that duplicated the names of siblings (since you said your sanitization only checks for nested duplication), this can cause you to incorrectly overwrite data into your variables.
Or what if the CDATA element had null characters in it, causing the entire XML document to get truncated in your string handling. Suppose the CDATA element also had just the right amount of XML in it, that it would all still pass your sanitization, as it had all valid characters and elements, and the right number of close tags... Only problem, is if the CDATA element was the first element, the CDATA element can essentially pass fake information into your code, and have incorrect data get persisted/loaded.
Might not sound like much, but if you are working with financial information, for example, this is a HUUUUGE breach.
And before you ask, it also sanitizes identical element names directly inside each other.
But it's perfectly legit to have embedded elements with the same name.
What you describe is fine if you have full control over the inputs, but in the real world, you'll find that other people as well as other groups could be interfacing with your API. In such a case, another person might see that your module takes XML as input, and code their module to output XML that is valid and spec compliant, but your module chokes on.
What if, for example, another group needs to use your modules, but they need to pass UTF-8 encoded strings, because they are using the module in China/Korea/etc? You wrote your module to only accept ASCII. Or what if they need to pass a Base64 or BinHex encoded byte array as part of their persisted state? What if they added prefixes to all your elements, and then defined that namespace to be the default namespace in your normal XML you expected. Your sanitization prohibits the colon character, so you are basically saying you don't allow any namespace definitions. That is very inflexible, because not only are you preventing the use of namespaces, you are preventing the declarations for schemas as well. Things that are actually HELPFUL with document validation.
So like I was saying. For a small operation where you control the inputs/outputs, that is fine, but even then, people may try to reuse your module later, and have to re-write it to get it to work. Then you'll have even more points of failure. In most real world scenarios, you have to think outside the box, and design some level of extensibility into your modules. Or do you think OS writers should also change their design paradigm and force you to re-compile the entire OS whenever you want to enable/disable features?
and the worst that can happen is the file gets rejected as containing invalid syntax
What if your parser accepted a CDATA element because it passed sanitization, but then because you didn't properly embed CDATA support misparsed an embedded close tag as a real close tag, than passed corrupt data as real data. Voila, that's how you encounter an unintended buffer over-run exploit....
The only problem with your approach, is that it's very inflexible. Just figure out what you actually need to complete your task. I can understand that approach, but at the last company I worked at, it was the undoing of the company. All the modules they outsourced to be implemented were all designed to solve the EXACT task at hand, and could not easily be extended or reused for other tasks. When new functionality had to be designed to work with existing functionality, it was pretty much determined that the entire system needed to be re-written, which caused the company to almost go through bankruptcy to try to fix their designs...
On the same token, I can see your point about not wanted to use existing libraries, as at this same company, when I first started, I wrote my own modules that duplicated some functionality of that company's core libraries, but I did it for architectural reasons. I trimmed a month-end process from taking 6 hours to run to under 10 minutes, because I saw the way they were storing and passing data was designed by a moron. I was reprimanded by said company for "re-inventing" the wheel. I now work for a company that advocates re-use when applicable, but re-design when necessary.:) However, I would never generalize and recommend people to always roll their own stuff, because at current company, I have a coworker that did just that, and facked up our whole project, with his idiotic assumptions and program logic.
If rolling your own, design before you begin to code.
Dude, you should be doing that with ALL your code, regardless if you are re-inventing something or not...
The rewrite it bit was actually referring to automatic updates and XML parsing. Those are pretty easy to implement properly in an app, without depending on Microsoft-coded services.
No necessarily. I work with lots of companies on various international standards. I've seen companies try to roll their own XML parser in their products. 9 times out of 10, they fack it up. Not necessarily from a exploitation point of view, but from a compliance point of view. I've seen home-grown xml parsers that failed to or improperly parsed comments, empty elements, white spaces,or most commonly, failed to properly handle namespaces.
Now I'm not saying it can't be done properly, as I've done it before. I'm just saying that in general it's bad advice to tell someone to roll something themselves when there exists a core library that they can take advantage of, because not everybody is as competent as you. Believe me, I've seen all sorts of crazy logic in people's XML parsing logic, when I was helping companies with interop testing of their products.
I have lots of friends that have both Mac and PC... Just because you own both does not mean you are proficient at both. Most people I know, call tech support when they have issues. Most people I know with Macs, take their computer back to the apple store when they have issues... A couple of my friends will call me, asking me to fix their computer, even when they know I don't own any Macs. That being said, One time a friend was having issues with his network on his Macbook, and I was able to figure out how to alter the routing tables on their Mac, so they can surf the net, and connect to their source control server at the same time... That person owns both windows and macs, yet he could not figure that out, and I could, and I don't even own any macs. Was he more computer literate than me? Nope...
Computers are just tools. Tools do not give you computer skills. One with computer skills uses any available tools at their disposal to accomplish a given task. This is the same reason I found it no more difficult to replace the intake manifold gaskets on a Nissan VQ DOHC V6 and a GM Supercharged overhead valve V6.
The experiments I was working on needed shielding from other 2.4 and 5 ghz devices. Besides, I usually "setup" once I get settled inside the cage. I didn't even unload my cart yet, and my phone started ringing...
Yes, the optical isolator is at the edge of the cage. Cell signal could have been exceptionally strong, because our building has a bunch of repeaters in the ceilings, as well as a bouncer in the main courtyard.
Seriously... I work with a WiFi lab at work... I have a Faraday Cage in our lab... One of the techs forgot to install an optical isolator on the network cables, so for a few days the ethernet cables went right into the cage... Well, first day I went to use it, I locked myself in the cage, and was surprised that my cellphone started ringing... Without the optical isolators on the network cable, the RF signals were able to find there way into the faraday cage through the ethernet cables....
with that being said, I highly doubt that simply painting your walls will keep RF signals at bay... Even when the grounding wire was simply loose on the door to the faraday cage, RF signals would leak in...
Your phone and/or battery sucks if you run down the battery in only 30 minutes. My battery drops 20% per hour while I'm actively streaming video over a 3G connection on my Windows Mobile phone. One time when the train broke down, I verified this, by seeing my battery at 35% after streaming videos for 3 hours straight. That was 3 hours of continuous 3G and LCD screen usage.
Slashdot Mirror
See above. Some devices need a phone signal to get their maps, but that's an implementation choice. Google's implementation caches the image data for the route.
If it does it sucks. When I was in San Jose, my cell signal cut out in a few places, and every time it did, Google would quit updating the map, causing me to miss my turn, because Google didn't know where I was. It was quite annoying.
Uh, don't most automobile radios have the antenna located OUTSIDE the passenger compartment?
Only if you have an antenna mast outside your car. All my cars have integrated antennas. They are on the INSIDE surface of the rear glass, mounted just like the rear-window defogger.
The car's radio has an antenna OUTSIDE THE CAR so its reception won't be affected
Actually many cars these days have integrated antennas. They are on the INSIDE surface of the rear glass, mounted in the same fashion as your defoggers.
not that it matters, but I meant 8kb not 8k.
By the way, I'm not trying to be an ass, I'm just going by experience. Using XML as an example, I've gone to a plugfest event that relied on XML where there were > 20 companies in attendance, and almost every company there had errors in their xml handling on their first showing at the event.
Not necessarily because the parser was flawed, but also because certain assumptions/shortcuts made about the document being parsed were invalid, resulting in interop failures.
For example, one company was looking at the element name, and wasn't resolving the namespace. They took the literal element name as being (for example) x:foo, and rejected foo and y:foo, even if the default namespace of foo or the defined namespace of y were equivalent to the defined namespace given by x.
You'd be surprised at the types of shortcuts I've seen companies employ. One company didn't even bother parsing the element names at all, they just did a string search for the element name they were looking for. Other companies assumed the order of the elements were static. etc etc...
My point is that a developer who thinks making an XML parsing library is difficult isn't a very good developer. Bugs always happen, and it is usually a good idea to use libraries that have already gone though the bug fixing process. I do agree about that much. But XML is about as simple a nesting grammar as you can get, and still (almost) generate a Turing complete language (XML lacks a lambda.). Indeed, XML's grammar is formally equivalent to S-expressions, via a trivial syntactic transformation.
I'm not saying it's difficult, I'm saying that if you think that you can get it done correctly in 2 days, you're being arrogant. Haxml started development in 1998. That was 11 years ago. If you look at the change logs, there were still bugs found much later than 1998, that were indeed parsing bugs. I'm not saying that's a bad thing, I'm just saying if it is really so "easy" to develop a compliant XML parser, how come the haxml guys couldn't do it in 2 days? Are you just more brilliant than all the guys that worked/contributed to haxml?
/. article was talking about windows. I really doubt someone that is writing win32 apps will necessarily be using or be familiar with Haskell.
Besides, the original point of this
and yes, I have seen Microsoft at many plugfest events, so at least they are doing their learning as well... I know of a few companies that release SDKs for such specs, that have never ever sent a rep to a plugfest. Than one day another company brings a device to a plugfest based on said SDK, and on day one, the engineer is trying to figure out why their device doesn't play well with others... Maybe that's why I'm jaded with blind recommendations for 3rd party libraries instead of MSFT libraries, when I've seen instances where such third party libraries are crap. Not saying MSFT libraries are necessarily better, just saying that at least they participate in many plugfest events with their toolkits. (Whether or not they apply patches based on their learnings is another thread)
I'm not trying to argue that writing an XML parser is going to take thousands of lines of code. I know, that XML parser I mentioned that I wrote, which was tested/certified for compliance, was only 8k in size, and that was written in C.
I'm just saying such tasks are almost never as trivial as originally thought. Sure, it might take a day or two, or even a few hours to write the initial version, but that initial version is almost always going to have bugs in it, that aren't going to be readily visible.
You may find that during testing a few months later, that some edge case scenario botches the parser. For example, even with haxml, an infinite loop condition was found in a later revision... Mishandling of certain characters were found later on as well.
You can try to test your module as well as you think you can, but some bugs just don't rear their ugly heads until you have a developer from another company using your module passing in data that they think is compliant to their interpretation of the spec, which is contradictory or unintended from you the creator of said module. I see it all the time.
This is why, generally it is usually better to reuse existing libraries than trying to roll your own... For example, I wrote an SDK for a particular spec, and it has since spent 5 years or so proliferating in the community participating in every international plugfest event. Fast forward a few years, and a newby coworker decides to roll their own implementation for a project they are working on rather than to even bothering to look at what is available, and the first plugfest they go to, their device suffers from basic conceptual bugs that I've known about for years. It's like they were starting over again in the learning process when they didn't need to.
In fact I see it with almost every company that sends a device to a plugfest for the first time. They all suffer from the same kinds of bugs, and go through the same evolutionary path. And the tools/SDKs our company releases, we give away for free and/or give to the opensource community, so I know all about OSS alternatives and such to things as well. So if you're going to recommend using an OSS alternative for something that's one thing, but to encourage a developer to home-brew their own implementation is quite another.
Your point? Look at the change log for haxml. There were many bug fixes along the way in each version. The whole project took more than "an hour" to get right. THAT was my point. You can't just sit down and roll your own XML parser thinking it's any easy one hour task to get right...
Do you play multiplayer online games? Cuz my PS3 will NOT allow me to play online unless I install updates
Maybe they also do what nintendo does, and include the firmware update on the Game disc itself... My Wii doesn't have a network connection, yet it installs updates all the time when I get new games.
according to the original linked article, it says the update is mandatory.
No, sony broke it, because yeah, you have to agree to the license, but to not agree to the license, means you can't use your PS3 anymore, as the games won't run without the update being installed. You can call it user choice, but in reality it's a forced decision.
That's the main point of well-formedness. XML is *designed* to be easy to parse.
XML was designed to by easily HUMAN READABLE, not to be easily parsed. For example, a binary encoded data blob is much easier to parse than XML. Parsing XML tokens is easy. Assembling the tokens into their proper form not so much, especially when you have to deal with custom schema defined data types and such.
Working on international standards, I find many companies have a hard time just agreeing on the correct interpretation of a particular spec. All it takes is two implementations to interpret the spec differently, and you can have two incompatible solutions.
Like I said, I know it can be done (I actually wrote an XML parser to be used on an embedded platform, that was tested/certified for compliance), I'm just saying it is NOT a trivial task, and cannot be done correctly in only an hour's time....
I could write an XML parser in Perl in an hour, if I didn't mind re-inventing something that already exists on the CPAN.
You can implement the entire W3C XML spec, in an hour, and have no mistakes in your logic? BULLSHIT. I've seen companies fack up an XML implementation that they spent weeks/months working on. I have real world experience, as I worked with these said companies doing interop testing of their products. Almost all of them screwed up basic XML parsing.
What if the CDATA element did not cause you to prematurely close a tag, but instead had valid XML elements, that duplicated the names of siblings (since you said your sanitization only checks for nested duplication), this can cause you to incorrectly overwrite data into your variables.
Or what if the CDATA element had null characters in it, causing the entire XML document to get truncated in your string handling. Suppose the CDATA element also had just the right amount of XML in it, that it would all still pass your sanitization, as it had all valid characters and elements, and the right number of close tags... Only problem, is if the CDATA element was the first element, the CDATA element can essentially pass fake information into your code, and have incorrect data get persisted/loaded.
Might not sound like much, but if you are working with financial information, for example, this is a HUUUUGE breach.
And before you ask, it also sanitizes identical element names directly inside each other.
But it's perfectly legit to have embedded elements with the same name.
What you describe is fine if you have full control over the inputs, but in the real world, you'll find that other people as well as other groups could be interfacing with your API. In such a case, another person might see that your module takes XML as input, and code their module to output XML that is valid and spec compliant, but your module chokes on.
What if, for example, another group needs to use your modules, but they need to pass UTF-8 encoded strings, because they are using the module in China/Korea/etc? You wrote your module to only accept ASCII. Or what if they need to pass a Base64 or BinHex encoded byte array as part of their persisted state? What if they added prefixes to all your elements, and then defined that namespace to be the default namespace in your normal XML you expected. Your sanitization prohibits the colon character, so you are basically saying you don't allow any namespace definitions. That is very inflexible, because not only are you preventing the use of namespaces, you are preventing the declarations for schemas as well. Things that are actually HELPFUL with document validation.
So like I was saying. For a small operation where you control the inputs/outputs, that is fine, but even then, people may try to reuse your module later, and have to re-write it to get it to work. Then you'll have even more points of failure. In most real world scenarios, you have to think outside the box, and design some level of extensibility into your modules. Or do you think OS writers should also change their design paradigm and force you to re-compile the entire OS whenever you want to enable/disable features?
and the worst that can happen is the file gets rejected as containing invalid syntax
What if your parser accepted a CDATA element because it passed sanitization, but then because you didn't properly embed CDATA support misparsed an embedded close tag as a real close tag, than passed corrupt data as real data. Voila, that's how you encounter an unintended buffer over-run exploit....
On the same token, I can see your point about not wanted to use existing libraries, as at this same company, when I first started, I wrote my own modules that duplicated some functionality of that company's core libraries, but I did it for architectural reasons. I trimmed a month-end process from taking 6 hours to run to under 10 minutes, because I saw the way they were storing and passing data was designed by a moron. I was reprimanded by said company for "re-inventing" the wheel. I now work for a company that advocates re-use when applicable, but re-design when necessary.
If rolling your own, design before you begin to code.
Dude, you should be doing that with ALL your code, regardless if you are re-inventing something or not...
The rewrite it bit was actually referring to automatic updates and XML parsing. Those are pretty easy to implement properly in an app, without depending on Microsoft-coded services.
No necessarily. I work with lots of companies on various international standards. I've seen companies try to roll their own XML parser in their products. 9 times out of 10, they fack it up. Not necessarily from a exploitation point of view, but from a compliance point of view. I've seen home-grown xml parsers that failed to or improperly parsed comments, empty elements, white spaces,or most commonly, failed to properly handle namespaces.
Now I'm not saying it can't be done properly, as I've done it before. I'm just saying that in general it's bad advice to tell someone to roll something themselves when there exists a core library that they can take advantage of, because not everybody is as competent as you. Believe me, I've seen all sorts of crazy logic in people's XML parsing logic, when I was helping companies with interop testing of their products.
I have lots of friends that have both Mac and PC... Just because you own both does not mean you are proficient at both. Most people I know, call tech support when they have issues. Most people I know with Macs, take their computer back to the apple store when they have issues... A couple of my friends will call me, asking me to fix their computer, even when they know I don't own any Macs. That being said, One time a friend was having issues with his network on his Macbook, and I was able to figure out how to alter the routing tables on their Mac, so they can surf the net, and connect to their source control server at the same time... That person owns both windows and macs, yet he could not figure that out, and I could, and I don't even own any macs. Was he more computer literate than me? Nope...
Computers are just tools. Tools do not give you computer skills. One with computer skills uses any available tools at their disposal to accomplish a given task. This is the same reason I found it no more difficult to replace the intake manifold gaskets on a Nissan VQ DOHC V6 and a GM Supercharged overhead valve V6.
The experiments I was working on needed shielding from other 2.4 and 5 ghz devices. Besides, I usually "setup" once I get settled inside the cage. I didn't even unload my cart yet, and my phone started ringing...
Yes, the optical isolator is at the edge of the cage. Cell signal could have been exceptionally strong, because our building has a bunch of repeaters in the ceilings, as well as a bouncer in the main courtyard.
Seriously... I work with a WiFi lab at work... I have a Faraday Cage in our lab... One of the techs forgot to install an optical isolator on the network cables, so for a few days the ethernet cables went right into the cage... Well, first day I went to use it, I locked myself in the cage, and was surprised that my cellphone started ringing... Without the optical isolators on the network cable, the RF signals were able to find there way into the faraday cage through the ethernet cables....
with that being said, I highly doubt that simply painting your walls will keep RF signals at bay... Even when the grounding wire was simply loose on the door to the faraday cage, RF signals would leak in...
Your phone and/or battery sucks if you run down the battery in only 30 minutes. My battery drops 20% per hour while I'm actively streaming video over a 3G connection on my Windows Mobile phone. One time when the train broke down, I verified this, by seeing my battery at 35% after streaming videos for 3 hours straight. That was 3 hours of continuous 3G and LCD screen usage.