Slashdot Mirror
Security Software Conflicts with AJAX?
ithyus needs help with the following: "My employer is running an e-commerce site that, until recently, our customers were quite happy to use. With increased traffic to the site we decided to implement AJAX to try to reduce the load on our database servers. In doing so, our customers have experienced all kinds of problems with security/privacy software such as Norton and McAfee. It seems that no matter what we do we can't make these programs happy. Bigger companies such as Google have documented work arounds for some of them, but we wouldn't be able to keep our docs current with all the software that's presently out there. I'd really like to know how Slashdot's readers have handled these issues. Since security programs don't appear to be compatible with the emerging features of the Internet, do you simply suggest that the customer disable the offending software or do you opt to offer some support for the more popular ones? Are those really the only two options? How do you justify your method?"
In general, you want to work around users' environments, not the other way around. That means if something you're using isn't compatible with a large number of your visitors' systems, you either work around it or use something else. Then your justification becomes "because it works". If you tell everyone to uninstall their antivirus software to use your site, I suspect you'll lose a substantial number of visitors.
If you're resorting to using AJAX only ameliorate your DB load, you may wish to try more conservative methods that will work on all client machines, such as optimizing your queries, first.
ummm.... You answered your own question, didn't you? The like to Google's "work arounds" seems to be the answer. These aren't work arounds, they're specific steps for authorizing your site with the AV software. Just make your own document similar to google's and direct customers to it.
Explain to me how creating more HTTP requests by using AJAX is going to decrease your database load.
Oh, while we're at it, explain how sheep's bladders may be employed to prevent earthquakes.
I can see how ajax would lighten the load to some extent but wouldn't using memcache be more appropriate?
First, Ajax is not going to reduce your database load, if you do not use it like a cache for already performed query results in remote client pc. And even in that case, if you use query caching (in mysql for example), as most common queries will have been already cached, they wouldnt incur as much load as they would as they will be pulled out of the cache without an extra new query. Even if you go for "using as a cache to store the content in remote client pc" route, than there is the matter of uploading loads of content that many clients will not be even attempting to view, and in congested network routes your site might seem as being slow to come up.
As for security matter, there is indeed no way out i believe :
You, no matter what you use, are utilizing a remote processing power in a remote pc to process the content. That is actual compilaton of some sort of code to do something, and that is not just getting a plain text format html code and visually putting it in place.
As we have experienced that even images can be used to embed viruses, it is no joke or no surprise that there will come up viruses/exploits that are going to use such and such procedures of Ajax in ways unimaginable now to do mischief. Java is already something that the anti-virus guys and people are wary about, it is widely exploited to plant a phletora of stuff in visitor pcs. Just type "porn" to google, and visit the 10 sites that come up, or go deeper from a link, and voila, an example for the above situation.
Anti virus is never going to be easy and happy with ajax, that is for sure.
Anything that takes the work to client pcs can be exploited for mischief.
Read radical news here
McAffee silently works as a proxy and adds javascript to downloaded pages -- an error handler to prevent javascript errors, maybe some other things as well.
Do you even lift?
These aren't the 'roids you're looking for.
...don't fix it!
I believe before AJAX should be fully deployed, HTTP will need to be rewritten to allow much more streamlined updating. I would also have a look at Norton and other virus detectors. I use AVG, but asking your clients to use another software isn't feasible. Is it possible to go back to the old way? If it isn't broken, I wouldn't fix it. I have read some topics from message board software deploying AJAX techniques and some large forums turn off many of the AJAX features because it just isn't resource savvy.
In the past I've seen problems that arise when Norton, McAfee, etc., block the referer* header in the name of security. Although it provides a modest gain in privacy, many sites use the referer header to make sure you're not using their form submission interfaces, images, and other data for third-party purposes. I had to work around it by modifying my referer-checks to include a check for a NULL referer.
(Yes, referers can be spoofed and needn't be specified in HTTP, but every browser sends them by default.)
* "Referrer" in English, but "referer" in HTTP.
For more information, click here.
Over the years I have had 3 on-line merchants ask me to disable or uninstall my network security software to access their site. I immediately stopped shopping at each of them. They were not selling anything worth the risk of being connected to the Internet with no protection, and I doubt that you are selling anything worth that risk either...
There is a quick and simple answer to this one. Detect if the AJAX is not working (or let the user specifiy they do not wish to use AJAX), and send a document that can do the same end results as the AJAX version, but (and wait for it), without AJAX.
If you are designing programs that can be potentially used by many thousands of users, you can not afford to write programs that only cater for those who wish to play by *your* game. A good few of them will refuse and use another software.
NeoThermic
Use my link above, or to view my server, NeoThermic.com
And as a web developer that tries to make good use of ajax style techniques, this is very troubling.
I'm always seeing articles about AJAX security issues, and they always puzzle me. AJAX is just another way of sending http requests to the server from the browser. If you're able to write secure server side scripts already, then you should have no trouble writing ajax responders. How do these security aps decide that these particular http requests from the browser are "bad"?
The kinds of things security software disables should be non-essential anyway. For instance, ActiveX disabled in Internet Explorer will stop you from using XMLHttpRequest, but that throws an error that you can catch, and your fallback behaviour for non-JavaScript users can be used.
Whenever I see somebody complaining about software interference with web applications, it's virtually always because they've cut corners and neglected to code appropriate fallback behaviour when browsers don't support a particular feature. Unfortunately, it's impossible to give you specific advice because you've unhelpfully neglected to mention anything specific at all about the problems you are having.
As somebody else mentioned, if your goal is to reduce load on your databases, then this can be achieved through other means. For instance, caching (both page fragments, and HTTP caching) can significantly reduce load if most of your transactions are reads that apply to multiple users.
Bogtha Bogtha Bogtha
Is the issue your customers are having in the actual ajax, or is it just in some of the new javascript implemented. Some javascript (ie: moving a window off of the screen) requires "a signed script," which caused me a lot of trouble untill I figured that out. Also, if your ajax is trying to call a page on another domain (www.google.com and google.com are different, in the eyes of ajax, at least on my system), then that will likely create security issues.
Scott Swezey
More importantly, Referer can be used to make sure users aren't coming from *other* sites, to prevent cross-site scripting. For example:
e te_all_my_stuff
http://www.your-site.com/do_action.php?action=del
Check the referer. It's not your-site.com? Don't delete all my stuff.
Last post!
With security like that, your stuff is so amazingly gone.
Information wants to be free.
Entertainment wants to be paid.
You just want to be cheap.
No, that's a poor use of Referer. What happens when you have a blank Referer header? You can either ignore the request, which breaks things for people who have disabled that header, or you can go ahead with the request, which causes security problems for people who have disabled that header.
Established best practice for avoiding attacks like this is to generate a random value tied to the user to be used as a hidden field in each legitimate form and check for valid values for each request.
Bogtha Bogtha Bogtha
Good points - I've worked mostly with internal systems though, and never had any complaints from people who have referers turned off. It's not a common practice. Still - I'll use the method you describe in the future.
It may not have been the "best practice", but it did keep my users safe, which is all I care about.
Last post!
If you're resorting to using AJAX only ameliorate your DB load, you may wish to try more conservative methods that will work on all client machines, such as optimizing your queries, first.
Unless you can predict the future, you can't "optimize queries" to only get the data the user is going to want. Random crap advice of "optimize your queries" is worthless and unhelpful.
Getting data on demand is how the web is going to work. I'd suggest you stick to the standards and the users will complain to the anti-virus makers. It's their job, too, to work in the user's environment. That includes sites that load content on demand.
You may have been lucky. Referer is set by the client, thus is trivial to spoof. Never rely on referer for any kind of security measure or request verification.
This was just for XSS protection - to make sure links from other sites (mysite.com?delete_all_my_stuff) couldn't harm the client. If the client was sending anything other than the expected referer, they were redirected to a known safe page (home page). Sure they could spoof it, but why would they want to?
Last post!
I suggest they replace it with something else. Almost anything else, really.
Norton is the only thing I've ever seen decide that outbound DNS queries were *all* suspicious and should be silently blocked.
fencepost
just a little off
> Unless you can predict the future, you can't "optimize queries"
Do you even know what he meant by "optimize queries"?
The actual data retrieved by a client is pretty much irrelevant. It mostly doesn't matter which data the user is going to want. It only matters how they get the data. No (sane) web site allows visitors to run custom SQL queries, so therefore there are a limited number of queries that are performed. Each of those can be examined to determine whether or not they are being done in the most efficient way, or in a way that works best with their current load profile.
Expecting your customers to complain to their Anti-Virus companies, and then wait for those companies to give the a fix(Ha!) sounds like an excellent way to not have any more customers.
Here's a simple flow -
1. Introduce new feature.
2. Customers complain feature doesn't work.
3. Try to fix feature.
4. Did fix work? If yes, then go to 6.
5. Fix didn't work, roll back new feature and search for an alternative way to solve the original problem.
6. Profit!
Your flow would be -
1. Introduce new feature
2. Customers complain feature doesn't work.
3. Tell customers they're wrong.
4. Tell customers they're still wrong.
5. LALALALALALALAI CAN'T HEAR YOULALALALALALALA
6. wonder where all the profit went...
Advanced users are users too!
Portable AJAX can only use the same host name, ip, and port (!), as the page the script was served from.
-I like my women like I like my tea: green-
...why and how would "implementing AJAX" cause problems with your "security/privacy software such as Norton and McAfee". Can you please explain? As far as I know AJAX is is just a friendly word for technique where a web browser sends regular HTTP requests dynamically based on action the user performs using web based user interface so that certain aspects of the user interface change accordingly to the performed action without fully reloading the page. In what way would these kind of HTTP requests/responses be distinct from the requests the browser performs when initially loading the page and resources that are needed to render the page from the point of view of "security/privacy software"?
And why do you think that "implementing AJAX" would magically reduce the load on database servers? The load on database server is purely dependant on the data you are storing and the queries you're doing in order to access it. Sure, you could be able to avoid certain queries hitting the database by firing the queries only after the user does something on the web interface. But on the other hand, this is not magic bullet for sure. If you don't know what you're trying, you might find yourself in a situation where your web and database servers are hammered with say 10x increase in requests/queries.
Way to go! You've just guaranteed that the owners of your-site.com now have thousands of 404s in their web logs, with slashdot referers ;)
-Jar.
Together, We Can Make Slashdot Better. I Do NOT Mod ACs. - Check Me Out
Please don't quote me out of context.
What I said was:
Unless you can predict the future, you can't "optimize queries" to only get the data the user is going to want.
It's very often that a website doesn't present all the data to the user that the user could request. Fetching the data on demand can result in database improvements that "optimized queries" can't touch because you don't even have to get the data often times.
And yes, I do know what it means, I do it for a living.
Really, it was presumptuous of you to try to foist your server load problems off onto your clients. Yeh, Ajax has "buzz" and all, but to most people it's just damned stupid.
This indicates a complete lack of understanding of why developers are turning to AJAX. The point is not that they want to foist their server load problems on the client. The point is that they are trying to give their users a more responsive experience. Reducing the burden on the server is one effective way to do this.
When we were all on the far end of 14K4 modems, a few seconds of latency at the server went more or less unnoticed. Now that most of us have broadband, people complain if the information does not at least appear to arrive instantly. AJAX, when used properly (and I concede that it is not always used properly) is about having web sites that only move the change information back and forth, without moving all of every page when little has changed. This reduces the traffic between the client and server in both directions and can improve responsiveness as a result. Google Maps is a classic example of this; by moving the map tiles around locally in JavaScript and only asking the server for the missing tiles as they are needed makes for a much more interactive experience. The fact that the map tiles are all static content is a bonus, but it's not the end goal.
If intelligent life is too complex to evolve on its own, who designed God?
In the case of Slashdot moderation, I think you're forgetting something. Working through a page of comments, reading each one, and assessing whether the existing moderation seems fair and accurate takes a significant amount of time. In that time it is entirely likely that others have also moderated the same page.
/. developers have made it unnecessary by invoking the full reload you describe.
I'm sure that all moderators will have been caught at least once by seeing a particular comment end up with an inappropriate score as a result. Conscientious moderators will probably then decide that when moderating, they should reload frequently.
I think that that is exactly the right thing to do; and that is what I _would_ do...except that I don't have to because the
If the page were served as you are suggesting, all that would happen is that I would immediately reload the entire page manually; and my action taken with your partial AJAX reload would add up to _more_ traffic, not less.
"We reject kings, presidents and voting. We believe in rough consensus and running code." Dave Clark, IETF
1. If we were going to do /. moderation in AJAX, we might change how it worked. Instead of modding an existing score up or down, you'd vote for what you think the true score is. Since this wouldn't be dependent on the existing score, you wouldn't need to refresh anything.
2. The time when you want to reload the page isn't *after* you've moderated, it's before. I.e. refreshing the page after you moderate only helps you for the next moderation, not the current moderation. AJAX could offer you a button that refreshed the moderation for just that post (actually, come to think of it, clicking on the post would be enough; generally you should be looking to moderate posts that show only the headline). A really small request/response. AJAX would also allow a popup that says "Hey, moderation changed here; did you really want this to be a +4? Or are you happy at +3?" A strictly better solution than reloading the entire page, as it takes away the race condition.
3. Even with the existing system, you don't necessarily need to refresh the page. What you want to do is refresh the changes. I.e. get all the moderations and new posts; leave all the existing posts.
It's certainly possible to implement AJAX in ways that are less efficient than the non-AJAX solution. However, properly implemented, AJAX will reduce database load because it does two things: remember previous state and allow one to load changes (to the previous state).
If getting ActiveX controls to work is a problem, you could fall back on loading your results by javascripting into an IFrame. Granted, its much more difficult to implement "POST" handlers that way, but with the right library, most of those differences can be hidden away. The main thing you lose is the automatic XML parsing - but if using Ajax as an RPC mechanism, you should try using JSON instead. The Javascript DOM implementation ain't the greatest.
"But remember, most lynch mobs aren't this nice." (H.Simpson)
-- Joe
parent, I wish I could mod you "Funny/Informative Satire" For those people who can't reverse your satire:
It is relatively difficult to add more database servers. It is relatively easy to add more webservers. Any caching structure you could possibly use in AJAX you could definitely use on your webserver - and many more, because you have power over those servers.
You MIGHT use AJAX to reduce the load on your webserver in some cases, but if it reduces load on your DB server you didn't have caching setup right. Generally you use AJAX for a better user experience.
I want to take this opportunity to mention Flash as an alternative. Certainly it has been abused, but over the last few years Actionscript (used in Flash and Flex) has become a full fledged OOP language. It is ideal for these sorts of applications, precisely because the results you get on one platform or browser are identical to the ones you get on another one, in Windows, OSX or Linux (i386) It has the highest installed userbase of any software in use by web users (higher than IE, because it's also installed on non-IE browsers) Applications without a lot of GUI can be quite small. A significant part of my living is creating database driven Flash/Flex apps.
The AJAX alternative would seem to be to use some kind of AJAX framework that abstracted the differences away for you. Offhand I can't recall a specific one, but I've seen a couple.
Looking for freelance Actionscript (Flash/Flex) or ColdFusion work and/or freelance developers. Email me, put Slashdot
Good. Hate those guys.
Last post!
My first thought was, "You use Microsoft." AJAX is HTML, CSS, Javascript, XML, and XLST. One can throw in a sprinkle of "XMLHttpRequest "; But by appling KISS, one will find XMLHttpRequest is at best, redundent. The reader should at this point start to notice that this is all Client Side handling; Basically, the Server spoon feeds the Client. If one ponders how to talk to the Server, consider the POST, and GET options for a <form ...> tag? If one wonders about those who would listen, consider the use of the HTTPS protocol.
As for the Server side of this reply, consider the following quote, "Regular Expression Your Inputs, Build Your Outputs."
If one needs Security solutions; Is buying from a Marketing company the best merchant?
I've seen alot from Microsoft about security problems with AJAX; But no examples. Without examples to support these concerns, this is FUD.
This comment will only be relevant if I guessed your intent from your problem definition.
When you say you are using Ajax to lighten your DB load, it implies might be using Ajax to request content from other servers than the source server your content comes from (cross-site scripting).
If that is the case, you can certainly expect your clients' antivirus systems to (rightfully) give you a headache.
You simply should not be doing that, and until something like http://json.org/JSONRequestJSONRequest (proposed for inclusion into ECMASCript core)is implemented in Ajax in an acceptable, core way, then you should stay far away from cross-site functionality. Go through a proxying solution to handle off-site content vian agents on your server.
-- ronan at roasp.com roasp.com
Of course, without more details on what you're trying to do, it's hard to be more specific. But you can do a lot without storing cookies, running ActiveX bits, etc.
- "History shows again and again how nature points out the folly of men" -- Blue Oyster Cult, 'Godzilla'
How AJAX reduced DB load.
Enlighten us.
At my work, we've chosen the XForms route using Chiba for a recent product. It's not that surprising that I use it, because I was one of the editors of the 1.0 spec a long time ago, but it progressed even in my absence ;-) and it does fit many of the needs people describe here, in particular security and accessibility.
We write our dynamic markup in XHTML+XForms, following W3C standards (including nascent accessibility standards), and then use Chiba server-side in Tomcat to translate it into HTML4 and JavaScript. Chiba also offers a translation to HTML4 with a "refresh" button to initiate the dynamic activity via plain old HTTP, so it's fully accessible, though not as dynamic.
Longer term, please check out the Mozilla XForms XPI, which will take the XHTML+XForms markup directly. The browser does all its security stuff already, and it follows the IETF HTTP RFC and other W3C specs directly, so there's no need for funky workarounds or security lifting. The Firefox implementation is at rev 0.5 or 0.6 about now, and it's definitely usable, but when it gets to 1.0 it will be really great for dynamic behavior.
There are other implementations as well, FormFaces, which is written entirely in JavaScript and is way cool, FormsPlayer for IE, with lots of advanced features and which is a plugin, not a native implementation (as in the Firefox one); X-Smiles, the open source standalone implementation from Finland, SolidApp, another OSS one with paid support for small devices (mobile phone vendors look here), PicoForms, also for embedded apps but not OSS, Orbeon, a complete back-end server solution incorporating XForms and pipelines. I'm sure there's more major implementations I've missed, and also partial ones as well.
If you want to know more check out the XForms FAQ and XForms or HTML authors Part 1 and Part 2.
Plus, Will Wheaton likes it.
...please check out the Mozilla XForms...
Ajax isn't a emerging feature, it's a garish hack to further extend html into doing things directly contrary to what it's good at. Yet another prime example of hype run amok.
I don't think many of the people who commented here have actually used AJAX. One person's complaint was "Explain to me how creating more HTTP requests by using AJAX is going to decrease your database load." The answer is two part: 1) It decreases overall load & bandwidth used by reducing the number of full pages sent. Not having to re-render a page significantly reduces the number of queries required; 2) It reduces database load in particular by running a greater number of low-load/complexity queries, instead of a lesser number of high-load/complexity queries (joins). I've never personally had my Norton or Avast prevent me from doing any AJAX. In fact, I've never even heard of this, and I've done a lot of AJAX and a lot of cross-browser testing. It's "just another request" as far as the browser is concerned. Are you talking about the flag Internet Explorer raises when you use AJAX to request "off site" content? (i.e., using AJAX to request data from a domain different from the one on which the javascript is hosted). There is a *very* simple workaround for this. The solution is to route remote AJAX requests through a local request forwarder. All it does is receive your AJAX request, and forward the request. You get the result back via your request forwarder. Voila!
geeks are cats who dig a certain kind of cool