Slashdot Mirror
Redundant Credit Card Processing Solution?
RokaMoka asks: "As I type this, I'm on hold with Verisign Payment Services, our (only) merchant services provider. I run several e-commerce sites, and how shall I say... 'tis the season. At the moment, VPS is totally down, and I am losing thousands of dollars per hour. Does anyone have any experience in designing and supporting e-commerce solutions with multiple vendors for CC processing? What other networks are out there, and what has been the customer experience with them? What should the strategy be, load-balance or fail-over?"
I don't think that kind of thing should be your problem. It's like web hosting. Who has redundant web hosts? I don't. My provider's job is to provide a level of service.
Do you have a service level agreement? If not, you might want to look into negotiating one.
Taral
WARN_(accel)("msg null; should hang here to be win compatible\n");
-- WINE source code
If you were losing thousands of dollars per hour, would you really be "asking Slashdot"?
He, like everyone else in that other huge credit card processor dependant industry, would have a backup processor.
- billn
Rather than failing to authorize the CC when VPS is down store all the needed information in a table, queue, etc. This allows the user's transaction to complete and it allows you to authorize CCs in a batch process once VPS is back up. Obviously this is not how you should run all the time, just when VPS is down.
BTW: I would not ship anything until I successfully authorized and charged the CC.
Well... I'd think it's simple. Which is cheaper for you to run, load balance of failover?
The World Wide Web is dying. Soon, we shall have only the Internet.
"Thousands of dollars per hour" and you can't afford a direct merchant account with the various CC companies?
They will guarantee a much higher level of service than going through some 3rd party.
If you need to, hire someone to hook your mechant account to your web sites. Simple as that, you got the money.
The ratio of people to cake is too big
Many sites have redundant hosting via multiple server locations. I own 4 servers, two on the East Coast, two in Vancouver BC. In a way, it serves to load balance, but also, if on or two go down, there are always the others, and it's very unlikely that both Washington DC and Vancouver BC will go totally down at the same time. There are many people who do this.
"Who are in control, they are not in control of anything - they don't even control themselves!" - Glen Beck
This is when people realize that the lowest bidder is not always the best choice.
All my clients use them, and I have heard them described several times in articles as the standard in e-commerce payment processing in the US.
Authorize.net
Our Payment Gateway has also been flaky occasionally. Whether it's a targeted ddos in the summertime (actually happened), or a massive surge in traffic (ddos?) during the holidays, our Payment Gateway will occasionally refuse to process transactions for hours at a time. We sell online services, with a business model similar to the "buy a download key to unlock our shareware" model, so storing CC #'s for later is not an option. Once we give the service, it's given, so we need to process the cards in real-time. I approached "upper management" about getting a 2nd Payment Gateway account for redundancy last year, but in the end they were more willing to risk losing the sales during the downtime than spending the $$ and time to implement a redundant payment system.
You're going to get killed in fees if you don't process a decent level of transactions through a backup processor when it does come into play.
As I type this I have a client who's CC processing has been down nearly 24 hours, and has resorted to a dial backup solution. Not exactly the way to process 5000+ orders a day. And to top it off they sent out a special email offer to 500,000 subscribers this morning, so they're dying as we speak, and if it's not resolved in the morning we may be switching providers in a hurry. Thank the stars that they choose their own provider...
Ignore the posts talking about why you don't need this, and SLA's. No SLA is going to replace lost revenues, and anyone who doesn't have a backup plan in place is just waiting to get burned.
This may be the only option. At a very high level, this would require two things. First that you have a merchant account with the various CC companies. Depending on what kind of business you are in, this could be very easy, or very hard. More difficult would be the software itself. You "talk to" the CC company through one of a few Processor networks.. And those networks only allow certified systems to talk to them, and getting a system certified is, I suspect, close to impossible.
Fortunatly, there are more then a few libraries/servers. RedHat once had such a system, and based on their referral, I once played with MCVE, from Main Street Software. I left the job before anything came of the project; I diddnt go very far with it, but it was infinitly better then a Java system, whose name I dont remember, that I also played with (Dammit, its Java. I should be able to run it under Linux just fine, asshats.)
MCVE bindings are included in stock PHP, which I think is a reasonable vote of confidence.
While doing it yourself would not really remove the SPOF, it would bring it under your control. While the system you build may be technically less secure then one of the third-party-processors, it would also be a smaller target. Your own system wouldnt be effected by a vendetta DDOS against a TPP.
I think, in the grand scheme of things, that the politics of getting merchant accounts with the CC companies would be easier then the technical implementation.
It's true that there are some regions and/or users who are unable or unwilling to use paypal. However there are also some users who would prefer to use it when given the chance. So they cancel each other out in my opinion.
Paypal is easy to set up and they have an automatic notification system that you can hook into to fufill all your needs.
A Multiplayer Strategy Game for Mac OS X, Windows, and Linux
...and even beyond credit cards. There's e-gold, Norfeds, even faxing a check direct. CCs just cost you money, any way you look at it. It's *handy* to have a CC vendor,and as you can see it can also cause a problem, but it's also handy to store your loot and work with your loot in something besides federal reserve note digits being handled by the CC company loan sharks that isn't dropping daily in worth. In case you ain't seen it, the US funny money printing press buck is sinking pretty severely now because the rest of the planet has noticed they don't really have to filter their reality through it any longer, it's become redundant and expensive.
I know this is tangentially off the direct question, but just wanted to point out there are alternatives, and it doesn't hurt to offer them to your customers, and it's easy enough to do as well.
when the service goes down.
The company I work for has a multiple country/currency/provider solution.
That allows for realtime swapping of payment providers (in the case of failure) in a way that is transparent to the merchant.
We have the ability to geolocate and cluster such that we are always reachable in the event of one server being down etc...
....... /
Firstly GGIAITCCI (golly Gee I AM in the credit card industry)
/hr is only 8.7mil a year. Your local MegaLoMart probably does more than that. They are probably paying around 100,000 for processing. Doing your own processing would require an investment over a million dollars and sponsorship from a bank.
To those posters who thing that "thousands of dollars per hour" is large enough to justify processing credit cards yourself, that is really not a large number. $1000
Having a second credit card processor would be a mild pain. it would probably require having two merchant accounts with two different banks (since a processor can only process merchants that are with banks they have arangments with). Not to mention that you would have support two different interfaces.
Your best bet would be to switch to a processor who takes downtime seriously.
Disclaimer: I work for Fifth Third Bank, in a credit card processing capacity (software testing and installation for authorization/settlement system).
First off, Verisign being totally down is completely unacceptable. Demand a refund for the service outage.
Second, why the hell are they totally down? The system that I work with (one of several owned by Fifth Third) is never completely down. We have three access methods; dial, SSL, and non-SSL TCP/IP. It's rare for one of them to have problems, virtually impossible for all of them to get hosed at once. We run on Tandems, which allow for "buddy" process running in seperate CPUs where the secondary takes over if the primary has a hardware problem; we have redundant access to our disk drives so that we can always get to the data. We also have a voice-menu system that you can use to authorize (not a good plan for e-commerce, but I figured if I was plugging the company I work for, why not?). Hell, we even have two identical systems in widely seperated locations! If you can't get through to us, you've probably got bigger things to worry about because there's been a major natural disaster.
Third, WTF did they change during the holiday season that blew up their system? We have a concept called "peak season freeze". Basically, we change *NO* software or hardware between mid-November and the end of September, except emergency fixes for things that are totally broken, and even that is rare.
Fourth, the guy who said you should running your own credit card processing solution is an idiot. He obviously does not know how the credit card processing world works and has never attempted a certification with one of the credit networks.
--Ender
PS I'll go write up an explanation of how the credit card processing world works in my journal now, so that you can go educate yourselves on the basics.
Loose things are easy to lose. You're getting your hair cut. They're going there to see their aunt.
We rely on monitoring the sites, and if there is a problem, switch from the processor to a simple CC capture. We would have to process the orders by hand, but we would only lose the sales that occur between the event of failure, and the switchover.
The key is knowing of a failure, and switching over.
As far as having two gateways/processors. This will be tough. You could have two of each, and just switch to the other one if the other goes down. Your store software should allow you to change this easily. Just disable one, activate the other. Problem with that is having a second gateway and processor which you don't use, then all of a sudden use, the will charge you out the ass.
Now you could get two gateways, and ONE processor. But if the processor goes down, you are completly screwed. But in my years of dealing with gateways and processors, it is 999/1000 the gateway's fault, not the processor.
Fear Is the Only God
I work for a company that does E-commerce, and I handle the CC processing. We've tried different software packages that use SSL direct to the processor -- even one written by the processor -- and they're all junk. Or if there is a good program, I haven't found it.
The best solution I've found is to load balance between Verisign and Authorize.net. Both send to the same processor, as we want to use only one merchant account. At times, I have seen any one of the three go down, sometimes for hours.
An improvement to this is to get a second merchant account on a different processor, but management doesn't want to do that. It will take a day-long outage at our processor to get that ball rolling, and it WILL happen, but I'm only allowed to fight fires, not smoke.
I feel your pain. Good luck.
Having actually worked on the VPS system at VeriSign, I have a bit of insight into many of the problems that are there. Not that this will help you solve your current problem, but maybe this will serve as a warning to others.
VPS came about when VeriSign bought a company called Signio a while back. Originally it was Windows NT servers with Cold Fusion code and MS SQL backend. Still have the same ancient Cold Fusion, but now on Windows 2000 and Oracle on Sun. So a lot of bits have been rigged and kludged together to scale the system and to keep it running. But worse than the antiquated processing system is the political infighting at VeriSign which keeps it from ever getting updated. There has been a plan to rewrite the entire VPS platform into something more modern (say Weblogic app servers) for more than three years. But nothing has yet come of that.
So if you are using VPS, you are using a five year+ old, duct taped together, barely held together system. It's a miracle it stays up during the holidays, ever, except for the work of some very dilligent and dedicated operations folks.
Seriously in violation of another NDA-
Cheers.
- Lots of different vendors wanted lots of different payment service providers (some were refused at certain providers but agreed to at others, some had to go with a certain one for their merchant account etc.)
- Lots of these different payment service providers offer similar types of interface options
- Sometimes payment providers go down, sometimes they go bust (myPaySystems), sometimes they get ddos'd
The upshot of this is that we designed our system to be able to:- use pretty much any payment provider out there (we haven't been presented with one that we couldn't integrate with)
- use multiple payment providers if needed (i.e. nochex vs paypal for the smaller sites)
- change primary payment provider in less than 10 seconds (a single entry in the site config is changed and bam, new payment processor live).
This isn't the kind of thing that's particularly fun to backport into an existing project, or stress free (the quality of most providers documentation is, with rare exceptions like worldpay and protx, shit) but it can really pay dividends in situations like yours.I am NaN
If you do not mind shopping abroad, try Ogone (www.ogone.com). It has different gateways: a Web terminal, batch upload, direct link to your website, ...
If you send me the complete credit card details by simple e-mails, I will quickly tell you if they work or not !
Google passes Turing test : see my journal
I don't understand why everyone beats on about reliability etc... The truth is, that they ALWAYS go down for some reason.
As mentioned earlier, the company I work for sells the use of a payment gateway.
Where I work we have redundancy via the fact that we can process via any provider depending on the merchants requirements (we can rollover to a different provider in the case of failure). Basically the merchant can integrate for one API and the rest is magic.
If a merchant wanted fallover in case our servers fell, then, we can simply duplicate our setup and have it hosted in multiple locations.
I am not really a sales manager, and, don't really have to worry about getting new customers. So, I only offer this info as I am sure that there has to be a US company that offers similar services.
www.ogone.com is another company that offers a similar solution, although, I think they are restricted by merchant Id whereas we are not.
....... /
... get a direct merchant account with a credit card processor. I used Authorize.net and it worked just fine. Of course you need a merchant bank account, a Duns & Bradstreet number etc. Spend the few grand it will take to get a programmer to write your own code to interface with the processor's API. I wrote the layer to talk to Authorize.net's API in less than a week.
Have the user enter his CC info on YOUR site, don't redirect him to the merchant site. This has the added bonus that you can save the CC numbers, so you can give the user one-click shopping next time (but better check Amazon's "patent" first). Then use your own interface to talk to the processor's API and authorize the credit card. If the processor happens to be down, store the transaction in a temporary DB table and tell the user the order has been accepted. Then later when the processor comes back up, authorize the CC. If it doesn't authorize correctly send the user an email saying there was a problem with the CC.
A site that makes 'thousands of dollars per hour' definitely justifies the nominal cost to build a robust, in-house solution. The solution above oughta get you through the holiday season.
There are 2 kinds of people in this world. Those that can keep their train of thought,
Boss:"Sam, where's our money??"
Sam:"It's o'gone!"
Ogone.com
"Leave your money with us."
No thanks.
-- @rjamestaylor on Ello
> MCVE bindings are included in stock PHP,
Which is why we used it, and why I included a chapter about it in my book on PHP. The software works very well, and I've used it for over five years.
The problem is that Red Hat screwed over their customers pretty hard when they just stopped supporting CCVS and gave the name to Main Street. Just call them and try to get documentation or additional keys for CCVS. They won't do it. A friend that works there claims they deleted the source-code for CCVS, because the management wanted to make sure they never spent any more time or effort on it. Before that, some of the employees were supporting it on their own time because they felt bad about how Red Hat and Main Street screwed-over their customers.
I've spent about $30k on CCVS (now owned by Main Street), and they won't provide a changed key so I can change my merchant account for two of my customers that want to change. I will never buy another product from Red Hat, and I will never deal with their proxy Main Street. It sucks when I have money to spend, and they won't take it. It sucks worse when I don't know of another option for Linux that will do the same thing I need.
http://www.TrustCommerce.Com
In some posts people have referred to CC processing "by hand". How does one go about doing this? I have a friend who has a small business in which he processes credit cards over a dial-up connection. The problem is, the software needed only runs on an outdated OS. I've tried looking for alternate solutions for him but I haven't found any site that allows by-hand processing at his current price level (very low).
I run a web site much smaller than yours. On a good month, I do thousands of dollars in traffic per month, rather than per hour. But, speaking as somebody who has written credit card processing code (I go through authorize.net's AIM system), I can say that it's not very hard. I wrote my own shopping card/connection software because the protocols to authorize.net were simpler than the APIs of any of the shopping cart systems I could find. It took about two weeks to write and test it. That's it.
So, one person (me) can write code to connect to a gateway in two weeks. I pay about $15 per month to the gateway. If I did enough business to need redundancy, I would do another two weeks work to write code for the 2nd gateway, another $15 or so per month, then have a switch so that system admins could choose gateway 1, 2, or both...done. 1 engineer month and a few dollars monthly to pay for the 2nd gateway, and there is your redundant credit card processing system.
I think your choice is clear: Pay a programmer to spend a month writing the code you need to handle both gateways from your web site. Pay for both gateways. Problem solved.
In a former life I was director of engineering for a company which sold/supported retail Point of Sales systems CC processing was a important part of these systems.
First we always used a payment gateway system these used dialup or 56k leased lines to the processor. I know that using the internet for transport is attractive because of low costs however since you have accepted the use of an "unreliable transport" for your CC authorization traffic to your merchant account you have essentially no recourse when the system goes down.
If on the other hand you use a processor provided link THEY are on the hook when the system goes belly up.
We used First Data http://www.firstdata.com
They had another name back then but that is unimportant. We used their payment gateway and interfacing to it was simple.
And if the system went down the transactions were still captured logged and if necessary the customer could voice authorize the transactions.
yeah, and their name sucks, too!! "Fifth Third"? Yep, I want a bank that not only isn't "First", they can't figure out if they're third or fifth!!
I also like authorize.net they are very reliable and good tech support.
I notice that TrustCommerce has debian packages for their APIs. My impression from reading their web site is that they're clueful. So far I've only used paypal, but would be interested in peoples' experience with TrustCommerce. I'm thinking of adding them as a secondary payment option some time next year.
I started off only running the process at 2am, but not all the cards would go through. When I added the 12pm and 4am spots, then everything always went through. The key is to avoid the times when people are at the malls competing for VeriSign's bandwidth via checkout card readers.
Great ideas often receive violent opposition from mediocre minds. - Albert Einstein
How to prevent leaked credit cards?
Seriously, I am trying to do self processing at a small shop. Just trying to get the store online with CC payments, and in store charge accounts... But everyone I have talked to says that unless your using a payment gateway to process the credit cards, it is a lawsuit waiting to happen. They say PGP emails are not good enough, and storing them on the server might be illegal or considered negligance.
What shopping cart software do you propose to use for this type of CC storage which would allow me to retrieve it and bill it using a hand processor? (being that it is a low volume web store which compliments a brick and mortar store...)
thanks if you can help!
Your ignorance is infinitely greater than you realize.
Interestingly enough, I wrote a software package from scratch that load balances our company's credit card processing across various networks, including Verisign and AuthorizeNet, as well as among some smaller parties.
It's a great method for not only avoiding blackouts when a gateway goes down, but also for load-balancing chargeback rates if you sign up with multiple banks as well.
Interestingly enough, sometimes a climbing chargeback rate means we send *more* traffic their way... thus increasing the denominator, so to speak.
AuthorizeNet got hit by a DDoS shortly after we deployed this system, so it was a lifesaver. AuthorizeNet hasn't been the same ever since, unfortunately...
Marshal