Royal Bank of Canada Software Upgrade Goes Awry
Reader mks113 writes "Many Canadians living payday to payday have been in for a shock this week. Canada.com along with many other sources is reporting how thousands of customers have been inconvenienced following an unsuccessful software upgrade at the Royal Bank of Canada on Monday. All government employees (including me) in several provinces had their direct deposits delayed by a day or more." RBC has a comment on the mess.
I guess that bad karma is pretty sticky. Even selling their preferred A-1 shares to Baystar didn't save them.
My Canadian friends are screaming bloody murder. I don't blame them.
"Do the Right Thing. It will gratify some people and astound the rest." - Mark Twain
"Do the Right Thing. It will gratify some people and astound the rest." - Mark Twain
Now you know where the old wisdom "Never touch a running system" comes from.
Geez, I'm showing my age again...
A feeling of having made the same mistake before: Deja Foobar
I've heard a lot about this here in Canada over the past few days. Does anybody know what systems RBC was using, and what upgrade they were doing? It definitely seems suspicious that they were doing an upgrade at the *end* of the month (May), which is the busiest time for a bank (I know, from having worked at one). Was this really an upgrade gone wrong? Are there any more technical details?
"Never attribute to malice what can be adequately explained by stupidity"
I don't typically work on these all-or-nothing type of systems. I usually work on embedded controllers, so we have the ability to put, say 10 units into the field for trials. Given the English air troubles and now this, isn't there a way to deploy a system such as this where it can be tested with real-world loads but not be the only system in use? Our controllers are not placed directly on the assembly line on the first day out, just for this purpose.
I for one welcome our new [insert main topic] overlords.
You don't get contacted three times by a collection agency for being one day late with your Visa payment.
How can we continue to believe in a just universe and freedom to eat crackers if we have no ale?
You know, if she's been contacted THREE TIMES this week about payment, then she's at LEAST 2 months overdue. Going to blame the other 2+ months on the computer glitch that started on Monday? Besides, their Visa bills aren't actually DUE until about 7 working days into the month, so if she was up to date, then her payment wouldn't even be due yet.
No, that's just somebody who thinks the world owes them everything taking the opportunity to complain because it might get them something they don't really deserve.
Dark Nexus
"Sanity is calming, but madness is more interesting."
Nice planning: an end-of-month upgrade at a financial institution when, by their own admission, transactions are at their peak.
Maybe they thought they would broaden their QA testing base to, say 20,000,000.
Have you Meta Moderated t
Something like this happened at my company with the direct deposit so I understand you situation but shouldn't you have made sure that you had the money in the account before you wrote the check?
I know it's a common practice to float a check, you take in to account that the check will take X days via us mail, it will then take x days to post, my paycheck will go in to my account in X - y days so I'll be fine.
Just because it's common practice doesn't make it right. At some point you have to take responsibility, you singed your name to that check and said that at from the date on this check forward I have the funds in my account to cover it.
I'm not trying to be an troll or anything, I just think that you share a part of the blame for your checks bouncing.
500 dollar reward for tip(s) leading to the arrest of the person(s) who stole my sig.
No, this is management's fault. The only time a programmer fails is if something wasn't delivered on time, or they just don't produce, or their stuff doesn't make it past QC. Then fire them.
If bad code makes it into the wild, then somebody signed off on it. Somebody cut corners on testing. Somebody decided deadline is more important than quality. Somebody insisted it had to run the newest Microsoft code.
That somebody is the programmer's boss.
I'd bet on managers pushing through the update before it was ready.
>Heads should roll for this one.
But are the "correct" heads going to roll?
>In cases like this, you should be lucky if you aren't held 100% liable.
If you were suppose to be held liable, do you think anything would change? Were any Professional Engineers held liable for the big blackout last year?
The surprise isn't how often we make bad choices; the surprise is how seldom they defeat us.
I totally agree. Most experts recommand that you have three to six months worth of funds saved up, which means that delay of payroll check deposit of a day, or even couple of weeks should be non-issue.
1f u c4n r34d th1s u r34lly n33d t0 g37 l41d
For the longest time I was living from paycheck to paycheck. Compared to my friends I made some pretty good money for being 20 (30K/Yr) but it didnt seem like much when you were living on your own with rent, insurance, car payments, electric, cable, phone, water, and a girlfriend. Its amazing how things add up. It just so happens that I receive the first paycheck of the month on the same day that rent is due. After paying rent I'm left with about $100. It is also convenient that the second and last paycheck of the month is received on the same day that all of the bills are due. After paying them I'm left with $300. Note I havent mentioned the G/F tax yet but that one is expensive.
Anyway, my tip is, next time you get a bonus, tax returns, some lump sum of money, spend it on next months rent before you can do anything else with it. Trust me on this. If you put it in your savings you can too easily transfer it to checking when you see Wizz-Bang4000 on pricewatch for only $499! I do this every chance I get and it really helps out a lot.
Now if I could only figure out what to do with the SO.
Im dreaming ofa big bndwdth, That can resist the
Can you imagine working in that IT department right now. My first reaction when I saw this story was that I felt incredibly sorry for those IT guys and gals. Must be hell over there right now!
I know I always sweat when releasing new software, at least I don't have to worry about effecting the bank accounts of millions of people. That would truly be scary!
Corporate Gadfly
Jonathan Archer: the most beaten up Enterprise captain in Star Trek history
Yeah, it was probably written in the US :)
I would disagree. Yes you should double check, but doing thier job is 100% thier responsibility.
Free Mac Mini Yeah, it's
This is insightful? I hope aardwolf was actually trying to be funny. Otherwise his "don't save money because it is too easy to spend savings" plan will have him working until the day he dies. Personally, my SO and I live off of one paycheck and put the other into some form of savings or another. Using this plan, we're aiming at having a million smackers in the bank before we're fifty.
Better still, this plan doesn't have us up Shit Creek when a paycheck (or twenty) is missed.
If you have such a serious problem with raiding your savings account direct deposit can be a great tool for you. Have a small chunk of each check sent into this savings account and never touch it. Never ever. Hell, you'd probably be better off if you didn't even open your bank statements for that account but once a year. Whatever you do, living month to month is not the answer.
Peter
Downsize DC Today!
It makes me wonder; if you piss off enough of the clueful folks in this industry, would they simply not apply at your organization, insuring that the only people your HR department sees are the dregs of the vocational schools? Since HR people can't tell the difference between good IT people and bad, no one would get wind of the situation until such time as there were a major failure in your systems brought on by your shitty IT department. Makes you stop and think, doesn't it?
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
Try being fresh out of college. How the hell are you supposed to have 6 months of funds saved up when you've only been working full-time for 2 or 3? I'm sure that will teach those pesky kids trying to pay their rent and student loans for not saving anything up. Bottom line not everyone has that luxury.
I've been saying that to IT people for years. While security is important, having a sound disaster recover plan is even more important. Your system cannot be 100% secure; it's practically impossible. It can be 99.99999999% secure, but that one time can still happen, and you had better be prepared for it.
I've seen too many IT people focus entirely on security, and forget about having a sound backup strategy. Those IT people need to get fired. Sure, playing with Arcserve or Backup Exec or whatever sux, compared to finding some new hash algorithm to further encrypt your ssh session, and let's face it, we all like learning new things, but IT isn't about that kinda stuff.
I've had yelling matches at my last employer with IT people; they insisted on not having ANY FTP server, were farting around with NDS, yet when we asked them for a restore from tape, every time there was an issue. They had a 100% failure rate on their backup tapes.
There's too many retarded IT people out there. and too many retarded IT managers out there too.
09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
Oh, excellent economic analysis.
... seems to me Canada was in WWI and WWII as well, rendering your analysis on even the most facile level moot and not a little idiotic.
Really, the only factor is whether the US is at war or not? Really!? Really, really!??
Let's see
So what do these "experts" recommend you spend during those 3 to 6 months?
Honestly, if everyone had the ability to just save up 6 months worth of funds then no one would be bouncing checks or investing in short term disability insurance. I had bills before I had a job, no one gave me a grace period to get together some emergency savings.
I save a pretty good amount, 20% of each check goes straight to savings, but it'll still take me most of a year to get 3 months of backup funds. Someone living on a more hand-to-mouth salary may never get that much saved.
-- I'm not a pessimist, I'm a realist. It's not my fault that life sucks so much. --
Since when does having a girlfriend mean having a money pit? It sounds like the girlfriend is more interested in material goods than you or you are out to make a big, but expensive impression on her.
See my sig...
"Gold still represents the ultimate form of payment in the world." - Alan Greenspan, 1999
It's quite sad, really: any disagreement with you automatically means that I don't like Americans and any and all arguments can and will result in violence, in this case coupled with a strange assortment of insults, what I can only guess is a swipe at my sexuality, and a comment that shows your profound misunderstanding of geography. I must, however, point out that your message, such as it is, is getting garbled by your obvious mastery of grammar and spelling.
That's a highly desirable breathing space but still leaves you shackled to your paycheck. If you can learn discipline -- where "savings" means "that stack of money that keeps growing and that I will never touch unless my child is dying", you will be FREE.
FREE, to take a six-month leave of absence to do something that's important to you.
FREE, to quit the job that is making you ill with stress, even though you have no prospects at this time.
FREE, where your boss and your company's CFO and any of the financial institutions you keep your money in -- all these have NO SAY in your life, except as far as you wish them to.
You can't protect against everything, but 30K ought to be enough to get ahead. I started out at 15K a year in 1987, and gave some of that away to charities. I know what humble beginnings are like. ESCAPE THEM!
Couldn't happen to a nicer, more deserving bunch of jerkwads.
(/former field-support rep for a vendor, who got *burned* by the incompetence and mendacity of RBC IT personnel who lied to their manager, and my manager, when THEY screwed up their evaluation of our product - AFTER they had dragged the evaluation out past the 12-month mark. . . how the hell do you justify evaluating a product for 12 months?).
These are my friends, See how they glisten. See this one shine, how he smiles in the light.
It's makes you wonder what "money" really is,
Money is a concept built on mutual trust. I trust that the money you give me will be honored at its face value in another place.
Be it dollars, euros, gold, or matchsticks.
Money, as such, is meaningless without that trust.
- - - - - - - - - - -
I am a programmer. I am paid to produce syntax not grammar. Deal with it.
On behalf of the United States, I would like to appologize to anyone who was offended by the moronic poster who decided to insult our Northern neighbor, as well as assume that Canada is smaller then Texas. To the insulting poster - buy a globe, buy a hammer. Study the globe, and smash your keyboard so you can't type anymore.
I mod down so you can mod up. Your welcome.
>If you can learn discipline -- where "savings"
>means "that stack of money that keeps growing and
>that I will never touch unless my child is dying"
That's a good strategy and everything, but I learned that it would be better to go ahead and let that savings account go to about zero, if I can use it to get completely out of debt.
If I had a "child is dying" incident, I'm sure it will mean going into debt anyway, and it can't hurt to have zero debt in a situation like that.
-fb Everything not expressly forbidden is now mandatory.
This comes down to poor testing. The manager(s) involved should (and probably will) be fired - they would have had to have signed off on the code change, and odds are that they rushed the programmers/QA people through testing.