Slashdot Mirror
Trading the Markets With FOSS Software?
Robert writes "Along with many other techies, I share an interest in the world of finance (bubble-era stock options pulled me in). Unfortunately, as someone with a strong preference for GNU/Linux as my operating system of choice, I have found that software in this area seems quite sparse. For awhile I have made do with Python, R, Gnumeric, Gnucash and a telephone, along with some small utilities I have written myself. What I would like to know is: what FOSS software do you use for financial analysis, trading, system development, and testing in a Un*x environment? Are there programs you would like to see written or ported? Do any brokerages, data providers, or other services provide good support for we the few? And finally, what commercial entities do you know of that are using FOSS software in their operation?"
is a dartboard.
It's unfortunately not available on most distros, but building yourself isn't too hard.
The dependencies are merely a wall, and Newton's three laws of motion.
SEE
Lehman, Merrill, AIG, HBOS all used lots of FOSS IIRC.
Screw automated trading; screw Ben Bernanke, screw McCain-Bush. I'm going to be foreclosed because I lost my job in the operations dept at Merrill and I can't refinance my mortgage. Why should they get a bailout? Quants screwed over my life and I want them to pay.
I've started looking at this too as i've picked up some stock recently, and it is a difficult proposition (given that i'm not really willing to pay for a commercial solution).
Personally, I absolutely love the interface of Google's stock ticker - the interface is nice, the information is top notch. The problem being of course that there's way in any of the nine layers i'd trust google with my portfolio information. The big advantage of a local program in my mind is that the information you put in, even if it is only "I want to track these stocks" is kept wholly to yourself and not stored on some remote server where you have to trust the hoster not to take a peek.
In the end i've been using the default stock program that came on the iPhone to watch the stock prices. Thats all it will do, that and a short graph history, and it uses the yahoo info instead of the google, but it's close to realtime and it's stored (I hope) on the iPhone. Course, Yahoo can still see which stocks i'm requesting, so maybe in the end it makes no difference.
Ideal would be a device-based solution that could draw down the information, either from google/yahoo or direct from the *sx, and hold information regarding you portfolio too - but locally, so theres no worry of the monetary values being shunted across the net to the infovores.
Doublethink is basically the power of holding two contradictory beliefs in one's mind simultaneously, and accepting both
There is none. You will get lots of recommendations to "hack it" with a hodgepodge of crap like the one you seem to be using right now, but that gets you nowhere.
There is no quality trading/management trading software on any OS other than Windows that bears even a passing mention.
Posting AC because people around here tend to get tender and defensive when someone dares suggest that their techno-religious experience is not absolutely perfect for everyone in the planet. Email and surfing the web? Linux is great. This? Don't waste your time and just stick with Windows.
(cue twitter to tell me how I "hate freedom")
EclipseTrader is probably the most advanced open source trading program. It interfaces with some trading platforms and intra-day data feeds. It has several hundred technical indicators. It also is very expandable and easy to write modules for (in Java). I wrote some technical analysis modules for the back-testing system and was fairly impressed with how well it worked as it is based on the very solid OSGI/Eclipse model. I'd say it actually competes fairly well with some of the proprietary trading platforms I have used, especially if you are a Java coder and want to add modules to it to aid in implementing your particular trading style.
I understand that the submitter doesn't know what is out there, but if you ever have a trading account you're likely to have API access to your broker's systems. I recommend Interactive Brokers as their TWS software has a lot of different language bindings.
In finance you don't look for "FOSS" tools, you go to your broker, get API access, and write them yourself.
Website Hosting
I find your views on investing interesting and would like to subscribe to your newsletter.
Sig this!
Is there anywhere a small can find a computerised market in the first place? Are there firms who allow specifies a communications protocol and lets you to trade by computer or does one have to do web scraping to automate things?
Swedish plasma phys. PhD student; MSc EE; knows maths, programming, electronics; finance interest; seeks opportunities
Interactive Brokers has a Java-based
Trader Workstation ("TWS") and they explicitly support Linux. They offer almost anything you can get anywhere, including mutual funds, stocks, options, futures (commodity & financial), currency, and foreign stocks. Commissions are 10x lower than Charles Schwab if you trade often (if you don't then a minimum monthly commission kicks in).
TWS is a large, cumbersome Java applet, but it works tolerably well on a fast machine (and there's not much alternative on Linux)
One annoyance is that they only support jdk 1.5.0_x (not the current 1.6.x), I think because of some concurrency bugs in their code (they claim the newer Java is buggy). However TWS generally does work with the latest jdk, but they won't support it.
IB's telephone support is sometimes rude, the opposite of "hand holding". I guess they have only a few over-worked support people to keep costs down. Also, they only provide on-line statements and never send physical mail except for annual 1099 tax forms. So, be sure your spouse/executor knows you have an IB account, because if you die there will be no monthly statements to clue them!
In summary, IB is good, despite their warts. If you trade a few times a month or more, it's worth the hassles.
Any trading platform is only as good as the data it gets.
Not to sound like an ass but anyone who is that serious about trading needs to invest in a Bloomberg terminal. If you are just dinking around - that's one thing. But the summary seems to indicate you are looking for a "serious" trading platform. Don't waste your time with FOSS. FOSS has it's place but a trading platform is not it. Go with a proven solution.
There are plenty of "active trader" platforms out there from a variety of brokers. Most that I have see are Java apps. I run Schwab's web-based active trading on Linux all the time but even it is a simplified version of the real thing.
Also, one more thing to consider: I would hate to find out that my FOSS trading platform had a bug. If it's bad enough, it could be devastating and totally wipe you out. Do you really want to take on that kind of risk? This really isn't the place to be "testing" your trading platform when real dollars are at stake.
Now....analysis is a different beast. FOSS might have a place there. I don't know but I see no reason to reinvent the wheel when there are FAR better solutions already out there. I have yet to see anything from FOSS that is compelling. And no -- MS Money/Quicken imitators are NOT what we are talking about here. Not even close.
Yes you're the only one. The rest of us don't mistake open source to mean some anti-capitalist campaign but rather a way to make and distribute good software.
I use TD Ameritrade's API (and really nice Java based real-time market tool) which has everything I need for gathering data. Beyond running some perl scripts on the data to generate some basic statistical plots for gnuplot I don't use many other tools. I found that most success in the markets isn't how well you read the past but rather how well you understand the present and can forecast reasonable risk/reward actions.
I know I don't actually mention any FOSS solutions but as a mostly FOSS user this is how I trade.
Being most of this stuff needs a network connection anyways I am sure you will find particular websites that will do the trick. Why do you want to find and download an app, run an update every time the regulations change. Most of the apps for this stuff are so old and usually try to get people to use the web anyways. Don't look for an app when the Web can do the work just as well if not better.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
That is the trading system I've spent several years working on is built entirely using Open Source tools and libraries. The system itself is not currently open, but that is a possibility we here certainly look favorably at.
As far as actual entire free trading systems, there is JavaTraders@googlegroups.com which is a good place to start. Also check out the quickfixj.org site, you will find some things there. There is also an Eclipse plugin which provides some level of GUI.
Frankly we didn't any of the code in any of those projects (although we do use ta-lib). But as I say, you can do a lot with ActiveMQ, any good open source RDBMS (PostgreSQL,MySQL) and your Enterprise Java framework bits of choice.
Basically if I were you I'd pick one of the java based projects that is kicking and does roughly what you want, the way you want to do it. For simple basic trading of one or two instrument classes you can probably put together something pretty workable.
"Malo periculosam, libertatem quam quietam servitutem." -- Jefferson
Say what you want about the origins of the problem but there is no doubt whatsoever that regulation of securities (via the SEC) has been totally and 100% completely absent during all of this.
/. crowd. Say you have a $10 watch. You go to a pawn shop for a loan and they loan you what? I'd say about $5 or so. Certainly something less than the value of the watch, right? Well, on Wall Street lately, they've been getting $300 loans based on that $10 watch.
That, most definitely, happened on Bush's watch. The "laissez-faire" philosophy of the republicans sounds and looks a lot like Hoover right now. They have, literally, let Wall Street run itself. And you can see the end result on the front pages everyday. Whats the saying? Power corrupts and absolute power corrupts absolutely.
Look, I am no regulation lover but even the staunchest of conservative economists recognize free markets must have some regulation to insure a fair playing field. Under this administration -- there has been NONE.
Do you realize that many firms were allowed to leverage up 30 to 1? Let me break down what that means for the
That is totally fucked up and should have never happened. End of story.
Just because the people you talk to appear to be inexplicably (and idiotically) uninterested in tracking their 401Ks, or watching the stock and options they got from their employer, or trying to make a buck by picking up cheap stock before we start climbing out of this hole, doesn't mean the rest of us are too. Maybe you should take your own advice and update your own worldview.
the coolest club on
If you're going the Interactive Brokers approach, JBookTrader and JSystemTrader are both worth a look: http://code.google.com/p/jbooktrader/ and http://groups.google.com/group/jsystemtrader
I have not seen any open source projects that you can use as a platform for building a trading system.
I have built an intraday (within one day) trading system in Java. I'm afraid that this system is not open source either. This system runs one or more models that look for intraday trading signals. The Java software submits buy and sell orders. It is multi-threaded and runs one thread per stock. I have been very happy with the software performance. A long running "server" like this seems to benefit from Sun's HotSpot compiler. The system is web services based (e.g., it runs on Tomcat).
I used Interactive Brokers for my market data and order infrastructure. I was concerned about the quality of the Interactive Brokers tick data (the trade by trade data). Interactive Brokers consolidates their tick data feed so you get a consolidated tick about ever 250 msec. For my system this has been adequate. If you want to run on Linux or use Java there are few inexpensive options for real time data feeds. Information may way to be free, but market data is expensive.
I have some web pages on the alternatives that I explored as platforms for a Java/Linux based trading system. These notes can be found on my web page Software for Constructing a Market Trading System
I never reply to AC's but I am making an exception.
The executive branch appoints people to run institutions like the SEC, FCC, etc. Those people appointed have totally abdicated their responsibility to the people. The entire purpose of the SEC is to make it a "level playing field" with respect to financial disclosure and equity trading. It is not.
Why? Because nobody in those posts is enforcing ANYTHING. And they haven't for a very long time. Yes, Congress makes the rules. But if nobody enforces them, are they really rules?
Because the normal regulations were absent! And why were they absent? They were changed... in Washington!
Those would be the "normal regulations" that brought us Enron, Worldcom, Global Crossing and friends? The Bush Administration has nothing to be proud of, but the concept of corporate fraud was not invented on their watch.
How to solve most of our problems: 1.Lots of nuclear plants. 2.Cure aging.
Wake up and smell the fascism ??? How about, wake up and smell the socialism?
Uh, how about, "Wake up and smell the Crony Capitalism"?
Goofy, Geeky Gifts and More!
All the little stockholders at AIG are getting the shaft.
Uh, I think that was a foregone conclusion when they hired inept management.
It was essential that every stockholder in AIG lose virtually everything they invested. Otherwise it becomes profitable to mismanage your company and let Uncle Sam buy you out.
I think that some of these resuces were necessary for the good of the greater economy. Sure, they shouldn't be necessary, but regulators messed up and now for the sake of not collapsing into a depression we need to clean up.
If I were in charge the only thing I'd do differently when doing bailouts like these would be:
1. Company is 100% taken over.
2. Stock is declared void. Stockholders get a 1-time eminent domain payment of (value of company assets)-(cost to taxpayers for bailout)/(# shares outstanding). Frankly the stockholders should be happy they don't end up owing money which is what the math certainly will work out to.
3. Corporate officers arrested and face heavy criminal penalties. Costing the taxpayers billions of dollars needs to be made a serious crime. It is certainly worse than robbing the corner store.
4. Government runs company in such a way to preserve the general economy.
5. Eventually company is either dissolved or IPO'ed - with all proceeds going to taxpayers.
If this were how bailouts worked you wouldn't see too many executives asking for them.
Don't get me wrong - the preference is in general to let companies just go bankrupt and not interfere. But, if interference is needed for the greater good than this is how it should be done.
Interesting speech from Senator Dorgan when the bill, Financial Services Modernization Act, was being discussed '99. Those who don't know history...
"I remember a couple of circumstances that existed more recently. I was not around during the bank failures of the 1930s. I was not around for the debate that persuaded a Congress to enact Glass-Steagall and a range of other protections. But I was here when, in the early 1980s, it was decided that we should expand the opportunities for savings and loans to do certain things. And they began to broker deposits and they took off. They would take a sleepy little savings and loan in some town, and they would take off like a Roman candle. Pretty soon they would have a multibillion-dollar organization, and they would decide they would use that organization to park junk bonds in. We had a savings and loan out in California that had over 50 percent of its assets in risky junk bonds.
Let me describe the ultimate perversion, the hood ornament on stupidity. The U.S. Government owned nonperforming junk bonds in the Taj Mahal Casino. Let me say that again. The U.S. Government ended up owning nonperforming junk bonds in the Taj Mahal Casino in Atlantic City. How did that happen? The savings and loans were able to buy junk bonds. The savings and loans went belly up. The junk bonds were not performing. And the U.S. Government ended up with those junk bonds.
Was that a perversion? Of course it was. But it is an example of what has happened when we decide, under a term called modernization, to forget the lessons of the past, to forget there are certain things that are inherently risky, and they ought not be fused or merged with the enterprise of banking that requires the perception and, of course, the reality--but especially the perception--of safety and soundness.
Last year, we had a failure of a firm called LTCM, Long-Term Capital Management. It was an organization run by some of the smartest people in the world, I guess, in the area of finance. They had Nobel laureates helping run this place. They had some of the smartest people on Wall Street. They put together a lot of money. They had this hedge fund, unregulated hedge fund. They had invested more than $1 trillion in derivatives in this fund--more than $1 trillion in derivatives value.
Then, with all of the smartest folks around, and all this money, and an enormous amount of leverage, when it looked as if this firm was going to go belly up, just flat out broke, guess what happened. On a Sunday, Mr. Greenspan and the Federal Reserve Board decided to convene a meeting of corresponding banks and others who had an interest in this, saying: You have to save Long-Term Capital Management. You have to save this hedge firm. If you don't, there will be catastrophic results in the economy. The hit will be too big.
You have this unregulated risky activity out there in the economy, and you have one firm that has $1 trillion in derivative values and enormous risk, and, with all their brains, it doesn't work. They are going to go belly up. Who bears the burden of that? The Federal Government, the Federal Reserve Board.
We have the GAO doing an investigation to find out the circumstances of all that. I am very interested in this no-fault capitalism that exists with respect to Long-Term Capital Management. Who decides what kind of capitalism is no-fault capitalism? And when and how and is there a conflict of interest here?
The reason I raise this point is, this will be replicated again and again and again, as long as we bring bills to the floor that talk about financial services modernization and refuse to deal with the issue of thoughtful and sensible regulation of things such as hedge funds and derivatives and as long as we bring bills to the floor that say we can connect and couple, we can actually hitch up, inherently risky enterprises with the core banking issues in this country.
I hear about fire walls and affiliates, all these issues. I probably know less about them than some others;
Leverage (aka: debt) is key to our economic model. However, that leverage is carefully managed.
Management of the leverage is what is missing here. NO FIRM should be allowed to leverage up 30:1. (30x your collateral)
You are comparing apples and oranges.
And since you went there with LTCM....allow me to quote you a figure, "At the beginning of 1998, the firm had equity of $4.72 billion and had borrowed over $124.5 billion with assets of around $129 billion. It had off-balance sheet derivative positions with a notional value of approximately $1.25 trillion"
Management of leverage was obviously missing there. 1.25 trillion? That is absolutely totally fucking insane. But because Merriweather was a "smart guy" --- the investment banks let him do this. And it put the entire system at risk.
That's not supposed to happen. Ever. Yet it did. Do we learn nothing from history?
While it's not open source you could give Personal Stock Streamer a try:
http://www.personalstockstreamer.com/
It's free to use for non-professionals if you use it with your TD AMERITRADE account. There's a VBscript and C++ API for creating your own indicators, reports and extensions.
... is Firefox and an index fund. Dollar-cost averaging is your friend.
what FOSS software do you use for financial analysis, trading, system development, and testing in a Un*x environment?
I am aware of TA-Lib, QuantLib and libraries implementing the FIX protocol to be used in commercial products and private Un*x trading applications.
Sadly, most people looking for a solid FOSS trading platform will find that they need to roll up their sleeve to get something decent working.
I have found that software in this area seems quite sparse.
Indeed. I closely monitor the FOSS community since 1998 for a trading platform. What I observed was a lot of well intended projects, but even to these days, none did reach satisfying maturity.
I think it relates to under estimation of the complexity. Most project starts with a lot of energy then goes idle because of unattainable short-term goals.
A general purpose platform will not emerge until someone put coherently together existing building blocks instead of starting yet another weak trading platform from scratch.
Example of building block is the FIX protocol. Many FIX libraries have matured. Quantlib is another solid example.
My contribution to the whole picture is TA-Lib. It is a set of functions for people who care about technical analysis (shameless plug).
\Mario
As a software engineer, I too share some interest in financial analysis and have researched it off and on over the years with an eye towards developing some of my own analysis tools as a hobby interest. However, I have continually been discouraged by the lack of decent public market data services of the type that would be suitable for feeding to analysis software. The decent market data services all want thousands of dollars per month for their streams so that basically precludes anyone but the trading professionals working for financial services companies or high net worth individuals (HNWI), who have enough at stake to justify the $20,000 per month Bloomberg Terminal, from doing anything more than dabbling. The free services are delayed 20 minutes or more and/or have limits on queries per unit time (usually both and always the query limit) etc that make them unsuited for automated analysis (probably what the companies want, since they are trying to sell you up to their professional streaming services). I stopped after some basic analysis and test programs because I knew that without access to relatively comprehensive streaming data services, much of the value of a more complex automated analysis system would be unrealized. The professional industry is already well served by Bloomberg, Reuters, and other terminal / market data businesses so it really isn't worth trying to compete against them professionally by starting a company (the existing services have a massive head start and are already firmly entrenched and unlikely to be unseated and the company would need millions to get off the ground in any case since exchanges are not going to give a new firm any breaks on prices they charge Bloomberg and other firms for access to the live streams). In the end I learned enough about finance to do my own research and analysis manually with the assistance of some basic tools that I cobbled together for myself and that is basically all there is to be done about it unless you want to get serious and plunk down the big bucks for access to the professional data services (you have $20,000 per month to burn?).
It's a complicated public/private structure, and basically anything less than a book-length explanation of it oversimplifies in one way or another. It's the de facto U.S. central bank, and the main federal agency regulating banking, but also has significant private components. Its funding sources are technically ownership and fees by various private banks, but restricted in such ways that it's de facto more like government regulation and bank taxes than ownership and fees between private entities. Its board of governors are appointed by the President, and confirmed by the Senate. It's also effectively an operational arm of the U.S. Treasury for everything from tax collection to paper-money supply to treasury-bond issuance and interest payments.
More to the point, its solvency is implicitly guaranteed by the U.S. government, much like that of Fannie Mae and Freddie Mac were (and those two quasi-private entities were actually de facto considerably more private than the Fed is).
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
Fundamentally FOSS and financial trading software are incompatible concepts. IE if you develop any software product of tangible use and value then it does not make sense for the author to leave it as a free product - when you can be making money out of it (and generally substantial amounts).
This was the pattern we saw. Many developers had started projects with good intentions, and as training exercises for themselves etc. but then after that they usually branched off to create a commercial product, or were hired by a financial institution which usually includes confidentiality in the contract.
Projects like EclipseTrader are a progression, but are quite a way behind the edge in terms of functionality - they are recreating tools which were commercial products some ten years ago - which is not to say it is not a useful tool, which it is.
It is the same reason that market data (share prices etc) is generally quite expensive in real time, but often free after 15 minutes. It is a bit like knowing who is the winner of a horse race - 5 minutes makes a big difference to the value of the information! For academic purposes, exchanges like Euronext (LIFFE) will provide tic level data all the way back if you ask nicely.
If you invented a trading system which could be shown to help generate a good income & profits - would you give it away for free?
- if you do please send me a copy!
Ben
Ummm, this looks like a windows app. The poster is looking for linux ones.