Re:Real-life Experience with functional programmin
on
OCaml For the Masses
·
· Score: 1
Our main application (LexiFi Apropos) is almost entirely written in OCaml. Some years ago we had kernel algorithms implemented in OCaml and some GUI + business logic in C#, and we decided to switch everything to OCaml. We are very happy about this choice.
In my opinion, the most important language features that exist in OCaml (and Haskell, etc) and which are still missing from mainstream languages are sum types (tagged union with parameters) and pattern-matching. Honestly, I don't understand why such powerful and simple constructs are still reserved to functional languages... Probably the bad heritage of C-like languages where you can easily multiply types (structures = product types) but not add them (sum types). Addition is as basic as multiplication!
Then I could cite various means that OCaml offers to factorize and share the code: structural objects, functors, first-class modules. These are great and really allowed us to reduce the size of our code base and increase its maintainability a lot.
For the UI, we use CSML which is a kind of high-level FFI between OCaml and.Net. We use directly the Windows Forms API from our OCaml code as we would do from C#. On top of that, we have developed higher-level layers to facilitate GUI development. In particular, we have some libraries to generate automatically entry screens from OCaml type definitions (see http://www.lexifi.com/product/technology/automatic-generation-of-user-interfaces).
Same for databases: we use exactly the same low-level programming model as we would do it we were programming in an imperative or OO language (OCaml is imperative and OO, btw). On top of that, we have an ORM again based on OCaml type definitions. Nothing really specific to functional programming here.
You seem to underestimate library availability for OCaml. Sure, there are gaps in some areas, especially under Windows, but there are good native OCaml libraries that cover many things, and their API is directly adapted to the ML style of programming.
Also, it is possible to use.NET libraries from OCaml, using the CSML tool we have been developing at LexiFi. Importing a.NET library only amounts to describing its API as an OCaml interface.
Our OCaml applications under Windows have native Windows GUI, with some 3rd party.NET widgets; this goes through CSML. Same for some Windows specific programming (like file system monitoring, or Office automation). For XML, HTTP, networking, basic system programming, database (Postgres, Oracle, Sqlite), generation of Office and PDF documents, we use OCaml libraries.
You should try CSML, the tool LexiFi (for which I'm working) has developed (first for internal use, and then released publicly). It provides a much higher-level bridge between OCaml and.Net than what you'd expect if you're used to COM.
Well, being integrated in.Net comes with a cost at the level of the language design itself. Because F# code runs on the CLR and can share its data with code written in other.Net language, I think it has no way to enforce invariants which cannot be expressed by the CLR itself. For instance, a really nice property of OCaml is that unless you explicitly say it, a field cannot be "null". As far as I know, F# cannot enforce this very simple but important constraint on its data structures as soon as they are potentially manipulated by code written in, say, C#.
Some innovations on type systems cannot be integrated easily in F#, because of its dependency on the CLR, without important runtime costs. When OCaml knows that its type invariants are enforcer, it can skip runtime tests, which is impossible on the CLR. Examples already in OCaml: structural object types, first-class modules, and more to come (like GADTs).
For what concerns 3rd party libraries, you can use.Net libraries from OCaml under Windows, using a tool (CSML) which we have developed for this purpose. We do it regularly, when we have no other choice, but frankly, we have had a lot less trouble with OCaml libraries than we.Net components! OCaml libraries don't cover every needs, especially on Windows, but they are of great quality and their APIs are directly adapted to the functional style (of course!), unlike.Net components.
You say: "Ocaml was developed by INRIA to support their own R&D program". This used to be true, but the story is different now. INRIA has created the Caml Consortium to streamline the interaction with industrial users, and the development model for OCaml is now more open. There are non-researcher core OCaml developers who work for companies using OCaml, and the general public has (read-only) access to the live Subversion repository of the project (including experimental branches).
We do also develop on Windows, and OCaml works perfectly well with.NET. We have released an internal development tool (google "csml lexifi") that allows very easy and safe bindings between.Net and OCaml. The bridge takes care of memory management, data mapping in a type-safe way (including first-class functions), exception mapping, etc.
Is it possible to create a mixed assembly, that contains code written both in C# and in F#? Last time I tried, I could not figure out how to do it. With CSML, we develop applications where some parts are implemented in C#, some in OCaml, and they interact in a bi-directional way (some data structure definitions span the two worlds).
The software editorI'm working for uses OCaml for its developments, even though most of our customers are running Windows. And we do actually rely on.Net for some aspects of our application, using a tool which we have developed for this purpose (we release this tool publicly). Being able to work with the same code base on Windows and Unix is a big plus for us, and we can use.Net components (and expose our technologies as.Net components as well) when we need to.
Individual double are indeed boxed, but doubles within arrays or structures of doubles are not. Same for doubles used as intermediates results in computations: not boxed.
In many cases, avoiding to specialize polymorphic code for specific types is a win (but in some cases, it's not). It allows for a smaller memory footprint for the code, which is good for the cache. Some programs and special data structures can really use a huge (unbounded) number of instantiations, in a dynamic way. The JIT approach makes it possible to specialize code also in these cases, at runtime, but this would be very costly. (Think about polymorphic recursion, where a polymorphic function calls itself at a different type.)
Core OCaml developers do care a lot about their users, including the industrial ones.
The Caml Consortium is the place where companies can get direct access to the developers, express their needs, and I can tell you that their voice is heard. As a matter of fact, there are now several non-academic members of the core OCaml development group, working directly on the "official version", who happen to work for companies using OCaml (I'm one of them).
LexiFi (the company I'm working for) has been founded by one of the authors of this academic paper and our core technology is based on the same principles. We use OCaml as our implementation language, but the approach is really language agnostic. The basic idea is to represent complex derivatives as terms of a small DSL, with a few well-understood basic blocks and combinators. Contracts are just simple pieces of data, that can be exchanged and manipulated by independent systems, and there is a clear semantics for the language.
Based on this approach, LexiFi has developed an implementation of the DSL, with many "universal" operations that works on arbitrary contracts described in the DSL: pricing, life-cycle management, risk-analysis, reporting tools. We expose these operations in GUI applications targeted to specific kinds of end-users, and the uniform representation of financial instruments gives us a lot of productivity to add new kinds of contracts.
Clearly, another great use of the representation language is to have "electronic term sheets", that is, executable descriptions of contracts that can be understood both by humans and by computers (due to the controlled expressivity of the DSL). This is very much in line with what the SEC proposes.
Indeed, a functional programming language is probably a better choice. Actually, a domain specific language, whose expressivity is well-controlled, is even better: it is more readable, and you can apply exact analysis. And this is exactly the approach we have taken at LexiFi (for which I work)! We have a small DSL to describe all kinds of derivatives, and various "universal" treatments that can be applied to any contract described in this DSL. Our implementation language is OCaml, but this is really an internal choice: contracts are just data, which can be represented in any syntax (XML for instance).
I work for LexiFi, a software provider whose technology is built upon a formal description of financial contracts. This is based on the academic work you mention, but our implementation now relies on the OCaml language. Note that OCaml is only the implementation language and our approach is really language agnostic: our idea is to represent terms and conditions of derivatives as data, not programs. More specifically, we have a small domain specific language, made of a few building blocks and combinators, that can express the semantics of all the existing and future structures of derivatives. Contracts represented in this DSL can be manipulated by various programs. It is a matter of syntax to export the contracts as XML tree or whatnot.
Based on our formal description of contracts, which is easier to read and understand than a program in a Turing-complete general purpose language, we have developed a number of "universal" operations (these operations can be applied on any structure described in the language): life-cycle management of the contracts, pricing, and all kinds of risk analysis and reporting.
We have developed full-blown applications for various kinds of end-users, and our approach gives us a huge productivity increase for adding new structures to these applications (and our users can also do it themselves), but the same technology could definitely be used to exchange formal descriptions of contracts between institutions, in a much safer way than with Python scripts.
Don't hesitate to contact us directly if you are interested!
Slashdot Mirror
Our main application (LexiFi Apropos) is almost entirely written in OCaml. Some years ago we had kernel algorithms implemented in OCaml and some GUI + business logic in C#, and we decided to switch everything to OCaml. We are very happy about this choice.
In my opinion, the most important language features that exist in OCaml (and Haskell, etc) and which are still missing from mainstream languages are sum types (tagged union with parameters) and pattern-matching. Honestly, I don't understand why such powerful and simple constructs are still reserved to functional languages... Probably the bad heritage of C-like languages where you can easily multiply types (structures = product types) but not add them (sum types). Addition is as basic as multiplication!
Then I could cite various means that OCaml offers to factorize and share the code: structural objects, functors, first-class modules. These are great and really allowed us to reduce the size of our code base and increase its maintainability a lot.
For the UI, we use CSML which is a kind of high-level FFI between OCaml and .Net. We use directly the Windows Forms API from our OCaml code as we would do from C#. On top of that, we have developed higher-level layers to facilitate GUI development. In particular, we have some libraries to generate automatically entry screens from OCaml type definitions (see http://www.lexifi.com/product/technology/automatic-generation-of-user-interfaces).
Same for databases: we use exactly the same low-level programming model as we would do it we were programming in an imperative or OO language (OCaml is imperative and OO, btw). On top of that, we have an ORM again based on OCaml type definitions. Nothing really specific to functional programming here.
You seem to underestimate library availability for OCaml. Sure, there are gaps in some areas, especially under Windows, but there are good native OCaml libraries that cover many things, and their API is directly adapted to the ML style of programming.
Also, it is possible to use .NET libraries from OCaml, using the CSML tool we have been developing at LexiFi. Importing a .NET library only amounts to describing its API as an OCaml interface.
Our OCaml applications under Windows have native Windows GUI, with some 3rd party .NET widgets; this goes through CSML. Same for some Windows specific programming (like file system monitoring, or Office automation). For XML, HTTP, networking, basic system programming, database (Postgres, Oracle, Sqlite), generation of Office and PDF documents, we use OCaml libraries.
You should try CSML, the tool LexiFi (for which I'm working) has developed (first for internal use, and then released publicly). It provides a much higher-level bridge between OCaml and .Net than what you'd expect if you're used to COM.
Well, being integrated in .Net comes with a cost at the level of the language design itself. Because F# code runs on the CLR and can share its data with code written in other .Net language, I think it has no way to enforce invariants which cannot be expressed by the CLR itself. For instance, a really nice property of OCaml is that unless you explicitly say it, a field cannot be "null". As far as I know, F# cannot enforce this very simple but important constraint on its data structures as soon as they are potentially manipulated by code written in, say, C#.
Some innovations on type systems cannot be integrated easily in F#, because of its dependency on the CLR, without important runtime costs. When OCaml knows that its type invariants are enforcer, it can skip runtime tests, which is impossible on the CLR. Examples already in OCaml: structural object types, first-class modules, and more to come (like GADTs).
For what concerns 3rd party libraries, you can use .Net libraries from OCaml under Windows, using a tool (CSML) which we have developed for this purpose. We do it regularly, when we have no other choice, but frankly, we have had a lot less trouble with OCaml libraries than we .Net components! OCaml libraries don't cover every needs, especially on Windows, but they are of great quality and their APIs are directly adapted to the functional style (of course!), unlike .Net components.
You say: "Ocaml was developed by INRIA to support their own R&D program". This used to be true, but the story is different now. INRIA has created the Caml Consortium to streamline the interaction with industrial users, and the development model for OCaml is now more open. There are non-researcher core OCaml developers who work for companies using OCaml, and the general public has (read-only) access to the live Subversion repository of the project (including experimental branches).
We do also develop on Windows, and OCaml works perfectly well with .NET. We have released an internal development tool (google "csml lexifi") that allows very easy and safe bindings between .Net and OCaml. The bridge takes care of memory management, data mapping in a type-safe way (including first-class functions), exception mapping, etc.
Is it possible to create a mixed assembly, that contains code written both in C# and in F#? Last time I tried, I could not figure out how to do it. With CSML, we develop applications where some parts are implemented in C#, some in OCaml, and they interact in a bi-directional way (some data structure definitions span the two worlds).
The software editorI'm working for uses OCaml for its developments, even though most of our customers are running Windows. And we do actually rely on .Net for some aspects of our application, using a tool which we have developed for this purpose (we release this tool publicly). Being able to work with the same code base on Windows and Unix is a big plus for us, and we can use .Net components (and expose our technologies as .Net components as well) when we need to.
Individual double are indeed boxed, but doubles within arrays or structures of doubles are not. Same for doubles used as intermediates results in computations: not boxed.
In many cases, avoiding to specialize polymorphic code for specific types is a win (but in some cases, it's not). It allows for a smaller memory footprint for the code, which is good for the cache. Some programs and special data structures can really use a huge (unbounded) number of instantiations, in a dynamic way. The JIT approach makes it possible to specialize code also in these cases, at runtime, but this would be very costly. (Think about polymorphic recursion, where a polymorphic function calls itself at a different type.)
Core OCaml developers do care a lot about their users, including the industrial ones.
The Caml Consortium is the place where companies can get direct access to the developers, express their needs, and I can tell you that their voice is heard. As a matter of fact, there are now several non-academic members of the core OCaml development group, working directly on the "official version", who happen to work for companies using OCaml (I'm one of them).
Thanks for mentioning this work!
LexiFi (the company I'm working for) has been founded by one of the authors of this academic paper and our core technology is based on the same principles. We use OCaml as our implementation language, but the approach is really language agnostic. The basic idea is to represent complex derivatives as terms of a small DSL, with a few well-understood basic blocks and combinators. Contracts are just simple pieces of data, that can be exchanged and manipulated by independent systems, and there is a clear semantics for the language.
Based on this approach, LexiFi has developed an implementation of the DSL, with many "universal" operations that works on arbitrary contracts described in the DSL: pricing, life-cycle management, risk-analysis, reporting tools. We expose these operations in GUI applications targeted to specific kinds of end-users, and the uniform representation of financial instruments gives us a lot of productivity to add new kinds of contracts.
Clearly, another great use of the representation language is to have "electronic term sheets", that is, executable descriptions of contracts that can be understood both by humans and by computers (due to the controlled expressivity of the DSL). This is very much in line with what the SEC proposes.
Indeed, a functional programming language is probably a better choice. Actually, a domain specific language, whose expressivity is well-controlled, is even better: it is more readable, and you can apply exact analysis. And this is exactly the approach we have taken at LexiFi (for which I work)! We have a small DSL to describe all kinds of derivatives, and various "universal" treatments that can be applied to any contract described in this DSL. Our implementation language is OCaml, but this is really an internal choice: contracts are just data, which can be represented in any syntax (XML for instance).
Thanks for mentioning this work.
I work for LexiFi, a software provider whose technology is built upon a formal description of financial contracts. This is based on the academic work you mention, but our implementation now relies on the OCaml language. Note that OCaml is only the implementation language and our approach is really language agnostic: our idea is to represent terms and conditions of derivatives as data, not programs. More specifically, we have a small domain specific language, made of a few building blocks and combinators, that can express the semantics of all the existing and future structures of derivatives. Contracts represented in this DSL can be manipulated by various programs. It is a matter of syntax to export the contracts as XML tree or whatnot.
Based on our formal description of contracts, which is easier to read and understand than a program in a Turing-complete general purpose language, we have developed a number of "universal" operations (these operations can be applied on any structure described in the language): life-cycle management of the contracts, pricing, and all kinds of risk analysis and reporting.
We have developed full-blown applications for various kinds of end-users, and our approach gives us a huge productivity increase for adding new structures to these applications (and our users can also do it themselves), but the same technology could definitely be used to exchange formal descriptions of contracts between institutions, in a much safer way than with Python scripts.
Don't hesitate to contact us directly if you are interested!