Slashdot Mirror
Erik Meijer: The Curse of the Excluded Middle
CowboyRobot (671517) writes "Erik Meijer, known for his contributions to Haskell, C#, Visual Basic, Hack, and LINQ, has an article at the ACM in which he argues that 'Mostly functional' programming does not work. 'The idea of "mostly functional programming" is unfeasible. It is impossible to make imperative programming languages safer by only partially removing implicit side effects. Leaving one kind of effect is often enough to simulate the very effect you just tried to remove. On the other hand, allowing effects to be "forgotten" in a pure language also causes mayhem in its own way. Unfortunately, there is no golden middle, and we are faced with a classic dichotomy: the curse of the excluded middle, which presents the choice of either (a) trying to tame effects using purity annotations, yet fully embracing the fact that your code is still fundamentally effectful; or (b) fully embracing purity by making all effects explicit in the type system and being pragmatic by introducing nonfunctions such as unsafePerformIO. The examples shown here are meant to convince language designers and developers to jump through the mirror and start looking more seriously at fundamentalist functional programming.'"
"The examples shown here are meant to convince language designers and developers to jump through the mirror and start looking more seriously at fundamentalist functional programming."
Or, perhaps, to acknowledge that it's very hard to do anything useful without side effects.
You can write beautiful, elegant, purely functional code, as long as it doesn't have to touch a storage system, a network, or a user. But, hey, other than that, it's great!
to interact with an imperfect world one needs monads. to have monads is to compromise functional programming. ipso-facto-quod-splut: i always did rarther fancy Fortran. (hsst: don't tell anyone, but Forth is the -only- way to go, (and by 'go' i don't mean "Go" (or "Dart")))
If your goal is "functional" and you wind up at "mostly functional"... aren't you admitting you've got a bug in your program?
With CloudI
I suspect that this was written by a computer because even though it is is grammatically correct it still seems like a bunch of technical words strung together.
After programming for 16 years, I finally realize I have no idea what I'm doing. I'm so glad these people are out there to point this out.
what does Bennett Haselton think about this topic?
I remember he used to lament the fact that we had to use computers to run programs, because they were always so impure. Hacked up with model-breaking input and output considerations. He loved APL. Had us write our programs as math equations and prove that they had no side effects. On paper. Step by step, like how elementary teachers used to have you write out long division. He was a computer scientist before they HAD computers, he'd point out.
To be fair, APL was a wonderful language, and perfect so long as you didn't want to actually /do/ anything.
Well, that's unfair. As long as you meant to do a certain type of thing, these languages work out fairly well. The issue is the old percent split issue you normally see with frameworks and libraries - by making it easy to do some percent, X, easily, you create a high barrier to performing the remaining percent, Y. The problem with adhering to pure functional languages is that Y is not only high, it's often the most common tasks. Iterating, direct input and output, multi-variable based behavior, a slew of what we'd call flow conditions - these are very hard to do in a pure functional language. The benefit you get is far outweighed by the fact that you could use C, or the non-functional aspects of OCaml, or some other so-called 'multi-paradigm' language to fix the problem in a fraction of the time, even with side-effect management.
Then, have you ever tried to maintain a complex functional program? There's no doubt you can implement those Y-items above. The problem is that it makes your code very specific and interrelated as you're forced to present a model that captures all the intended behaviors. It's a lot of work. Work that will then need to be repeated each time you need to make additional changes. Adding a mechanism to - for example - play a sound at the end of a processing job based on the status - that's a line of code in most languages. Not so in a functional language.
The problem here isn't the oft-cited 'Devs just have to think of things differently, and they'll see it's better.'. It's more basic. It's simple throughput. Functional languages might be a theoretical improvement, but they're a practical hindrance. That, in a nutshell, is why they're not in common use in a corporate environment, where "value" loses it's theoretical polish and is compared to hard metrics like time and cost for a given quality.
It does if you're a government contractor, apparently...
http://www.forbes.com/sites/je...
http://www.computerworld.com/s...
(and thousands more...)
I use function techniques even when I'm using Java or C. Writing functions that have no side effects is useful and achievable, even when the language doesn't strictly enforce it.
Furthermore it can be mixed with imperative, or even object, programming. It's a useful technique for minimizing bugs.
"First they came for the slanderers and i said nothing."
Removing side effects from the equation is only one advantage of functional programming. The solutions to many problems are far more succinctly written in terms of functions. That's why the reason why I like using the paradigm. "Mostly functional programming" gives you the flexibility of imperative programming with the powerful expressiveness of functional. Who said it was trying to make it safer?
This subject had us working in groups to code a structured editor in a functional language - we called this the "specification" but it was essentially a functional implementation of an editor with a few concessions to do the actual IO. We then "implemented" in Java. I learnt exactly 4 things from that course.
1) NEVER EVER try to build an editor in a structured language. Functional languages are a poor fit for procedural tasks.
2) Having built something in a functional language do not then try to translate directly to an object oriented language. Plenty will be lost in the translation (It's like trying to translate a book of jokes from one language to another, only worse) and you'll wind yourself up in a knot trying to define a subset of a functional language in an object oriented one. What you'll end up in the object oriented language will be an ugly inelegant unreadable mess!!!
3) The idea that you can build a provable system using a functional language is bat crap insane. In terms of proof it'll give you nothing that good unit tests wouldn't give you.
4) Spend a lot of time ensuring you work in groups where for the most part everyone pulls their own weight.
Since input and output are side effects, and externals like running time, or even whether a program is running could be used to signal output, the only Pure Functional program is such that it is undecidable whether or not the program is even running, or can be run. But maybe reality is is a simulation run by a functional program, and the only observers are inside the system, and there's no way to tell if reality is a simulation from inside the system. So studying Pure Functional programming is just cheesy metaphysics by another name.
I'm not an expect in functional programming, but I am an expert in other (object, etc) styles. While I appreciate the functional toolbox in languages such as Scala (which I use every day), I don't really see a way to do my day to day job in a purely functional way. Others have mentioned the I/O dilemma, but I think it goes deeper than that. Functional != Efficient for many of the tasks I perform, which are rather iterative. For many of my tasks, the overhead of the functional structures required are either much more memory intensive, or impose a run-time overhead that isn't acceptable. In the end, when what I have to do is move 300 fields from one data structure to another with edits, COBOL would be sufficient...
}#q NO CARRIER
The synopsis completely misses the qualification, made in the first sentence, that TFA is discussing "concurrency, parallelism (manycore), and, of course, Big Data". Purely functional programming eliminates some significant issues in this type of programming (while introducing its own set of limitations). Meijer's point is that mostly functional programming is not really better than imperative here
For other types of programming, mostly functional style (using multi-paradigm languages) can be very nice. At least that's my position.
I like to use a combination of procedural, functional, and OO. I find some things are better for different things. Restricting yourself from options doesn't necessarily make you a better programmer.
If you want to hear a good one that will make you cringe on side effects: When I write personal code for myself that no one else will touch, I use a globally mutable systems and I have no issues. Globally mutable systems are bad in teams if you're trying to track down who's code is causing the problem because side effects could happen anywhere! But when your a solo guy, you know right away.
God spoke to me
I'll break it down into Retardese for you.
Side effects are things that change the state of the program. We don't want to store the state of the program because it gives it a memory. If it is not memoryless, then it is difficult to reason about. For instance, the equation f(x)=2x+1 is memoryless, because it does not matter what was observed the last time you observed f(x). As such, we can reason very easily with this function, and as long as we always supply the same input, we always get the same output.
Now think what would happen if we have g(x)=h(x)*x+1, where h(x) returns the previous value that it was supplied with (assume that it returns 0 on the first call if you like). That complicates things greatly, because we now have to consider everything that has been called before. You evaluate g(10) then g(5) to get 51, then reset the environment and evaluate g(5) then g(5) to get 26. That means it's no longer a function and cannot be reasoned like it is one. This means that you cannot formally prove the code correct, but you have to use a debugger to hunt down things like some sort of code monkey. It's intolerable!
If Republicans are elected, expect fundamentalist programming to become mandatory.
I write lots of state machines that control external world gadgets, which input new results to my program to use to compute the next state. Consequently I can only wonder about this idea of removing side effects as: "WTF?"
That said, I tend to write modules so that they only write to globals within that module.
Writing an IIR filter without memory is a pretty funny idea.
My new one language to rule them all, Ruperelix, will solve all that, for 1 week only, until it's taken apart. As you rightly stated, computers are not paper.
This just a bunch of academic rambling that has nothing to do with the real world. Real world business software developers just don't talk that way. They could care less if the language they use is functional or not and probably don't know. They just want to know how to implement a business requirement. If something doesn't work it is a bug, not an effect.
Are you gay?
Am I the only person whose first thought on reading the headline was that Erik Meijer (of all people) should know that the law of the excluded middle is not a theorem in the type theories that he advocates?
sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
Sorry, I'm taken.
I support this topic,
Bennett.
So Erik Meijer is arguing that mostly functional programming which is widely used in industry "does not work" and suggest instead purely functional programs which have been around for decades and gone nowhere?
Through the looking glass indeed.
From TFA:
mostly secure does not work
Spoken like a true academic. Mostly secure does work in practice. My house is mostly secure, my car is mostly secure, my bank is mostly secure. None of them are perfectly secure, as all of them would fail to a sufficiently strong attack, but generally they do fine.
So does mostly functional programming. It works great in practice even though it is not 100% safe but neither is functional programming once you allow monads which are needed to make FP Turing complete.
It's frustrating. Functional programming is painful when you actually have to do something, not just compute some result. But the real problem is older. We never got concurrency right in imperative languages.
Classic pthread-type concurrency suffers from the problem that the language has no idea what's locked by a lock. This problem is in C, wasn't fixed in C++, and isn't even fixed properly in Go. It was addressed more seriously in Modula and Ada, where the language knew which variables where shared and which were not. The Ada rendezvous approach was too limiting for anything otther than hard real-time, but it was on the right track.
Java addressed this with synchronized objects. This was a step in the right direction. The basic concept of a synchronized object is that, when executing a method of the object, nothing else can affect the state of the object. Java's synchronized objects don't quite get that right - you can call out of an object, then back into it, from within the same thread. This can break the object's invariant, in that the callback function is entered while the object is not in its stable, nobody-inside state. This is a classic cause of trouble in GUI systems, which involve lots of objects calling each other through dynamically changing collections. (If some unusual order of clicks crashes a program, there's a good chance the bug is of this type.)
The inside/outside issue for state protected by locks is a big one. This also comes up when a thread blocks. Many programs have sections where a thread unlocks a lock, blocks, then relocks the lock. This constitutes control leaving the block, but the compiler doesn't understand this. There's no syntax that says "I am now leaving this object to wait", with the language checks to insure that no internal object state gets passed to the code outside the object. The Spec# group at Microsoft (Spec# is a proof of correctness project using a form of C#) attacked this problem, and came up with a solution of sorts, but it never went mainstream. It's hard to fix this with a language bolt-on.
Objects ought to be either immutable, synchronized, or part of something that's synchronized. Then you're safe from low level race conditions. (You can still deadlock. However, deadlock bugs tend to be detectable and repeatable, unlike race condition bugs. So they get caught and fixed.) if this is built into the language, the compiler can check and optimize. Compilers are good at catching things like a local variable being passed to something that might save a reference to it and mess with it concurrently. Humans suck at that. Machines are good at global analysis of big data.
I had great hopes that the Go crowd would have a solution. They claim to, but there's a lot of hand-waving. They claim "share by communicating, not by sharing memory", but the examples in "Effective Go" all share memory. It's also really easy to share memory between goroutines in Go inadvertantly, because slices and dicts are reference objects. Pass them through a pipe and you've shared data and can have race conditions. The problem is bad enough that Google AppEngine limits Go programs to one thread.
Mixed functional/imperative programming has all these problems, plus the illusion that the problem has been solved. It hasn't.
Functional programming has gotten so much buzz the past few years, despite the fact that it's as old as math. It didn't take me too long to realize that any real computer is going to have memory, which is to say... it's going to have STATE. Furthermore, much of the world is stateful. If you don't model state directly in the language, you just model it in some klutzy way. Yeah, let's tie the brains of new users in knots making the comprehend monads so we can do... what any old imperetive language does.
I am a low level programmer, and found this paper quite hard to grasp. In my opinion it terribly fails on the vulgarization front.
I gave up after I understood the problem to fight was that there were state in objects. I understand removing state makes the program simpler (and easier to mathematically prove), but I suspect this brings false assumptions, since the underlying environment (OS, computer, network) is full of hidden states. For instance a memory allocator may succeed or fail regardless of the parameter it was called with.
Twenty years ago a CS grad student and fellow at the research center that I worked at, told me that he was certain that after object orientation the next big thing would turn out to be functional programming.
I thought that made perfect sense and bought into the notion. So I am still waiting.
The other way to make code safer, of course, is to eliminate the programmers.
You would think so, but as a programmer I can assure you that over time code changes itself.
No way *I* wrote that...
"There is more worth loving than we have strength to love." - Brian Jay Stanley
There is no programming language that is both useful and can ensure there are no bugs, so you'll have to excuse those of us who think it looks rather like fighting windmills.
This might be the first time ever I have seen someone use 'excluded middle' in a true to the definition ironic sense.
The article just notes that the hybrid approach doesn't magically address the problems in concurrent and parallel programming that other have claimed that they do. No where in the article does the author say that these approaches aren't useful. And I agree, you do need that purity to get those advantages for concurrent programming.
But, purity comes at a cost. Monads add a order of magnitude of complexity compared to imperative models of I/O. The author notes that other approaches to the problem of side effects are complex that "one shouldn't need a PhD in computer science to code", but I think he fails to notice that monadic I/O is also incredibly complicated to the average programmer as well. It does end becoming a domain specific language very quickly, and that can be a big hammer for a very small problem.
But, the fundamental argument is sound. If you really want those gains that pure functional languages can bring to concurrency, you have to embrace that model completely.
However, I don't buy the argument that other approaches won't have merit. I found this paper (http://research.microsoft.com/apps/pubs/default.aspx?id=170528/) to show that type extensions really may have some promise in augmenting imperative OO languages for concurrency.
Haven't these guys ever heard this saying? I have never seen a more comprehensive and obvious example of letting the perfect be the enemy of the good.
I remember the "OOP everywhere" fad. Turned out OOP works well in some parts of applications but not others. Functional Programming probably has a similar profile in that it's a nice tool to have for certain applications or parts of applications, but that doesn't mean it should be everywhere. The skill is in knowing when to use it and when not to.
Table-ized A.I.
I am happy this absolutist is talking computer languages and not means of production - else a bunch of us would be facing a purge.
I'm too lazy to evaluate the arguments.
Well, in the end even the languages itself have to be created using lowlevel languages which don't have all the lazy crap languages have been bloated with these days..
So it all depends on which level you write your application.
And there are many ways to do stuff, there is no one right way, and 'good' code is all in the eye of the beholder..
for programming, is that it solves nothing.
Let us imagine for example web servers. Any http daemon should be easy for a single C/C++ programmer to implement.Yes the more all the bells and whistles will take a lot more work, but something functional. --no problem. These things run massively parallel yet have no problems? Why. There is no need for the threads to communicate.
Where do web servers have conncurency problems?
Answer, when the execute processes that access shared data For example order processing programs that access the same database. Then all the concurrency problems come back.
Generally shared data causes enough problems that a fairly intelligent programmer avoids sharing data unless absolutely necessary, like the order processing system. So that any shared data in a concurrent system written by intelligent people is necessary. What's more the machinery required by functional programming requires that the programmer be of average intelligence. So functional programming most likely solves a problem that isn't there.
To top it off, the part of coding where functional programming is best used, is precisely the easier part of any programming problem. So all the extra effort in functional programming is spent solving the easiest part of the problem which is the part that doesn't need it.
Just out of interest: have you seen the wages for good FP programmers recently? (Disclaimer: I am one.) If not, you might want to take a look. The only reason for FP fanboys to "get defensive" these days is when all you rabble start using the word "monad" correctly! :-)
This is just wrong. Care to explain how you think a monad "compromises" a functional program?
I'd suggest that anyone who believes you on this needs to spend a lot more time on simple monads like Maybe / Option, before tackling things like IO and ST. You've obviously got the wrong end of the (non-)burrito. :-)
Well, to be frank, a lack of knowledge and experience sounds like the problem. Imagine hiring a bunch of graphic designers to write C code. To me, that's only slightly more insane than hiring people who only have experience with imperative, mutable languages to write code in a functional language. It's really a different skill set. There are obviously some overlaps, but shifting over to FP often requires substantially more effort than moving between the various imperative languages. Acknowledging the extra work required to "pick up" true FP thinking is part of the problem it faces in the broader industry.
How about when you pass your code to someone else, or they pass theirs to you? Sure, YOU know which functions are pure, but how could anyone else possibly know that without reading either all the code or trusting the comments?
What you say is true, of course, and it's a key theme in many books like "Effective Java". However, it's no substitute for a pure, functional language.
Paul Phillips has lots of good points, but after watching his recent presentations, I worry far more about the health of Paul Phillips than about the health of the Scala ecosystem.
Seriously, I invite you actually to take a look at the Scala compiler next to something like gcc or clang and tell me, with a straight face, that it's a "fucking mess". The reality is that it's a thing of refined beauty by comparison, and improving all the time. Really, look for yourself!
Functional programming for mere mortals coming soon. Erlang tamed for the new century: http://elixir-lang.org/ Listen here http://programming.oreilly.com... and here elixir computer language
Artificial intelligence is the study of how to make real computers act like the ones in the movies.
An IIR filter is easy, using the Direct Form II Transposed Structure. You just maintain the current state space (z vector) as part of your evaluation:
(y, z') = iir(x, z)
What's difficult and / or funny about that? It's a direct transcription of the equations!
You can wrap it in an ST monad if you want to evaluate the filter over a signal without having to allocate a new z vector for each sample. I ran a benchmark a few years ago and found that an ST version in Haskell was faster than Matlab's shipping implementation... MUCH more readable too!
That link is giving me a blank page :( What's the name of the paper?
So "Mostly functional programming is unfeasable"? Oh, really? No shit.
There's a name for 'mostly funcitonal programming' - it's called 'I-just-started-with-programming-and-Basic imperative spagetti code'.
There is one situation were functional programming makes sense, and that is when you're not sure which segment of which procedure will come first, either because you can't wrap your head around it due to the complexity of the domain you're just programming your way into or because you really can't know. UI state and workflow procedure is one of those things. It's basically information hiding when building and tying up complex interdependant procedures, and functional programming is the intelligent hack to deal with that. Well-built Spreadsheets of course being *the* classic example of that sort of thing.
Doing functional programming outside of its domain, like, for instance, modelling a business process or a gameworld, is not only counter-productive, it's flat out stupid/bad software development.
So, yeah, doing everything functional is unfeasable. Thanks for the news pal.
That being said, every programmer should look into functional programming and know when to apply it. Switching your mind to functional mode at the right time is a skill that can save a programmer lots of headaches. Quite litteraly actually.
My 2 cents.
We suffer more in our imagination than in reality. - Seneca
The problem with pure functional programming is you get abominations like Erlang (the only language besides Intercal to have a COME FROM statement - code mysteriously starts executing in some other place for no obvious reason), and hacks to work around the limitations of functional programming. Pure functional programming is so mind-bending that it's hard for humans to do. A little functional programming here and there with a closure or anonymous class helps make programs cleaner, but really most programming is easier and more straightforward in a sequential way. Same with asynchronous programming - sometimes you need to do one thing before something else, and if you can't, the result is a huge mess of hacks and pages of checks to see if objects are null because they may not be available. Forcing one style doesn't work. Purists can say what they want, but programmers have voted for hybrid languages like Java to get work done.
Why are you talking about tomorrow? There are many FP jobs available today (disclaimer: I work in FP), and they pay VERY well. "Flying cars" suggests that we're still waiting on a demand for FP to materialise, which is a bit nuts if you ask me. If you've got good FP skills, you're in demand RIGHT NOW!
Dude did you smoke the mushrooms then drink the Kool-aid?
I only look human.
My mother is a halfling and my dad is an ogre, so that makes me an Ogreling
Mostly functional. Side-effect tracing. This is the border where the imperative and the functional styles meet. And on the border, everything lies in the details of the tools and the problems.
Programs in languages like OCaml are predominantly of a functional style. This gives some of the benefits of functoinal code for that part of the program that can be clearly expressed so. But OCaml does have time-varying variables, and if you use them you can be exposed to the problems therewith. If you go concurrent, you may have race conditions.
The everyday solution to this is careful coding. When that fails, you'll have bugs.
There are less common ways of dealing with these bugs. The main one is formal analysis of the critical section structure of the code. Modula 3, (an imperative-style language) had a sophisticated analysis tool for this once upon a time, but that tool is lost to the ages. There are comments in the Modula 3 code of the Trestle user-interface system that serve as archaeological rellics of this analysis.
And there are starting to be languages with elaborate side-effect declaration systems that mitigate the problem. I've been pointed to the language Idris as an example. It remains to be seen to what extend this mitigates the issue, but it is plainly a next step in formal ways of dealing with these issues.
But it remains the case that the majority of code today is written in technically obsolete languages. And it looks like the situation will continue for a long, long time, because knowledge of the better ways to do things spreads more slowly than the knowledge how to do things at all. The extremist fundamentalist functionalists are only frightening ordinary folks from even trying functional techniques when those are appropriate. It may seem like an all-or-nothing excluded-middle question if you are trying to reach perfection, but perfection is probably not reachable with practical means anyway.
There's nothing wrong with following a middle way in the real world. There's everything wrong with sticking with purely imperative code in the real world. Or for imposing purely functional code on all real-world problems,
Maybe the situation will change with improved real-world technology in the future, but it's lot helpful to take the attitude if "If you're not with us, you're against us".
Tail recursion is just a different way to spell iteration, and a compiler should understand that.
Unless the language provides security semantics built around inspecting the call stack to see what I/O capabilities are available to a method. For example, a particular function might be allowed to open network connections only to the hostname and port that the caller (or the caller's caller, etc.) has authorized. When I last looked into why Java didn't support O(1)-memory tail calls, ability to check whether or not to throw a SecurityException for I/O was a large part of the reason. Each tail call thus potentially adds another element to the list of security contexts that must be checked.
And there's the rub: no computer is Turing complete. Physical computers have limited memory, making them equivalent to linear bounded automata. Storing every brush stroke in a paint program adds up.
The synopsis completely misses the qualification, made in the first sentence, that TFA is discussing "concurrency, parallelism (manycore), and, of course, Big Data".
Apart from "Big Data", that isn't much of a "qualification" anymore. Since CPU clock speeds reached 2 GHz years ago, speed as expressed in operations per second per thread has more or less stagnated in favor of packing multiple cores onto a die. Another trend has been to use slower cores in a device so that the CPU draws less current from the battery, letting the computation complete without having to recharge the battery halfway through. This is why "concurrency [and] parallelism (manycore)" will become even more important.
When I write personal code for myself that no one else will touch, I use a globally mutable systems and I have no issues. Globally mutable systems are bad in teams if you're trying to track down who's code is causing the problem because side effects could happen anywhere! But when your a solo guy, you know right away.
Not necessarily. Practically, you now and you in six months behave almost as if separate people. So if you're going to maintain your program longer than a few weeks, you're going to need to comment what can modify what and make sure to keep the comments updated.
The Tao gave birth to machine language. Machine language gave birth to the assembler.
The assembler gave birth to the compiler. Now there are ten thousand languages.
Each language has its purpose, however humble. Each language expresses the Yin and Yang of software. Each language has its place within the Tao.
But do not program in Java if you can avoid it.
Socialism: a lie told by totalitarians and believed by fools.
That's why isolating state and lazy is so awesome in FP. Algorithms that open network connections are isolated from algorithms that want to act on a network connection. So if you have a loop of 10,000 elements that need to open 50 network connections the 50 connections calls get grouped:
List of stuff to send to A
List of stuff to send to B
List of stuff to send to C
the actual opening to A, B and C end up in another module which would have security and which doesn't have the looping structure.
That is, IF the programmers and administration are willing to spend the money and take the time to bulletproof the code. to do so, would take at least a second team to do nothing but debug the code as in FDA validation. Every step had to be proven and probably cost as much or more than the original programming. This is far from what hey required for the ACA, Had we tried to pass off that kind of code, the fines and jail sentences would have been huge. The Govt violated their own FDA requirements for rolling out that code,