Yeah, I think that was mentioned in Luke Hoban's talk on ECMAScript at Lang.NEXT (if not, I saw it elsewhere that I can't recall). That would certainly help and even make calling LAPACK possible in JavaScript, but I still think it wouldn't be much fun. It would be pretty cool though:-)
Yeah, at some point if you need to put code on something like an embedded processor or DSP you just got to get down and dirty with C code and probably some ASM. But there's still some room in the middle for less pain with good performance:-)
Dynamic typing doesn't add any overhead when you can determine which specific method you need when generating code — which, in a dynamic language with a JIT, is very late, meaning that you can most of the time. Julia uses tons of small method definitions that call other small methods and so on, even for basic things like adding two integers, but the compiler is smart enough to compile addition into a single machine instruction. The notion that dynamic languages are slow because of their dynamism is very outdated in light of modern compiler techniques.
The C/C++ benchmarks are intentionally written in C; the only reason that's it's a C++ files instead of C is so that we can use C++'s complex template in the Mandelbrot benchmark. Otherwise the whole thing would just be done in C. The clock_now function is only used to time other code, so its performance is irrelevant.
You can't do this just yet, but we're working on it. Should be possible in the near future. At some point further into the future, you'll be able to compile Julia code into a.so file, load it from C code and just call it as though it were written in C —except that the person writing the code gets the benefits of a high-level numerical language.
That's a fair question, and one I asked myself after writing the benchmark code for JavaScript and seeing just how incredibly fast the V8 engine is. Should we be doing numerical computing in JavaScript? The biggest problem that I can see is that JavaScript doesn't have a good story for calling external C/Fortran libraries. Some sort of FFI could be implemented, but there are deeper issues —especially the paucity if numerical types in JavaScript. JavaScript doesn't even have integers —every number is a double. So how can you distinguish between an array of integers versus an array of doubles that happen to have integer values? Can the compiler be smart enough to know that it can store those numerical arrays inline in a format that can be passed to LAPACK? To do numerical computing, you really need more control over memory layout, at the very least for the sake of calling external libraries.
As I mentioned in the interview, we're working on a compiler, at which point you would even be able to use compiled Julia code in embedded systems. So you get a nice productive, interactive development environment, then you invoke the static compiler and presto! you have a compiled.so files that you can just call from C. That eliminates the need to prototype in one language and then re-write everything in another language when you want to actually deploy it. I've done that before and it's deeply annoying, time-consuming, and hard to get right (it's generally harder to write correct C code than correct Matlab code). I've also deployed Matlab using their compiler. That's workable if you want to avoid re-writing your prototype, but it's also pretty annoying.
1. Julia is entirely dynamic, so there's no need to think about compile time versus run time, simplifying the mental model (but the performance is like that of compiled languages). It's as easy as Python or Matlab in that respect, but much faster. 2. There are just a few powerful language features (e.g. ubiquitous, fast multiple dispatch, supported with an expressive type system), rather than a lot of features that interact in complicated ways. 3. Good for general programming stuff: working with strings, calling external programs and other things that are generally pretty awkward in R and Matlab (one of the reasons why NumPy is gaining popularity).
In general, the motivation (expressed in a previous Julia blog post) is to have something that's easy to use and learn, but fast and powerful (you *can* go deep if you want to), and designed expressly for numerical work —which means, among other things, that it has to be able to store large arrays of numeric values in-line and call libraries like LAPACK on them.
It's difficult to fathom how the authors interpret the data on page 14 as *not* supporting the hypothesis that there is a male/female variance ratio of about 1.1. Figure 1A is a bell-like curve which is clearly centered around 1.1. In Figure 1B, almost all of the points are below the 1:1 line, whereas if you plot a 1.1:1 line, its a perfect fit for the data. In Figure 1C, the x value where the regression line intersects a zero gender gap (i.e. no evidence of cultural bias), is at a variance ratio of about 1.1. All of the evidence the authors present points to an underlying variance ratio near 1.1, yet somehow they conclude the opposite.
For a fairly mature project like wikipedia (everybody knows about them, they have more pagerank than god, ignorance is unlikely to be the reason behind most non-contributors), focusing on anomalies in your contributor statistics is a good way of identifying potential issues that might be standing in the way of your growth.
I'm pretty sure that god is not the gold standard when it comes to PageRank. Yup: god — top hit, wikipedia.
computer science beach reading
on
Beautiful Data
·
· Score: 5, Informative
I'm a data scientist at company that is big enough to have a six-person data science team. Our CTO bought the data science team all copies of this book (mine is sitting on my desk right now). The best thing about this book is the cover. Which is not to say that the book is terrible —the cover is a really pretty picture of a kiwi. The only chapter in this book that was really interesting was Chapter 5, "Information Platforms and the Rise of the Data Scientist", by Jeff Hammerbacher, who edited the book. The rest is pretty fluffy. Nice easy reading, but nothing really useful or all that interesting.
Very true. On the seeding issue, note that even for events with a substantial amount of randomness inherent, like key presses or other I/O events, the high-order bits are still quite predictable. For example, the intervals between keyboard events tend to follow a Pareto distribution (power law basically). That's why the kernel only uses the very lowest few bits of data from these sources to generate entropy.
I may be wrong, but I believe that the difference between/dev/random and/dev/urandom is that the former only provides true hardware-generated entropy and blocks when that's not available, whereas the latter uses true entropy and cryptographically strong PRNG to generate stream of data that has a variable level of true entropy mixed into it over time. Can anyone corroborate that and/or refute that? (References a plus!)
I'm not concerned that it's not pseudo-random. My point is that's not how any modern pseudo-random number generators actually work. Maybe during WWII, but not today. The common PRNG techniques are:
linear congruential generators
lagged Fibonacci generators
linear feedback shift registers
generalised feedback shift registers
Mersenne twister
Fortuna (if you need one that's cryptographically secure)
Blum Blum Shub (likewise)
These are all pure mathematical algorithms. Nowhere in any of these is there any sort of pre-generated random lookup tables. (Unless you count the S-boxes used in some block ciphers with Fortuna.) Pre-generated "random" lookup tables only hide poor randomness in the generation process and don't actually improve the situation cryptographically at all; I suspect that for most other applications there would be problems as well. If your generated numbers don't cover the entire domain space uniformly, then they still won't no matter how many lookup tables you use to transform them.
According to the article, people are sitting around rolling dice to generate random number sequences. Really? REALLY?!? Who wrote this article?
The intuitive disconnect here is that humans have a well-documented inability to understand or true randomness. By true randomness, I mean in a mathematical sense: uniformly distributed values over some range, with each value independent from the next (uncorrelated). Just try it: whatever you come up with, whether it be some algorithm, hardware, whatever — it will probably fail all the statistical tests for true randomness. And if it can't pass those tests, then it will be useless for most of the applications that one needs random things for... simulations, encryption, authentication, etc. And mind you, even if one can pass all the existing tests with some method, it doesn't mean that someone will invent a new statistical test tomorrow that the method will fail spectacularly. Like many apparently simple problems, random number generation is surprisingly deep and very, very difficult to do right.
Has anyone noticed the bizzaro definition of pseudo-random in the article:
Ordinary random number generators found in most computers in use today are 'pseudo-random' numbers that use various algorithms to pick the numbers from large pre-compiled databases of numbers obtained by methods such as rolling the dice. Um, that's not how any pseudo-random number generator I know of works...
Already done and fully deployed: HTTP 1.1 includes persistent HTTP connections and pipelining of requests within the same TCP session. This means that the common conception of how HTTP actually functions—i.e. one TCP setup/teardown per HTTP request—is just incorrect. Of course, this depends on both ends of the connections supporting persistent connections and having the option to use them turned on.
I'm all for regression testing. Refusing to regression test, and insisting on using only debugging would be silly. The converse, however — refusing to use a debugger, and insisting on using only regression testing — is just as silly. Debuggers are extremely useful tools that are very appropriate in many programming situations. Any professional programmer who flat out refuses to use a debugger — and moreover derides those who do — is someone whose general views on programming I would find suspect.
It's not exactly what you want, but "c " will continue until hitting the line number given. Combined with "l" to list the code in the lines after the current line, this allows you get the effect you want. Another useful trick is setting conditional breakpoints: the syntax is "b ". As always, the condition can be arbitrary perl code.
[B/v/L]
Re:Obsolete before printing?
on
Pro Perl Debugging
·
· Score: 2, Insightful
I've been a serious Perl programmer for about eight years, and I never use an IDE. I always do my debugging at the terminal. [I do use a syntax highlighting editor.] I understand that a lot of people like IDEs, but I find them annoying and hampering. I'd prefer to have a text mode debugger pipe a large data structure to less and let me use / and ? to find various things in the pager. I haven't found an IDE yet that lets you do that easily. This is a very useful technique--especially in the perl debugger, where you can run arbitrary perl code at any point in the program and pipe the output to a pager.
Moveover, you're assuming that you can run an IDE in your development environment. I often have to work on deployed server systems that have neither a windowing system not an IDE installed. However, I can always run the perl debugger and interact with it in an SSH session.
Slashdot Mirror
Yeah, I think that was mentioned in Luke Hoban's talk on ECMAScript at Lang.NEXT (if not, I saw it elsewhere that I can't recall). That would certainly help and even make calling LAPACK possible in JavaScript, but I still think it wouldn't be much fun. It would be pretty cool though :-)
Yeah, at some point if you need to put code on something like an embedded processor or DSP you just got to get down and dirty with C code and probably some ASM. But there's still some room in the middle for less pain with good performance :-)
Dynamic typing doesn't add any overhead when you can determine which specific method you need when generating code — which, in a dynamic language with a JIT, is very late, meaning that you can most of the time. Julia uses tons of small method definitions that call other small methods and so on, even for basic things like adding two integers, but the compiler is smart enough to compile addition into a single machine instruction. The notion that dynamic languages are slow because of their dynamism is very outdated in light of modern compiler techniques.
The C/C++ benchmarks are intentionally written in C; the only reason that's it's a C++ files instead of C is so that we can use C++'s complex template in the Mandelbrot benchmark. Otherwise the whole thing would just be done in C. The clock_now function is only used to time other code, so its performance is irrelevant.
Stefan Karpinski
I didn't write the title the interview article. It's definitely inaccurate since C is already the C of scientific computing.
Stefan Karpinski
You can't do this just yet, but we're working on it. Should be possible in the near future. At some point further into the future, you'll be able to compile Julia code into a .so file, load it from C code and just call it as though it were written in C —except that the person writing the code gets the benefits of a high-level numerical language.
Stefan Karpinski
That's a fair question, and one I asked myself after writing the benchmark code for JavaScript and seeing just how incredibly fast the V8 engine is. Should we be doing numerical computing in JavaScript? The biggest problem that I can see is that JavaScript doesn't have a good story for calling external C/Fortran libraries. Some sort of FFI could be implemented, but there are deeper issues —especially the paucity if numerical types in JavaScript. JavaScript doesn't even have integers —every number is a double. So how can you distinguish between an array of integers versus an array of doubles that happen to have integer values? Can the compiler be smart enough to know that it can store those numerical arrays inline in a format that can be passed to LAPACK? To do numerical computing, you really need more control over memory layout, at the very least for the sake of calling external libraries.
Stefan Karpinski
As I mentioned in the interview, we're working on a compiler, at which point you would even be able to use compiled Julia code in embedded systems. So you get a nice productive, interactive development environment, then you invoke the static compiler and presto! you have a compiled .so files that you can just call from C. That eliminates the need to prototype in one language and then re-write everything in another language when you want to actually deploy it. I've done that before and it's deeply annoying, time-consuming, and hard to get right (it's generally harder to write correct C code than correct Matlab code). I've also deployed Matlab using their compiler. That's workable if you want to avoid re-writing your prototype, but it's also pretty annoying.
Stefan Karpinski
There's a few points:
1. Julia is entirely dynamic, so there's no need to think about compile time versus run time, simplifying the mental model (but the performance is like that of compiled languages). It's as easy as Python or Matlab in that respect, but much faster.
2. There are just a few powerful language features (e.g. ubiquitous, fast multiple dispatch, supported with an expressive type system), rather than a lot of features that interact in complicated ways.
3. Good for general programming stuff: working with strings, calling external programs and other things that are generally pretty awkward in R and Matlab (one of the reasons why NumPy is gaining popularity).
In general, the motivation (expressed in a previous Julia blog post) is to have something that's easy to use and learn, but fast and powerful (you *can* go deep if you want to), and designed expressly for numerical work —which means, among other things, that it has to be able to store large arrays of numeric values in-line and call libraries like LAPACK on them.
Stefan Karpinski
It is under active development.
Karpinski does at the moment have a beard. – Karpinski
Evidence: http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012/Julia
It's difficult to fathom how the authors interpret the data on page 14 as *not* supporting the hypothesis that there is a male/female variance ratio of about 1.1. Figure 1A is a bell-like curve which is clearly centered around 1.1. In Figure 1B, almost all of the points are below the 1:1 line, whereas if you plot a 1.1:1 line, its a perfect fit for the data. In Figure 1C, the x value where the regression line intersects a zero gender gap (i.e. no evidence of cultural bias), is at a variance ratio of about 1.1. All of the evidence the authors present points to an underlying variance ratio near 1.1, yet somehow they conclude the opposite.
For a fairly mature project like wikipedia (everybody knows about them, they have more pagerank than god, ignorance is unlikely to be the reason behind most non-contributors), focusing on anomalies in your contributor statistics is a good way of identifying potential issues that might be standing in the way of your growth.
I'm pretty sure that god is not the gold standard when it comes to PageRank. Yup: god — top hit, wikipedia.
;-)
I'm a data scientist at company that is big enough to have a six-person data science team. Our CTO bought the data science team all copies of this book (mine is sitting on my desk right now). The best thing about this book is the cover. Which is not to say that the book is terrible —the cover is a really pretty picture of a kiwi. The only chapter in this book that was really interesting was Chapter 5, "Information Platforms and the Rise of the Data Scientist", by Jeff Hammerbacher, who edited the book. The rest is pretty fluffy. Nice easy reading, but nothing really useful or all that interesting.
Mod up parent. I was going to point out the same thing. How the heck could this not be causation?
Very true. On the seeding issue, note that even for events with a substantial amount of randomness inherent, like key presses or other I/O events, the high-order bits are still quite predictable. For example, the intervals between keyboard events tend to follow a Pareto distribution (power law basically). That's why the kernel only uses the very lowest few bits of data from these sources to generate entropy.
/dev/random and /dev/urandom is that the former only provides true hardware-generated entropy and blocks when that's not available, whereas the latter uses true entropy and cryptographically strong PRNG to generate stream of data that has a variable level of true entropy mixed into it over time. Can anyone corroborate that and/or refute that? (References a plus!)
I may be wrong, but I believe that the difference between
[B.v.L]
These are all pure mathematical algorithms. Nowhere in any of these is there any sort of pre-generated random lookup tables. (Unless you count the S-boxes used in some block ciphers with Fortuna.) Pre-generated "random" lookup tables only hide poor randomness in the generation process and don't actually improve the situation cryptographically at all; I suspect that for most other applications there would be problems as well. If your generated numbers don't cover the entire domain space uniformly, then they still won't no matter how many lookup tables you use to transform them.
According to the article, people are sitting around rolling dice to generate random number sequences. Really? REALLY?!? Who wrote this article?
[BvL]
The intuitive disconnect here is that humans have a well-documented inability to understand or true randomness. By true randomness, I mean in a mathematical sense: uniformly distributed values over some range, with each value independent from the next (uncorrelated). Just try it: whatever you come up with, whether it be some algorithm, hardware, whatever — it will probably fail all the statistical tests for true randomness. And if it can't pass those tests, then it will be useless for most of the applications that one needs random things for... simulations, encryption, authentication, etc. And mind you, even if one can pass all the existing tests with some method, it doesn't mean that someone will invent a new statistical test tomorrow that the method will fail spectacularly. Like many apparently simple problems, random number generation is surprisingly deep and very, very difficult to do right.
[B.v.L]
[B.v.L]
Already done and fully deployed: HTTP 1.1 includes persistent HTTP connections and pipelining of requests within the same TCP session. This means that the common conception of how HTTP actually functions—i.e. one TCP setup/teardown per HTTP request—is just incorrect. Of course, this depends on both ends of the connections supporting persistent connections and having the option to use them turned on.
[B.v.L]
I'm all for regression testing. Refusing to regression test, and insisting on using only debugging would be silly. The converse, however — refusing to use a debugger, and insisting on using only regression testing — is just as silly. Debuggers are extremely useful tools that are very appropriate in many programming situations. Any professional programmer who flat out refuses to use a debugger — and moreover derides those who do — is someone whose general views on programming I would find suspect.
[B/v/L]
Even shorter is "perl -de0". Actually, I do this so much that I have it aliased to "perld".
[B/v/L]
It's not exactly what you want, but "c " will continue until hitting the line number given. Combined with "l" to list the code in the lines after the current line, this allows you get the effect you want. Another useful trick is setting conditional breakpoints: the syntax is "b ". As always, the condition can be arbitrary perl code.
[B/v/L]
I've been a serious Perl programmer for about eight years, and I never use an IDE. I always do my debugging at the terminal. [I do use a syntax highlighting editor.] I understand that a lot of people like IDEs, but I find them annoying and hampering. I'd prefer to have a text mode debugger pipe a large data structure to less and let me use / and ? to find various things in the pager. I haven't found an IDE yet that lets you do that easily. This is a very useful technique--especially in the perl debugger, where you can run arbitrary perl code at any point in the program and pipe the output to a pager.
Moveover, you're assuming that you can run an IDE in your development environment. I often have to work on deployed server systems that have neither a windowing system not an IDE installed. However, I can always run the perl debugger and interact with it in an SSH session.
[B/v/L]