That's not planned for the moment, but with PreferenceStats.jsm (currently in Firefox Nightly), it is already possible to write an add-on that does monitors each tab.
Firefox doesn't have a feature for stopping all scripts in a page yet (that's https://bugzilla.mozilla.org/s..., if you're curious), but reloading the tab without scripts (or other features) shouldn't be too hard.
Yep, I'm working on it.
Firefox Nightly should have a usable display of CPU by the end of August. Memory usage is planned, but I have many other things to do first.
Am I understanding correctly that you have reimplemented threads on an application level, instead of the OS level? So your application's threads (fibers, whatever) can be pre-empted by the OS, because the OS doesn't care about your scheduler when it's handling its own processes?
This is absolutely correct. And, if you check, this is what any system that wants to be able to manage tens of thousands of threads does (not necessarily with the exact same techniques). Including all major databases, all large scale virtual machines, etc. Recent versions of MacOS X even provide system libraries to simplify the implementation of such user-level threading (under the name of Grand Central Dispatch).
Am I also understanding correctly that there's no priority system for your threading implementation?
Actually, there is one. It is just not exported to userland yet.
Ah, I think I finally understand your point. It is about compatibility, not about paradigm, is it?
If so, you may be happy to learn that the low-level layers of Opa can already access other databases. The feature is not accessible at high-level yet because we are going to make a number of breaking changes before long, but eventually, we do intend to give access to most mainstream databases. Would this solve your problem?
Actually, it's a graph database and one of the most powerful database engines I have seen on the net. With this engine, you store complete data structures, thinking in terms of references instead of keys – except when you want to think of keys, of course. What is the problem with that?
Good point. Unfortunately, as the CSO, I have strictly no say in the license. If you wish to discuss it with someone who does, you should ping phy_si_cal, our CEO and the OP of this thread, or send an e-mail to Mathieu Baudet, our COO.
because html, javascript, and browsers are soooo robust and versatile
Good point. We had really hard times tweaking the compiler until it produced JavaScript that worked on all browsers (note: in our books, IE7 is not a browser), or html that worked in all browsers, and with linking with all mainstream JS libraries (note: thanks Facebook for the nice code provided, that explicitly depends on server misconfigurations and/or browser bugs and suddenly stops working when confronted to a standards-compliant web browser and a well-configured server), and with getting our Comet to work with all the different possible interpretations of "asynchronous".
But that is not the point. The point is that this is a problem that *every* non-trivial web application needs to face. The point is that we faced it and we beat it, so that Opa developers do not need to. Should a new browser version arise and cause new incompatibilities, *we* will be the ones doing the heavy debugging until we can produce a new version of the compiler and runtime that lets developers forget about the incompatibilities.
So yes, these brittle bases made our work difficult (and interesting:) ). But they also constitute an important part of the interest of Opa.
Caveat I'm part of the Opa team. Well, worse than that, I am the architect-in-chief.
Well, in my experience, most of Scheme (i.e. not call/cc) ports quite well to JavaScript and performs quite well once compiled to JS. Performing a CPS transform on JS cuts performances by roughly 50%, which is not *that* bad.
The problem is that you've just created a monolithic application that ties one single database to one single front-end. What happens when you want to swap out databases?[...] It's just another language that corporate developers not only need to learn, but need to sell to management [...] But as it is, it's an interesting idea, nothing more - I'd never let it in my development shop, and even if I did, the architectural committee would hang me out to dry.
I can understand the issue. Would this calm your concerns if I told you that
on the Opa git, there is an Opa version for Windows – it's not quite ready for prime-time, but it's there;
Opa is designed to be able to link with other server-side technologies – for the moment, OCaml and C, other technologies will arrive in time;
at low-level, the Opa database can access other, mainstream, database – the feature is not quite exported at high-level yet, because we are not quite satisfied with it yet, but in time, it will;
and of course, there is always the option of modularizing your code through the use of web services, which is a nice way of allowing [hot]swapping of components.
And "compiling to JavaScript" is just ignorant - there's no such thing. Maybe the author means "translating to JavaScript"?
Of course, what do I know about compilation? I only have a PhD on the topic and 9 years of professional experience.
Interesting question. Let's forget about the framework for the moment and concentrate on what the compiler should do:
analyze statically the whole code and determine what goes on the client and what goes on the server (I tend to believe that this is not feasible with a dynamic language, but of course, feel free to prove me wrong);
compile client-side code to JavaScript or to code that can be interpreted by a JS-based VM (lots of work, but no theoretical barrier, emscripten can even make your life somewhat easier at least for C++);
inject Ajax entry points on the server (no real problem there);
inject Comet re-entry points on the server (requires a server-side language that supports either CPS or tens of thousands of lightweight non-blocking threads – theoretically possible with custom implementation of the languages).
That's without security or distribution, without database, without a syntax for css or html, but it looks somewhat possible for some languages, with considerable effort. Say, if you attempt to do this, don't hesitate to ping me, it might be interesting.
Is there any reason that a language like Java, C#, C++, Python, Perl or Ruby, or with the appropriate framework and compiler couldn't do the same thing that Opa is aiming to do?
Frameworks definitely would not suffice. Reaching the level of features of Opa, even without security, requires quite complex static analysis that strictly eliminates Python, Perl or Ruby, and in practice, even C++. Now, it might just be possible with Java or C#, with non-trivial changes to the language and compiler.
On Hanselminutes, Scott Hanselman interviewed a guy who made the statement "Javascript is the machine language of the Internet", alluding to the fact that it is better from a productivity and performance perspective to develop in a lot of other languages (C# in this case) and have compilers that know how to create optimized, minimized, and cross browser javascript.
Unsurprisingly, we agree.
Caveat I am part of the Opa team. Well, worse than that, I am the architect-in-chief.
[...] experience from 15+ years of experienced developers writing scalable systems with Object Oriented Programming Languages, and concluding that OO is not the right paradigm for the task, and that other paradigms need to be hammered upon OO languages to make them scalable
More experience would tell you that these technologies evolved organically to meet a rapidly changing market need - thats why they were "bolted on" to also-emerging OO server-side platform. There was no attempt at central planning. A single cohesive OO approach could just be as scalable.
That's an interesting claim. Do you have any example to back it?
We decided to choose a paradigm not on its popularity, but rather on its suitability for the task of writing highly-scalable, highly-secure, highly-dynamic web applications. This paradigm is comparable to that of Erlang or Scala.
Uhhh the Java lesson, which like it or not is widely adopted demonstrates the opposite. Erlang and Scala at the moment remain in the realm of the hobbyist or bleeding edge experiment - Java saw rapid widespread adoption because it was so similar to the oh-so popular C++ - it both syntax and semantic terms.
Er... the opposite of what? Still, I think I understand your point – see a past discussion on my blog on the topic of syntax.
Good luck!...and change the license ouch!
Not my call, unfortunately. You may wish to convince phy_si_cal (our CEO) or Mathieu Baudet (or COO), though.
Well, they could sort of work, but with reduced UI and reduced features (no drag&drop, no disconnection warnings, no offline reading, no real-time chat, etc.). I do not believe that may people would like to return to this age.
Well, cedrics answer should help answer your question. I will try and add a few details.
Indeed, epoll et al. are based on polling and as such are non-preemptive. However, with proper language/runtime support, they can be coupled with a scheduler and turned effectively into lightweight preemptive multi-tasking. In Opa, the compiler inserts CPS breaks (i.e. opportunities for the scheduler to perform context-switching and/or to poll for the completion of lengthy I/O) wherever appropriate, with guarantees that any lengthy computation will be thus broken on a regular basis.
By opposition to pthreads/system threads, this mechanism lets us scale to millions of lightweight threads, all of them effectively executed concurrently. By opposition to the Node.js-style event-based approach, we are certain that no task will block and the style is more natural.
How does opa handle multiple threads of execution? Any decent server nowadays just needs to have multiple threads of execution, sometimes even thousands of them to work in a non-blocking way. Opa seems to be "non-blocking" (I read from the tutorial), but you have non-blocking and non-blocking... One version simply uses one thread and an event-loop (aka non-preemptive multitasking). This is not truly non-blocking, as large I/O operations in one task still prevent other tasks from working.
We have lightweight threads can be preempted any time they are not in (Opa) kernel mode, and rely on asynchronous I/O (epoll/kqueue/completion ports) to achieve non-blocking I/O. Does this answer your question?
And does opa allow us to control the priority of different threads?
Not at the moment.
Also, does Opa have support for fallback to the underlying systems (javascript/databases etc.)? If something is not supported by Opa (very likely in the beginning), I sure want to be able to fix it myself without having to understand all the opa internals.
I am not certain I understand the question. Does this chapter answer your question? There's the same thing server-side, of course.
Caveat I'm a member of the Opa team. Well, worse than that, I'm the architect-in-chief.
Well, for one line of C++ (or of one of the compiled-to-native dialects of Python, if you prefer), how many lines of ASM do you get? And is that so bad?
Slashdot Mirror
It can work both with and without Electrolysis. However, you are right that it will take time, so don't expect it before 42 (current Nightly) or 43.
That's not planned for the moment, but with PreferenceStats.jsm (currently in Firefox Nightly), it is already possible to write an add-on that does monitors each tab. Firefox doesn't have a feature for stopping all scripts in a page yet (that's https://bugzilla.mozilla.org/s..., if you're curious), but reloading the tab without scripts (or other features) shouldn't be too hard.
Yep, I'm working on it. Firefox Nightly should have a usable display of CPU by the end of August. Memory usage is planned, but I have many other things to do first.
Just a quick highlight: Firefox is open-source. Last time I checked, the only licensing difference between Firefox and Iceweasel concerned the logo.
I think that you are missing the obvious: it worked.
Am I understanding correctly that you have reimplemented threads on an application level, instead of the OS level? So your application's threads (fibers, whatever) can be pre-empted by the OS, because the OS doesn't care about your scheduler when it's handling its own processes?
This is absolutely correct. And, if you check, this is what any system that wants to be able to manage tens of thousands of threads does (not necessarily with the exact same techniques). Including all major databases, all large scale virtual machines, etc. Recent versions of MacOS X even provide system libraries to simplify the implementation of such user-level threading (under the name of Grand Central Dispatch).
Am I also understanding correctly that there's no priority system for your threading implementation?
Actually, there is one. It is just not exported to userland yet.
Ah, I think I finally understand your point. It is about compatibility, not about paradigm, is it?
If so, you may be happy to learn that the low-level layers of Opa can already access other databases. The feature is not accessible at high-level yet because we are going to make a number of breaking changes before long, but eventually, we do intend to give access to most mainstream databases. Would this solve your problem?
Self-promotion on Slashdot? Nah, this never happens.
Actually, it's a graph database and one of the most powerful database engines I have seen on the net. With this engine, you store complete data structures, thinking in terms of references instead of keys – except when you want to think of keys, of course. What is the problem with that?
Good point. Unfortunately, as the CSO, I have strictly no say in the license. If you wish to discuss it with someone who does, you should ping phy_si_cal, our CEO and the OP of this thread, or send an e-mail to Mathieu Baudet, our COO.
because html, javascript, and browsers are soooo robust and versatile
Good point. We had really hard times tweaking the compiler until it produced JavaScript that worked on all browsers (note: in our books, IE7 is not a browser), or html that worked in all browsers, and with linking with all mainstream JS libraries (note: thanks Facebook for the nice code provided, that explicitly depends on server misconfigurations and/or browser bugs and suddenly stops working when confronted to a standards-compliant web browser and a well-configured server), and with getting our Comet to work with all the different possible interpretations of "asynchronous".
But that is not the point. The point is that this is a problem that *every* non-trivial web application needs to face. The point is that we faced it and we beat it, so that Opa developers do not need to. Should a new browser version arise and cause new incompatibilities, *we* will be the ones doing the heavy debugging until we can produce a new version of the compiler and runtime that lets developers forget about the incompatibilities.
So yes, these brittle bases made our work difficult (and interesting :) ). But they also constitute an important part of the interest of Opa.
Caveat I'm part of the Opa team. Well, worse than that, I am the architect-in-chief.
Well, in my experience, most of Scheme (i.e. not call/cc) ports quite well to JavaScript and performs quite well once compiled to JS. Performing a CPS transform on JS cuts performances by roughly 50%, which is not *that* bad.
The problem is that you've just created a monolithic application that ties one single database to one single front-end. What happens when you want to swap out databases?[...] It's just another language that corporate developers not only need to learn, but need to sell to management [...] But as it is, it's an interesting idea, nothing more - I'd never let it in my development shop, and even if I did, the architectural committee would hang me out to dry.
I can understand the issue. Would this calm your concerns if I told you that
And "compiling to JavaScript" is just ignorant - there's no such thing. Maybe the author means "translating to JavaScript"?
Of course, what do I know about compilation? I only have a PhD on the topic and 9 years of professional experience.
Yeah, well, nobody in the team speaks Lithuanian :)
That's without security or distribution, without database, without a syntax for css or html, but it looks somewhat possible for some languages, with considerable effort. Say, if you attempt to do this, don't hesitate to ping me, it might be interesting.
Just for clarity: I'm David, the Chief Scientific Officer. Mathieu is the Chief Of Operations.
How is using Pepper different than ActiveX with Internet Explorer?
One word: sandbox.
Is there any reason that a language like Java, C#, C++, Python, Perl or Ruby, or with the appropriate framework and compiler couldn't do the same thing that Opa is aiming to do?
Frameworks definitely would not suffice. Reaching the level of features of Opa, even without security, requires quite complex static analysis that strictly eliminates Python, Perl or Ruby, and in practice, even C++. Now, it might just be possible with Java or C#, with non-trivial changes to the language and compiler.
On Hanselminutes, Scott Hanselman interviewed a guy who made the statement "Javascript is the machine language of the Internet", alluding to the fact that it is better from a productivity and performance perspective to develop in a lot of other languages (C# in this case) and have compilers that know how to create optimized, minimized, and cross browser javascript.
Unsurprisingly, we agree.
Caveat I am part of the Opa team. Well, worse than that, I am the architect-in-chief.
[...] experience from 15+ years of experienced developers writing scalable systems with Object Oriented Programming Languages, and concluding that OO is not the right paradigm for the task, and that other paradigms need to be hammered upon OO languages to make them scalable
More experience would tell you that these technologies evolved organically to meet a rapidly changing market need - thats why they were "bolted on" to also-emerging OO server-side platform. There was no attempt at central planning. A single cohesive OO approach could just be as scalable.
That's an interesting claim. Do you have any example to back it?
We decided to choose a paradigm not on its popularity, but rather on its suitability for the task of writing highly-scalable, highly-secure, highly-dynamic web applications. This paradigm is comparable to that of Erlang or Scala.
Uhhh the Java lesson, which like it or not is widely adopted demonstrates the opposite. Erlang and Scala at the moment remain in the realm of the hobbyist or bleeding edge experiment - Java saw rapid widespread adoption because it was so similar to the oh-so popular C++ - it both syntax and semantic terms.
Er... the opposite of what? Still, I think I understand your point – see a past discussion on my blog on the topic of syntax.
Good luck! ...and change the license ouch!
Not my call, unfortunately. You may wish to convince phy_si_cal (our CEO) or Mathieu Baudet (or COO), though.
Well, they could sort of work, but with reduced UI and reduced features (no drag&drop, no disconnection warnings, no offline reading, no real-time chat, etc.). I do not believe that may people would like to return to this age.
Well, cedrics answer should help answer your question. I will try and add a few details.
Indeed, epoll et al. are based on polling and as such are non-preemptive. However, with proper language/runtime support, they can be coupled with a scheduler and turned effectively into lightweight preemptive multi-tasking. In Opa, the compiler inserts CPS breaks (i.e. opportunities for the scheduler to perform context-switching and/or to poll for the completion of lengthy I/O) wherever appropriate, with guarantees that any lengthy computation will be thus broken on a regular basis.
By opposition to pthreads/system threads, this mechanism lets us scale to millions of lightweight threads, all of them effectively executed concurrently. By opposition to the Node.js-style event-based approach, we are certain that no task will block and the style is more natural.
Believe it or not, roughly half of the Opa team speaks German, at least one (me) speaks Argentine Spanish and at least one speaks Greek.
And guess what? We decided that the puns were funny.
How does opa handle multiple threads of execution? Any decent server nowadays just needs to have multiple threads of execution, sometimes even thousands of them to work in a non-blocking way. Opa seems to be "non-blocking" (I read from the tutorial), but you have non-blocking and non-blocking... One version simply uses one thread and an event-loop (aka non-preemptive multitasking). This is not truly non-blocking, as large I/O operations in one task still prevent other tasks from working.
We have lightweight threads can be preempted any time they are not in (Opa) kernel mode, and rely on asynchronous I/O (epoll/kqueue/completion ports) to achieve non-blocking I/O. Does this answer your question?
And does opa allow us to control the priority of different threads?
Not at the moment.
Also, does Opa have support for fallback to the underlying systems (javascript/databases etc.)? If something is not supported by Opa (very likely in the beginning), I sure want to be able to fix it myself without having to understand all the opa internals.
I am not certain I understand the question. Does this chapter answer your question? There's the same thing server-side, of course.
Caveat I'm a member of the Opa team. Well, worse than that, I'm the architect-in-chief.
Plus I've never cared for indentation defining blocks.
Well, neither does Opa.
(Note: Your other points are valid)
Caveat I'm a member of the Opa team. Well, worse than that, I'm the architect-in-chief.
Well, for one line of C++ (or of one of the compiled-to-native dialects of Python, if you prefer), how many lines of ASM do you get? And is that so bad?