Slashdot Mirror

Your post is long, so I'll only address a few snippets by lack of time.

Also, why do people keep making new languages with new syntax (generally with poor to missing error messages, debuggers, IDEs, documentation, and libraries) when we would so much more benefit from improved FOSS libraries for existing ones? I'd be a lot more excited about Tao if it was a JavaScript library that just supported some special format INI files.

The answer is complex. I try to address it in this article. It's not just for the sake of inventing a new language, it's because what I wanted to do I could not do with JavaScript.

Looking at a bit of the Tao overview video on SourceForge, it looks like variables don't need to be declared (ick!).

Variables need to be declared, but an assignment does declare a variable in that scope.

Also, requiring "locally" seems to imply that it has one of the worst "features/bugs" of JavaScript design for default globals? Or maybe I did not understand "locally".

The 'locally' function does not concern variables, but graphic state. It's a way to say "I don't want this rotation to escape this block". In OpenGL terminology, you can think of it as a PushMatrix/PopMatrix pair (and same for other attributes).

The main aspects of the language (like what is a code block, what is an argument) don't seem clear at first glance to me, perhaps because of various keywords being defined or seeing commas some places and not others?

I find the use of a comma without inner parentheses interesting for functions and arguments, where the comma in a sense is doing what Smalltalk keyword colons are doing. Still, it misses labelling arguments like Smalltalk, and why not drop the commas and just have all arguments separated by spaces and instead require nested expressions to be surrounded by parentheses if you are going in that direction? For example: "translate -500 100 (10 + x)"

If you define something like:

        translate X Y Z -> translate X, Y, Z

then you probably are close to what you want. In practice, this forced me to add many parentheses (as you just did), and I found using a low-priority comma to separate arguments was much more practical.

I like the clean looking syntax without semicolons at the end of lines. I'm assuming it uses indentation after a comma to define code blocks?

Yes.

It it ran on JavaScript, maybe I'd try it today...

If someone wants to give Emscripten a try ;-)

If you're the author, despite any criticism above (just half-baked opinions from watching the video for a few minutes on-and-off while writing this), I'd still encourage you to keep moving forwards with it. Looks like a lot of fun! And it is exploring some new ideas and the library looks amazing. There is no question popular computer languages (Java, JavaScript, C++) have many warts and someday it would be great to have better languages (again though, Smalltalk and message passing is my favorite, even as I move to JavaScript now for various reasons).

Thanks for the encouragements.

Still, if you haven't already, you might want to make an ANTLR ( http://www.antlr.org/ ) grammar for Tao3D and generate JavaScript for a backend to the animations which uses asm.js for speed and so it can run easily in a web browser. Then people with a compatible recent browser could just click on a link and be up and running with Tao.

I have other plans to achieve that same effect. What out for Buddda ;-)

Write a generic ETL app. Quite useful. Might be many out there, though. Probably few good free ones..

That's actually a very good idea, because I've found the exact opposite of your suspicion: there's few out there and they're all bad!

The kind of insanity I see regularly:

* Visual programming languages, which are known to be inefficient and just all-round bad. They promise a lot, but fail to deliver.
* Poor re-use or re-factoring of common tasks, such as consistent handling of groups of columns from disparate sources.
* Poor parallelism. I suspect that there's no ETL tool out there that can parse a CSV file in parallel. It's hard, because all but the first thread has to "hold" its results and potentially back-track. There are organizations out there that import multi-gigabyte text files!
* Poor adherence to standards. For example, SQL Server 2008 R2 and earlier don't support the CSV standard. No joke!
* Poor scaffolding or get-started-quick importing. Lots of ETL tools make you drag & drop at least each table once. Performing an "upsert" merge (or similar) between a database and a subset involving many tables is almost always months of fiddly work. God help you if you need to perform more complex merges...

Essentially, writing an advanced ETL tool in a high-level and safe language like C# or Java wouldn't be too hard, and would be useful to a lot of people. There's also great tools out there now for developing new Domain Specific Languages (DSLs), which would allow a full-fledged ETL language to be developed quickly.

Lots of good programming practice in a project like that: parallelism, databases, and parsers. Yet, it's easy to get started, and even a very simple version might be useful for some things!

Wikipedia, the encyclopedia that anyone can edit - in my ass.

Actually, it's possible to make a wysiwyg editor for Wiki markup in HTML with a little Javascript, there's no need for Flash!

It's not even hard, I did one for a corporate project in about a week, and I'm by no means an expert at Javascript.

It's even possible to do a split-screen view where it shows you the markup AND the preview, and the user can edit either.

The trick is that doing this has a prerequisite: the wiki syntax has to have a nice unambiguous grammar, and you need a parser generator that can emit Javascript parsers for it. At first, I tried to base my wiki grammar on Wikipedia's syntax, but it turns out that it's ambigious and difficult to parse, so I made one from scratch, and used ANTLR to generate a parser for it. The actual project used the C# parser (it was for an ASP.NET web site), but I experimented with real-time parsing in web pages using the JS parser. It works well, and is fast enough for pages of about 1KB. It would need some clever programming to scale past that, like incremental parsing.

This wouldn't work for Wikipedia though, because the way it has been written is typical PHP spaghetti code. It relies heavily on repeated "search & replace" operations and regular expressions, which sounds not-too-bad, until you have to figure out the formal grammar or the page object model. It's got layers of crap on top of each other, with no rhyme or reason. For example:

Both of these are: '''bold''' <b>bold</b>

You can have nested code too: ''italic <b>italic-bold''bold<b>

This complex mess of Wiki markup, legacy HTML markup, and some XML-like elements makes parsing Wikipedia a royal pain. The nested syntax is especially nasty. Everyone else with any sense is moving toward XHTML-like syntax, where open tags have to be closed in a strict reverse sequence.

When I did my undergrad at the University of Minnesota in 2000, they let us know that they took our code that we submitted and stored it in a program with a database. Whenever a student submitted new code, it went through this program. Essentially, some really fancy Hamming Distance that I think might have been similar to FASTA or BLAST algorithms for genetics were employed to score assignments against all the other ones.

If it was common for students to write assignments -- say they had been given a design template -- then all of the scores would come back rather high. If the TA noticed an outlier, they would investigate. If two submissions came up sufficiently similar, they would investigate.

It was (of course) never explained in detail how it worked but I bet that today one could take this to many new levels with things like ANTLR that might allow the program to check the inherent structure in code to avoid something trivial like different comments or variable names skewing the results.

Was it me who was in the professor's situation, I would bite the bullet and code the very basic above application using a web form submission for TAs and Professors. Then I would ask for help from other members of the department and make it a customizable growing project to protect the academic integrity of my school and students. Then I myself would put the opened source on there and run all my students assignments against it.

Problem solved, you can keep all your assignments static, you lazy bastards :)

... notes that 'Linus Torvalds, a Finn, comments his code in English (it apparently never occurred to him to do otherwise).

I thought I had read/heard somewhere (might have even been the documentary Revolution OS) that Finns & Swedes grow up with English Sesame Street available to them and as a result many of them are bilingual from a young age.

I've also ready that being bilingual or a polyglot is beneficial to thinking and memory skills. So I would caution thinking that because Linus Torvalds chooses comments in English for any reason other than more people speak it than Finnish. I would also caution you to assume that Linus learned English in order to increase his hacking skills. And I might even be inclined to argue that Linus' bilingualism aided or enabled him to reach such great heights with programming languages.

After toying with tools like ANTLR, it's not too far of a jump to say that understanding another language (even a dead one like Latin) helps you understand that information & logic can be portrayed multiple different ways with different vocabularies & grammar rules. Thus priming you for many software languages.

I cannot attest as to whether or not English buys you anything over Russian or Chinese as far as resources available on the web but I will argue that someone who has Russian as a first language and Chinese as a second will most likely be better off to code than someone with merely English as a first language (Disclaimer: I am the latter).

'advocating the adoption of English as the de-facto standard language of software development is simple pragmatism, the most virtuous of all hacker traits'

I don't think that makes you an 'ugly American programmer' but I sure do think it sets you up for some surprises in life.

After using ANTLR for a class long ago and being so impressed with it, I just returned to ANTLR today. I was shocked at the lack of documentation on the web site. I eventually typed "antlr reference" into google and found the following PDF: http://www.antlr.org/share/1084743321127/ANTLR_Reference_Manual.pdf It's outdated and had many no-longer-supported constructs, but paired with the changes from 2.x to 3.0 it was adequate for what I needed to do. I can see nothing comparable linked from the ANTLR homepage. It seems like an obvious attempt to get you to buy the book. Oh well.

I certainly appreciate that a lexer/parser generator can save me months of work. And as I mentioned in my original post, I'm using one. When I did my Google search for parser generators, I found several, and I chose the one that was public domain over BSD and GPL ones because the license makes it easy to use. I was unable to judge merit having not used any of them before, but ANTLR did not look markedly inferior to the alternatives. (Some lexer/parser generators, including ANTLR, require distributing a runtime with your application, not just the output code; thus the license is relevant. Bison/Flex was not an option as we are not using C).

I think you are overestimating my aversion to BSD code. Like I said, the situation where using BSD code would save me months of work has not yet come up (because there is more public domain code out there than you might think), and in that situation I would use it. Also, if our project had already incorporated some BSD code and therefore had precedent (approval from management/legal, a CREDITS file or equivalent to put the attribution in) I would be more comfortable.

As a developer on an internal project, I am reluctant to use even BSD licensed code. I have no problem with giving credit where credit is due, I just can't personally guarantee that it will happen. At the moment we don't have formal user documentation or any other good place to put the attribution required such that it will stick around. There's no possible way I can guarantee, when I put something in, that it will remain in the future. The code I'm writing, though it is internal for now, could end up anywhere in a few years. If anyone forgets to copy all the correct attributions when moving the code around, all of a sudden bam! You're illegal.

That's why I only use public domain code, of which a surprising amount exists. At the moment, I'm using SQLite and ANTLR (though v3 is moving to BSD...).

It's 2006, we shouldn't have any trouble parsing natural language, let alone programming languages by now. There's good tools available, it's just absurd.

apparently antlr does python (first entry) too. i stick by my claim that lex/bison are really antiquated (you have to do some majorly crufty and poorly documented stuff to make bison output thread safe, for example). lalr is a powerful approach to parsing. it's by no means the only approach, or, for that matter, necessarily the best approach. antlr is capable of compiling a superset of the languages that lalr tools (specifically yacc/bison) are capable of ( see here). does the average joe care about the difference betwee ll(k) and lalr(k)? probably not. still, theoretically, antlr is a more powerful tool than yacc/bison

if you have never taken a compiler class before or written a compiler on your own, i suggest the following:

• while i encourage re-use, if your purpose is to learn how to write a compiler, don't extend someone else's. find a grammar for a language you're comfortable with (e.g. pascal) and start from there. you'll find that getting just plain pascal to compile properly will be quite a challenge. oo-ness just adds another layer of complexity to the compiler.
• to aid your debugging, you'll want to spend some time thinking up good ways of a) visualizing your parse tree, b) representing your IL in a human readable format, c) representing the entire state of your interpreter in human readable format. these just by themselves can be very challenging projects.
• acquire a copy of the dragon book. this is only a starting point, you may also want to peruse some of andrew appel's compiler books (such as this one or this one)
• lex/flex and bison/yacc are rather antiquated, you may want to check out terence parr's antlr (formerly pccts) instead. this allows you to implement your compiler+interpreter in your language of choice, rather than being forced to use c. my compiler classes all required that we used lex/yacc, so that's what i did; however, i would have really liked to have the option of doing it all in java or c++.
• how you setup your symbol table will have a large determining factor in your success. i've used trees of hashtables in the past quite successfully (each node in the tree corresponding to a lexical context such that all symbols visible in the current scope are on the path between the present node and the root). i expect that extending this to support an OO language shouldn't be too hard. e.g: augment nodes representing a type to include back pointers to parent types. you will have to modify search to do lookups as appropriate.

there are several "toy" grammars out there which allow you to do useful stuff (recursion, 'interesting' data structures [i.e. self referential], etc.) without wading through a lot of useless cruft (implementing huge amounts of runtime support, for example). i'd go with one of those. once you're comfortable that you can make one of these learning languages work, then try to hack one of your own.

this all said, good luck! i am by no means an expert on compiler construction (worked on a custom in-house scripting language as an intern a couple years back and had to take compiler classes to satisfy breadth requirements for my m.s. c.s.) but i do hope this is a little bit useful.

I was investigating whether to start from an existing compiler and extend it. The compiler will be based on yacc, or bison.

Perl also has the right license (GPL). Is Python out of the question for such a project, since it's not GPL?

Antlr is simply the java port of what was originally called the Purdue Compiler Construction Tool Set

And I fail to understand where you've come up with the idea that webstart ties you down with some licensing restrictions.

I'm no fan of Java3d either (I much prefer the opengl bindings), but please get your facts straight... or at least cite your sources.

Well...the Dragon book for starters, as mentioned earlier. That's probably the ur-source for most of the theory behind the magic. Makes my head hurt, though.

Terence Parr's book, Practical Computer Language Recognition and Translation (out of print). His doctoral dissertation is a useful thing too (try the Purdue University library).

comp.compilers is another useful resource. It's archived at http://compilers.iecc.com.

Alan Holub's Compiler Design in C is a classic.

The ACM's SIGPLAN ("Special Interest Group On Programming Languages") and it's journal SIGPLAN Notices of the ACM are all fine resources. So is ACM Transactions on Programming Languages and Systems.

Don't forget the IEEE as well.

Not to mention Abelman and Sussman: Structure and Interpretation of Computer Programs.

The garbage collection page is a good source for information on memory management and garbage collection.

Your university's library is another good resource.

Well. That should keep you out of trouble.

You've got a couple choices -- finding yourself a good regular expression library seems like a good start ;-) If you're looking to do something a little more interesting than just lexical analysis, check out the red dragon book (better known as Compilers: Principles, Techniques, and Tools by Aho, Sethi & Ullman. I used it in my compiler course and I can tell you that they hit all the various parsing techniques (recursive descent, LA, LALR, SLR, etc.) very well, along with some other stuff. They concentrate on Lex/Yacc as tools -- you may prefer to check out ANTLR -- Terrence Parr's parser generator. It can be targeted at a bunch of languages and can also produce tree walkers for when it comes time to use your parsed data.

Another thought, partially off-topic, pertains to a previous poster's comments about working from a DTD and then migrating to XML Schema. I have to wonder how much of that is simply habit; I know that I've certainly had to solve problems that DTD's just can't handle. In my mind, even though habit may dictate starting with a DTD, starting with something that clearly will not accomplish the task at hand seems inherently flawed.

Anyway, instead of XML Schemas or DTDs I'd go for canonical description possibilities every time. E.g., annotated grammars in EBNF like in ANTLR with automatic transformation to XML Schemas for usage with XML tools.

And besides, XML Schemas are not very powerful if one compares them to real syntax description systems (i.e., parser specs) and real data constraint systems (i.e., formal spec languages, be they axiomatic or algebraic).