[TclinaPickle] - 2012-07-05 14:51:06

That Tcl is a toy language is not a misperception, but a patently obvious observation by a very, very large number of programmers.

I've been using it against my will for the last few months, and recently discovered, to my complete shock, that braces in comments are syntactically significant.

So I google it and find all these very helpful posts from Tclers helping newbies to understand why.

The fact that braces in comments are syntactically significant is not something I want explained to me. I want it to behave in a reasonable way. It's a fucking comment and I want the interpreter to ignore it, for fuck's sake!

Tcl is little more than an extremely poor implementation of Lisp.

AMG: Tcl is basically several languages stacked on top of each other. The outer language doesn't know what a comment is; it only knows how to match up braces and quotes and stuff to form a list of words. Only when that's done do things get passed to the next level language: command dispatch, which does know comments. Then each command forms yet again its own language, e.g. [expr]. What you're suggesting is to make the brace matcher recognize comments. FB and I have proposed this too, and I hope to further develop this idea (and others) at the next Tcl/Tk conference. More on this in a second.

However, saying that comments which don't work reliably (i.e. don't work in a way that's common among other languages) doesn't make Tcl a toy language, just an odd-man-out. Would you prefer APL comments? But yes, I definitely agree that this is something that needs to be addressed. There are many places I've wanted to put comments but can't, e.g., embedded in a list. One example of such a list is the second argument to [switch], which would be the list of match patterns and scripts. I can put the comment in the script part, but not alongside the match pattern. It's confusing until you learn that the Dodekalogue says what it means and means what it says, despite any (sometimes deceptive) similarities to other languages.

Okay, back to comments in the brace matcher. I've found that calling something "simple" isn't sufficient to describe it. Tcl is said to be simple. Because of criticisms such as yours, I've even said that Tcl is too simple to understand. Tcl's simplicity is in its design, but (except to those who really "get it") this translates to complexity in practice. I have no trouble using it, but I've seen many folks struggle. Why can't I put comments there? / Because that's not where a command would go. Or: Why don't the braces match up right? / Because they don't--- see that brace right there, the one with a pound sign right in front of it? / I thought that was a comment. / Read your Dodekalogue, the brace matcher doesn't care. Seriously, making the brace matcher recognize comments actually was quite difficult to hammer out and explain (only sounds simple in summary, the trick is figuring out what constitutes a comment without having to actually execute it), so I definitely can see why it wasn't done as part of the original design. It sounds like I'm saying a system can either be simple for the implementer or for the user, but never really both. I don't know if that's actually true, but this case is certainly evidence of such.

AMG, update: Here, let me throw a little more fuel on the fire. I know to avoid mismatched unbackslashed braces in comments, but I still get surprised sometimes by mismatched unbackslashed braces in double quotes. The latter can easily happen in code generators. For instance, let's say that a Tcl script is generating some C code. It always emits a certain code snippet, but sometimes it sometimes wraps it inside a for loop. Well, that would consist of the following code, right?

if {$for} {
    emit "for (i = 0; i < $limit; ++i) {"
}
emit $code
if {$for} {
    emit "}"
}

Well, what does Tcl actually do with this code? The second argument to if runs all the way from the opening brace at the end of the first line to the closing brace at the very last line. The [emit $code] line only executes if $for is true. And that's only if there happens to be a command called "}", since the single "}" on the line above the [emit $code] is interpreted as a command name.

The fix is simple: put backslashes in front of the braces inside quotes. But still it surprises me.

This is another case my proposal handles, but it was by no means easy. Not only does the brace matcher have to figure out what constitutes a comment, but it also has to get quoted strings. The rule for a comment isn't simple: it can only start where a command can start; and the rule for quoted strings is only slightly easier: it can only start at a word start.

Cloverfield - Parser shows a years-old and incomplete attempt I made at solving these issues.

[TclinaPickle]: The response merely emphasizes the original contention. Perhaps it would be helpful to define what is meant by "toy language"

"Tcl is a toy language" doesn't mean it's not expressive, or not a practical means of getting certain things done. It is both of these things. No bones there.

"Tcl is a toy language" really means that its philosophical underpinnings are unsound - it makes design decisions that are easy for the implementer, not the user. In my original rant, I pointed out that I don't want to have certain behaviour rationalized; I want it fixed. Simplicity in a language can be a very good thing, but the problem here is that Tcl's implementation is simple, and such decisions have very unfortunate implications to the user without tangible benefit. Lisp is conceptually even simpler - it is mathematically elegant and reasonably correct. Implementations, though, are not.

If you're not aware of it, there's a famous article, Worse Is Better, on some of the ideas you raised. Well worth a read.

AMG: Tcl has a couple core principles. It would be nice for someone to spell them all out, like a mission statement. I could attempt so here, but for now I don't want to distract from your point, so I'll just focus on one thing. Tcl tries to be a simple yet powerful way to glue together tools, so it doesn't have any syntax of its own besides the bare minimum needed for invoking (externally defined) commands. That leaves it with a very minimal core (the Dodekalogue) upon which any command can be implemented in a way that makes it look like it was part of the language all along.

The issue you raise, and I raise the same, is that the core language's quoting mechanism is underdesigned and interacts badly with other core language features (comments) and even itself (braces versus double quotes). This bad interaction is regarded as a corner case, and the language contains a workaround: put backslashes in front of nested braces if they can appear mismatched. If I read you right, you think this is a symptom of a bigger problem, that Tcl is really just another example of the New Jersey approach. (By the way, thanks for the link; I am familiar with the paper but didn't know about its context.)

(I'm not trying to stray off-topic here, only supply you with another example to chew on.) There's a bad interaction between the word separator and the way array variables are named. Witness:

puts $foo(a b c)
set foo(a b c) zzz

The first line works, the second does not. That's because the parser only recognizes parentheses in the context of variable substitution. Consequently, [set]'s arguments are: (1) foo(a (2) b (3) c) (4) z. To fix the script, either quote the spaces between a, b, and c (put double quotes or braces around the whole variable name or put backslashes in front of the spaces) or put the array element name inside a separate variable. Again, a corner case with easy workarounds, but still a pitfall for folks who are expecting things to be simple for the user.

My proposal deals with this issue too, though I'm sure my method for this will be poorly received because it breaks compatibility with all existing scripts. I fully expect it will be derided as "Perlish" due to the addition of (gasp!) new syntax. At this moment I'm not prepared to give my proposal the treatment it deserves, so I'll refrain from writing more about it. If you're interested, we can collaborate in private, exchange drafts, etc. But I will say that my approach for naming variables not only solves the above problem but also offers powerful new functional programming possibilities while bolstering Tcl's everything is a string philosophy.

So, let's return to the concept of underlying philosophy, or lack thereof. At first JO didn't think it would be possible to write a complete program in Tcl; Tcl could only glue together and/or be embedded within systems written in other languages. Only after people added commands for file I/O and such did Tcl become useful unescorted. This put Tcl in a new territory for which it had no specific philosophy, but its original goals served it well enough, particularly "everything is a string". The dynamic nature of Tcl helped it to achieve some functional programming goals. Since Tcl isn't mired in memory allocation, procedures can create and return complex data structures with abandon, trusting that copy-on-write will appropriately optimize everything. Adding [dict] made this even easier to do. But as functional goes, that's quite basic. We've been able to add lambdas with [apply], though the fact that they have to be shepherded by the [apply] command makes me sad. [apply] is a very clever way to integrate a new concept into an existing language in such a way that existing commands (e.g. [bind]) can cope, but it also shows the limitations of the language, which become even more apparent when trying to take on closures and continuations. NRE is a brilliant improvement, and the [coroutine] command made Wibble possible, but it doesn't take us all the way there. So we're trying to bolt functional concepts on a system that doesn't 100% support them, and we're constantly banging up against the stops.

---