Updated 2018-05-21 05:39:01 by pooryorick

Tcl_Obj is the C structure that implements every Tcl value. The name is misleading. Tcl_Value would better describe its purpose, but that term was already taken by Tcl_CreateMathFunc().

AMG: Tcl 9 renames Tcl_Obj to Tcl_Value [1].
Tcl_Obj's are like storks. They have two legs, the internal representation and the string representation. They can stand on either leg, or on both.
-- attributed to DKF
Any Tcl_Obj that is not isomorphic to a string is a bug.
-- dgp, Tcl Chatroom, 2015-07-01

See Also  edit

Creating and Using Tcl Handles in C Extensions
How to embed Tcl in C applications
Managing the reference count of Tcl objects
Tcl_Obj Deep Copy
Tcl_Obj refCount HOWTO
Tcl_Obj types list
Blessed Tcl_Obj Values
A Tcl_Obj Command Machine Code Generator
Tcl_Obj proposals
Discussions of changes to the Tcl_Obj structure and its semantics are referred to
Tcl_Obj vs Command
Extending Tcl
Islist Extension
32-bit integer overflow
Category Tcl Library
Hack on Tcl objects from the script level.
An extension for Tcl 8.6+ to reflect the Tcl_Obj API into the script level.

Documentation  edit

An On-the-fly Bytecode Compiler for Tcl, Brian T. Lewis, 1996
introduces Tcl_Obj
official reference for Tcl_Obj and friends
official reference for object types

Description  edit

Tcl_Obj provides for both a string (UTF8) representation and an "internal" representation of the value, which is limited only by the constraints of the C language itself. Each Tcl_Obj carries information about the type of its internal representation, and how to perform a conversion from the string representation to the internal representation, and vice-versa. In this way, the string becomes the universal intermediate representation for conversions between types. To enhance performance, Tcl knows how to perform direct conversions between certain often-used types.

A Tcl_Obj is is reference counted, and the allocator for it is very heavily tuned. It has a deeply unfortunate name, but the far more apt Tcl_Value was previously taken for handling user-defined expr functions - a now obsolete facility.

RS: thinks that the name is ok if one does not expect OO features, class membership etc. Objects have been there long before OO, and the name is certainly not under a monopoly (I'd object against that ;-). But the basic feature of Tcl_Obj's is that they have a string representation and possibly a problem-oriented one, but each can be regenerated from the other (also if you define your obj Obj types). If such type conversions occur frequently, this costs performance - the so-called shimmering occurs. E.g. see what happens to $i below:
for {set i 0} {$i<10} {incr i} { #here we need the integer rep
    puts [string length $i]      ;#here the string rep..
    puts [llength $i]            ;# and here the list rep, so int rep goes away 

CMcC: I've put together a summary page of Tcl_Objs current for 8.4, containing information culled from the source.

A Tcl_Obj is defined as a structure containing:
an integer refCount
representing the number of references to this Tcl_Obj
a char *bytes
being the string representation of the object (under the doctrine `everything's a string'). For a non-empty string, objv[i]->bytes points to Tcl_Alloc()ated memory. For an empty string, objv[i]->bytes points to a static char in the Tcl library that holds a NUL byte.
an integer length
being the length of the string representation in bytes (minus the extra byte for the terminating NUL)
a pointer to a Tcl_ObjType
which contains the type's name, and pointers to functions implementing the four fundamental operations which all Tcl_Obj instances are expected to implement.
a union internalRep
which is used to store up to two pointers of information which is opaque to Tcl.

Each Tcl_ObjType structure contains the following four function pointers plus a name.
Called to free any storage for the type's internal rep. NULL if the internal rep does not need freeing.
Called to create a new object as a copy of an existing object; NULL indicates that the default strategy (memcpy the whole internalRep union) is sufficient.
Called to update the string rep from the type's internal representation. (Not sure what NULL means for this; IME that's not an especially good idea. DKF: It's OK provided you never ever set the bytes field to NULL.)
Called to convert the object's internal rep to this type. Frees the internal rep of the old type. Returns TCL_ERROR on failure. NULL indicates that objects of this type can't normally be created (typically because extra context is needed.)

Allocating a Tcl_Obj  edit

DKF: You must not allocate a Tcl_Obj manually. Always call Tcl_NewObj (or one of its close relatives, such as Tcl_NewIntObj()) to do it for you. This is because Tcl uses a special memory management engine for them that is tuned to be extra efficient -- useful because Tcl uses these things a lot -- and that's only accessible through Tcl_NewObj (or some wholly internal APIs that aren't exposed outside the Tcl library).

Reference Counting  edit

AMG: In C extension code for Tcl you have several options with respect to reference counting:

  1. Manually invoke Tcl_IncrRefCount() on the Tcl_Objs you create. This protects them from being freed, but you're also responsible for calling Tcl_DecrRefCount() or else they'll leak.
  2. Give your Tcl_Objs to something that increments their reference counts. For example, put them in a Tcl variable, list, or dict. Don't call Tcl_IncrRefCount() unless your code is also retaining pointers that you expect to be valid sometime in the future.
  3. Don't fuss with reference counting because you're not the one creating the Tcl_Objs. This is the case when all you do is read arguments passed to your function which is implementing an extension command.
  4. Don't call Tcl_IncrRefCount() because you like to live dangerously and "know" that you're only passing your Tcl_Objs to things that won't pull the rug out from under you. Call Tcl_DecrRefCount() when you're done, and the Tcl_Objs will be freed when their refcounts go negative just as surely as when they go zero.

There's no way Tcl can remotely zap your Tcl_Objs with nonpositive refcount unless you've passed your Tcl_Obj pointers to Tcl library functions. Tcl doesn't keep a list of Tcl_Objs in existence, so it can't sweep.

Discarding the Internal Cached Interpretation of a Value  edit

string length causes the internal cached interpretation of a value to be discarded :
% incr i
% ::tcl::unsupported::representation $i
value is a int with a refcount of 2, object pointer at 0x6000660e0, internal representation 0x1:0x600066320, string representation "1"
% string length $i
% ::tcl::unsupported::representation $i
value is a string with a refcount of 2, object pointer at 0x6000660e0, internal representation 0x6000ad6a0:0x600066320, string representation "1"

Threads  edit

Data members of Tcl_Obj, particularly internalRep, can be mutated, so a Tcl_Obj should be exclusively owned by one thread. See Thread safety in tclZipfs.c.

Nested Tcl_Obj Structures  edit

PYK 2018-05-12: Sometimes a Tcl_Obj is stored in the internal representation of another Tcl_Obj. This can lead to tricky issue such as this memory leak in foreach. The fix involved clearing the internal representation of the nested Tcl_Obj, but couldn't something else set the internal representation back to a problematic value again?