Updated 2016-02-13 16:54:34 by escargo

gcov is the gcc coverage testing tool. It shows, how often each line of code executes and especially what lines of code are actually executed. It can be used, to see which code paths are not tested by the tcl test suite.

To instrument tcl for usage with gcov is easy. The gcov manual advises to compile without compiler optimization, to get better results. Therefore, I simply configured tcl with --enable-symbols (which disables optimization).
    ./configure --enable-symbols

Now, edit the generated Makefile, and add the special gcc options -fprofile-arcs -ftest-coverage to CFLAGS. Then, just call make test. I've done this with tcl8.4.7. Without modifying the Makefile the test suite runs without failed test for me (on linux). If I run the test suite with the described Makefile modifications I get 11 failed tests out of the files exec.test, io.test and unixInit.test. I haven't taken a deeper look, why this tests fail if tcl was compiled with that flags.

The compilation and the test run creates a bunch of new data files. Now, you're able to run gcov. You should run gcov in the same directory you've invoked the compiler for. 'gcov' is called for every source file in this way:
    gcov ../generic/tclBasic.c

For convenience, I did
    for file in ../generic/*.c; do
    gcov $file >> gcov.results 2>&1
    done

After that, gcov.results look like:
    regc_color.bbg:cannot open graph file
    regc_cvec.bbg:cannot open graph file
    regc_lex.bbg:cannot open graph file
    regc_locale.bbg:cannot open graph file
    regc_nfa.bbg:cannot open graph file
    File `../generic/regcomp.c'
    Lines executed:80.22% of 1097
    ../generic/regcomp.c:creating `regcomp.c.gcov'

    File `../generic/regc_lex.c'
    Lines executed:96.75% of 554
    ../generic/regc_lex.c:creating `regc_lex.c.gcov'

    File `../generic/regc_color.c'
    Lines executed:86.34% of 344
    ../generic/regc_color.c:creating `regc_color.c.gcov'

    File `../generic/regc_nfa.c'
    Lines executed:92.06% of 642
    ../generic/regc_nfa.c:creating `regc_nfa.c.gcov'

    File `../generic/regc_cvec.c'
    Lines executed:58.90% of 73
    ../generic/regc_cvec.c:creating `regc_cvec.c.gcov'

    File `../generic/regc_locale.c'
    Lines executed:65.13% of 195
    ../generic/regc_locale.c:creating `regc_locale.c.gcov'

    rege_dfa.bbg:cannot open graph file
    File `../generic/regerror.c'
    Lines executed:64.86% of 37
    ../generic/regerror.c:creating `regerror.c.gcov'

    File `../generic/regexec.c'
    Lines executed:76.47% of 476
    ../generic/regexec.c:creating `regexec.c.gcov'

    File `../generic/rege_dfa.c'
    Lines executed:89.59% of 365
    ../generic/rege_dfa.c:creating `rege_dfa.c.gcov'

    File `../generic/regfree.c'
    Lines executed:75.00% of 4
    ../generic/regfree.c:creating `regfree.c.gcov'

    regfronts.bbg:cannot open graph file
    File `../generic/tclAlloc.c'
    Lines executed:100.00% of 6
    ../generic/tclAlloc.c:creating `tclAlloc.c.gcov'

    File `../generic/tclAsync.c'
    Lines executed:96.72% of 61
    ../generic/tclAsync.c:creating `tclAsync.c.gcov'

    File `../generic/tclBasic.c'
    Lines executed:83.75% of 1495
    ../generic/tclBasic.c:creating `tclBasic.c.gcov'

    File `../generic/tclBinary.c'
    Lines executed:96.61% of 678
    ../generic/tclBinary.c:creating `tclBinary.c.gcov'

    File `../generic/tclCkalloc.c'
    Lines executed:39.62% of 53
    ../generic/tclCkalloc.c:creating `tclCkalloc.c.gcov'

    File `../generic/tclClock.c'
    Lines executed:92.06% of 126
    ../generic/tclClock.c:creating `tclClock.c.gcov'

etc.

The percent numbers show, how much lines of code with actual operations (that is: not counted pre-processor directives like defines, empty lines etc.) were actually called in the test run. A look at the generated *.gcov files provides much more detailed information. For example, take a look at this snippet out of tclCompile.c.gcov:
        -: 1850: *----------------------------------------------------------------------
        -: 1851: *
        -: 1852: * TclInitCompiledLocals --
        -: 1853: *
        -: 1854: *        This routine is invoked in order to initialize the compiled
        -: 1855: *        locals table for a new call frame.
        -: 1856: *
        -: 1857: * Results:
        -: 1858: *        None.
        -: 1859: *
        -: 1860: * Side effects:
        -: 1861: *        May invoke various name resolvers in order to determine which
        -: 1862: *        variables are being referenced at runtime.
        -: 1863: *
        -: 1864: *----------------------------------------------------------------------
        -: 1865: */
        -: 1866:
        -: 1867:void
        -: 1868:TclInitCompiledLocals(interp, framePtr, nsPtr)
-: 1869
Tcl_Interp *interp; /* Current interpreter. */
-: 1870
CallFrame *framePtr; /* Call frame to initialize. */
-: 1871
Namespace *nsPtr; /* Pointer to current namespace. */
  1025735: 1872:{
  1025735: 1873:    register CompiledLocal *localPtr;
  1025735: 1874:    Interp *iPtr = (Interp*) interp;
  1025735: 1875:    Tcl_ResolvedVarInfo *vinfo, *resVarInfo;
  1025735: 1876:    Var *varPtr = framePtr->compiledLocals;
  1025735: 1877:    Var *resolvedVarPtr;
  1025735: 1878:    ResolverScheme *resPtr;
  1025735: 1879:    int result;
        -: 1880:
-: 1881
/*
-: 1882
* Initialize the array of local variables stored in the call frame.
-: 1883
* Some variables may have special resolution rules. In that case,
-: 1884
* we call their "resolver" procs to get our hands on the variable,
-: 1885
* and we make the compiled local a link to the real variable.
-: 1886
*/
        -: 1887:
  5676824: 1888:    for (localPtr = framePtr->procPtr->firstLocalPtr;
-: 1889
localPtr != NULL;
-: 1890
localPtr = localPtr->nextPtr) {
        -: 1891:
-: 1892
/*
-: 1893
* Check to see if this local is affected by namespace or
-: 1894
* interp resolvers. The resolver to use is cached for the
-: 1895
* next invocation of the procedure.
-: 1896
*/
        -: 1897:
  4651089: 1898:        if (!(localPtr->flags & (VAR_ARGUMENT|VAR_TEMPORARY|VAR_RESOLVED))
-: 1899
&& (nsPtr->compiledVarResProc || iPtr->resolverPtr)) {
#####: 1900
resPtr = iPtr->resolverPtr;
        -: 1901:
#####: 1902
if (nsPtr->compiledVarResProc) {
#####: 1903
result = (*nsPtr->compiledVarResProc)(nsPtr->interp,
-: 1904
localPtr->name, localPtr->nameLength,
-: 1905
(Tcl_Namespace *) nsPtr, &vinfo);
-: 1906
} else {
#####: 1907
result = TCL_CONTINUE;
-: 1908
}
        -: 1909:
#####: 1910
while ((result == TCL_CONTINUE) && resPtr) {
#####: 1911
if (resPtr->compiledVarResProc) {
#####: 1912
result = (*resPtr->compiledVarResProc)(nsPtr->interp,
-: 1913
localPtr->name, localPtr->nameLength,
-: 1914
(Tcl_Namespace *) nsPtr, &vinfo);
-: 1915
}
#####: 1916
resPtr = resPtr->nextPtr;
-: 1917
}
#####: 1918
if (result == TCL_OK) {
#####: 1919
localPtr->resolveInfo = vinfo;
#####: 1920
localPtr->flags |= VAR_RESOLVED;
-: 1921
}
-: 1922
}
        -: 1923:
-: 1924
/*
-: 1925
* Now invoke the resolvers to determine the exact variables that
-: 1926
* should be used.
-: 1927
*/
        -: 1928:
  4651089: 1929:        resVarInfo = localPtr->resolveInfo;
  4651089: 1930:        resolvedVarPtr = NULL;
        -: 1931:
  4651089: 1932:        if (resVarInfo && resVarInfo->fetchProc) {
#####: 1933
resolvedVarPtr = (Var*) (*resVarInfo->fetchProc)(interp,
-: 1934
resVarInfo);
-: 1935
}
        -: 1936:
  4651089: 1937:        if (resolvedVarPtr) {
#####: 1938
varPtr->name = localPtr->name; /* will be just '\0' if temp var */
#####: 1939
varPtr->nsPtr = NULL;
#####: 1940
varPtr->hPtr = NULL;
#####: 1941
varPtr->refCount = 0;
#####: 1942
varPtr->tracePtr = NULL;
#####: 1943
varPtr->searchPtr = NULL;
#####: 1944
varPtr->flags = 0;
#####: 1945
TclSetVarLink(varPtr);
#####: 1946
varPtr->value.linkPtr = resolvedVarPtr;
#####: 1947
resolvedVarPtr->refCount++;
-: 1948
} else {
  4651089: 1949:            varPtr->value.objPtr = NULL;
  4651089: 1950:            varPtr->name = localPtr->name; /* will be just '\0' if temp var */
  4651089: 1951:            varPtr->nsPtr = NULL;
  4651089: 1952:            varPtr->hPtr = NULL;
  4651089: 1953:            varPtr->refCount = 0;
  4651089: 1954:            varPtr->tracePtr = NULL;
  4651089: 1955:            varPtr->searchPtr = NULL;
  4651089: 1956:            varPtr->flags = localPtr->flags;
-: 1957
}
  4651089: 1958:        varPtr++;
-: 1959
}
        -: 1960:}

The numbers before the line numbers show you, how often that line of code was called. This information may be helpful for optimization efforts. The lines with '####' are especially interesting: this lines of code are not called at all. This may help to find additional tests (or even to detect dead code).

See the 'gcov' man page for a few more output options.

Diving thru the results I got the impression, that the tcl test suite is already pretty good. Of course, some code paths are only called, if tcl was compiled with according flags (e.g.: tcl compiled with block allocator or not). Some debugging facilities seems not to be tested at all, but that may be intentionally. You've to search for relevant code paths, which are not called. One the other hand, gcov may be helpful, to spot that few code areas, which are currently not coveraged by the test suite. de

jmn 2004-08-04 It would be great to have something like this to use in conjunction with tcltest on Tcl scripts. Does such a thing exist? - (update: Coverage Analysis answers my question)