This page is about benchmarking Tcl's
proc vs
Itcl-based code to see how much overhead Itcl adds and how to make its performance better.
I decided to compare plain Tcl recursive calling vs Itcl inheriting, using
itcl chain and delegating.
If Itcl class has no variables and procs do not use any variables:
ITcl: Chain 10 level deep 51.347
ITcl: Chain 11 level deep 55.769
ITcl: Chain 12 level deep 61.921
ITcl: Chain 13 level deep 72.064
ITcl: Chain 14 level deep 76.513
ITcl: Chain 15 level deep 77.732
ITcl: Chain 16 level deep 84.874
ITcl: Chain 17 level deep 91.932
ITcl: Chain 18 level deep 121.868
ITcl: Chain 19 level deep 114.08
ITcl: Chain 20 level deep 108.025
ITcl: Delegate 10 level deep 32.479
ITcl: Delegate 11 level deep 36.436
ITcl: Delegate 12 level deep 39.776
ITcl: Delegate 13 level deep 41.741
ITcl: Delegate 14 level deep 45.113
ITcl: Delegate 15 level deep 49.032
ITcl: Delegate 16 level deep 52.265
ITcl: Delegate 17 level deep 54.443
ITcl: Delegate 18 level deep 93.945
ITcl: Delegate 19 level deep 71.611
ITcl: Delegate 20 level deep 70.01
ITcl: Inherit 10 level deep 25.959
ITcl: Inherit 11 level deep 28.699
ITcl: Inherit 12 level deep 31.088
ITcl: Inherit 13 level deep 33.196
ITcl: Inherit 14 level deep 35.765
ITcl: Inherit 15 level deep 38.178
ITcl: Inherit 16 level deep 44.797
ITcl: Inherit 17 level deep 43.085
ITcl: Inherit 18 level deep 54.318
ITcl: Inherit 19 level deep 47.477
ITcl: Inherit 20 level deep 50.636
Tcl: Recursive 10 level deep 10.403
Tcl: Recursive 11 level deep 11.743
Tcl: Recursive 12 level deep 12.324
Tcl: Recursive 13 level deep 14.245
Tcl: Recursive 14 level deep 14.721
Tcl: Recursive 15 level deep 16.241
Tcl: Recursive 16 level deep 16.997
Tcl: Recursive 17 level deep 17.485
Tcl: Recursive 18 level deep 18.763
Tcl: Recursive 19 level deep 22.154
Tcl: Recursive 20 level deep 21.667
So it seems that inheriting and calling methods from the same object seem to be the fastest way and using
[chain
] is the worst way.
But things change when we define 20 variables at each level - so a 20-level-deep test actually increments
19 * 20 = 380 variables:
ITcl: Chain 10 level deep 104.602
ITcl: Chain 11 level deep 112.974
ITcl: Chain 12 level deep 126.078
ITcl: Chain 13 level deep 138.874
ITcl: Chain 14 level deep 147.253
ITcl: Chain 15 level deep 161.369
ITcl: Chain 16 level deep 173.584
ITcl: Chain 17 level deep 188.603
ITcl: Chain 18 level deep 198.702
ITcl: Chain 19 level deep 242.112
ITcl: Chain 20 level deep 226.422
ITcl: Delegate 10 level deep 59.411
ITcl: Delegate 11 level deep 68.117
ITcl: Delegate 12 level deep 70.796
ITcl: Delegate 13 level deep 78.662
ITcl: Delegate 14 level deep 84.386
ITcl: Delegate 15 level deep 90.361
ITcl: Delegate 16 level deep 96.495
ITcl: Delegate 17 level deep 103.415
ITcl: Delegate 18 level deep 109.857
ITcl: Delegate 19 level deep 116.133
ITcl: Delegate 20 level deep 121.538
ITcl: Inherit 10 level deep 75.653
ITcl: Inherit 11 level deep 105.711
ITcl: Inherit 12 level deep 92.505
ITcl: Inherit 13 level deep 104.654
ITcl: Inherit 14 level deep 107.463
ITcl: Inherit 15 level deep 120.157
ITcl: Inherit 16 level deep 150.662
ITcl: Inherit 17 level deep 134.11
ITcl: Inherit 18 level deep 144.124
ITcl: Inherit 19 level deep 154.595
ITcl: Inherit 20 level deep 163.931
Tcl: Recursive 10 level deep 61.822
Tcl: Recursive 11 level deep 68.536
Tcl: Recursive 12 level deep 79.564
Tcl: Recursive 13 level deep 82.756
Tcl: Recursive 14 level deep 93.726
Tcl: Recursive 15 level deep 96.324
Tcl: Recursive 16 level deep 102.629
Tcl: Recursive 17 level deep 109.943
Tcl: Recursive 18 level deep 118.434
Tcl: Recursive 19 level deep 123.855
Tcl: Recursive 20 level deep 129.817
The most interesting part is that delegates now seem to work much faster than the other ones. Still, 25% performance loss using inheritance and
[chain
] having the biggest loss. The interesting part is that delegates now caught up with procs.
Now, for 50 variables (so 1000 variable references with 20th level):
ITcl: Chain 10 level deep 172.473
ITcl: Chain 11 level deep 194.999
ITcl: Chain 12 level deep 244.775
ITcl: Chain 13 level deep 242.553
ITcl: Chain 14 level deep 273.848
ITcl: Chain 15 level deep 304.169
ITcl: Chain 16 level deep 337.07
ITcl: Chain 17 level deep 370.41
ITcl: Chain 18 level deep 432.148
ITcl: Chain 19 level deep 466.136
ITcl: Chain 20 level deep 510.95
ITcl: Delegate 10 level deep 98.282
ITcl: Delegate 11 level deep 113.818
ITcl: Delegate 12 level deep 118.42
ITcl: Delegate 13 level deep 128.619
ITcl: Delegate 14 level deep 140.015
ITcl: Delegate 15 level deep 153.629
ITcl: Delegate 16 level deep 171.915
ITcl: Delegate 17 level deep 175.932
ITcl: Delegate 18 level deep 193.936
ITcl: Delegate 19 level deep 212.568
ITcl: Delegate 20 level deep 259.58
ITcl: Inherit 10 level deep 146.439
ITcl: Inherit 11 level deep 164.215
ITcl: Inherit 12 level deep 183.119
ITcl: Inherit 13 level deep 199.634
ITcl: Inherit 14 level deep 232.69
ITcl: Inherit 15 level deep 269.736
ITcl: Inherit 16 level deep 270.989
ITcl: Inherit 17 level deep 336.297
ITcl: Inherit 18 level deep 338.85
ITcl: Inherit 19 level deep 369.274
ITcl: Inherit 20 level deep 423.055
Tcl: Recursive 10 level deep 130.904
Tcl: Recursive 11 level deep 156.326
Tcl: Recursive 12 level deep 160.326
Tcl: Recursive 13 level deep 179.524
Tcl: Recursive 14 level deep 189.009
Tcl: Recursive 15 level deep 208.528
Tcl: Recursive 16 level deep 219.666
Tcl: Recursive 17 level deep 232.773
Tcl: Recursive 18 level deep 253.428
Tcl: Recursive 19 level deep 287.717
Tcl: Recursive 20 level deep 304.345
This is very interesting because the test claims that delegates using Itcl can actually be faster than non-
OO programming - which usually should not be the case.
The
tclbench-based test that did the trick:
package require Itcl
set itclvars 0
set minlevel 10
set maxlevel 20
set iter 10000
proc ::level1 {a b c d e} {
}
itcl::class ::class1 {
public method level1 {a b c d e} {
return 1
}
}
itcl::class ::classI1 {
inherit ::class1
}
itcl::class ::classD1 {
inherit ::class1
}
itcl::class ::classC1 {
public method level {a b c d e} {
return 1
}
}
for {set i 2} {$i <= $maxlevel} {incr i} {
set varcode ""
set varproccode ""
set varitclcode ""
set varlist [list]
for {set j 0} {$j < $itclvars} {incr j} {
set vn v${i}c${j}
lappend varlist $vn
append varcode "protected variable $vn 0" \n
set ::$vn 0
append varproccode "incr ::$vn" \n
append varitclcode "incr $vn" \n
}
set ip [expr {$i-1}]
proc ::level$i {a b c d e} "$varproccode \; return \[::level$ip \$a \$b \$c \$d \$e\]"
itcl::class ::classD$i "
$varcode
public method level$i \{a b c d e\} \{
$varitclcode
return \[::oD$ip level$ip \$a \$b \$c \$d \$e\]
\}
"
itcl::class ::classI$i "inherit ::classI$ip
$varcode
public method level$i \{a b c d e\} \{
$varitclcode
return \[level$ip \$a \$b \$c \$d \$e\]
\}
"
itcl::class ::classC$i "inherit ::classC$ip
$varcode
public method level \{a b c d e\} \{
$varitclcode
return \[chain \$a \$b \$c \$d \$e\]
\}
"
}
for {set i 1} {$i <= $maxlevel} {incr i} {
::classI$i ::oI$i
::classD$i ::oD$i
::classC$i ::oC$i
}
if {[catch {
::oI$maxlevel level$maxlevel 1 2 3 4 5
::oD$maxlevel level$maxlevel 1 2 3 4 5
::oC2 level 1 2 3 4 5
::level2 1 2 3 4 5
}]} {
puts stderr $::errorInfo
exit 1
}
for {set i $minlevel} {$i <= $maxlevel} {incr i} {
bench -desc "Tcl: Recursive [format %2d $i] level deep" -iter $iter \
-body "::level$i a b c d e"
bench -desc "ITcl: Inherit [format %2d $i] level deep" -iter $iter \
-body "::oI$i level$i a b c d e"
bench -desc "ITcl: Delegate [format %2d $i] level deep" -iter $iter \
-body "::oD$i level$i a b c d e"
bench -desc "ITcl: Chain [format %2d $i] level deep" -iter $iter \
-body "::oC$i level a b c d e"
}If someone wishes to make improvements to the code, please also update the benchmarks.
