Updated 2015-08-08 06:59:09 by juef ▲

PS 2003-08-20:

File and directory change notifications/events are useful for (G)UIs showing a list of files, for mirroring directory(trees), tail applications, and some other things I can't think of right now. I asked if anyone knew of an extension for win32 that uses the directory notification API of windows. The quick consensus on the chat was that there wasn't one.

It'd also be nice to have this functionality for a lot of platforms. Pat Thoyts quickly found [FindFirstChangeNotification] and friends on MSDN and created a working extension.

See Also  edit

directory notification package in Tcl
for Linux
Tcl-Inotify
an efficient Linux-only solution
TWAPI
has the begin_filesystem_monitor/cancel_filesystem_monitor [1] functions to allow you to monitor file system changes by registering a callback. Like the rest of TWAPI, WIndoes NT 4.0 and later only. See [2] for an example.
watchman
A service that can watch for changes on Linux, OS X, FreeBSD, OpenBSD and Illumos/Solaris and abstracts away the differences between those platforms. It can be controlled using the command line or a JSON API spoken over a UNIX socket. In both cases watchman returns JSON output.
filewatcherd
FreeBSD daemon

Generic Solution  edit

Not a complete solution, but on the systems where [file mtime] returns the last update of a directory/file a reasonably efficient solution is just glob all files in a directory tree and storing their mtime. If you are only interested in create/rename/delete events (like most UIs are) you only need to call [file mtime $dir] to see if a file has been created/renamed/deleted since you last checked. Doing this every two seconds gives good interactive response in most cases (for UIs)

US: Just to emphasize: This does not work for write/update operations on a file!
namespace eval fschanges {
    if {0} {
        #if debugging:
        interp alias {} [namespace current]::dputs {} puts
    } else {
        proc dputs {args} { }
    }

    variable watchId 0

    proc watch {file_or_dir} {
        variable watchId
        incr watchId

        upvar [namespace current]::watch$watchId watching
        if {[info exists [namespace current]::watch$watchId]} {
            array unset [namespace current]::watch$watchId
        }

        set watching(watching) [list]


        if {[file isdir $file_or_dir]} {
            addDir watch$watchId $file_or_dir
        } else {
            add watch$watchId $file_or_dir
        }

        #set initial scan time
        set watching(last) [clock seconds]

        return watch$watchId
    }

    proc add {id name} {
        dputs "add $name"
        upvar [namespace current]::$id watching

        if {[info exists watching(watch.$name)]} {
            dputs "add exists $name"
            #no watching twice!
            return
        }

        lappend watching(watching) $name [file isdir $name]

        #and determine initial time (if any)
        if {[file exists $name]} {
            set itime [file mtime $name]
        } else {
            set itime 0
        }

        set watching(watch.$name) $itime

        return $name
    }

    proc addDir {id dir} {
        dputs "Add dir $dir"
        upvar [namespace current]::$id watching
        if {[info exists watching(watch.$dir)]} {
            dputs "Adddir exists $dir"
            #no watching twice!
            return
        }


        #puts "Add dir $dir"

        lappend new [add $id $dir]
        #puts "glob: [glob -nocomplain -path $dir/ *]"
        foreach file [glob -nocomplain -path $dir/ *] {

            if {[file isdir $file]} {
                dputs "Recurse into $file"
                set new [concat $new [addDir $id $file]]
            } else {
                lappend new [add $id $file]
            }
        }
        return $new
    }

    proc newfiles {id time} {
        upvar [namespace current]::$id watching
        set newer [list]
        foreach {file isdir} $watching(watching) {
            if {$watching(watch.$file) >= $time} {
                lappend newer $file
            }
        }
        return $newer
    }

    proc changes {id} {
        upvar [namespace current]::$id watching
        set changes [list]
        set new [list]
        #puts $watching(watching)
        foreach {file isdir} $watching(watching) {
            #puts "$isdir && [file mtime $file] > $watching(watch.$file)"
            if {$isdir && [file exists $file] && [file mtime $file] > $watching(watch.$file)} {
                set watching(watch.$file) [file mtime $file]
                lappend changes $file update
                foreach item [glob -nocomplain -dir $file *] {
                    if {![info exists watching(watch.$item)]} {
                        if {[file isdir $item]} {
                            set new [concat $new [addDir $id $item]]
                        } else {
                            lappend new [add $id $item]
                        }
                    }
                }
            }
        }
        foreach item $new {
            lappend changes $item created
        }

        return $changes
    }
    namespace export watch changes newfiles
}
package provide fschanges 0.5

Sample usage
package require fschanges
namespace import fschanges::*

#watch a directory:
set w [watch /tmp]

puts "Files created within the last hour: [newfiles $w [expr [clock seconds]-3600]]"

exec touch /tmp/testfile

#Show the new file and directory update:
puts [changes $w]

file delete -force /tmp/testfile

#file deletions are not noted (yet) but it will show an updated directory.
puts [changes $w]

Platform specific Which platforms can do this? It would be nice to create a (core?) extension to do this.

  • Windows 95, 98, ME, NT 2000, XP: Yes, [FindFirstChangeNotification]() and NTFS can do [file mtime $dir]
  • Windows CE: unknown but also likely to support FindFirstChangeNotification.
  • Linux <2.2: Unknown, but ext2 can do [file mtime $dir]
  • Linux 2.2+: Yes, see linux/Documentation/dnotify.txt using fcntl(fd, F_NOTIFY,DN_MODIFY|DN_CREATE|DN_MULTISHOT); See directory notification package in Tcl for more.
  • FreeBSD: Yes, [kqueue]: http://people.freebsd.org/~jlemon/papers/kqueue.pdf
  • OpenBSD: Supports kqueue in version 3.1 (unknown when first supported)/
  • (Other)BSD: Same as FreeBSD? NetBSD supports kqueue since 2.0
  • Classic Mac: Unknown
  • MacOSX: Supports kqueue starting with 10.3
  • Solaris: stevel thought so.
  • HP-UX: unknown
  • irix: yes. there's a file monitor.
  • dec: unknown
  • others: ???

elfring 2003-08-26: How do you think about the tool "File Alteration Monitor and Inode Monitor" (http://oss.sgi.com/projects/fam/links.html)? I do not know when a TCL programming interface will be available for it.

ps 2003-08-27: Well, by the looks of it, the IMon project is complementary to the DNotify stuff, and will probably be used if we get round to make a Tcl package.

elfring 2003-11-01: Can the function library "liboop" help to dispatch file events easier?

MHo: See ffcn for several (incomplete) solutions. I think, the drawback of tcl-only-solutions are that they are based on polling, whereas the windows-apis are event-based. Because the MS-APIs are, as usual, pure horror, TWAPI seems to be the most elegant way, but because TWAPI is a big monolithic block, the code blows up.

APN: MHo, could you explain what you mean by the code blows up? I'd like to fix any bugs in TWAPI. MHo: sorry, perhaps this was the wrong phrase. I meant that: the code one have to write is usually clear and small, because the commands of TWAPI are very powerfull. What I forgot to mention is that I almost always deploy programs as starpacks (=executables). And the twapi dll is very heavy in size.

Category ConceptCategory File