| comm(n) | Remote communication | comm(n) |
comm - A remote communication facility for Tcl (8.3 and later)
package require Tcl 8.3
package require comm ?4.6.1?
::comm::comm send ?-async? ?-command callback? id cmd ?arg arg ...?
::comm::comm self
::comm::comm interps
::comm::comm connect ?id?
::comm::comm new chan ?name value ...?
::comm::comm channels
::comm::comm config
::comm::comm config name
::comm::comm config ?name value ...?
::comm::comm shutdown id
::comm::comm abort
::comm::comm destroy
::comm::comm hook event ?+? ?script?
::comm::comm remoteid
::comm::comm_send
::comm::comm return_async
$future return ?-code code? ?value?
$future configure ?-command ?cmdprefix??
$future cget -command
The comm command provides an inter-interpreter remote execution facility much like Tk's send(n), except that it uses sockets rather than the X server for the communication path. As a result, comm works with multiple interpreters, works on Windows and Macintosh systems, and provides control over the remote execution path.
These commands work just like send and winfo interps :
::comm::comm send ?-async? id cmd ?arg arg ...?
::comm::comm interps
This is all that is really needed to know in order to use comm
The package initializes ::comm::comm as the default chan.
comm names communication endpoints with an id unique to each machine. Before sending commands, the id of another interpreter is needed. Unlike Tk's send, comm doesn't implicitly know the id's of all the interpreters on the system. The following four methods make up the basic comm interface.
The evaluation semantics of ::comm::comm send are intended to match Tk's send exactly. This means that comm evaluates arguments on the remote side.
If you find that ::comm::comm send doesn't work for a particular command, try the same thing with Tk's send and see if the result is different. If there is a problem, please report it. For instance, there was had one report that this command produced an error. Note that the equivalent send command also produces the same error.
% ::comm::comm send id llength {a b c}
wrong # args: should be "llength list"
% send name llength {a b c}
wrong # args: should be "llength list"
The eval hook (described below) can be used to change from send's double eval semantics to single eval semantics.
More than one comm channel (or listener) can be created in each Tcl interpreter. This allows flexibility to create full and restricted channels. For instance, hook scripts are specific to the channel they are defined against.
The default configuration parameters for a new channel are:
"-port 0 -local 1 -listen 0 -silent 0"
The default channel ::comm::comm is created with:
"::comm::comm new ::comm::comm -port 0 -local 1 -listen 1 -silent 0"
The config method acts similar to fconfigure in that it sets or queries configuration variables associated with a channel.
These configuration variables can be changed (descriptions of them are elsewhere in this manual page):
These configuration variables are read only:
When config changes the parameters of an existing channel (with the exception of -interp and -events), it closes and reopens the listening socket. An automatically assigned channel id will change when this happens. Recycling the socket is done by invoking ::comm::comm abort, which causes all active sends to terminate.
comm uses a TCP port for endpoint id. The interps (or ids) method merely lists all the TCP ports to which the channel is connected. By default, each channel's id is randomly assigned by the operating system (but usually starts at a low value around 1024 and increases each time a new socket is opened). This behavior is accomplished by giving the -port config option a value of 0. Alternately, a specific TCP port number may be provided for a given channel. As a special case, comm contains code to allocate a a high-numbered TCP port (>10000) by using -port {}. Note that a channel won't be created and initialized unless the specific port can be allocated.
As a special case, if the channel is configured with -listen 0, then it will not create a listening socket and will use an id of 0 for itself. Such a channel is only good for outgoing connections (although once a connection is established, it can carry send traffic in both directions). As another special case, if the channel is configured with -silent 0, then the listening side will ignore connection attempts where the protocol negotiation phase failed, instead of throwing an error.
A communication channel in its default configuration will use the current interpreter for the execution of all received scripts, and of the event scripts associated with the various hooks.
This insecure setup can be changed by the user via the two options -interp, and -events.
When -interp is set all received scripts are executed in the slave interpreter specified as the value of the option. This interpreter is expected to exist before configuration. I.e. it is the responsibility of the user to create it. However afterward the communication channel takes ownership of this interpreter, and will destroy it when the communication channel is destroyed. Note that reconfiguration of the communication channel to either a different interpreter or the empty string will release the ownership without destroying the previously configured interpreter. The empty string has a special meaning, it restores the default behaviour of executing received scripts in the current interpreter.
Also of note is that replies and callbacks (a special form of reply) are not considered as received scripts. They are trusted, part of the internal machinery of comm, and therefore always executed in the current interpreter.
Even if an interpreter has been configured as the execution environment for received scripts the event scripts associated with the various hooks will by default still be executed in the current interpreter. To change this use the option -events to declare a list of the events whose scripts should be executed in the declared interpreter as well. The contents of this option are ignored if the communication channel is configured to execute received scripts in the current interpreter.
By default, each channel is restricted to accepting connections from the local system. This can be overridden by using the -local 0 configuration option For such channels, the id parameter takes the form { id host }.
WARNING: The host must always be specified in the same form (e.g., as either a fully qualified domain name, plain hostname or an IP address).
These methods give control over closing connections:
"::comm::comm destroy; ::comm::comm new ::comm::comm"
When a remote connection is lost (because the remote exited or called shutdown), comm can invoke an application callback. This can be used to cleanup or restart an ancillary process, for instance. See the lost callback below.
This is a mechanism for setting hooks for particular events:
When an event occurs, the script associated with it is evaluated with the listed variables in scope and available. The return code (not the return value) of the script is commonly used decide how to further process after the hook.
Common variables include:
These are the defined events:
This hook is invoked before making a connection to the remote named in id. An error return (via error) will abort the connection attempt with the error. Example:
% ::comm::comm hook connecting {
if {[string match {*[02468]} $id]} {
error "Can't connect to even ids"
}
}
% ::comm::comm send 10000 puts ok
Connect to remote failed: Can't connect to even ids
%
This hook is invoked immediately after making a remote connection to id, allowing arbitrary authentication over the socket named by fid. An error return (via error ) will close the connection with the error. host and port are merely extracted from the id; changing any of these will have no effect on the connection, however. It is also possible to substitute and replace fid.
Hook invoked when receiving an incoming connection, allowing arbitrary authentication over socket named by fid. An error return (via error) will close the connection with the error. Note that the peer is named by remport and addr but that the remote id is still unknown. Example:
::comm::comm hook incoming {
if {[string match 127.0.0.1 $addr]} {
error "I don't talk to myself"
}
}
This hook is invoked after collecting a complete script from a remote but before evaluating it. This allows complete control over the processing of incoming commands. cmd contains either send or async. buffer holds the script to evaluate. At the time the hook is called, $chan remoteid is identical in value to id.
By changing buffer, the hook can change the script to be evaluated. The hook can short circuit evaluation and cause a value to be immediately returned by using return result (or, from within a procedure, return -code return result). An error return (via error) will return an error result, as is if the script caused the error. Any other return will evaluate the script in buffer as normal. For compatibility with 3.2, break and return -code break result is supported, acting similarly to return {} and return -code return result.
Examples:
% ::comm::comm send [::comm::comm self] pid
5013
% ::comm::comm hook eval {puts "going to execute $buffer"}
% ::comm::comm send [::comm::comm self] pid
going to execute pid
5013
% ::comm::comm hook eval {puts "would have executed $buffer"; return 0}
% ::comm::comm send [::comm::comm self] pid
would have executed pid
0
% ::comm::comm send [::comm::comm self] llength {a b c}
wrong # args: should be "llength list"
% ::comm::comm hook eval {return [uplevel #0 $buffer]}
return [uplevel #0 $buffer]
% ::comm::comm send [::comm::comm self] llength {a b c}
3
% interp create foo
% ::comm::comm hook eval {return [foo eval $buffer]}
% ::comm::comm send [::comm::comm self] set myvar 123
123
% set myvar
can't read "myvar": no such variable
% foo eval set myvar
123
% ::comm::comm hook eval {return [eval foo eval $buffer]}
% ::comm::comm hook eval {
switch -- $buffer {
a {return A-OK}
b {return B-OK}
default {error "$buffer is a no-no"}
}
}
% ::comm::comm send [::comm::comm self] pid
pid is a no-no
% ::comm::comm send [::comm::comm self] a
A-OK
This hook is invoked after collecting a complete reply script from a remote but before evaluating it. This allows complete control over the processing of replies to sent commands. The reply buffer is in one of the following forms
For safety reasons, this is decomposed. The return result is in ret, and the return switches are in the return array:
Any of these may be the empty string. Modifying these four variables can change the return value, whereas modifying buffer has no effect.
Similar to reply, but used for callbacks.
This hook is invoked when the connection to id is lost. Return value (or thrown error) is ignored. reason is an explanatory string indicating why the connection was lost. Example:
::comm::comm hook lost {
global myvar
if {$myvar(id) == $id} {
myfunc
return
}
}
These interfaces may change or go away in subsequence releases.
proc send {args} {
eval ::comm::comm send $args
}
rename winfo tk_winfo
proc winfo {cmd args} {
if {![string match in* $cmd]} {
return [eval [list tk_winfo $cmd] $args]
}
return [::comm::comm interps]
}
Starting with version 4.6 of the package an option -socketcmd is supported, allowing the user of a comm channel to specify which command to use when opening a socket. Anything which is API-compatible with the builtin ::socket (the default) can be used.
The envisioned main use is the specification of the tls::socket command, see package tls, to secure the communication.
# Load and initialize tls package require tls tls::init -cafile /path/to/ca/cert -keyfile ... # Create secured comm channel ::comm::comm new SECURE -socketcmd tls::socket -listen 1 ...
The sections Execution Environment and Callbacks are also relevant to the security of the system, providing means to restrict the execution to a specific environment, perform additional authentication, and the like.
There is one outstanding difference between comm and send. When blocking in a synchronous remote command, send uses an internal C hook (Tk_RestrictEvents) to the event loop to look ahead for send-related events and only process those without processing any other events. In contrast, comm uses the vwait command as a semaphore to indicate the return message has arrived. The difference is that a synchronous send will block the application and prevent all events (including window related ones) from being processed, while a synchronous ::comm::comm send will block the application but still allow other events to get processed. In particular, after idle handlers will fire immediately when comm blocks.
What can be done about this? First, note that this behavior will come from any code using vwait to block and wait for an event to occur. At the cost of multiple channel support, comm could be changed to do blocking I/O on the socket, giving send-like blocking semantics. However, multiple channel support is a very useful feature of comm that it is deemed too important to lose. The remaining approaches involve a new loadable module written in C (which is somewhat against the philosophy of comm) One way would be to create a modified version of the vwait command that allow the event flags passed to Tcl_DoOneEvent to be specified. For comm, just the TCL_FILE_EVENTS would be processed. Another way would be to implement a mechanism like Tk_RestrictEvents, but apply it to the Tcl event loop (since comm doesn't require Tk). One of these approaches will be available in a future comm release as an optional component.
By default the result returned by a remotely invoked command is the result sent back to the invoker. This means that the result is generated synchronously, and the server handling the call is blocked for the duration of the command.
While this is tolerable as long as only short-running commands are invoked on the server long-running commands, like database queries make this a problem. One command can prevent the processing requests of all other clients for an arbitrary period of time.
Before version 4.5 of comm the only solution was to rewrite the server command to use the Tcl builtin command vwait, or one of its relatives like tkwait, to open a new event loop which processes requests while the long-running operation is executed. This however has its own perils, as this makes it possible to both overflow the Tcl stack with a large number of event loop, and to have a newer requests block the return of older ones, as the eventloop have to be unwound in the order of their creation.
The proper solution is to have the invoked command indicate to comm that it cannot or will not deliver an immediate, synchronous result, but will do so later. At that point the framework can put sending the actual result on hold and continue processing requests using the main event loop. No blocking, no nesting of event loops. At some future date the long running operation delivers the result to comm, via the future object, which is then forwarded to the invoker as usual.
The necessary support for this solution has been added to comm since version 4.5, in the form of the new method return_async.
The result of return_async is an object. This object, called a future is where the result of the script has to be delivered to when it becomes ready. When that happens it will take all the necessary actions to deliver the result to the invoker of the script, and then destroy itself. Should comm have lost the connection to the invoker while the result is being computed the future will not try to deliver the result it got, but just destroy itself. The future can be configured with a command to call when the invoker is lost. This enables the user to implement an early abort of the long-running operation, should this be supported by it.
An example:
# Procedure invoked by remote clients to run database operations.
proc select {sql} {
# Signal the async generation of the result
set future [::comm::comm return_async]
# Generate an async db operation and tell it where to deliver the result.
set query [db query -command [list $future return] $sql]
# Tell the database system which query to cancel if the connection
# goes away while it is running.
$future configure -command [list db cancel $query]
# Note: The above will work without problem only if the async
# query will nover run its completion callback immediately, but
# only from the eventloop. Because otherwise the future we wish to
# configure may already be gone. If that is possible use 'catch'
# to prevent the error from propagating.
return
}
The API of a future object is:
The future will deliver this information to invoker, if the connection was not lost in the meantime, and then destroy itself. If the connection was lost it will do nothing but destroy itself.
comm exports itself as a package. The package version number is in the form major . minor, where the major version will only change when a non-compatible change happens to the API or protocol. Minor bug fixes and changes will only affect the minor version. To load comm this command is usually used:
package require comm 3
Note that requiring no version (or a specific version) can also be done.
The revision history of comm includes these releases:
The after idle can spill into a forked child process if there is no event loop between its setup and the fork. This may bork the child if the next event loop is the vwait of comm's send a few lines above the after idle, and the child used the same serial number for its next request. In that case the parent's after idle unset will delete the very array element the child is waiting for, unlocking the vwait, causing it to access a now missing array element, instead of the expected result.
comm send id break
catch {comm send id break}
comm send id expr 1 / 0
Added a new hook for reply messages. Reworked method invocation to avoid the use of comm:* procedures; this also cut the invocation time down by 40%. Documented comm config (as this manual page still listed the defunct comm init!)
John LoVerso, John@LoVerso.Southborough.MA.US
http://www.opengroup.org/~loverso/tcl-tk/#comm
Please see the file comm.LICENSE that accompanied this source, or http://www.opengroup.org/www/dist_client/caubweb/COPYRIGHT.free.html.
This license for comm, new as of version 3.2, allows it to be used for free, without any licensing fee or royalty.
The following items can be implemented with the existing hooks and are listed here as a reminder to provide a sample hook in a future version.
The following are outstanding todo items.
This man page is bigger than the source file.
Tcl7.5 under Windows contains a bug that causes the interpreter to hang when EOF is reached on non-blocking sockets. This can be triggered with a command such as this:
"comm send $other exit"
Always make sure the channel is quiescent before closing/exiting or use at least Tcl7.6 under Windows.
Tcl7.6 on the Mac contains several bugs. It is recommended you use at least Tcl7.6p2.
Tcl8.0 on UNIX contains a socket bug that can crash Tcl. It is recommended you use Tcl8.0p1 (or Tcl7.6p2).
Tcl-DP provides an RPC-based remote execution interface, but is a compiled Tcl extension. See http://www.cs.cornell.edu/Info/Projects/zeno/Projects/Tcl-DP.html.
Michael Doyle <miked@eolas.com> has code that implements the Tcl-DP RPC interface using standard Tcl sockets, much like comm.
Andreas Kupries <andreas_kupries@users.sourceforge.net> uses comm and has built a simple nameserver as part of his Pool library. See http://www.purl.org/net/akupries/soft/pool/index.htm.
This document, and the package it describes, will undoubtedly contain bugs and other problems. Please report such in the category comm of the Tcllib SF Trackers [http://sourceforge.net/tracker/?group_id=12883]. Please also report any ideas for enhancements you may have for either package and/or documentation.
send(n)
comm, communication, ipc, message, remote communication, remote execution, rpc, secure, send, socket, ssl, tls
Programming tools
Copyright (c) 1995-1998 The Open Group. All Rights Reserved. Copyright (c) 2003-2004 ActiveState Corporation. Copyright (c) 2006-2009 Andreas Kupries <andreas_kupries@users.sourceforge.net>
| 4.6.1 | comm |