Description

Run multiple subprocesses connected via pipes.

Author

Hans Bulfone

Version

Usage

(require-extension pipeline)

Download

pipeline.egg

Documentation

This extension can start multiple subprocesses arbitrarily connected via pipes and scheme ports.

Running a pipeline

procedure: (run-pipeline PROCESSES ...)
Run a pipeline consisting of PROCESSES and return a vector containing the termination status of each process as pair (ok? . status). ok? is #t if the process exited normally and #f otherwise. status is either the exit status or the signal number that killed the process.

Defining a process

procedure: (pipeline:process CMD ARGS ENV REDIRECTIONS ...)
Defines a process of a pipeline. CMD, ARGS and ENV are passed to process-execute when the pipeline is run. REDIRECTIONS define I/O-redirections to be set up when executing the process (see below).

Defining I/O-redirections

procedure: (pipeline:input-from-pipe FD PIPE-NAME)
procedure: (pipeline:output-to-pipe FD PIPE-NAME)
These functions are for defining pipes beetween two processes that belong to the same pipeline. FD is an integer naming the file descriptor to redirect. PIPE-NAME is a symbol naming the pipe. A single PIPE-NAME should be used for exactly one pipeline:input-from-pipe and one pipeline:output-to-pipe definition (in two different processes of course).
procedure: (pipeline:input-from-file FD FILENAME [MODE])
procedure: (pipeline:output-to-file FD FILENAME [MODE])
procedure: (pipeline:append-output-to-file FD FILENAME [MODE])
These functions define a redirection from the given FD (an integer file descriptor) to the given FILENAME. pipeline:output-to-file truncates the output file if it already exists, pipeline:append-output-to-file appends to it. Both procedures create the output file (with permission bits MODE defaulting to #o666) if it does not already exist.
procedure: (pipeline:input-from-port FD PRODUCER)
procedure: (pipeline:output-to-port FD CONSUMER)
These functions define redirections from file descriptors to scheme ports. FD is the integer file descriptor to be redirected. PRODUCER is a procedure taking an output port. CONSUMER is a procedure taking an input port. PRODUCER and CONSUMER are run in their own threads; blocking reads/writes from/to the given port do not block other threads. The procedures should not close the ports as they are closed automatically after they return. The threads spawned for the procedures are joined before run-pipeline returns.

Examples

This example invokes tar to create an archive of some files and pipes the output through gpg for encryption. The encryption passphrase is given to gpg on file descriptor 3. The output of tar and gpg are saved.

(let ((passphrase "foobar")
      (tar-stderr #f)
      (gpg-stdout #f)
      (gpg-stderr #f))
  (run-pipeline
   (pipeline:process
    "tar" (append '("-czf" "-") files) #f
    (pipeline:input-from-file 0 "/dev/null")
    (pipeline:output-to-pipe 1 'tar-out)
    (pipeline:output-to-port 2 (lambda (port)
				 (set! tar-stderr (read-all port)))))

   (pipeline:process
    "gpg" (list "-o" outfile "--symmetric" "--passphrase-fd=3" "-") #f
    (pipeline:input-from-pipe 0 'tar-out)
    (pipeline:output-to-port 1 (lambda (port)
				 (set! gpg-stdout (read-all port))))
    (pipeline:output-to-port 2 (lambda (port)
				 (set! gpg-stderr (read-all port))))
    (pipeline:input-from-port 3 (lambda (port)
				  (display passphrase port))))))

License

Copyright (c) 2005-2006, Hans Bulfone
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

    * Redistributions of source code must retain the above copyright notice,
      this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.
    * Neither the name of the author nor the names of his contributors may
      be used to endorse or promote products derived from this software
      without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.