process-queue🔗ℹ

This library implements a queue to manage the execution of many OS level processes in parallel.

The process will describes what to do with a process when it terminates. For instance, the will might collect the results of the process, perform cleanup, and even enqueue a new process to "follow up" on the one that just terminated. In detail, the will is a function accepting the current state of the process queue and the process’s process-info and producing an updated process queue.

The process queue keeps track of the processes actively running and those waiting to run. The operations on the process queue add processes to the waiting list, manage process termination and will execution, and wait for all processes to terminate.

For convenience, the process queue also has a field for storing client data. This is useful for storing information related to processes on the side, since the queue is accessible to process wills.

This library provides several implementations of this basic concept which all conform to the same Process queue interface. First, Functional process queues implements the basic process queue in a functional style (such that the process queue is immutable). Second, Functional process priority queue implements a priority-queue version in a functional style. Third, Imperative process queues implements an imperative version of the basic process queue. Finally, Imperative process priority queue implements a priority-queue version in an imperative style.

1 Example usage🔗ℹ

This is an illustrative example of the usage of the Imperative process queues implementation.

It creates a queue that runs up to two processes simultaneously, and then enqueues three processes to be run (1, 2, and 4).

The first two are launched immediately. The second process terminates quickly, and its process will enqueues a followup process (3).

Process 4 is then launched, and after it terminates process 3 is launched and terminates.

Finally process 1, which had been running since the start, terminates.

2 Process queue interface🔗ℹ

All of the implementations below provide the following operations on process queues.

extra-data provides optional extra information that may or may not be used depending on the implementation (for example, a priority value).

Returns the updated process queue.

3 Imperative process queues🔗ℹ

These imperative process queue implementations mutate a single process queue in-place. Hence, all of the Process queue interface operations that return a new process queue simply return the input queue after mutating it. (The interface returns the queue to support the Functional process queues.)

3.1 Plain imperative process queue🔗ℹ

active-limit is the maximum number of processes that can be active at once.

data initializes the data field of the queue which can be accessed with process-queue-get-data and process-queue-set-data.

process-timeout-seconds, if non-false, specifies a "best effort" limit on the real running time of each process in seconds. Best effort here means that the timeout is not strictly enforced in terms of timing — i.e. a process may run for longer than process-timeout-seconds. Instead, the library checks for over-running processes periodically while performing other operations on the queue and kills any that it finds. This is useful as a crude way to ensure that no process runs forever. If you need precise/strict timeout enforcement, you might consider using the timeout unix utility or other racket tools, and using process-timeout-seconds as a fallback.

3.2 Imperative process priority queue🔗ℹ

Imperative process priority queues prioritize processes to run first using a sorting function, provided when creating the queue. These queues use the third argument of process-queue-enqueue as the priority, which defaults to 0 if not provided.

See Plain imperative process queue for more details on the remaining arguments.

4 Functional process queues🔗ℹ

These implementations mirror the imperative versions, but all queue operations functionally transform the queue instead of mutating it in-place.

That means that you (the user of this library) must thread the queue around your program and take care never to use a stale queue: only the latest queue is valid, because queue values reflect the state of external and stateful processes. Hence, the functional implementations are a little strange, and you’re probably better off using the Imperative process queues. That said, sometimes a functional interface fits better with the rest of one’s program structure, caveats and all.

4.1 Plain functional process queue🔗ℹ

See Plain imperative process queue for more details on the remaining arguments.

4.2 Functional process priority queue🔗ℹ

Functional process priority queues prioritize processes to run first using a sorting function, provided when creating the queue. These queues use the third argument of process-queue-enqueue as the priority, which defaults to 0 if not provided.

See Plain imperative process queue for more details on the remaining arguments.