The Circus package is composed of a high-level get_arbiter()
function and many classes. In most cases, using the high-level function
should be enough, as it creates everything that is needed for Circus to
run.
You can subclass Circus’ classes if you need more granularity than what is offered by the configuration.
get_arbiter()
is just a convenience on top of the various
circus classes. It creates an arbiter (class Arbiter
) instance
with the provided options, which in turn runs a single Watcher
with a
single Process
.
circus.
get_arbiter
(watchers, controller=None, pubsub_endpoint=None, statsd=False, stats_endpoint=None, statsd_close_outputs=False, multicast_endpoint=None, env=None, name=None, context=None, background=False, stream_backend='thread', httpd=False, plugins=None, debug=False, proc_name='circusd', loop=None, check_delay=1.0, **kw)¶Example:
from circus import get_arbiter
arbiter = get_arbiter([{"cmd": "myprogram", "numprocesses": 3}])
try:
arbiter.start()
finally:
arbiter.stop()
Circus provides a series of classes you can use to implement your own process manager:
Process
: wraps a running process and provides a few helpers on top
of it.Watcher
: run several instances of Process
against the same
command. Manage the death and life of processes.Arbiter
: manages several Watcher
instances.circus.process.
Process
(name, wid, cmd, args=None, working_dir=None, shell=False, uid=None, gid=None, env=None, rlimits=None, executable=None, use_fds=False, watcher=None, spawn=True, pipe_stdout=True, pipe_stderr=True, close_child_stdin=True, close_child_stdout=False, close_child_stderr=False)¶Wraps a process.
Options:
shlex.split()
. Defaults to None.age
()¶Return the age of the process in seconds.
children
(recursive=False)¶Return a list of children pids.
info
()¶Return process info.
The info returned is a mapping with these keys:
is_child
(pid)¶Return True is the given pid is a child of that process.
pid
¶Return the pid
send_signal
(sig)¶Sends a signal sig to the process.
send_signal_child
(pid, signum)¶Send signal signum to child pid.
send_signal_children
(signum, recursive=False)¶Send signal signum to all children.
status
¶Return the process status as a constant
stderr
¶Return the stdout stream
stdout
¶Return the stdout stream
stop
()¶Stop the process and close stdout/stderr
If the corresponding process is still here (normally it’s already killed by the watcher), a SIGTERM is sent, then a SIGKILL after 1 second.
The shutdown process (SIGTERM then SIGKILL) is normally taken by the watcher. So if the process is still there here, it’s a kind of bad behavior because the graceful timeout won’t be respected here.
Example:
>>> from circus.process import Process
>>> process = Process('Top', 'top', shell=True)
>>> process.age()
3.0107998847961426
>>> process.info()
'Top: 6812 N/A tarek Zombie N/A N/A N/A N/A N/A'
>>> process.status
1
>>> process.stop()
>>> process.status
2
>>> process.info()
'No such process (stopped?)'
circus.watcher.
Watcher
(name, cmd, args=None, numprocesses=1, warmup_delay=0.0, working_dir=None, shell=False, shell_args=None, uid=None, max_retry=5, gid=None, send_hup=False, stop_signal=<Signals.SIGTERM: 15>, stop_children=False, env=None, graceful_timeout=30.0, prereload_fn=None, rlimits=None, executable=None, stdout_stream=None, stderr_stream=None, priority=0, loop=None, singleton=False, use_sockets=False, copy_env=False, copy_path=False, max_age=0, max_age_variance=30, hooks=None, respawn=True, autostart=True, on_demand=False, virtualenv=None, stdin_socket=None, close_child_stdin=True, close_child_stdout=False, close_child_stderr=False, virtualenv_py_ver=None, use_papa=False, **options)¶Class managing a list of processes for a given command.
Options:
name: name given to the watcher. Used to uniquely identify it.
cmd: the command to run. May contain $WID, which will be replaced by wid.
args: the arguments for the command to run. Can be a list or
a string. If args is a string, it’s splitted using
shlex.split()
. Defaults to None.
numprocesses: Number of processes to run.
working_dir: the working directory to run the command in. If not provided, will default to the current working directory.
shell: if True, will run the command in the shell environment. False by default. warning: this is a security hazard.
uid: if given, is the user id or name the command should run with. The current uid is the default.
gid: if given, is the group id or name the command should run with. The current gid is the default.
send_hup: if True, a process reload will be done by sending the SIGHUP signal. Defaults to False.
stop_signal: the signal to send when stopping the process. Defaults to SIGTERM.
stop_children: send the stop_signal to the children too. Defaults to False.
env: a mapping containing the environment variables the command will run with. Optional.
rlimits: a mapping containing rlimit names and values that will be set before the command runs.
stdout_stream: a mapping that defines the stream for the process stdout. Defaults to None.
Optional. When provided, stdout_stream is a mapping containing up to three keys:
This mapping will be used to create a stream callable of the specified class. Each entry received by the callable is a mapping containing:
This is not supported on Windows.
stderr_stream: a mapping that defines the stream for the process stderr. Defaults to None.
Optional. When provided, stderr_stream is a mapping containing up to three keys: - class: the stream class. Defaults to circus.stream.FileStream - filename: the filename, if using a FileStream - max_bytes: maximum file size, after which a new output file is
opened. defaults to 0 which means no maximum size (only applicable with FileStream)
This mapping will be used to create a stream callable of the specified class.
Each entry received by the callable is a mapping containing:
This is not supported on Windows.
priority – integer that defines a priority for the watcher. When the Arbiter do some operations on all watchers, it will sort them with this field, from the bigger number to the smallest. (default: 0)
singleton – If True, this watcher has a single process. (default:False)
use_sockets – If True, the processes will inherit the file descriptors, thus can reuse the sockets opened by circusd. (default: False)
on_demand – If True, the processes will be started only at the first connection to the socket (default: False)
copy_env – If True, the environment in which circus is running run will be reproduced for the workers. This defaults to True on Windows as you cannot run any executable without the SYSTEMROOT variable. (default: False)
copy_path – If True, circusd sys.path is sent to the process through PYTHONPATH. You must activate copy_env for copy_path to work. (default: False)
max_age: If set after around max_age seconds, the process is replaced with a new one. (default: 0, Disabled)
max_age_variance: The maximum number of seconds that can be added to max_age. This extra value is to avoid restarting all processes at the same time. A process will live between max_age and max_age + max_age_variance seconds.
hooks: callback functions for hooking into the watcher startup and shutdown process. hooks is a dict where each key is the hook name and each value is a 2-tuple with the name of the callable or the callable itself and a boolean flag indicating if an exception occuring in the hook should not be ignored. Possible values for the hook name: before_start, after_start, before_spawn, after_spawn, before_stop, after_stop., before_signal, after_signal or extended_stats.
options – extra options for the worker. All options found in the configuration file for instance, are passed in this mapping – this can be used by plugins for watcher-specific options.
respawn – If set to False, the processes handled by a watcher will not be respawned automatically. (default: True)
virtualenv – The root directory of a virtualenv. If provided, the watcher will load the environment for its execution. (default: None)
stdin_socket: If not None, the socket with matching name is placed at file descriptor 0 (stdin) of the processes. default: None.
close_child_stdin: If True, closes the stdin after the fork. default: True.
close_child_stdout: If True, closes the stdout after the fork. default: False.
close_child_stderr: If True, closes the stderr after the fork. default: False.
use_papa: If True, use the papa process kernel for this process. default: False.
kill_process
(process, stop_signal=None, graceful_timeout=None)¶Kill process (stop_signal, graceful_timeout then SIGKILL)
kill_processes
(stop_signal=None, graceful_timeout=None)¶Kill all processes (stop_signal, graceful_timeout then SIGKILL)
manage_processes
()¶Manage processes.
notify_event
(topic, msg)¶Publish a message on the event publisher channel
reap_and_manage_processes
()¶Reap & manage processes.
reap_processes
()¶Reap all the processes for this watcher.
send_signal_child
(pid, child_id, signum)¶Send signal to a child.
spawn_process
(recovery_wid=None)¶Spawn process.
Return True if ok, False if the watcher must be stopped
spawn_processes
()¶Spawn processes.