mozilla

The Plugin System

Circus comes with a plugin system which let you interact with circusd.

Note

We might add circusd-stats support to plugins later on

A Plugin is composed of two parts:

  • a ZMQ subscriber to all events published by circusd
  • a ZMQ client to send commands to circusd

Each plugin is run as a separate process under a custom watcher.

A few examples of some plugins you could create with this system:

  • a notification system that sends e-mail alerts when a watcher is flapping
  • a logger
  • a tool that add or remove processes depending on the load
  • etc.

Circus itself provides a few plugins:

  • a statsd plugin, that sends to statsd all events emited by circusd
  • the flapping feature which avoid to re-launch processes infinitely when they die too quickly.
  • many more to come !

The CircusPlugin class

Circus provides a base class to help you implement plugins: circus.plugins.CircusPlugin

class circus.plugins.CircusPlugin(endpoint, pubsub_endpoint, check_delay, ssh_server=None, **config)

Base class to write plugins.

Options:

  • context – the ZMQ context to use
  • endpoint – the circusd ZMQ endpoint
  • pubsub_endpoint – the circusd ZMQ pub/sub endpoint
  • check_delay – the configured check delay
  • config – free config mapping
call(command, **props)

Sends to circusd the command.

Options:

  • command – the command to call
  • props – keywords argument to add to the call

Returns the JSON mapping sent back by circusd

cast(command, **props)

Fire-and-forget a command to circusd

Options:

  • command – the command to call
  • props – keywords argument to add to the call
handle_recv(data)

Receives every event published by circusd

Options:

  • data – a tuple containing the topic and the message.
handle_stop()

Called right before the plugin is stopped by Circus.

handle_init()

Called right befor a plugin is started - in the thread context.

When initialized by Circus, this class creates its own event loop that receives all circusd events and pass them to handle_recv(). The data received is a tuple containing the topic and the data itself.

handle_recv() must be implemented by the plugin.

The call() and cast() methods can be used to interact with circusd if you are building a Plugin that actively interacts with the daemon.

handle_init() and handle_stop() are just convenience methods you can use to initialize and clean up your code. handle_init() is called within the thread that just started. handle_stop() is called in the main thread just before the thread is stopped and joined.

Writing a plugin

Let’s write a plugin that logs in a file every event happening in circusd. It takes one argument which is the filename.

The plugin could look like this:

from circus.plugins import CircusPlugin


class Logger(CircusPlugin):

    name = 'logger'

    def __init__(self, filename, **kwargs):
        super(Logger, self).__init__(**kwargs)
        self.filename = filename
        self.file = None

    def handle_init(self):
        self.file = open(self.filename, 'a+')

    def handle_stop(self):
        self.file.close()

    def handle_recv(self, data):
        topic, msg = data
        self.file.write('%s::%s' % (topic, msg))

That’s it ! This class can be saved in any package/module, as long as it can be seen by Python.

For example, Logger could be found in a plugins module in a myproject package.

Async requests

In case you want to make any asynchronous operations (like a Tornado call or using periodicCall) make sure you are using the right loop. The loop you always want to be using is self.loop as it gets set up by the base class. The default loop often isn’t the same and therefore code might not get executed as expected.

Trying a plugin

You can run a plugin through the command line with the circus-plugin command, by specifying the plugin fully qualified name:

$ circus-plugin --endpoint tcp://127.0.0.1:5555 --pubsub tcp://127.0.0.1:5556 myproject.plugins.Logger
[INFO] Loading the plugin...
[INFO] Endpoint: 'tcp://127.0.0.1:5555'
[INFO] Pub/sub: 'tcp://127.0.0.1:5556'
[INFO] Starting

Another way to run a plugin is to let Circus handle its initialization. This is done by adding a [plugin:NAME] section in the configuration file, where NAME is a unique name for your plugin:

[plugin:logger]
use = myproject.plugins.Logger
filename = /var/myproject/circus.log

use is mandatory and points to the fully qualified name of the plugin.

When Circus starts, it creates a watcher with one process that runs the pointed class, and pass any other variable contained in the section to the plugin constructor via the config mapping.

You can also programmatically add plugins when you create a circus.arbiter.Arbiter class or use circus.get_arbiter(), see Circus Library.

Performances

Since every plugin is loaded in its own process, it should not impact the overall performances of the system as long as the work done by the plugin is not doing too many calls to the circusd process.