Ansible Integration
Zuul contains Ansible modules and plugins to control the execution of Ansible
Job content.
Zuul provides realtime build log streaming to end users so that users
can watch long-running jobs in progress.
Streaming job output
All jobs run with the zuul.ansible.base.callback.zuul_stream callback
plugin enabled, which writes the build log to a file so that the
zuul.lib.log_streamer.LogStreamer can provide the data on demand
over the finger protocol. Finally, zuul.web.LogStreamHandler
exposes that log stream over a websocket connection as part of
- class zuul.ansible.base.callback.zuul_stream.CallbackModule
This is the Zuul streaming callback. It’s based on the default callback plugin, but streams results from shell commands.
- class zuul.lib.log_streamer.LogStreamer(host, port, jobdir_root)
Class implementing log streaming over the finger daemon port.
- class zuul.web.LogStreamHandler(*args, **kw)
- class zuul.web.ZuulWeb(config, connections, authenticators, info=None)
In addition to real-time streaming, Zuul also installs another callback module,
zuul.ansible.base.callback.zuul_json.CallbackModule that collects all
of the information about a given run into a json file which is written to the
work dir so that it can be published along with build logs.
- class zuul.ansible.base.callback.zuul_json.CallbackModule(display=None)
Since the streaming log is by necessity a single text stream, choices
have to be made for readability about what data is shown and what is
not shown. The json log file is intended to allow for a richer more
interactive set of data to be displayed to the user.
Capturing live command output
As jobs may execute long-running shell scripts or other commands,
additional effort is expended to stream stdout and stderr of
shell tasks as they happen rather than waiting for the command to
finish.
The global job configuration should run the zuul_console task as a
very early prerequisite step.
This will start a daemon that listens on TCP port 19885 on the testing
node. This daemon can be queried to stream back the output of shell
tasks as described below.
Zuul contains a modified version of Ansible’s
Ansible command module module that overrides the default
implementation.
This library will capture the output of the running
command and write it to a temporary file on the host the command is
running on. These files are named in the format
/tmp/console-<uuid>-<task_id>-<host>.log
The zuul_stream callback mentioned above will send a request to
the remote zuul_console daemon, providing the uuid and task id of
the task it is currently processing. The zuul_console daemon will
then read the logfile from disk and stream the data back as it
appears, which zuul_stream will then present as described above.
The zuul_stream callback will indicate to the zuul_console
daemon when it has finished reading the task, which prompts the remote
side to remove the temporary streaming output files. In some cases,
aborting the Ansible process may not give the zuul_stream callback
the chance to send this notice, leaking the temporary files. If nodes
are ephemeral this makes little difference, but these files may be
visible on static nodes.