Gerrit

Gerrit is a code review system. The Gerrit driver supports sources, triggers, and reporters.

Zuul will need access to a Gerrit user.

Give that user whatever permissions will be needed on the projects you want Zuul to report on. For instance, you may want to grant Verified +/-1 and Submit to the user. Additional categories or values may be added to Gerrit. Zuul is very flexible and can take advantage of those.

If change.submitWholeTopic is configured in Gerrit, Zuul will honor this by enqueing changes with the same topic as circular dependencies. However, it is still necessary to enable circular dependency support in any pipeline queues where such changes may appear. See queue.allow-circular-dependencies for information on how to configure this.

Zuul interacts with Gerrit in up to three ways:

Receiving trigger events
Fetching source code
Reporting results

Trigger events arrive over an event stream, either SSH (via the gerrit stream-events command) or other protocols such as Kafka, or AWS Kinesis.

Fetching source code may happen over SSH or HTTP.

Reporting may happen over SSH or HTTP (strongly preferred).

The appropriate connection methods must be configured to satisfy the interactions Zuul will have with Gerrit. The recommended configuration is to configure both SSH and HTTP access.

The section below describes commond configuration settings. Specific settings for different connection methods follow.

Connection Configuration

The supported options in zuul.conf connections are:

<gerrit connection>

<gerrit connection>.driver (required)

gerrit: The connection must set driver=gerrit for Gerrit connections.

<gerrit connection>.server (required): Fully qualified domain name of Gerrit server.

<gerrit connection>.canonical_hostname: The canonical hostname associated with the git repos on the Gerrit server. Defaults to the value of <gerrit connection>.server. This is used to identify projects from this connection by name and in preparing repos on the filesystem for use by jobs. Note that Zuul will still only communicate with the Gerrit server identified by server; this option is useful if users customarily use a different hostname to clone or pull git repos so that when Zuul places them in the job’s working directory, they appear under this directory name.

<gerrit connection>.baseurl Default: https://{server}: Path to Gerrit web interface. Omit the trailing /.

<gerrit connection>.gitweb_url_template Default: {baseurl}/gitweb?p={project.name}.git;a=commitdiff;h={sha}

Url template for links to specific git shas. By default this will point at Gerrit’s built in gitweb but you can customize this value to point elsewhere (like cgit or github).

The three values available for string interpolation are baseurl which points back to Gerrit, project and all of its safe attributes, and sha which is the git sha1.

<gerrit connection>.max_dependencies: This setting can be used to limit the number of dependencies that Zuul will consider when processing events from Gerrit. If used, it should be set to a value that is higher than the highest number of dependencies that are expected to be encountered. If, when processing an event from Gerrit, Zuul detects that the dependencies will exceed this value, Zuul will ignore the event with no feedback to the user. This is meant only to protect the Zuul server from resource exhaustion when excessive dependencies are present. The default (unset) is no limit. Note that the value 0 does not disable this option; instead it limits Zuul to zero dependencies. This is distinct from tenant.max-dependencies.

<gerrit connection>.user Default: zuul: User name to use when accessing Gerrit.

SSH Configuration

To prepare for SSH access, create an SSH keypair for Zuul to use if there isn’t one already, and create a Gerrit user with that key:

cat ~/id_rsa.pub | ssh -p29418 review.example.com gerrit create-account --ssh-key - --full-name Zuul zuul

Note

If you use an RSA key, ensure it is encoded in the PEM format (use the -t rsa -m PEM arguments to ssh-keygen).

If using Gerrit 2.7 or later, make sure the user is a member of a group that is granted the Stream Events permission, otherwise it will not be able to invoke the gerrit stream-events command over SSH.

<gerrit ssh connection>

<gerrit ssh connection>.ssh_server: If SSH access to the Gerrit server should be via a different hostname than web access, set this value to the hostname to use for SSH connections.

<gerrit ssh connection>.port Default: 29418: Gerrit SSH server port.

<gerrit ssh connection>.sshkey Default: ~zuul/.ssh/id_rsa: Path to SSH key to use when logging into Gerrit.

<gerrit ssh connection>.keepalive Default: 60: SSH connection keepalive timeout; 0 disables.

<gerrit ssh connection>.git_over_ssh Default: false: This forces git operation over SSH even if the password attribute is set. This allow REST API access to the Gerrit server even when git-over-http operation is disabled on the server.

HTTP Configuration

<gerrit ssh connection>

<gerrit ssh connection>.password: The HTTP authentication password for the user. This is optional, but if it is provided, Zuul will report to Gerrit via HTTP rather than SSH. It is required in order for file and line comments to reported (the Gerrit SSH API only supports review messages). Retrieve this password from the HTTP Password section of the Settings page in Gerrit.

<gerrit ssh connection>.auth_type Default: basic

The HTTP authentication mechanism.

basic: HTTP Basic authentication; the default for most Gerrit installations.

digest: HTTP Digest authentication; only used in versions of Gerrit prior to 2.15.

form: Zuul will submit a username and password to a form in order to authenticate.

gcloud_service: Only valid when running in Google Cloud. This will use the default service account to authenticate to Gerrit. Note that this will only be used for interacting with the Gerrit API; anonymous HTTP access will be used to access the git repositories, therefore private repos or draft changes will not be available.

<gerrit ssh connection>.verify_ssl Default: true: When using a self-signed certificate, this may be set to false to disable SSL certificate verification.

Kafka Event Support

Zuul includes support for Gerrit’s events-kafka plugin. This may be used as an alternative to SSH for receiving trigger events.

Kafka does provide event delivery guarantees, so unlike SSH, if all Zuul schedulers are unable to communicate with Gerrit or Kafka, they will eventually receive queued events on reconnection.

All Zuul schedulers will attempt to connect to Kafka brokers. There are some implications for event delivery:

All events will be delivered to Zuul at least once. In the case of a disrupted connection, Zuul may receive duplicate events.
Events should generally arrive in order, however some events in rapid succession may be received by Zuul out of order.

<gerrit kafka connection>

<gerrit kafka connection>.kafka_bootstrap_servers (required): A comma-separated list of Kafka servers (optionally including port separated with :).

<gerrit kafka connection>.kafka_topic Default: gerrit: The Kafka topic to which Zuul should subscribe.

<gerrit kafka connection>.kafka_client_id Default: zuul: The Kafka client ID.

<gerrit kafka connection>.kafka_group_id Default: zuul: The Kafka group ID.

<gerrit kafka connection>.kafka_tls_cert: Path to TLS certificate to use when connecting to a Kafka broker.

<gerrit kafka connection>.kafka_tls_key: Path to TLS certificate key to use when connecting to a Kafka broker.

<gerrit kafka connection>.kafka_tls_ca: Path to TLS CA certificate to use when connecting to a Kafka broker.

AWS Kinesis Event Support

Zuul includes support for Gerrit’s events-aws-kinesis plugin. This may be used as an alternative to SSH for receiving trigger events.

Kinesis does provide event delivery guarantees, so unlike SSH, if all Zuul schedulers are unable to communicate with Gerrit or AWS, they will eventually receive queued events on reconnection.

All Zuul schedulers will attempt to connect to AWS Kinesis, but only one scheduler will process a given Kinesis shard at a time. There are some implications for event delivery:

All events will be delivered to Zuul at least once. In the case of a disrupted connection, Zuul may receive duplicate events.
If a connection is disrupted longer than the Kinesis retention period for a shard, Zuul may skip to the latest event ignoring all previous events.
Because shard processing happens in parallel, events may not arrive in order.
If a long period with no events elapses and a connection is disrupted, it may take Zuul some time to catch up to the latest events.

<gerrit aws kinesis connection>

<gerrit aws kinesis connection>.aws_kinesis_region (required): The AWS region name in which the Kinesis stream is located.

<gerrit aws kinesis connection>.aws_kinesis_stream Default: gerrit: The AWS Kinesis stream name.

<gerrit aws kinesis connection>.aws_kinesis_access_key: The AWS access key to use.

<gerrit aws kinesis connection>.aws_kinesis_secret_key: The AWS secret key to use.

Trigger Configuration

pipeline.trigger.<gerrit source>

The dictionary passed to the Gerrit pipeline trigger attribute supports the following attributes:

pipeline.trigger.<gerrit source>.event (required): The event name from gerrit. Examples: patchset-created, comment-added, ref-updated. This field is treated as a regular expression.

pipeline.trigger.<gerrit source>.branch: The branch associated with the event. Example: master. This field is treated as a regular expression, and multiple branches may be listed.

pipeline.trigger.<gerrit source>.ref: On ref-updated events, the branch parameter is not used, instead the ref is provided. Currently Gerrit has the somewhat idiosyncratic behavior of specifying bare refs for branch names (e.g., master), but full ref names for other kinds of refs (e.g., refs/tags/foo). Zuul matches this value exactly against what Gerrit provides. This field is treated as a regular expression, and multiple refs may be listed.

pipeline.trigger.<gerrit source>.ignore-deletes Default: true: When a branch is deleted, a ref-updated event is emitted with a newrev of all zeros specified. The ignore-deletes field is a boolean value that describes whether or not these newrevs trigger ref-updated events.

pipeline.trigger.<gerrit source>.approval: This is only used for comment-added events. It only matches if the event has a matching approval associated with it. Example: Code-Review: 2 matches a +2 vote on the code review category. Multiple approvals may be listed.

pipeline.trigger.<gerrit source>.email: This is used for any event. It takes a regex applied on the performer email, i.e. Gerrit account email address. If you want to specify several email filters, you must use a YAML list. Make sure to use non greedy matchers and to escapes dots! Example: email: ^.*?@example\.org$.

pipeline.trigger.<gerrit source>.username: This is used for any event. It takes a regex applied on the performer username, i.e. Gerrit account name. If you want to specify several username filters, you must use a YAML list. Make sure to use non greedy matchers and to escapes dots. Example: username: ^zuul$.

pipeline.trigger.<gerrit source>.comment: This is only used for comment-added events. It accepts a list of regexes that are searched for in the comment string. If any of these regexes matches a portion of the comment string the trigger is matched. comment: retrigger will match when comments containing retrigger somewhere in the comment text are added to a change.

pipeline.trigger.<gerrit source>.require-approval

Warning

This is deprecated and will be removed in a future version. Use pipeline.trigger.<gerrit source>.require instead.

This may be used for any event. It requires that a certain kind of approval be present for the current patchset of the change (the approval could be added by the event in question). It follows the same syntax as pipeline.require.<gerrit source>.approval. For each specified criteria there must exist a matching approval.

This is ignored if the pipeline.trigger.<gerrit source>.require attribute is present.

pipeline.trigger.<gerrit source>.reject-approval

Warning

This is deprecated and will be removed in a future version. Use pipeline.trigger.<gerrit source>.reject instead.

This takes a list of approvals in the same format as pipeline.trigger.<gerrit source>.require-approval but the item will fail to enter the pipeline if there is a matching approval.

This is ignored if the pipeline.trigger.<gerrit source>.reject attribute is present.

pipeline.trigger.<gerrit source>.require: This may be used for any event. It describes conditions that must be met by the change in order for the trigger event to match. Those conditions may be satisfied by the event in question. It follows the same syntax as Requirements Configuration.

pipeline.trigger.<gerrit source>.reject: This may be used for any event and is the mirror of pipeline.trigger.<gerrit source>.require. It describes conditions that when met by the change cause the trigger event not to match. Those conditions may be satisfied by the event in question. It follows the same syntax as Requirements Configuration.

Reporter Configuration

pipeline.reporter.<gerrit reporter>

The dictionary passed to the Gerrit reporter is used to provide label values to Gerrit. To set the Verified label to 1, add verified: 1 to the dictionary.

The following additional keys are recognized:

pipeline.reporter.<gerrit reporter>.submit Default: False: Set this to True to submit (merge) the change.

pipeline.reporter.<gerrit reporter>.comment Default: True: If this is true (the default), Zuul will leave review messages on the change (including job results). Set this to false to disable this behavior (file and line commands will still be sent if present).

A connection that uses the gerrit driver must be supplied to the trigger.

Requirements Configuration

As described in pipeline.require and pipeline.reject, pipelines may specify that items meet certain conditions in order to be enqueued into the pipeline. These conditions vary according to the source of the project in question. To supply requirements for changes from a Gerrit source named my-gerrit, create a configuration such as the following:

pipeline:
  require:
    my-gerrit:
      approval:
        - Code-Review: 2

This indicates that changes originating from the Gerrit connection named my-gerrit must have a Code-Review vote of +2 in order to be enqueued into the pipeline.

pipeline.require.<gerrit source>

The dictionary passed to the Gerrit pipeline require attribute supports the following attributes:

pipeline.require.<gerrit source>.approval

This requires that a certain kind of approval be present for the current patchset of the change (the approval could be added by the event in question). Approval is a dictionary or a list of dictionaries with attributes listed below, all of which are optional and are combined together so that there must be an approval matching all specified requirements.

pipeline.require.<gerrit source>.approval.username: If present, an approval from this username is required. It is treated as a regular expression.

pipeline.require.<gerrit source>.approval.email: If present, an approval with this email address is required. It is treated as a regular expression.

pipeline.require.<gerrit source>.approval.older-than: If present, the approval must be older than this amount of time to match. Provide a time interval as a number with a suffix of “w” (weeks), “d” (days), “h” (hours), “m” (minutes), “s” (seconds). Example 48h or 2d.

pipeline.require.<gerrit source>.approval.newer-than: If present, the approval must be newer than this amount of time to match. Same format as “older-than”.

Any other field is interpreted as a review category and value pair. For example Verified: 1 would require that the approval be for a +1 vote in the “Verified” column. The value may either be a single value or a list: Verified: [1, 2] would match either a +1 or +2 vote.

pipeline.require.<gerrit source>.open: A boolean value (true or false) that indicates whether the change must be open or closed in order to be enqueued.

pipeline.require.<gerrit source>.current-patchset: A boolean value (true or false) that indicates whether the change must be the current patchset in order to be enqueued.

pipeline.require.<gerrit source>.wip: A boolean value (true or false) that indicates whether the change must be wip or not wip in order to be enqueued.

pipeline.require.<gerrit source>.status: A string value that corresponds with the status of the change reported by Gerrit.

pipeline.reject.<gerrit source>

The reject attribute is the mirror of the require attribute. It also accepts a dictionary under the connection name. This dictionary supports the following attributes:

pipeline.reject.<gerrit source>.approval

This requires that a certain kind of approval not be present for the current patchset of the change (the approval could be added by the event in question). Approval is a dictionary or a list of dictionaries with attributes listed below, all of which are optional and are combined together so that there must be no approvals matching all specified requirements.

Example to reject a change with any negative vote:

reject:
  my-gerrit:
    approval:
      - Code-Review: [-1, -2]

pipeline.reject.<gerrit source>.approval.username: If present, an approval from this username is required. It is treated as a regular expression.

pipeline.reject.<gerrit source>.approval.email: If present, an approval with this email address is required. It is treated as a regular expression.

pipeline.reject.<gerrit source>.approval.older-than: If present, the approval must be older than this amount of time to match. Provide a time interval as a number with a suffix of “w” (weeks), “d” (days), “h” (hours), “m” (minutes), “s” (seconds). Example 48h or 2d.

pipeline.reject.<gerrit source>.approval.newer-than: If present, the approval must be newer than this amount of time to match. Same format as “older-than”.

Any other field is interpreted as a review category and value pair. For example Verified: 1 would require that the approval be for a +1 vote in the “Verified” column. The value may either be a single value or a list: Verified: [1, 2] would match either a +1 or +2 vote.

pipeline.reject.<gerrit source>.open: A boolean value (true or false) that indicates whether the change must be open or closed in order to be rejected.

pipeline.reject.<gerrit source>.current-patchset: A boolean value (true or false) that indicates whether the change must be the current patchset in order to be rejected.

pipeline.reject.<gerrit source>.wip: A boolean value (true or false) that indicates whether the change must be wip or not wip in order to be rejected.

pipeline.reject.<gerrit source>.status: A string value that corresponds with the status of the change reported by Gerrit.

Reference Pipelines Configuration

Here is an example of standard pipelines you may want to define:

- pipeline:
    name: check
    description: |
      Newly uploaded patchsets enter this pipeline to receive an
      initial +/-1 Verified vote.
    manager: independent
    precedence: low
    require:
      gerrit:
        open: True
        current-patchset: True
    trigger:
      gerrit:
        - event: patchset-created
        - event: change-restored
        - event: comment-added
          comment: (?i)^(Patch Set [0-9]+:)?( [\w\\+-]*)*(\n\n)?\s*recheck
    success:
      gerrit:
        # Note that gerrit keywords are case-sensitive.
        Verified: 1
    failure:
      gerrit:
        Verified: -1

- pipeline:
    name: gate
    description: |
      Changes that have been approved by core reviewers are enqueued
      in order in this pipeline, and if they pass tests, will be
      merged. For documentation on how gating with Zuul works, please see
      https://zuul-ci.org/docs/zuul/latest/gating.html
    manager: dependent
    precedence: normal
    supercedes: check
    post-review: True
    require:
      gerrit:
        open: True
        current-patchset: True
        approval:
          - Workflow: 1
    trigger:
      gerrit:
        - event: comment-added
          approval:
            - Workflow: 1
    start:
      gerrit:
        Verified: 0
    success:
      gerrit:
        Verified: 2
        submit: true
    failure:
      gerrit:
        Verified: -2
    window-floor: 20
    window-increase-factor: 2

- pipeline:
    name: post
    description: |
      This pipeline runs jobs that operate after each change is
      merged. Queue items are identified by the abbreviated hash (git
      log --format=%h) of the merge commit.
    manager: supercedent
    precedence: high
    post-review: True
    trigger:
      gerrit:
        - event: ref-updated
          ref: ^refs/heads/.*$

- pipeline:
    name: promote
    description: |
      This pipeline runs jobs that operate after each change is merged
      in order to promote artifacts generated in the gate
      pipeline.
    manager: supercedent
    precedence: high
    post-review: True
    trigger:
      gerrit:
        - event: change-merged
    success:
      gerrit: {}
    failure:
      gerrit: {}

- pipeline:
    name: deploy
    description: |
      This pipeline runs jobs that operate after each change is merged
      in order to deploy to production.
    manager: serial
    precedence: high
    post-review: True
    trigger:
      gerrit:
        - event: change-merged
    success:
      gerrit: {}
    failure:
      gerrit: {}

- pipeline:
    name: release
    description: |
      When a commit is tagged as a release, this pipeline runs jobs
      that publish archives and documentation.
    manager: independent
    precedence: high
    post-review: True
    trigger:
      gerrit:
        - event: ref-updated
          ref: ^refs/tags/[0-9]+(\.[0-9]+)*$

- pipeline:
    name: tag
    post-review: true
    description: This pipeline runs jobs in response to any tag event.
    manager: independent
    precedence: high
    trigger:
      gerrit:
        - event: ref-updated
          ref: ^refs/tags/.*$

Checks Plugin Support (Deprecated)

The Gerrit driver has support for Gerrit’s checks plugin. Due to the deprecation of the checks plugin in Gerrit, support in Zuul is also deprecated and likely to be removed in a future version. It is not recommended for use.

Caveats include (but are not limited to):

This documentation is brief.
Access control for the checks API in Gerrit depends on a single global administrative permission, administrateCheckers. This is required in order to use the checks API and can not be restricted by project. This means that any system using the checks API can interfere with any other.
Checkers are restricted to a single project. This means that a system with many projects will require many checkers to be defined in Gerrit – one for each project+pipeline.
No support is provided for attaching checks to tags or commits, meaning that tag, release, and post pipelines are unable to be used with the checks API and must rely on stream-events.
Sub-checks are not implemented yet, so in order to see the results of individual jobs on a change, users must either follow the buildset link, or the pipeline must be configured to leave a traditional comment.
Familiarity with the checks API is recommended.
Checkers may not be permanently deleted from Gerrit (only “soft-deleted” so they no longer apply), so any experiments you perform on a production system will leave data there forever.

In order to use the checks API, you must have HTTP access configured in zuul.conf.

There are two ways to configure a pipeline for the checks API: directly referencing the checker UUID, or referencing it’s scheme. It is hoped that once multi-repository checks are supported, that an administrator will be able to configure a single checker in Gerrit for each Zuul pipeline, and those checkers can apply to all repositories. If and when that happens, we will be able to reference the checker UUID directly in Zuul’s pipeline configuration. If you only have a single project, you may find this approach acceptable now.

To use this approach, create a checker named zuul:check and configure a pipeline like this:

- pipeline:
    name: check
    manager: independent
    trigger:
      gerrit:
        - event: pending-check
          uuid: 'zuul:check'
    enqueue:
      gerrit:
        checks-api:
          uuid: 'zuul:check'
          state: SCHEDULED
          message: 'Change has been enqueued in check'
    start:
      gerrit:
        checks-api:
          uuid: 'zuul:check'
          state: RUNNING
          message: 'Jobs have started running'
    no-jobs:
      gerrit:
        checks-api:
          uuid: 'zuul:check'
          state: NOT_RELEVANT
          message: 'Change has no jobs configured'
    success:
      gerrit:
        checks-api:
          uuid: 'zuul:check'
          state: SUCCESSFUL
          message: 'Change passed all voting jobs'
    failure:
      gerrit:
        checks-api:
          uuid: 'zuul:check'
          state: FAILED
          message: 'Change failed'

For a system with multiple repositories and one or more checkers for each repository, the scheme approach is recommended. To use this, create a checker for each pipeline in each repository. Give them names such as zuul_check:project1, zuul_gate:project1, zuul_check:project2, etc. The part before the : is the scheme. Then create a pipeline like this:

- pipeline:
    name: check
    manager: independent
    trigger:
      gerrit:
        - event: pending-check
          scheme: 'zuul_check'
    enqueue:
      gerrit:
        checks-api:
          scheme: 'zuul_check'
          state: SCHEDULED
          message: 'Change has been enqueued in check'
    start:
      gerrit:
        checks-api:
          scheme: 'zuul_check'
          state: RUNNING
          message: 'Jobs have started running'
    no-jobs:
      gerrit:
        checks-api:
          scheme: 'zuul_check'
          state: NOT_RELEVANT
          message: 'Change has no jobs configured'
    success:
      gerrit:
        checks-api:
          scheme: 'zuul_check'
          state: SUCCESSFUL
          message: 'Change passed all voting jobs'
    failure:
      gerrit:
        checks-api:
          scheme: 'zuul_check'
          state: FAILED
          message: 'Change failed'

This will match and report to the appropriate checker for a given repository based on the scheme you provided.