CI configuration

Jenkins pipelines, GitHub workflows, direct integrations with the CLI can use a .yontrack/ci.yaml file for an easy setup of the Yontrack items based on the available information.

Note

Prior to Yontrack V5, the usual way of feeding data into Yontrack was to use the GraphQL CLI, either directly or via integrations like Jenkins pipeline steps, GitHub actions, CLI, etc.

This is still possible, but the new "CI configuration" is the recommended way to feed data into Yontrack.

This .yontrack/ci.yaml file acts a unique way to configure the elements of your pipeliens into Yontrack, and is compatible with all the support CI tools.

For example, you can just use one unique endpoint to send a configuration file to Yontrack:

version: v1
configuration:
  defaults:
    branch:
      validations:
        BUILD:
          tests: { }
        UI_UNIT:
          tests: { }
        KDSL.ACCEPTANCE:
          tests: { }
        PLAYWRIGHT:
          tests: { }
      promotions:
        BRONZE:
          validations:
            - BUILD
            - UI_UNIT
            - KDSL.ACCEPTANCE
            - PLAYWRIGHT
        RELEASE:
          promotions:
            - BRONZE
          validations:
            - GITHUB.RELEASE
      notificationsConfig:
        notifications:
          - name: On validation error
            events:
              - new_validation_run
            keywords: failed
            channel: slack
            channelConfig:
              channel: '#notifications'
              type: 'ERROR'
            contentTemplate: |
              Build ${build} has failed on ${validationStamp}.
          - name: On BRONZE
            promotion: BRONZE
            events:
              - new_promotion_run
            channel: slack
            channelConfig:
              channel: '#notifications'
              type: 'SUCCESS'
            contentTemplate: |
              Build ${build} has been promoted to ${promotionLevel}.
          - name: On internal release
            promotion: RELEASE
            events:
              - new_promotion_run
            channel: slack
            channelConfig:
              channel: '#internal-releases'
              type: 'SUCCESS'
            contentTemplate: |
              Yontrack ${build} has been released.

              ${promotionRun.changelog?title=true&commitsOption=ALWAYS}
  custom:
    configs:
      - conditions:
          name: branch
          condition: '^release/\d+\.\d+$'
        branch:
          notificationsConfig:
            notifications:
              - name: On RELEASE
                promotion: RELEASE
                events:
                  - new_promotion_run
                channel: slack
                channelConfig:
                  channel: '#releases'
                  type: 'SUCCESS'
                contentTemplate: |
                  Yontrack ${build} has been released.

                  ${promotionRun.changelog?title=true&commitsOption=OPTIONAL}

Integrations

There are several ways to integrate this Yontrack CI Configuration from your tools.

• GitHub actions
• Jenkins pipeline
• Bitbucket Pipelines
• GitLab CI/CD
• Yontrack CLI
• Direct GraphQL call

GitHub actions

You can use the nemerosa/ontrack-github-actions-cli-config GitHub action to configure Yontrack from your GitHub workflows:

  - name: "Yontrack configuration"
    id: yontrack-config
    uses: nemerosa/ontrack-github-actions-cli-config@v1.0.3
    env:
      YONTRACK_URL: ${{ vars.YONTRACK_URL }}
      YONTRACK_TOKEN: ${{ secrets.YONTRACK_TOKEN }}
    with:
      github-token: ${{ secrets.GITHUB_TOKEN }}

It expects the .yontrack/ci.yaml file to be present in the root of the repository.

Jenkins pipeline

You can use the https://github.com/nemerosa/ontrack-jenkins-cli-pipeline Jenkins pipeline library to configure Yontrack from your Jenkins pipelines:

ontrackCliCIConfig()

It expects the .yontrack/ci.yaml file to be present in the root of the repository.

Bitbucket Pipelines

In Bitbucket Pipelines, use the Yontrack CLI and pass it the pipeline's BITBUCKET_* variables. The bitbucket-pipelines CI engine and the bitbucket-cloud SCM engine are then detected automatically:

pipelines:
  default:
    - step:
        name: Yontrack configuration
        script:
          # YONTRACK_URL and YONTRACK_TOKEN are defined as repository or workspace variables
          - yontrack ci config --file .yontrack/ci.yaml --env-all BITBUCKET_

It expects the .yontrack/ci.yaml file to be present in the root of the repository.

GitLab CI/CD

In GitLab CI/CD, use the Yontrack CLI and pass it the pipeline's CI_* variables. The gitlab-ci CI engine and the gitlab SCM engine are then detected automatically:

yontrack-config:
  stage: setup
  script:
    # YONTRACK_URL and YONTRACK_TOKEN are defined as project or group CI/CD variables
    - yontrack ci config
      --file .yontrack/ci.yaml
      --env GITLAB_CI="$GITLAB_CI" --env CI_PROJECT_URL="$CI_PROJECT_URL" --env CI_PROJECT_PATH="$CI_PROJECT_PATH" --env CI_PROJECT_NAME="$CI_PROJECT_NAME"
      --env CI_COMMIT_SHA="$CI_COMMIT_SHA" --env CI_COMMIT_REF_NAME="$CI_COMMIT_REF_NAME" --env CI_MERGE_REQUEST_IID="$CI_MERGE_REQUEST_IID"
      --env CI_PIPELINE_ID="$CI_PIPELINE_ID" --env CI_PIPELINE_IID="$CI_PIPELINE_IID" --env CI_PIPELINE_URL="$CI_PIPELINE_URL"

It expects the .yontrack/ci.yaml file to be present in the root of the repository.

Warning

Unlike for Bitbucket Pipelines, do not pass the whole namespace with --env-all CI_: CI_REPOSITORY_URL and CI_JOB_TOKEN carry the GitLab job token and are not used by Yontrack.

See Feeding Yontrack from GitLab CI/CD for a complete pipeline.

Yontrack CLI

You can use the Yontrack CLI to configure Yontrack from your CI pipelines:

yontrack ci config \
  --file .yontrack/ci.yaml \
  --env ... \
  --env ...

Note

See environment variables for the list of inputs.

Direct GraphQL call

Finally, you can always use the GraphQL API to configure Yontrack from your CI pipelines:

mutation {
    configureBuild(input: {
        config: "...",
        ci: "",
        scm: "",
        env: [...]
    }) {
        build {
            id
        }
        errors {
            message
        }
    }
}

In this mutation's input:

• config is the YAML for the CI Configuration
• ci is the name of the CI engine to use
• scm is the name of the SCM engine to use
• env is the list of environment variables to use

Reference

The minimal CI Configuration file is:

version: v1
configuration: { }

The following elements will be configured automatically in Yontrack:

• a project will be created based on the detected SCM URL (for example, if the SCM URL is https://github.com/yontrack/yontrack, a project named ontrack will be created and associated with the https://github.com/yontrack/yontrack GitHub repository)
• a branch will be created based on the detected SCM branch (for example, if the SCM branch is release/1.0, a branch named release-1.0 will be created and associated with the release/1.0 SCM branch)
• a build will be created and associated with the detected SCM commit. The name of the build is generated using a timestamp. If a VERSION environment variable is defined, its value will be used as the build release label.
• the project is set up for the automatic creation of validation stamps and promotion levels, so that the validations and promotions declared in the CI configuration do not need to be predefined beforehand.

Auto-creation of validation stamps and promotion levels

Concretely, the project gets the auto validation stamps property with both autoCreate and autoCreateIfNotPredefined enabled, and the auto promotion levels property with autoCreate enabled.

This is done on every run of the CI configuration, so editing these two properties in the UI has no lasting effect. To use different values, set them explicitly as project properties — they are applied after the automatic setup and therefore take precedence:

version: v1
configuration:
  defaults:
    project:
      properties:
        net.nemerosa.ontrack.extension.general.AutoValidationStampPropertyType:
          autoCreate: true
          autoCreateIfNotPredefined: false

The configuration section can have two sub-sections:

• defaults contains the configurations which are used by default.
• custom contains a list of configurations, each of them associated with number of conditions. These configurations act as overrides to the defaults ones if the conditions are met.

Configuration

Each configuration contains the following optional elements:

• project
• branch
• build

Project configuration

The project configuration is optional.

It may contain the following elements:

• properties - list of properties to set for the project
• name - if set, overrides the name of the project instead of computing it from the environment
• scmConfig - name of the SCM configuration to use for this project instead of computing it from the environment
• issueServiceIdentifier - issue service configuration for this project
• scmIndexationInterval - SCM indexation interval (in minutes)
• extensions - list of extensions to apply to the project

Branch configuration

The branch configuration is optional.

It may contain the following elements:

• properties - list of properties to set for the branch
• validations - list of validation stamps to define for the branch
• promotions - list of promotion levels to define for the branch
• extensions - list of extensions to apply to the branch

Validations

The validations property is a map of all the validation stamps to define for the branch.

The simplest configuration is to define only a list of names for these validation stamps:

branch:
  validations:
    - build: { }
    - ui-unit: { }
    - acceptance: { }
    - ui-acceptance: { }

This code would define four validation stamps named build, ui-unit, acceptance and ui-acceptance, not typed.

The validation stamps can be typed using their configurations. For example, to make the build validation stamp typed for a test summary:

branch:
  validations:
    - build:
        tests:
          warningIfSkipped: true

Instead of the fully qualified class name of the validation stamp type, the following aliases can be used:

Alias	Data type
tests	Test summary
chml	CHML
percentage	Percentage
number	Number
metrics	Metrics
security-findings	Security findings

For example, to fail a validation as soon as at least one secret is detected (okIfGreater: false means that the lower the value, the better):

branch:
  validations:
    - SECURITY.SECRETS:
        number:
          failureThreshold: 0
          okIfGreater: false

As another example, a security scan which fails on any critical issue and warns on any high one, but lets a warning through to the auto promotion:

branch:
  validations:
    - SECURITY.SCAN:
        chml:
          failedLevel: CRITICAL
          failedValue: 1
          warningLevel: HIGH
          warningValue: 1
          warningPassesAutoPromotion: true

Note

Use the JSON schema to see which other types are available.

Promotions

The promotions property is a map of all the promotion levels to define for the branch.

The simplest configuration is to define only a list of names for these promotion levels:

branch:
  promotions:
    - BRONZE: { }
    - SILVER: { }

Each promotion level can be associated with a list of validation stamps and/or other promotion levels:

branch:
  promotions:
    - BRONZE:
        validations:
          - build
          - ui-unit
    - SILVER:
        validations:
          - acceptance
          - ui-acceptance
        promotions:
          - BRONZE

A promotion can also have dependencies. For example:

branch:
  promotions:
    - BRONZE:
        validations:
          - build
          - ui-unit
    - SILVER:
        validations:
          - acceptance
          - ui-acceptance
        dependsOn:
          - BRONZE

The semantic of dependsOn is different from promotions:

• promotions define the list of other promotions that participate in the auto-promotion of this promotion ("if BRONZE is granted and the listed validations are OK, then SILVER is granted automatically")
• dependsOn defines the list of promotions that must be completed before the promotion can be completed ("trying to promote to SILVER without having completed BRONZE" will fail)

Auto revocation

By default, a promotion which has been granted stays granted, even if the validations which triggered it later fail. Setting autoRevoke to true on a promotion revokes it as soon as one of its prerequisites - a required validation stamp or a required promotion - is no longer valid. See auto promotion for the full behaviour.

branch:
  promotions:
    - BRONZE:
        validations:
          - build
        autoRevoke: true

Warning

Revoking a promotion deletes it, but does not undo its effects: any notification or workflow already triggered by the promotion remains fired.

autoRevoke is not set by default, and a layer which does not mention it leaves the value set by the previous layer untouched. It is ignored when the promotion declares neither validations nor promotions, since there is then no prerequisite to check.

Two behaviours are worth knowing before enabling it:

• deleting a promotion by hand also deletes every downstream promotion which required it and has autoRevoke enabled - a prerequisite is no longer valid whoever made it untrue
• adding a new prerequisite which then fails on an already-promoted build revokes that old promotion; nothing happens on the configuration change itself, only when a real failing run appears

Enabling autoRevoke does not re-evaluate the builds which are already promoted: revocation is forward-looking.

Agents

Setting agents to true on a promotion sets Agents admitted on its promotion level: a registered agent may then promote a build to it, provided its owner may.

branch:
  promotions:
    - GOLD:
        agents: true

agents is not set by default. true admits agents, false stops admitting them, and leaving it out keeps the setting of the promotion level as it is - set in the UI or through CasC, say. A layer which does not mention it leaves the value set by the previous layer untouched.

Fields

A promotion level can define a list of fields that must be filled in when a build is promoted. Fields are defined under the fields key of a promotion:

branch:
  promotions:
    BRONZE:
      validations:
        - build
      fields:
        - name: ticket
          displayName: Ticket
          description: "JIRA ticket reference"
          type: TEXT
          required: true
        - name: env
          displayName: Environment
          type: CHOICE
          options:
            - staging
            - prod
        - name: hotfix
          displayName: Hotfix?
          type: BOOLEAN

Each field supports the following properties:

Property	Required	Description
name	Yes	Technical identifier for the field (unique per promotion level)
displayName	Yes	Human-readable label shown in the UI
description	No	Optional description shown as a tooltip
type	Yes	One of TEXT, NUMBER, BOOLEAN, CHOICE, LINK
required	No	Whether the field must be filled in before promoting (default: false)
options	No	List of allowed values — only valid when type is CHOICE

When the CI config is applied multiple times (e.g. on different branches), fields are merged by their name: existing fields are updated, new fields are added, and no fields are removed.

Build configuration

The build configuration is optional.

It may contain the following elements:

• properties - list of properties to set for the build
• buildName - overrides the computed build name using a template ( see below)
• extensions - list of extensions to apply to the build

When using the Bitbucket or GitHub SCM, the GIT_COMMIT environment variable is used to set a Git Commit property on the build.

For the GitHub CI engine, the build is linked to the GitHub workflow run.

For the Jenkins CI engine, the build is linked to the Jenkins build.

If the VERSION environment variable is defined, its value will be used as the build release label.

Build name

By default, the build name is computed using:

{timestamp}-{suffix}

where:

• the timestamp is the current UTC timestamp using the yyyyMMddHHmmss format
• the suffix depends on the CI engine being used, but is usually the value of the BUILD_NUMBER environment variable or the GITHUB_RUN_NUMBER environment variable in a GitHub workflow.

This build name can be overridden using the buildName property, which is a template with only one context, env, allowing to access the environment variables defined in the CI configuration.

For example, if you want to use your custom timestamp format and a build number, you can use:

buildName: '${#.datetime?format=yyyy-MM-dd-HH-mm-ss}-${env.BUILD_NUMBER}'

Conditions

Each custom configuration is associated with a list of conditions. For example:

custom:
  configs:
    - conditions:
        - name: branch
          config: 'main'
        - name: environment-regex
          config:
            name: VERSION
            regex: '\d+\.\d+\.\d+'
      branch:
      # Branch configuration

In this case, the branch configuration will be used to patch the default configuration for any SCM branch matching the regular expression main and having a VERSION environment variable with a value matching the regular expression \d+\.\d+\.\d+.

The custom configuration is applied only if all the conditions in the list are met.

If you need some or condition, you can use the or condition.

See the conditions reference for the list of supported conditions and their parameters.

Merging the configurations

When a custom configuration conditions are met, it is merged with the default configuration.

The rules to merge a custom configuration into a default one are:

• properties, validations and promotions are merged at key level
• in lists:
  • custom elements not present in the default list are added
  • default elements not present in the custom list are left
  • elements present in both lists are merged. The rules depend on the type of extension

CI engines

In the context of the CI Configuration, a CI engine is the client which will call Yontrack to configure the elements like the project, branch and build.

It's responsible to collect some environment and allow Yontrack to perform its actions.

If a CI engine is not specified, Yontrack will do its best to guess it from the provided environment. Each CI engine has its own way of checking the environment to see if it matches.

See the CI engines reference for the list of supported CI engines and the way they work.

Note

If the way you inject the CI Configuration does not support the detection of the CI engine, you can pass it explicitly using the ci parameter. For example with the Yontrack CLI:

yontrack \
  ... \
  --ci github

SCM engines

In the context of the CI Configuration, a SCM engine is responsible for linking the project, branch and build to a SCM repository.

If a SCM engine is not specified, Yontrack will do its best to guess it from the provided environment. Each SCM engine has its own way of checking the environment to see if it matches. It usually uses the SCM URL provided by the CI engine.

See the SCM engines reference for the list of supported SCM engines and the way they work.

Note

If the way you inject the CI Configuration does not support the detection of the SCM engine, you can pass it explicitly using the scm parameter. For example with the Yontrack CLI:

yontrack \
  ... \
  --scm bitbucket-server

The Bitbucket Cloud SCM engine is selected the same way, with --scm bitbucket-cloud.

Including files

It can be very cumbersome to define the whole CI Configuration in a single file.

Most of the integrations allow to include other files using the @<file path> syntax.

For example, given a .yontrack/branch.yaml file:

validations:
  BUILD:
    tests: { }
  UI_UNIT:
    tests: { }
  KDSL.ACCEPTANCE:
    tests: { }
  PLAYWRIGHT:
    tests: { }
promotions:
  BRONZE:
    validations:
      - BUILD
      - UI_UNIT
      - KDSL.ACCEPTANCE
      - PLAYWRIGHT
  RELEASE:
    promotions:
      - BRONZE
    validations:
      - GITHUB.RELEASE

you can include it at the default branch configuration level:

version: v1
configuration:
  defaults:
    branch: '@branch.yaml'

This inclusion trick can be used anywhere in the CI Configuration file.

Environment variables

Each CI engine is responsible for collecting the environment variables it needs to configure Yontrack.

See the list of CI engines to check what environment variables they expect.

Extensions

Extensions are set directly as keys of the project, branch or build configurations, using their ID:

Extension	ID	Level
Auto-disabling of branches	auto-disabling	Project
Environments	environments	Project
Auto-versioning	autoVersioning	Branch
Notifications	notificationsConfig	Branch
Workflows	workflows	Branch
Auto-versioning check	autoVersioningCheck	Build

An extension can only be used at the level(s) it supports; using it anywhere else is rejected when the CI configuration is parsed.

Note

The extensions available on a given Yontrack instance depend on the extensions which are installed and enabled. Use the JSON schema to see the exact list and the configuration they accept.

Auto-disabling of branches

This extension is set at the project level to configure the pattern-based auto-disabling of the branches of this project.

Syntax:

project:
  auto-disabling:
    patterns: [ list of patterns ]

Each pattern has the following optional properties:

Property	Description
includes	List of regular expressions matched against the Yontrack branch names. If empty, all branches match.
excludes	List of regular expressions matched against the Yontrack branch names. A branch matching any of them is excluded.
mode	What to do with the matching branches: KEEP, DISABLE or KEEP_LAST. Defaults to KEEP.
keepLast	With DISABLE and KEEP_LAST, number of the most recent matching branches (semantic version order) to keep enabled. Defaults to 2.

Branches are matched against the patterns in order, and the mode of the first matching pattern is applied. See stale branch management for the exact semantics of the modes.

For example, to disable all the v* branches but the most recent one:

version: v1
configuration:
  defaults:
    project:
      auto-disabling:
        patterns:
          - includes:
              - 'v.*'
            mode: DISABLE
            keepLast: 1

Note

Because this is a project-level configuration, it is applied again every time the CI configuration runs, from any branch. To make sure the policy is set from one authoritative branch only, put it in a custom configuration guarded by a branch condition:

version: v1
configuration:
  custom:
    configs:
      - conditions:
          - name: branch
            config: main
        project:
          auto-disabling:
            patterns:
              - includes:
                  - 'v.*'
                mode: DISABLE
                keepLast: 1

Auto-versioning

This extension is set at the branch level to set its auto-versioning configuration.

Note

See the auto-versioning reference for the list of available configurations.

Syntax:

branch:
  autoVersioning:
    configurations: [ list of configurations ]

For example:

branch:
  autoVersioning:
    configurations:
      - sourceProject: yontrack
        sourceBranch: 'release/.*'
        sourcePromotion: RELEASE
        targetPath: versions.properties
        targetProperty: yontrack

sourceBranch is a regular expression selecting the latest matching branch of the source project. See selecting the source branch for the other available strategies.

If the auto-versioning configuration must not be applied to all branches, you can use the branchFilter property to restrict it to a list of branch names using regular expressions. For example:

version: v1
configuration:
  defaults:
    branch:
      autoVersioning:
        branchFilter:
          includes:
            - main
            - 'release/.*'
          excludes:
            - 'release/1\..*'

This restricts the auto-versioning configuration to the main and release/* branches, but not the release/1.* ones.

Note

The branchFilter regular expressions are matched, in full and ignoring the case, against the SCM branch name of the branch being configured. When a branch is excluded, any auto-versioning configuration it may already have is removed.

Auto-versioning check

This extension is set at the build level to check the compliance with the latest versions using the auto-versioning check.

Syntax:

build:
  autoVersioningCheck: [ true | false ]

For example:

build:
  autoVersioningCheck: true

When this is set (to `true), an auto-versioning check will be performed automatically after the build is created.

Environments

This extension is set at the project level to configure environments and slots for this project.

Syntax:

project:
  environments:
    environments: [ list of environments ]
    slots: [ list of slots ]

Each environment of a slot takes agentsAdmitted: true to admit agents in this slot (false by default).

See the environments reference for some examples.

Notifications

This extension is set at the branch level to configure the notifications for this branch.

Syntax:

branch:
  notificationsConfig:
    notifications: <list of notifications>

Each notification configuration has the following properties:

• name - Unique name of the subscription in its scope.
• promotion - Targeting a promotion instead of the branch. The promotion must have been configured first.
• events - List of events types to subscribe to
• keywords - Optional space-separated list of tokens to look for in the events, including actor:agent, actor:human and agent:<slug> - see Keywords
• channel - Name of the channel to send the notification to
• channelConfig - Configuration of the channel
• contentTemplate - Optional template to use for the message

Example:

branch:
  notificationsConfig:
    notifications:
      - name: On validation error
        events:
          - new_validation_run
        keywords: failed
        channel: slack
        channelConfig:
          channel: '#notifications'
          type: 'ERROR'
        contentTemplate: |
          Build ${build} has failed on ${validationStamp}.

Workflows

Whereas workflows can be specified using the notificationsConfig extension, you can also use the workflows extension to define them directly in the CI Configuration, in a more straightforward way.

Syntax:

branch:
  workflows:
    <PROMOTION>:
      <workflows>

This configuration is used to associate a list of named workflows to some promotions.

Each workflow is defined as usual.

Example:

workflows:
  RELEASE:
    - name: On internal release
      nodes:
        - id: notification
          executorId: notification
          data:
            channel: slack
            channelConfig:
              channel: '#internal-releases'
              type: 'SUCCESS'
            template: |
              Yontrack ${build} has been released.

              ${promotionRun.changelog?title=true&commitsOption=ALWAYS}
        - id: docs.yontrack.com
          executorId: notification
          data:
            channel: github-workflow
            channelConfig:
              config: github.com
              owner: yontrack
              repository: doc.yontrack.com
              workflowId: doc.yml
              reference: main
              callMode: ASYNC

Properties

Arbitrary properties can be set on projects, branches and builds.

The general syntax is (for example for a build):

build:
  properties:
    <fqcn>: <config>

where:

• <fqcn> is the fully qualified class name of the property type
• <config> is the configuration of the property

Note

See the properties reference for the list of available properties.

Some properties provide aliases to simplify the configuration.

For example:

• release for the release property - note that most of the time, you won't need this since it is automatically set when the VERSION environment variable is defined.
• useLabel for the use label property

For example, for a project:

project:
  properties:
    useLabel: false

Note

You can use the JSON schema to see which other properties are available and what is their configuration.

JSON schema

The JSON Schema for the CI Configuration is available in the UI in the user menu, under User information > Resources in the JSON schemas section:

Once downloaded, you can use it to validate your CI Configuration files. See the appendixes to learn more about using JSON schemas for edition and validation.

Metrics

Yontrack exports some operational metrics about the CI config injection.

See the reference for the list of these metrics.