GitLabTech&Tips

부모-자식 파이프라인

Source: GitLab Blog | Author: Chris Ward

애플리케이션과 앱 리포지토리 구조가 복잡해지면서 리포지토리의 .gitlab-ci.yml 파일은 관리, 협업 및 이점을 보기 어려워집니다. 이것은 여러 관련 서비스에 대한 코드를 하나의 저장소에 보관하는 팀들이 사용하고 있는 “모노레포” 패턴에서 특히 중요한 문제입니다. 현재 이 패턴을 사용할 때 개발자들은 모두 동일한 .gitlab-ci.yml  파일을 사용하여 서로 다른 애플리케이션 구성요소에 대해 서로 다른 자동화된 프로세스를 트리거하여 병합 충돌 및 생산성 저하를 유발합니다.

To help large and complex projects manage their automated workflows, we’ve added two new features to make pipelines even more powerful: Parent-child pipelines, and the ability to generate pipeline configuration files dynamically.

Meet Parent-child pipelines

So, how do you solve the pain of many teams collaborating on many inter-related services in the same repository? Let me introduce you to Parent-child pipelines, released with with GitLab 12.7. Splitting complex pipelines into multiple pipelines with a parent-child relationship can improve performance by allowing child pipelines to run concurrently. This relationship also enables you to compartmentalize configuration and visualization into different files and views.

Creating a child pipeline

You trigger a child pipeline configuration file from a parent by including it with the include key as a parameter to the trigger key. You can name the child pipeline file whatever you want, but it still needs to be valid YAML.

The parent configuration below triggers two further child pipelines that build the Windows and Linux version of a C++ application.

#include <iostream>
int main()
{
  std::cout << "Hello GitLab!" << std::endl;
  return 0;
}

The setup is a simple one but hopefully illustrates what is possible.

stages:
  - triggers

build_windows:
  stage: triggers
  trigger:
    include: .linux-gitlab-ci.yml
  rules:
    - changes:
      - cpp_app/*

build_linux:
  stage: triggers
  trigger:
    include: .win-gitlab-ci.yml
  rules:
    - changes:
      - cpp_app/*

The important values are the trigger keys which define the child configuration file to run, and the parent pipeline continues to run after triggering it. You can use all the normal sub-methods of include to use local, remote, or template config files, up to a maximum of three child pipelines.

Another useful pattern to use for parent-child pipelines is a rules key to trigger a child pipeline under certain conditions. In the example above, the child pipeline only triggers when changes are made to files in the cpp_app folder.

The Windows build child pipeline (.win-gitlab-ci.yml) has the following configuration, and unless you want to trigger a further child pipeline, it follows standard a configuration format:

image: gcc
build:
  stage: build
  before_script:
    - apt update && apt-get install -y mingw-w64
  script:
    - x86_64-w64-mingw32-g++ cpp_app/hello-gitlab.cpp -o helloGitLab.exe
  artifacts:
    paths:
      - helloGitLab.exe

Don’t forget the -y argument as part of the apt-get install command, or your jobs will be stuck waiting for user input.

The Linux build child pipeline (.linux-gitlab-ci.yml) has the following configuration, and unless you want to trigger a further child pipeline, it follows standard a configuration format:

image: gcc
build:
  stage: build
  script:
    - g++ cpp_app/hello-gitlab.cpp -o helloGitLab
  artifacts:
    paths:
      - helloGitLab

In both cases, the child pipeline generates an artifact you can download under the Job artifacts section of the Job result screen.

Push all the files you created to a new branch, and for the pipeline result, you should see the two jobs and their subsequent child jobs.

Parent-child pipeline result

The result of a parent-child pipeline

Dynamically generating pipelines

Taking Parent-child pipelines even further, you can also dynamically generate the child configuration files from the parent pipeline. Doing so keeps repositories clean of scattered pipeline configuration files and allows you to generate configuration in your application, pass variables to those files, and much more.

Let’s start with the parent pipeline configuration file:

stages:
  - setup
  - triggers

generate-config:
  stage: setup
  script:
    - ./write-config.rb
    - git status
    - cat .linux-gitlab-ci.yml
    - cat .win-gitlab-ci.yml
  artifacts:
    paths:
      - .linux-gitlab-ci.yml
      - .win-gitlab-ci.yml

trigger-linux-build:
  stage: triggers
  trigger:
    include:
      - artifact: .linux-gitlab-ci.yml
        job: generate-config
       
trigger-win-build:
  stage: triggers
  trigger:
    include:
      - artifact: .win-gitlab-ci.yml
        job: generate-config

During our self-defined setup stage the pipeline runs the write-config.rb script. For this article, it’s a Ruby script that writes the child pipeline config files, but you can use any scripting language. The child pipeline config files are the same as those in the non-dynamic example above. We use artifacts to save the generated child configuration files for this CI run, making them available for use in the child pipelines stages.

As the Ruby script is generating YAML, make sure the indentation is correct, or the pipeline jobs will fail.

#!/usr/bin/env ruby

linux_build = <<~YML
    image: gcc
    build:
        stage: build
        script:
            - g++ cpp_app/hello-gitlab.cpp -o helloGitLab
        artifacts:
            paths:
                - helloGitLab
YML

win_build = <<~YML
    image: gcc
    build:
        stage: build
        before_script:
            - apt update && apt-get install -y mingw-w64
        script:
            - x86_64-w64-mingw32-g++ cpp_app/hello-gitlab.cpp -o helloGitLab.exe
        artifacts:
            paths:
                - helloGitLab.exe
YML

File.open('.linux-gitlab-ci.yml', 'w'){ |f| f.write(linux_build)}
File.open('.win-gitlab-ci.yml', 'w'){ |f| f.write(win_build)}

Then in the triggers stage, the parent pipeline runs the generated child pipelines much as in the non-dynamic version of this example but instead using the saved artifact files, and the specified job.

Push all the files you created to a new branch, and for the pipeline result, you should see the three jobs (with one connecting to the two others) and the subsequent two children.

Dynamic parent-child pipeline result

The result of a dynamic parent-child pipeline

Pipeline flexibility

This blog post showed some simple examples to give you an idea of what you can now accomplish with pipelines. With one parent, multiple children, and the ability to generate configuration dynamically, we hope you find all the tools you need to build CI/CD workflows you need.

You can also watch a demo of Parent-child pipelines below:

댓글 남기기