如何在 CI 构建中访问多个存储库？

我们有一个由多个(非公共)存储库组成的项目。

要构建整个项目，构建系统需要拥有所有存储库(master分支)的文件。

有没有一种方法可以配置 GitLab CI 来提供我需要的存储库？

我想我可以在 CI 构建期间执行 git fetch或类似的操作，但是如何处理身份验证呢？

100552 次浏览
小开
最佳答案

You can add a deploy key to all projects. Then configure the deploy key's private key on the runner(s). Use normal git commands in your build process to clone the repositories on the runner. This may require a bit of configuration on your runners, but it should work.

You can either generate one SSH key pair and use that on all runners or generate one key pair per runner. To generate an SSH key pair follow SSH key documentation. The private key should be placed in 'gitlab-runner' user's .ssh directory so the git command can present it at clone time. The public key should be added to the GitLab project as a deploy key. Add a deploy key in the project settings -> 'Deploy Keys'.

小开

A couple of workarounds (I hate this word!) that work-around-ed for me:

  1. Using git submodule, see https://docs.gitlab.com/ce/ci/git_submodules.html

  2. Re-using $CI_REPOSITORY_URL defined by Gitlab and available even inside child Docker containers. This env var already contains username and password, that can be used for another repo on the same server. See snippet from .gitlab-ci.yml:

- BASE_URL=`echo $CI_REPOSITORY_URL | sed "s;\/*$CI_PROJECT_PATH.*;;"`
- REPO_URL="$BASE_URL/thirdparty/gtest.git"
- REPO_DIR=thirdparty/gtest
- rm -fr $REPO_DIR
- git clone $REPO_URL $REPO_DIR
  1. Even storing that URL with username\password in ~/.git-credentials file and configuring git to use it via credential.helper. All further "git clone" commands will use it.
- echo Storing git credentials to be used by "git clone" commands without username and password ...
- GIT_CREDENTIALS_FILE=~/.git-credentials
- BASE_URL=`echo $CI_REPOSITORY_URL | sed "s;\/*$CI_PROJECT_PATH.*;;"`
- echo $BASE_URL > $GIT_CREDENTIALS_FILE
- git config --global credential.helper store --file=$GIT_CREDENTIALS_FILE

HOWEVER !

Having spent quite a few years in CI \ CD field, I don't think it's a good design that requires linking repositories as sources.

Yes, in classic CI tools like Jenkins or TeamCity you can create a job that fetches several Git repos in different subdirectories.

But I like GitLab CI way of Pipeline As Code, where .gitlab-ci.yml controls the build of this very repo and you don't even have to think about that whole pre-build step of getting sources. Then such build would publish binary artifacts and downstream projects\repos can use those instead of sources of dependencies. It's also faster.

Separation of concerns.

I don't there is an official way in my .gitlab-ci.yml to use artifacts of another project. But there are other ways like hooks, Gitlab API, though such bespoke solutions require maintenance.

There's better way - to publish\fetch artifacts to\from external widely-adopted Package Manager. Depending on your language it could be Maven, NuGet, npm, jFrog Artifactory, Nexus, etc. Another advantage of this method is that developers can follow the same process in their local builds, which is not easily done if dependencies are defined in .gitlab-ci.yml

It's a bigger problem for native code (Cxx) mainly due to Binary Interface compatibility, but things like Conan.io etc are slowly catching up.

小开

If you are running gitlab version 8.12 or later, the permissions model was reworked. Along with this new permission model comes the the CI environment variable CI_JOB_TOKEN. The premium version of GitLab uses this environment variable for triggers, but you can use it to clone repos.

dummy_stage:
script:
- git clone https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.instance/group/project.git

提交答案