Container Registry : features

– gitlab projects have their own space to store Docker images.
– graphical view and management of the registry in gitlab (« container registry » button in group/project settings) – a registry bound to Gitlab projects : restrictions on what we can push on it but reachable inside or outside Gitlab pipelines.
– a registry served by a nginx reverse proxy (helpful to change the plumbing of the registry without disruptive changes for clients of the registry.

Gitlab container registry : internal or external

Using the integrated registry is quite simple to setup/configure while using it may be annoying if we already have a container registry and we want to multiply them.
Gitlab allows to use our own container registry but it warns that some features associated with the container registry may be unavailable (personally I never tried that way).

Setup the container registry (With a Gitlab installed by package manager)

Check that the registry service is enabled and running

Beware : the Gitlab gui may give the feeling that the registry is available even if it is not.
Indeed whatever its state, the registry tab appears in the gtilab gui.

To check whether the service is active, we have mainly two ways :
– On the gui : go to the http://gitlab.foo/admin url
There is a feature table that displays some features. These active and these inactive have a distinct icon, that is a green check or a power-off icon.
– the gitlab-ctl command : it is the more reliable way
gitlab-ctl service-list : displays all configured service and shows a * character if it is enabled. So if the registry is enabled we should see : registry*
gitlab-ctl status : displays status of each service with some details (running or not, pid…).
So if the registry is running we should see something like :
run: registry: (pid 11988) 78739s; run: log: (pid 6132) 90634s

Registry configuration files

The configuration template to specify/edit is here : /etc/gitlab/gitlab.rb.
Executing gitlab-ctl reconfigure will use that template and generate the configuration of each service, which the registry.
Here the generated configurations :
Docker registry : /var/opt/gitlab/registry/*
Nginx registry : /var/opt/gitlab/nginx/conf/gitlab-registry.conf
Don’t edit directly these files but for testing.

Configure the registry

We need to edit gitlab.rb.
We need to uncomment keys-values only when their default value don’t match to your requirement.
We work on keys with prefix registry (docker registry) and registry_nginx (nginx instance for registry)
Many registry_nginx keys are not declared in the files. To override them, we need to add them.
Here most important things :

# not related to registry. Just to provide the gitlab context
external_url 'http://gitlab.david.com:8585'
 
# registry_external_url = the external url of the registry. 
# In clear : the nginx reverse-proxy url/port.
# Which host choose ? 
# It is up to us : either the same host than the gitlab instance or a distinct
# For distinct, we may need to update /etc/hosts consequently.
 
# Which port choose ?
# Whichever port while it is not used.
 
# Http or Https ? 
# It is up to us.
# If we prefix the value by https : a ssl nginx configuration is generated
# If we prefix the value by http : a plain http nginx configuration is generated
 
registry_external_url 'https://registry.david.com:5050'
 
#  registry_http_addr = the docker registry host/port
# the host and port may be whatever but we must not specify the protocol in prefix. It is http.
registry['registry_http_addr'] = "registry.david.com:5051"
 
# If we specified https in registry_external_url, we should ensure that crt/pem and key are at the correct location
# by default it is /etc/gitlab/ssl/foo_registry_external_url.crt and /etc/gitlab/ssl/foo_registry_external_url.key.
# But if we specified a registry_external_url in another domain than the gitlab the crt/key, the files will not be found
# So we need to add these lines to specify their locations
registry_nginx['ssl_certificate'] = "/etc/gitlab/ssl/gitlab.david.com.crt"
registry_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/gitlab.david.com.key"

Use gitlab registry in a pipeline

We suppose that docker-image-version.json is a file that was built in a earlier stage and used by that stage.
It contains the version of the docker image to build.
Here a docker job that :
– build a docker image from a application code built in a earlier stage
– login to the registry
– tag the image with a label conform to Gitlab container registry : registryUrl/groupName/projectName
– push the tagged image to the registry

Docker Build:
  stage: docker-build
  tags:
    - docker
#  image: docker:19.03.12
  script:
    - imageName="$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME:$(cat docker-image-version.json)"
    - docker build -f docker/Dockerfile -t $imageName .
    - docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY
    - docker tag $imageName $CI_REGISTRY/$imageName
    - docker push $CI_REGISTRY/$imageName
  when: always
  only:
    refs:
      - master

Common errors with gitlab registry access

Problem : docker login to the registry returns an access denied error such as :

Error response from daemon: Get http://registry.david.org:5050/v2/: unauthorized: HTTP Basic: Access denied

Reason A : The user or token are incorrect.
Solution B : check the user/token
Reason B : The url of the exposed registry (nginx) has changes or it switched from http to https.
As a consequence, the deploy token will not match any longer.
Solution B : regenerate a user/token

Problem : docker login to the registry returns a forbidden error

Error response from daemon: Get http://registry.david.org:5050/v2/ : denied: access forbidden

Reason : The user is very authenticated but these rights are not enough to execute the operation.
Solution : check the user rights on the registry

Problem : I can login to the registry (outside and inside gitlab) but i have an denied error when I push an image such as :

docker push denied: requested access to the resource is denied gitlab

Reason : Gitlab tokens (Deploy tokens, as well as Access tokens) are scoped concerning the registry access. We can indeed push only image which name matches to the gitlabNamespace(s)-gitlabProject(s).
Solution : Change the docker tag that we push to be conform to the namespaces/project.