Configure OpenID Connect in AWS to retrieve temporary credentials

In this tutorial, we’ll show you how to use a GitLab CI/CD job with a JSON web token (JWT) to retrieve temporary credentials from AWS without needing to store secrets. To do this, you must configure OpenID Connect (OIDC) for ID federation between GitLab and AWS. For background and requirements for integrating GitLab using OIDC, see Connect to cloud services.

To complete this tutorial:

1. Add the identity provider
2. Configure the role and trust
3. Retrieve a temporary credential

Add the identity provider

Create GitLab as a IAM OIDC provider in AWS following these instructions.

Include the following information:

• Provider URL: The address of your GitLab instance, such as https://gitlab.com or http://gitlab.example.com. This address must be publicly accessible. If this is not publicly available, see how to configure a non-public GitLab instance
• Audience: The address of your GitLab instance, such as https://gitlab.com or http://gitlab.example.com.
  • The address must include https://.
  • Do not include a trailing slash.

Configure a role and trust

After you create the identity provider, configure a web identity role with conditions for limiting access to GitLab resources. Temporary credentials are obtained using AWS Security Token Service, so set the Action to sts:AssumeRoleWithWebIdentity.

You can create a custom trust policy for the role to limit authorization to a specific group, project, branch, or tag. For the full list of supported filtering types, see Connect to cloud services.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::AWS_ACCOUNT:oidc-provider/gitlab.example.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "gitlab.example.com:sub": "project_path:mygroup/myproject:ref_type:branch:ref:main"
        }
      }
    }
  ]
}

After the role is created, attach a policy defining permissions to an AWS service (S3, EC2, Secrets Manager).

Retrieve temporary credentials

After you configure the OIDC and role, the GitLab CI/CD job can retrieve a temporary credential from AWS Security Token Service (STS).

assume role:
  id_tokens:
    GITLAB_OIDC_TOKEN:
      aud: https://gitlab.example.com
  script:
    # this is split out for correct exit code handling
    - >
      aws_sts_output=$(aws sts assume-role-with-web-identity
      --role-arn ${ROLE_ARN}
      --role-session-name "GitLabRunner-${CI_PROJECT_ID}-${CI_PIPELINE_ID}"
      --web-identity-token ${GITLAB_OIDC_TOKEN}
      --duration-seconds 3600
      --query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]'
      --output text)
    - export $(printf "AWS_ACCESS_KEY_ID=%s AWS_SECRET_ACCESS_KEY=%s AWS_SESSION_TOKEN=%s" $aws_sts_output)
    - aws sts get-caller-identity

• ROLE_ARN: The role ARN defined in this step.
• GITLAB_OIDC_TOKEN: An OIDC ID token.

Working examples

• See this reference project for provisioning OIDC in AWS using Terraform and a sample script to retrieve temporary credentials.
• OIDC and Multi-Account Deployment with GitLab and ECS.
• AWS Partner (APN) Blog: Setting up OpenID Connect with GitLab CI/CD.
• GitLab at AWS re:Inforce 2023: Secure GitLab CD pipelines to AWS w/ OpenID and JWT.

Configure a non-public GitLab instance

If your GitLab instance is not publicly accessible, configuring OpenID Connect in AWS is not possible by default. You can use a workaround to make some specific configuration publicly accessible, enabling OpenID Connect configuration for the instance:

1. Store authentication details for your GitLab instance at a publicly available location, for example in S3 files:

   • Host the OpenID configuration for your instance in an S3 file. The configuration is available at /.well-known/openid-configuration, like http://gitlab.example.com/.well-known/openid-configuration. Update the issuer: and jwks_uri: values in the configuration file to point to the publicly available locations.
   • Host the public keys for your instance URL in an S3 file. The keys are available at available at /oauth/discovery/keys, like http://gitlab.example.com/oauth/discovery/keys.

   For example:

   • OpenID configuration file: https://example-oidc-configuration-s3-bucket.s3.eu-north-1.amazonaws.com/.well-known/openid-configuration.
   • JWKS (JSON Web Key Sets): https://example-oidc-configuration-s3-bucket.s3.eu-north-1.amazonaws.com/oauth/discovery/keys.
   • The issuer claim iss: in the ID Tokens and the issuer: value in the OpenID configuration would be: https://example-oidc-configuration-s3-bucket.s3.eu-north-1.amazonaws.com

2. Optional. Use an OpenID configuration validator like the OpenID Configuration Endpoint Validator to validate your publicly available OpenID configuration.

3. Configure a custom issuer claim for your ID tokens. By default, GitLab ID tokens have the issuer claim iss: set as the address of your GitLab instance, for example: http://gitlab.example.com.

4. Update the issuer URL:

   Linux package (Omnibus)

   1. Edit /etc/gitlab/gitlab.rb:

      gitlab_rails['ci_id_tokens_issuer_url'] = 'public_url_with_openid_configuration_and_keys'

   2. Save the file and reconfigure GitLab for the changes to take effect.

   Helm chart (Kubernetes)

   1. Export the Helm values:

      helm get values gitlab > gitlab_values.yaml

   2. Edit gitlab_values.yaml:

      global:
        appConfig:
          ciIdTokens:
            issuerUrl: 'public_url_with_openid_configuration_and_keys'

   3. Save the file and apply the new values:

      helm upgrade -f gitlab_values.yaml gitlab gitlab/gitlab

   Docker

   1. Edit docker-compose.yml:

      version: "3.6"
      services:
        gitlab:
          environment:
            GITLAB_OMNIBUS_CONFIG: |
              gitlab_rails['ci_id_tokens_issuer_url'] = 'public_url_with_openid_configuration_and_keys'

   2. Save the file and restart GitLab:

      docker compose up -d

   Self-compiled (source)

   1. Edit /home/git/gitlab/config/gitlab.yml:

       production: &base
         ci_id_tokens:
           issuer_url: 'public_url_with_openid_configuration_and_keys'

   2. Save the file and reconfigure GitLab for the changes to take effect.

5. Run the ci:validate_id_token_configuration Rake task to validate the CI/CD ID token configuration.

Troubleshooting

Error: Not authorized to perform sts:AssumeRoleWithWebIdentity

If you see this error:

An error occurred (AccessDenied) when calling the AssumeRoleWithWebIdentity operation:
Not authorized to perform sts:AssumeRoleWithWebIdentity

It can occur for multiple reasons:

• The cloud administrator has not configured the project to use OIDC with GitLab.
• The role is restricted from being run on the branch or tag. See configure a conditional role.
• StringEquals is used instead of StringLike when using a wildcard condition. See related issue.

Could not connect to openid configuration of provider error

After adding the Identity Provider in AWS IAM, you might get the following error:

Your request has a problem. Please see the following details.
  - Could not connect to openid configuration of provider: `https://gitlab.example.com`

This error occurs when the OIDC identity provider’s issuer presents a certificate chain that’s out of order, or includes duplicate or additional certificates.

Verify your GitLab instance’s certificate chain. The chain must start with the domain or issuer URL, then the intermediate certificate, and end with the root certificate. Use this command to review the certificate chain, replacing gitlab.example.com with your GitLab hostname:

echo | /opt/gitlab/embedded/bin/openssl s_client -connect gitlab.example.com:443

Couldn't retrieve verification key from your identity provider error

You might receive an error similar to:

• An error occurred (InvalidIdentityToken) when calling the AssumeRoleWithWebIdentity operation: Couldn't retrieve verification key from your identity provider, please reference AssumeRoleWithWebIdentity documentation for requirements

This error might be because:

• The .well_known URL and jwks_uri of the identity provider (IdP) are inaccessible from the public internet.
• A custom firewall is blocking the requests.
• There’s latency of more than 5 seconds in API requests from the IdP to reach the AWS STS endpoint.
• STS is making too many requests to your .well_known URL or the jwks_uri of the IdP.

As documented in the AWS Knowledge Center article for this error, your GitLab instance needs to be publicly accessible so that the .well_known URL and jwks_uri can be resolved. If this is not possible, for example if your GitLab instance is in an offline environment, see how to configure a non-public GitLab instance