How to use OpenID Connect (OIDC) provided by Bitbucket Pipelines in aws-eks-kubectl-run pipe

[#1]

About OIDC

You can use Bitbucket Pipelines OpenID Connect Provider (OIDC IDP) to allow your pipelines to access your resource server, such as AWS, GCP, or Vault. This means that you and your team can access the resource server without storing credentials in Bitbucket.

OpenID Connect is a simple identity layer on top of the OAuth 2.0 protocol. It allows clients to:

• Verify the identity of the end user based on the authentication performed by Bitbucket Pipelines.

• Obtain basic profile information about the end-user in an interoperable and REST-like manner.

Configuring Bitbucket Pipelines OpenID Connect and AWS infrastructures

• configure Bitbucket Pipelines as a Web Identity Provider in AWS

• attach to provider your AWS role with required policies in AWS

• create an access entries for your AWS EKS cluster

• setup a build step with oidc: true in your Bitbucket Pipelines

• provide AWS_OIDC_ROLE_ARN variable to the atlassian/aws-eks-kubectl-run

Bitbucket Pipelines OpenID Connect

Retrieve unique identifiers and claims

In order to configure Bitbucket Pipelines as an external OIDC IDP, you will need to retrieve the workspace, repository, and deployment environment UUID.

1. Select Repository settings on the left navigation sidebar of the repository you want to associate with an ODIC provider.

2. Select OpenID Connect under Pipelines on the left sidebar.

3. Copy the Identity provider URL and Audience. Keep those somewhere accessible as you will need them to complete the steps necessary to configure Pipelines as a External OpenID Connect Provider (OIDC IDP) and set up the build to assume the created role prior to running your build in your cloud environment.

OpenID Connect on AWS

In order to use OpenID Connect on AWS, you will need to configure Bitbucket Pipelines as a Web Identity Provider, create an IAM role, and configure the build to assume the created role prior to running your build.

AWS EKS cluster and OpenID Connect

There are multiple options to create a cluster on AWS EKS. The most reliable way is to use eksctl cli (a simple CLI tool for creating and managing clusters on AWS EKS):

eksctl create cluster --name my-cluster --region us-east-1 --version 1.29

When using AWS EKS, you have to remember, that AWS IAM principals (roles and users) don't make any sense inside an AWS EKS cluster. So, the next step is to connect an AWS IAM service with a Kubernetes service account.

You can allow IAM principals to access Kubernetes objects on your cluster using one of the following methods:

1. Creating Access entries (option recommended by AWS if your cluster is at or later than the platform version listed in the Prerequisites section).

Use access entries to manage the Kubernetes permissions of IAM principals from outside the cluster. You can add and manage access to the cluster by using the EKS API, AWS Command Line Interface, AWS SDKs, AWS CloudFormation, and AWS Management Console. This means you can manage users with the same tools that you created the cluster with.

2. Adding entries to the aws-auth ConfigMap as described in Managing Users or IAM Roles for your Cluster (In a future to be determined Kubernetes version of Amazon EKS, the aws-auth configMap will be removed as a supported authentication source, so migrating to access entries is strongly encouraged).

It is important to know, that only the IAM user who created the cluster has permissions to access it. Refer to the page linked above for how to grant your IAM users and roles permissions within the EKS cluster. From AWS docs: When you create an Amazon EKS cluster, the IAM entity user or role, such as a federated user that creates the cluster, is automatically granted system:masters permissions in the cluster's RBAC configuration. To grant additional AWS users or roles the ability to interact with your cluster, you must edit the aws-auth ConfigMap within Kubernetes. 

Access entries for AWS EKS

Amazon EKS creates an access entry with administrator permissions to the cluster for the IAM principal that originally created the cluster. The cluster must have a platform version that is the same or later than the version listed in the Prerequisites section, or a Kubernetes version that is later than the versions listed in the table.

To create an access entry via AWS Management console:

1. Open the Amazon EKS console at https://console.aws.amazon.com/eks/home#/clusters.

2. Choose the name of the cluster that you want to create an access entry in.

3. Choose the Access tab.

4. Choose Create access entry: add IAM principal, add access policy, select Type.

or use AWS CLI:

aws eks create-access-entry --cluster-name my-cluster --principal-arn arn:aws:iam::111122223333:role/bbci-pipes-oidc --type EC2_Linux

The pipe usage

Started from the version 2.7.0 the atlassian/aws-eks-kubectl-run pipe supports OIDC and you could run kubectl commands against a AWS EKS cluster with OpenID Connect (OIDC) alternative authentication:

Parameter "oidc: true" in the step configuration and variable `AWS_OIDC_ROLE_ARN` are required for OIDC authentication

An example of CI/CD workflow in bitbucket-pipelines.yml:

image: atlassian/default-image:4

pipelines:
  default:
    - step:
        oidc: true
        name: "Get OIDC"
        script:
          - echo "Hello OIDC"
          - pipe: atlassian/aws-eks-kubectl-run:2.7.0
          variables:
            AWS_DEFAULT_REGION: 'us-east-1'
            AWS_OIDC_ROLE_ARN: 'arn:aws:iam::111122223333:role/bbci-pipes-oidc'
            CLUSTER_NAME: 'my-cluster'
            KUBECTL_COMMAND: 'get svc'

Outcomes

Since the initial Amazon Elastic Kubernetes Service (Amazon EKS) launch, it has supported AWS Identity and Access Management principals as entities that can authenticate against a cluster. This was done to remove the burden from administrators of having to maintain a separate identity provider. Using AWS IAM also allows AWS customers to use their AWS IAM knowledge and experience and enables administrators to use AWS IAM security features, such as audit logging or multi-factor authentication.

Before AWS EKS access entries, administrators used Amazon EKS APIs to create clusters, then switched to the Kubernetes API to manage mappings of AWS IAM principals and their Kubernetes permissions. This manual and multi-step process complicated the way users were granted access to Amazon EKS clusters. It prevented administrators from revoking cluster-admin (root-like) permissions from the principal that was used to create the cluster. The need to make calls to different APIs (AWS and Kubernetes) to manage access also increased the likelihood of misconfiguration.

So, now with Bitbucket Pipelines OIDC and AWS EKS access entries your could easily and smoothly integrate and manage AWS and Kubernetes resources in secure way.

The future readings

The next guides provides more details about supported features and configuration Bitbucket Pipelines OIDC and Access entries for AWS EKS:

• Integrate Pipelines with resource servers using OIDC | Bitbucket Cloud | Atlassian Support

• Deploy on AWS using Bitbucket Pipelines OpenID Connect | Bitbucket Cloud | Atlassian Support

• Creating an Amazon EKS cluster - Amazon EKS

• Allowing IAM roles or users access to Kubernetes objects on your Amazon EKS cluster - Amazon EKS

• Enabling IAM principal access to your cluster - Amazon EKS

• A deep dive into simplified Amazon EKS access management controls | Amazon Web Services

Enjoy!