How can I use SSH in Bitbucket Pipelines?

Philip Hodder
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 3, 2016

I would like to use SSH in Bitbucket Pipelines to clone another repository from Bitbucket.

7 answers

2 accepted

6 votes
Answer accepted
Philip Hodder
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 3, 2018

Up to date as of 04/04/2018.

For example. I want to clone repo-2 into a build from repo-1

To use SSH to clone another repository from Bitbucket you need to do the following.

  1. In repo-1 generate a Pipelines SSH key to be injected into your build container. Follow Step 1 here: https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html
    • Note that if the known hosts information for bitbucket.org and github.com are injected by default. If you are using any other site you will need to configure Known Hosts, by following Step 2 in the above documentation.
  2. Add an Access Key to repo-2. Paste in the public key that was generated in the previous step.
  3. Configure your bitbucket-pipelines-yml to use git.

An example bitbucket-pipelines.yml is:

image: ubuntu:16.04

pipelines:
default:
- step:
- apt-get update -y
- apt-get install -y git
- git clone <your_repo_here> # For example, git@bitbucket.org:username/repo-2.git
Khue Bui July 10, 2018

Hi Philip,

I followed your steps above to add submodule from another bitbucket repo to my pipeline, but unfortunately encounter error:

 Host key verification failed.fatal: Could not read from remote repository.

 

GC August 26, 2018

Hi @Philip Hodder! I work at my client's office and they have provided me access to their private repositories. All repositories that i have access to are private. I am trying to run a pipeline in repo A and want to checkout some build scripts from repo B so i could use them during the pipeline run in repo A. I followed your steps described above and added public key (from the key-pair i generated in repo A) to "Access keys" section in repo B. Now when i try building code, i get the following message in pipeline for repo A:

+ git clone -b dev git@bitbucket.org:company_name/devops.git
Cloning into 'devops'...
Warning: Permanently added the RSA host key for IP address '18.205.93.0' to the list of known hosts.

To access this repository, add your IP address to the whitelist.
For details, see https://confluence.atlassian.com/x/TY5qMw

fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.

I checked the link mentioned in the log but it seems whitelisting is a Premium feature. When i checked settings in my avatar, it shows free account. Is there no way i can access one private repo's content in another repo's pipeline without a Premium account?

Philip Hodder
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
August 26, 2018

Hi @GC,

The IP whitelist needs to be configured on your client's account (the one that owns the repository you're trying to clone). Not on your own user account.

The IPs that need to be whitelisted are listed here: https://confluence.atlassian.com/bitbucket/what-are-the-bitbucket-cloud-ip-addresses-i-should-use-to-configure-my-corporate-firewall-343343385.html

Specifically the "Valid IP addresses for Bitbucket Pipelines build environments" section.

Thanks,

Phil

GC August 26, 2018

Thanks for the response @Philip Hodder. I'll request the client for whitelisting however, the IP mentioned in my error message is 18.205.93.0 but the section you've pointed out isn't having that range. On the other hand, under section "Valid IP addresses for bitbucket.org, api.bitbucket.org, and altssh.bitbucket.org", i see
"(Since July 28, 2018) IPv4 inbound for bitbucket.org, api.bitbucket.org, and altssh.bitbucket.org" which has my range. Can you kindly confirm which range do i need to whitelist?

Philip Hodder
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
August 26, 2018

I think you still need to use the IP range I suggested.

The IP mentioned in your build is being added to the known_hosts inside your pipeline. i.e. Your pipelines is talking to Bitbucket to perform the clone (IP 18.205.93.0 in this case)

If the range I suggest doesn't work, then try the other ones and report back. :)

GC August 26, 2018

That makes sense. Thanks for clarifying it. I've sent both the range sets to the client. Thanks Philip! :)

Philip Hodder
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
August 26, 2018

No problem :)

André Ferro Cadete August 27, 2018

@GC

If both repos reside in the same owner (i.e. the team) you don't need much to check-out a Repo within another Repo's pipeline. All you need is to configure the SSH key pairs within Pipeline itself, this will give you a public key that you can add to the authorized keys of the other Repo and it is enough.

Creating the SSH key specifically for Pipelines is described here:

https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html

 

The last part, instead of adding the public key to a "remote server" you will add it in the authorized keys of the repository you want to clone in the pipeline of the other repository.

(i.e. Repo A creates Pipeline key, Repo B adds as authorized key. Now Repo A can clone repo B in it's pipelines without explicitly pointing to a key).

Edit: answering because it does not sound like a whitelist problem, maybe something else you missed? You can also add the fingerprint of bitbucket in the same place you create the pipelines ssh key, but it shouldn't be needed (bitbucket already accepts its own host).

GC August 27, 2018

@André Ferro CadeteThanks for the response but when you said adding public key to authorized keys file in repo B (that i want to clone inside repo A's pipeline), where exactly will i do that? In case of a running pipeline, it's possible but where can this be done in a repo which needs to be cloned?

GC August 28, 2018

@Philip Hodder: It's working now after whitelisting the IPs. Thank you! :)

Alejandro Cotilla November 4, 2018

This solution works for me if I make git operations in my yml file, but, it doesn't work if my Docker image makes git operations from within. How can solve that? Thanks!

This is the error I get:

Please make sure you have the correct access rights
and the repository exists.
Warning: Permanently added the RSA host key for IP address '(...)' to the list of known hosts.
repository access denied. deployment key is not associated with the requested repository.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

Alejandro Cotilla November 5, 2018

Never mind, the issue was because the SSH keys defined under Settings / Pipelines / SSH keys have read-only access and I was trying to push.

3 votes
Answer accepted
Samuel Dorsaz January 29, 2018

There is today an official way to use keys without env variables

https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html

Elliot Borst February 1, 2018

How do you actually use that key in a command?


Philip Hodder
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
February 1, 2018

The keys are injected into your build and should be available to commands that need them (in a similar way to how it would run locally).

Note that you'll still need to add the corresponding public key to whichever service you're trying to contact.

Elliot Borst February 1, 2018

Yeah ive dropped the pipeline pub key onto my server in .ssh/authorized_keys

A simple "ssh -i user@domain.com" doesn't work

Ill keep trying

Philip Hodder
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
February 1, 2018

What error message are you getting? Key not found, or authentication errors?

Samuel Dorsaz February 2, 2018

@Elliot Borstthe -i option let you change the default key but you don't specify any file. You should remove this option.

Elliot Borst February 2, 2018

"ssh root@server.domain.com  Pseudo-terminal will not be allocated because stdin is not a terminal.Host key verification failed."  

 

The pub key is on the other box, I can ssh fine from my Mac, I can see the connection come through our firewall ok.  Seems like pipelines isn't sending the private key with my command

Elliot Borst February 2, 2018

Oh I worked it out, I wasn't actually adding the known host after doing the lookup.

Samuel Dorsaz February 5, 2018

Pseudo-terminal will not be allocated because stdin is not a terminal

take a look on ssh -t option if necessary

Hassange August 12, 2018

Oh I worked it out, I wasn't actually adding the known host after doing the lookup.

Same issue, thanks for the tip

4 votes
Steven Vaccarella
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 12, 2016

For a slightly more general guide for using ssh and ssh-based tools from a pipeline (not just for cloning another Bitbucket repository) see this other question:

https://answers.atlassian.com/questions/39429257

It is similar to Philip's answer but also covers server key fingerprints and uploading your public key to other types of servers.

3 votes
Philip Hodder
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 3, 2016

 

Out of date response. Content here still works, but is much more work to set up.

See this for an up to date solution: https://community.atlassian.com/t5/Bitbucket-questions/Re-How-can-I-use-SSH-in-Bitbucket-Pipelines/qaq-p/764824/comment-id/25670#M25670

--------------------------

Old method

There's a few steps required to do this.

First, you need to generate a key-pair to use for Bitbucket.

Generate a new SSH key-pair for use in Bitbucket. 

$ ssh-keygen -t rsa -b 4096 -C "

Do not use a passphrase, when prompted.

Once you have generated the key-pair, add the public key to your account:

https://confluence.atlassian.com/display/BITBUCKET/Add+an+SSH+key+to+an+account

If you already had an SSH key, and skipped the set up, start reading from here.

Now, we can set up SSH in Bitbucket Pipelines:

First, we will add the SSH key as an environment variable.

As of right now, Pipelines does not support line breaks in the environment variable, so we need to encode the private key first:

$ base64 &lt;path&gt;/&lt;to&gt;/id_rsa | pbcopy # probably path is ~/.ssh/id_rsa

Now create a secured environment variable in Bitbucket Pipelines called PRIVATE_KEY, with the contents of the base64 encoded private key:

https://confluence.atlassian.com/display/BITBUCKET/Environment+variables+in+Bitbucket+Pipelines

Now, create a bitbucket-pipelines.yml with the following content (I can't find the original source of this example sorry, share if you know smile):

# You can use a Docker image from Docker Hub or your own container
# registry for your build environment.
pipelines:
  default:
    - step:
        script: # Modify the commands below to build your repository.
          - echo $PRIVATE_KEY &gt; ~/.ssh/id_rsa.tmp
          - base64 -d ~/.ssh/id_rsa.tmp &gt; ~/.ssh/id_rsa
          - chmod 600 ~/.ssh/id_rsa
          - base64 ~/.ssh/id_rsa
          - git clone git@bitbucket.org:&lt;account_name&gt;/&lt;repo_name&gt;.git

You should see that the build successfully cloned your repository.

Troubleshooting

Q: The clone asks for a passphrase:

A: There are two things you should double check. When you created the SSH key-pair, did you create them without a passphrase? (If there's a passphrase, things get much more complicated than expected). If you did not have a passphrase, double check that the key (the private key) you have stored is base64 encoded.

------------------------

Out of date response. Content here still works, but is much more work to set up.

See: https://community.atlassian.com/t5/Bitbucket-questions/Re-How-can-I-use-SSH-in-Bitbucket-Pipelines/qaq-p/764824/comment-id/25670#M25670

Rob Jones July 5, 2016

 This line is superfluous and can be deleted: 
- base64 ~/.ssh/id_rsa

 

Keith Webster August 14, 2016

I added the above to my config file but on the first statement, echo $PRIVATE_KEY > ~/.ssh/id_rsa.tmp, I received the following: bash: /root/.ssh/id_rsa.tmp: No such file or directory

Philip Hodder
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
August 14, 2016

Add this statement before you echo the PRIVATE_KEY:

mkdir ~/.ssh

Does that help? smile

Pintea Cosmin August 17, 2016

Maybe this will help... I have configured the  SSH key-pair without passphrase but when i tried to clone the repo with the above code i get the the request to enter some passphrase . For me the solution was to add -i to the command that write the key   . I dont know why i have this behavior. The encoding was done with version  8.25 and decoded with 8.21.

base64 -d  -i ~/.ssh/id_rsa.tmp > ~/.ssh/id_rsa

Riyanka Purkayastha August 25, 2016

after running
git clone git@bitbucket.org:account_name/repo_name.git

+ git clone git@bitbucket.org:account_name/repo_name.git
Cloning into 'repo_name'...
The authenticity of host 'bitbucket.org (104.192.143.3)' can't be established.
RSA key fingerprint is 97:8c:1b:f2:6f:14:6b:5c:3b:ec:aa:46:46:74:7c:40.
Are you sure you want to continue connecting (yes/no)?

and can't do anything else

Pintea Cosmin August 26, 2016

try this : 

 - ssh-keyscan bitbucket.org >> ~/.ssh/known_hosts

Steven Vaccarella
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
August 26, 2016

That will work, but it's not very secure (it's essentially bypassing host key checking because the key scan is performed on every build). A better option is to do the key scan just once locally and then include the full host key in your script or add a known_hosts file to your repository. That way your build will fail if the host key changes (which would indicate a potential security issue). You can find more details in step 4 of the following answer: How do I set up ssh public-key authentication so that I can use ssh, sftp or scp from my Bitbucket Pipelines pipeline?

André Ferro Cadete July 10, 2018

As of right now, Pipelines does not support line breaks in the environment variable 

@Steven Vaccarella
Would you PLEASE add this potentially time-saving information to your ENV var documentation page?

https://confluence.atlassian.com/bitbucket/environment-variables-794502608.html

This info is crucial and nowhere to be found in the docs. Wasted a few hours trying to understand what the hell was wrong with my keys. 

For Docker within Docker + composer install, your new "easy" solution does not work (my docker build will run composer install and inside the built container your Bitbucket keys are nowhere to be found, nor do I want to rely on it - my build tool should work anywhere, not specifically in bitbucket pipelines).

Thanks

Linette
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 10, 2018

@André Ferro CadeteGood call! I've updated the environment variables page accordingly. Sorry you had such a painful experience, but hopefully now others won't have the same.

Linette
(Pipelines writer)

André Ferro Cadete July 12, 2018

Hi @Linette,

 

Thanks for the quick response and action. And no problem, was a learning experience as always ;)

0 votes
Nguyễn Hồng Quân January 27, 2019

Please support SSH key of ed25519 type.

Philip Hodder
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
January 28, 2019

Hi,

Can you please open a feature request for that here: https://bitbucket.org/site/master/issues/new

0 votes
Cory Robinson January 2, 2018

I'm attempting to use `npm` packages installed from private repositories in the same team account as the pipelines repo is running. I tried the steps for adding ssh but no luck any advice?

Screenshot 2018-01-02 21.00.48.png

anton_bakai_twinwin May 29, 2018

I have the same issue when I'm trying to install a dependency from a private repo. Did you find the solution?

Like Cory Robinson likes this
Cory Robinson May 28, 2019

@Philip Hodder any ideas on how we could solve this? 

Philip Hodder
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
June 10, 2019

Can you list out the steps you've tried for setting this up at the moment? It'll make it easier for me to see if you've missed anything in particular.

From the error message, it looks like NPM isn't using SSH. It's calling an HTTPS URL, and mentions an invalid username and password, rather than an invalid SSH key.

Raja Periyasamy July 12, 2019

@Cory Robinson Hope you resolved this issue. I had the very same issue and resolved by:

- changed the dependency git url in the package file from https to ssh. i.e. https://bitbucket.org/<company>/<project>.git to git@bitbucket.org:<company>/<project>.git.

- added the pipeline ssh public key to the target project.

0 votes
Keith Webster August 15, 2016

Great, this worked. Thanks! 

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events