[SOLVED] Vagrant fails to authenticate with GCP service account in provision

Issue

I tried to fetch some files from GCS in provision, but it didn't work as expected.

I created a service account and allowed it to access the target resources.
Everything works correctly when the script below is executed interactively. However, I got the following warning and errors when executed in provision.

1. Could not open the configuration file: [/root/.config/gcloud/configurations/config_default].
   I assumed it was because .config directory didn't exist in root environment.
   I made .config directory as a test, but it didn't solve the issue.

2. You do not currently have an active account selected.
   I don't want to authenticate as user account, so I don't think I have to implement "gcloud auth login".

Stacktrace and script are as follows.

default: WARNING: Could not open the configuration file: [/root/.config/gcloud/configurations/config_default].
    default: ERROR: (gcloud.iam.service-accounts.keys.create) You do not currently have an active account selected.
    default: Please run:
    default: 
    default:   $ gcloud auth login
    default: 
    default: to obtain new credentials.
    default: 
    default: If you have already logged in with a different account:
    default: 
    default:     $ gcloud config set account ACCOUNT
    default: 
    default: to select an already authenticated account to use.
    default: ERROR: (gcloud.auth.activate-service-account) Could not read json file /root/key.json: Expecting value: line 1 column 1 (char 0)
    default: ServiceException: 401 Anonymous caller does not have storage.objects.get access to the Google Cloud Storage object.
    default: /tmp/vagrant-shell: line 45: : command not found
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.

# Install Google Cloud SDK
## Add URI to package source list
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list

## Install packages needed
sudo apt install -y apt-transport-https ca-certificates gnupg

## Import Google Cloud public key
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -

## Update and Install Google Cloud SDK
sudo apt update && sudo apt install -y google-cloud-sdk

## Create service account key
cd ~ && gcloud iam service-accounts keys create key.json [email protected]

## Activate service account
gcloud auth activate-service-account \
         [email protected] \
         --key-file ~/key.json \
         --project example-project

• Host OS: MacOS 11.5.1
• Guest OS: Ubuntu 18.04 LTS(ubuntu/bionic64)
• Vagrant 2.2.17
• Virtualbox 6.1.26

Solution

IIUC, the error isn't the result of gcloud auth activate-service-account but from gcloud iam service-accounts keys create.

When you attempt gcloud iam service-accounts keys create, you've not authenticated gcloud, so you're not permitted to create a key.

You should create the key outside of the flow and (securely) transfer the key to the Vagrant process so that it may gcloud auth activate-service-account

Answered By - DazWilkin

Answer Checked By - Mildred Charles (WPSolving Admin)