minikube running on google cloud shell is unable to pull image from artifact registry - denied: Unauthenticated request

[anant-ca]

What Happened?

I am running minikube on google cloud shell. When I deploy my app using a command

kubectl apply -k k8/config/app/app-name

the deployment is unable to pull image from artefact registry. This is the error I receive:

Failed to pull image "europe-docker.pkg.dev/sample-dev-xxxxxx/github-cr/org-name/app:v0.0.46": Error response from daemon: Head "https://europe-docker.pkg.dev/v2/sample-dev-xxxxxx/github-cr/org-name/app/manifests/v0.0.46": denied: Unauthenticated request. Unauthenticated requests do not have permission "artifactregistry.repositories.downloadArtifacts" on resource "projects/sample-dev-xxxxxx/locations/europe/repositories/github-cr" (or it may not exist)

Please note the following:

• The registry is a remote registry that pulls artefacts from GitHub Container registry. This remote registry is duly authenticated while configuration.
• As mentioned above that the above registry is duly authenticated during configuration,
• I am able to pull the image using docker (on google cloud console),
• I am able to pull the image and it deploy on the Cloud Run,
• I am able to pull the image, using the the same kubernetes configuration, and deploy on GKE (google kubernetes engine).
• I also tried using the Automated Google Cloud Platform Authentication addon for minikube but it said It seems that you are running in GCE, which means authentication should work without the GCP Auth addon. If you would still like to authenticate using a credentials file, use the --force flag.

Attach the log file

log.txt

Operating System

Ubuntu

Driver

Docker

[medyagh]

anant-ca do you mind trying "gcloud auth login" in cloud shell and see if that fixes the problem (and then enable the gcp-auth addon )

minikube addons enable gcp-auth

[anant-ca]

@medyagh thanks for your prompt response. I tried your suggestion but it didn't solve the problem (minikube logs contain the same error as mentioned above).

Here's the console output for your reference:

anant@cloudshell:~/gh/deploy (sample-dev-434415)$ gcloud auth login

You are already authenticated with gcloud when running
inside the Cloud Shell and so do not need to run this
command. Do you wish to proceed anyway?

Do you want to continue (Y/n)?  y

Go to the following link in your browser, and complete the sign-in prompts:
...

anant@cloudshell:~/gh/deploy (sample-dev-434415)$ gcloud auth list
Credentialed Accounts

ACTIVE: *
ACCOUNT: [email protected]

To set the active account, run:
    $ gcloud config set account `ACCOUNT`
anant@cloudshell:~/gh/deploy (sample-dev-434415)$ minikube start
😄  minikube v1.33.1 on Ubuntu 22.04 (amd64)
    ▪ MINIKUBE_FORCE_SYSTEMD=true
    ▪ MINIKUBE_HOME=/google/minikube
    ▪ MINIKUBE_WANTUPDATENOTIFICATION=false
✨  Automatically selected the docker driver. Other choices: none, ssh
📌  Using Docker driver with root privileges
👍  Starting "minikube" primary control-plane node in "minikube" cluster
🚜  Pulling base image v0.0.44 ...
💾  Downloading Kubernetes v1.30.0 preload ...
    > gcr.io/k8s-minikube/kicbase...:  481.58 MiB / 481.58 MiB  100.00% 79.67 M
    > preloaded-images-k8s-v18-v1...:  342.90 MiB / 342.90 MiB  100.00% 40.98 M
🔥  Creating docker container (CPUs=2, Memory=4000MB) ...
🐳  Preparing Kubernetes v1.30.0 on Docker 26.1.1 ...
    ▪ kubelet.cgroups-per-qos=false
    ▪ kubelet.enforce-node-allocatable=""
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
anant@cloudshell:~/gh/deploy (sample-dev-434415)$ minikube addons enable gcp-auth
💡  gcp-auth is an addon maintained by Google. For any concerns contact minikube on GitHub.
You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS
❗  It seems that you are running in GCE, which means authentication should work without the GCP Auth addon. If you would still like to authenticate using a credentials file, use the --force flag.
anant@cloudshell:~/gh/deploy (sample-dev-434415)$ kubectl apply -k k8/config/app/user/
namespace/user created
configmap/user-config created
service/user-service created
deployment.apps/user-deployment created
horizontalpodautoscaler.autoscaling/user-hpa created
anant@cloudshell:~/gh/deploy (sample-dev-434415)$ kubectl get all -n user
NAME                                  READY   STATUS         RESTARTS   AGE
pod/user-deployment-999854dbc-jtsr4   0/1     ErrImagePull   0          28s
pod/user-deployment-999854dbc-rmn4w   0/1     ErrImagePull   0          28s

NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
service/user-service   LoadBalancer   10.109.100.15   <pending>     8080:31890/TCP   28s

NAME                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/user-deployment   0/2     2            0           28s

NAME                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/user-deployment-999854dbc   2         2         0       28s

NAME                                           REFERENCE         TARGETS              MINPODS   MAXPODS   REPLICAS   AGE
horizontalpodautoscaler.autoscaling/user-hpa   Deployment/user   cpu: <unknown>/80%   2         20        0          28s

[medyagh]

@anant-ca I see , are you sure you are pulling image from Google AR ? or is that "GitHub Container Registry" I think that would require a different auth creds and would need to login to github