Infinite dependency resolving for kfp and google-cloud-aiplatform

[HenriqueAJNB]

• Poetry version: 1.7.1
• Python version: 3.10.12
• OS version and name: Ubuntu 22.04.3 LTS
• pyproject.toml

[tool.poetry]
name = "my-project"
version = "0.0.1"
description = ""
authors = []
readme = "README.md"
packages = [{ include = "my_pkg" }]

[tool.poetry.dependencies]
python = ">=3.10,<3.12"
kfp = "1.8.22"
google-cloud-aiplatform = "1.25.0"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

To reproduce the issue using the same environment, just run poetry install

It will fall into infinite dependency resolving.

$ poetry install
Creating virtualenv my-project in /home/.../.venv
Updating dependencies
Resolving dependencies... (2003.3s)

The logs using -vvv option shows what is the package that is causing the problem:

Found 0 compatible packages for grpcio-status
   1: fact: no versions of grpcio-status match >1.49.1,<1.50.0 || >1.50.0,<1.51.0rc1 || >1.51.0rc1,<1.51.1 || >1.51.1,<1.51.3 || >1.51.3,<1.52.0rc1 || >1.52.0rc1,<1.53.0 || >1.53.0,<1.53.1 || >1.53.1,<1.53.2 || >1.53.2,<1.54.0 || >1.54.0,<1.54.2 || >1.54.2,<1.54.3 || >1.54.3,<1.55.0rc1 || >1.55.0rc1,<1.55.3 || >1.55.3,<1.56.0 || >1.56.0,<1.56.2 || >1.56.2,<1.57.0 || >1.57.0,<1.58.0 || >1.58.0,<1.59.0 || >1.59.0,<1.59.2 || >1.59.2,<1.59.3 || >1.59.3,<1.60.0 || >1.60.0,<1.60.1 || >1.60.1,<2.0.dev0
   1: conflict: no versions of grpcio-status match >1.49.1,<1.50.0 || >1.50.0,<1.51.0rc1 || >1.51.0rc1,<1.51.1 || >1.51.1,<1.51.3 || >1.51.3,<1.52.0rc1 || >1.52.0rc1,<1.53.0 || >1.53.0,<1.53.1 || >1.53.1,<1.53.2 || >1.53.2,<1.54.0 || >1.54.0,<1.54.2 || >1.54.2,<1.54.3 || >1.54.3,<1.55.0rc1 || >1.55.0rc1,<1.55.3 || >1.55.3,<1.56.0 || >1.56.0,<1.56.2 || >1.56.2,<1.57.0 || >1.57.0,<1.58.0 || >1.58.0,<1.59.0 || >1.59.0,<1.59.2 || >1.59.2,<1.59.3 || >1.59.3,<1.60.0 || >1.60.0,<1.60.1 || >1.60.1,<2.0.dev0
   1: ! grpcio-status (>1.49.1,!=1.50.0,!=1.51.0rc1,!=1.51.1,!=1.51.3,!=1.52.0rc1,!=1.53.0,!=1.53.1,!=1.53.2,!=1.54.0,!=1.54.2,!=1.54.3,!=1.55.0rc1,!=1.55.3,!=1.56.0,!=1.56.2,!=1.57.0,!=1.58.0,!=1.59.0,!=1.59.2,!=1.59.3,!=1.60.0,!=1.60.1,<2.0.dev0) is partially satisfied by not grpcio-status (==1.51.0rc1)
   1: ! which is caused by "grpcio-status (1.51.0rc1) depends on protobuf (>=4.21.6)"
   1: ! thus: grpcio-status (>1.49.1,<1.50.0 || >1.50.0,<1.51.1 || >1.51.1,<1.51.3 || >1.51.3,<1.52.0rc1 || >1.52.0rc1,<1.53.0 || >1.53.0,<1.53.1 || >1.53.1,<1.53.2 || >1.53.2,<1.54.0 || >1.54.0,<1.54.2 || >1.54.2,<1.54.3 || >1.54.3,<1.55.0rc1 || >1.55.0rc1,<1.55.3 || >1.55.3,<1.56.0 || >1.56.0,<1.56.2 || >1.56.2,<1.57.0 || >1.57.0,<1.58.0 || >1.58.0,<1.59.0 || >1.59.0,<1.59.2 || >1.59.2,<1.59.3 || >1.59.3,<1.60.0 || >1.60.0,<1.60.1 || >1.60.1,<2.0.dev0) requires protobuf (>=4.21.6)
   1: ! grpcio-status (>1.49.1,!=1.50.0,!=1.51.1,!=1.51.3,!=1.52.0rc1,!=1.53.0,!=1.53.1,!=1.53.2,!=1.54.0,!=1.54.2,!=1.54.3,!=1.55.0rc1,!=1.55.3,!=1.56.0,!=1.56.2,!=1.57.0,!=1.58.0,!=1.59.0,!=1.59.2,!=1.59.3,!=1.60.0,!=1.60.1,<2.0.dev0) is partially satisfied by not grpcio-status (==1.52.0rc1)
   1: ! which is caused by "grpcio-status (1.52.0rc1) depends on protobuf (>=4.21.6)"
...
   1: conflict: grpcio-status (1.56.2) depends on protobuf (>=4.21.6)
...
   1: conflict: grpcio-status (1.60.0) depends on protobuf (>=4.21.6)
...

Tons of those conflits appears, but no error or exception is raised.

The pip way

But, when I use built-in package venv instead of poetry it runs smootly.

$ python3 -m venv .venv
$ source .venv/bin/activate
(.venv) $ pip3 install kfp==1.8.22 google-cloud-aiplatform==1.25.0

And the most important, pip can found a version for grpcio-status...

(.venv) $ pip3 show grpcio-status
Name: grpcio-status
Version: 1.48.2
Summary: Status proto mapping for gRPC
Home-page: https://grpc.io
Author: The gRPC Authors
Author-email: [email protected]
License: Apache License 2.0
Location: /home/.../.venv/lib/python3.10/site-packages
Requires: googleapis-common-protos, grpcio, protobuf
Required-by:

Attempts

I've tried to use the following attempts with no sucess:

• Force grpcio-status = "1.48.2" in pyproject.toml
• Be flexible with dependencies using kfp = "^1.8" and google-cloud-aiplatform = "^1"
• Clear all caches by running poetry cache clear pypi --all and rm -rf ~/.cache/pypoetry/cache

Sum up

In short terms: this dependencies in pyproject.toml can't be installed using poetry, but they can using pip

Extra information

This dependencies could be installed normally in macOS Sonoma 14.1.1. This seems to be a poetry and Ubuntu problem.

[dimbleby]

Duplicate #8475 please close

[HenriqueAJNB]

Closing, I'll follow in the #8475.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.