Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pyinfra.operations.files.get raises UnicodeDecodeError when attempting to get a file that is not utf8 encoded #1208

Open
evoldstad opened this issue Sep 18, 2024 · 0 comments

Comments

@evoldstad
Copy link

Describe the bug

pyinfra.operations.files.get fails with an exception when attempting to get a file that is not utf-8 encoded.

To Reproduce

Added an example file that demonstrates the issue by cloning and building ipxe: example.zip
Had to zip it due to github not allowing .py files.

Run it with: pyinfra @local encoding-bug-example.py

Expected behavior

The behavior of files.get is the same regardless of encoding, or documentation clearly explains that the operation only supports utf-8 encoded files.

Meta

  • Include output of pyinfra --support.
    System: Linux
    Platform: Linux-6.10.8-1-default-x86_64-with-glibc2.40
    Release: 6.10.8-1-default
    Machine: x86_64
    pyinfra: v3.1
    black: v24.4.2
    black: v24.4.2
    click: v8.1.7
    configparser: v7.1.0
    coverage: v7.5.1
    coverage: v7.5.1
    distro: v1.9.0
    flake8: v7.0.0
    flake8: v7.0.0
    flake8-black: v0.3.6
    flake8-black: v0.3.6
    flake8-isort: v6.1.1
    flake8-isort: v6.1.1
    flake8-spellcheck: v0.12.1
    gevent: v24.2.1
    importlib-metadata: v8.5.0
    ipdb: v0.13.13
    ipdbplugin: v1.5.0
    ipython: v8.27.0
    isort: v5.13.2
    isort: v5.13.2
    jinja2: v3.1.4
    mypy: v1.11.2
    mypy: v1.11.2
    myst-parser: v2.0.0
    myst-parser: v2.0.0
    packaging: v24.1
    paramiko: v3.5.0
    pyinfra-guzzle-sphinx-theme: v0.16
    pyinfra-guzzle-sphinx-theme: v0.16
    pytest: v8.2.1
    pytest: v8.2.1
    pytest-cov: v5.0.0
    pytest-cov: v5.0.0
    python-dateutil: v2.9.0.post0
    pywinrm: v0.5.0
    redbaron: v0.9.2
    setuptools: v65.5.0
    sphinx: v6.2.1
    sphinx: v6.2.1
    twine: v5.1.1
    typeguard: v4.3.0
    types-PyYAML: v6.0.12.20240917
    types-PyYAML: v6.0.12.20240917
    types-cryptography: v3.3.23.2
    types-cryptography: v3.3.23.2
    types-paramiko: v3.5.0.20240918
    types-paramiko: v3.5.0.20240918
    types-python-dateutil: v2.9.0.20240906
    types-python-dateutil: v2.9.0.20240906
    types-setuptools: v75.1.0.20240917
    types-setuptools: v75.1.0.20240917
    typing-extensions: v4.12.2
    wheel: v0.44.0
    Executable: /home/evo/pyinfra-dev/pyinfra/.venv/bin/pyinfra
    Python: 3.11.9 (CPython, GCC)

  • How was pyinfra installed (source/pip)?
    python -m venv .venv && source .venv/bin/activate && pip install pyinfra

  • Include pyinfra-debug.log (if one was created)
    File "/home/evo/pyinfra-dev/test-project/venv/lib64/python3.11/site-packages/pyinfra_cli/main.py", line 222, in cli
    _main(*args, **kwargs)
    File "/home/evo/pyinfra-dev/test-project/venv/lib64/python3.11/site-packages/pyinfra_cli/main.py", line 392, in _main
    run_ops(state, serial=serial, no_wait=no_wait)
    File "/home/evo/pyinfra-dev/test-project/venv/lib64/python3.11/site-packages/pyinfra/api/operations.py", line 334, in run_ops
    _run_single_op(state, op_hash)
    File "/home/evo/pyinfra-dev/test-project/venv/lib64/python3.11/site-packages/pyinfra/api/operations.py", line 302, in _run_single_op
    if not greenlet.get():
    ^^^^^^^^^^^^^^
    File "src/gevent/greenlet.py", line 805, in gevent._gevent_cgreenlet.Greenlet.get
    File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception
    File "/home/evo/pyinfra-dev/test-project/venv/lib64/python3.11/site-packages/gevent/_compat.py", line 49, in reraise
    raise value.with_traceback(tb)
    File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run
    File "/home/evo/pyinfra-dev/test-project/venv/lib64/python3.11/site-packages/pyinfra/api/operations.py", line 184, in _run_host_op_with_context
    return run_host_op(state, host, op_hash)
    ^^^^^^^^^^^^^^^^^
    File "/home/evo/pyinfra-dev/test-project/venv/lib64/python3.11/site-packages/pyinfra/api/operations.py", line 53, in run_host_op
    return _run_host_op(state, host, op_hash)
    ^^^^^^^^^^^^^^^^^
    File "/home/evo/pyinfra-dev/test-project/venv/lib64/python3.11/site-packages/pyinfra/api/operations.py", line 121, in _run_host_op
    status = command.execute(state, host, connector_arguments)
    ^^^^^^^^^^^^^^^^^
    File "/home/evo/pyinfra-dev/test-project/venv/lib64/python3.11/site-packages/pyinfra/api/command.py", line 193, in execute
    return host.get_file(
    ^^^^^^^^^^^^^^^^^
    File "/home/evo/pyinfra-dev/test-project/venv/lib64/python3.11/site-packages/pyinfra/api/host.py", line 435, in get_file
    return self.connector.get_file(*args, **kwargs)
    ^^^^^^^^^^^^^^^^^
    File "/home/evo/pyinfra-dev/test-project/venv/lib64/python3.11/site-packages/pyinfra/connectors/local.py", line 191, in get_file
    data = temp_f.read()
    ^^^^^^^^^^^^^^^^^
    File "", line 322, in decode
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x86 in position 69: invalid start byte

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant