Fix Builds (python_gnupg-0.5.2-py2.py3-none-any.whl.asc 404 not found) #78
Comments
maltfield
changed the title
|
In the past, we've had build issues due to the verification of this same In the above issues, the issue was a tooling issue with the availability of curl/wget on MacOS. But in this case, it appears that the file actually has disappeared from the server. I cannot fetch it, for example, on my local machine |
|
I checked the package's page on the PyPI project's website. If there's a newer version, then it might make sense that the archived version somehow got orphaned from its signature? If so, it's a bug that we should report Anyway, it appears that the version hasn't changed since the one we pegged as a dependency. The latest version is still v0.5.2 |
|
I was digging through my notes and found that my old link to hash checking in PyPI is stale The above link points to a new location which, after some digging, brings you here Woah, that's dangerous. The above link suggests that hash checking in pip somehow adds security to verify packages are not malicious when turned-on. This is dangerous misinformation. pip doesn't have any built-in mechanism to cryptographically verify the authenticity of packages (although they've been working on this since 2019). The whole reason we have this awkward manual gpg check is because pip doesn't have a secure way to download software. Fortunately, Anyway, I opened a ticket with PyPI to fix their documentation to move the hashing section outside of the |
|
I opened an upstream bug about this here: |
|
I switched to fetching the files from GitHub, but now I have an issue with verifying the signature of the libusb1 dependency |
|
I opened an upstream bug about this here: |
|
Per the above ticket, the |
maltfield
added a commit
that referenced
this issue
Mar 6, 2024
it looks like the upstream PyPI signature bugs are more widespread than just python_gnupg I've worked with the devs to ensure releases are getting uploaded to GitHub so our builds aren't blocked by PyPI bugs breaking PGP signatures * #78 * vsajip/python-gnupg#234 * vpelletier/python-libusb1#96 Now we download from GitHub's website instead of PyPI's website, so 404 errors from PyPI won't break the builds. TODO: update this to query the GitHub API and grab the latest release (currently they're just hardcoded & pinned as a quick fix)
maltfield
added a commit
that referenced
this issue
Mar 6, 2024
maltfield
added a commit
that referenced
this issue
Mar 7, 2024
|
I've replaced the download URL for the signed python deps from PyPI to GitHub in all three platform's build scripts, but it looks like MacOS is still having some distinct issues finding the path to |
|
I'm also struggling with the PowerShell on Windows. For some reason I can't get the subcommand to work on this curl call. But if I take the same output from the subcommand and paste it into the curl command manually, it works fine I've asked about this on SE here: |
|
Update: issue with the windows builds in previous comment was a trailing newline. I fixed it by saving the output of |
|
Windows builds are fixed! I fixed the issue with the MacOS builds not being able to find PIP_PATH by adding a wildcard glob at the end of /usr/local/Cellar/python* But, unfortunately, lots of other pip depends are now failing to install It even fails at the end (in the "print debugging info" section) when trying to get the pip version |
|
Looks like I had this issue before, and it was fixed by making sure we used the right python/pip version |
|
Google seems to suggest that these Currently we install Looks like the latest version is |
|
TOFU 1/3 (VPN, exit in Hong Kong) gives sha256sum = |
|
TOFU 2/3 (Tor, exit in Poland) gives sha256sum = |
|
TOFU 3/3 (ISP, exit in Ecuador) gives sha256sum = |
|
Ok, 3/3 TOFU on 3 distinct days matches; adding to the repo |
maltfield
added a commit
that referenced
this issue
Mar 10, 2024
Upgrading pip in attempt to fix the SCHEME_KEYS errors on macos builds. For more info, see: * #78 (comment)
|
After upgrading, the linux & windows builds are still working but the MacOS builds are still failing We fixed the issue with SCHEME_KEYS, however, this is something new. For some reason when it installs |
|
The first noticeable error appears to be when it tries to install PyInstaller |
|
There's also issues installing Kivy. I think I should just go-through and update all the python packages to their latest versions. |
|
While I'm at it, I'll update python to Python v3.12. Python is already securely installed in our linux CI runner, so no manual updates needed there. Python comes installed on our MacOS CI runner. Possibly not secure, but we're relying on shared GitHub runners for MacOS builds. Anyway, no manual updates needed. We do have to install Python on our Windows CI runners, and manually update our installer in this repo. Fortunately, it looks like Python does sign their windows releases with PGP, but I don't have the key in my keyring. It looks like Python has a designated person as a Release Manager for each version of Python. Additionally, they have a designated person just for signing their Windows releases:
I'm going to have to 3TOFU those fingerprints. Here's the commands for this |
|
The documentation on how to verify the signature on python releases wasn't linked-to from the actual download page (where the link to download the signature is available), so I opened this ticket with the Python project to address this: |
|
I also updated the build script to output the version of My local Windows 10 VM (which verifies the GOOD signature correctly) has gpg 2.4.5 (same as above), but with libgcrypt 1.10.3 (slightly newer than above) |
|
I manually re-ran the build, and it's working now. So, yep, adding Curiously, though, it's still detecting it as a failure and exiting 1 |
maltfield
added a commit
that referenced
this issue
Jun 6, 2024
this commit updates our documentation such that, when signing text files, we use the '--textmode' argument. This ensures that the signatures will still be valid on dumb systems (cough Windows cough) that insert additional carriage return characters before our newline characters. * #78 (comment) * BusKill/buskill-app-deps@63ccc12
|
ok, I fixed the error by making sure to execute |
|
cool, looks like our windows builds deps are now being installed OK. But we're getting an error on PyInstaller |
|
Well, I'm not familiar with the Curiously, it looks like PyInstaller is bitching about a module that it itself uses. So my best-guess here is that I need to update PyInstaller. It appears that we're currently installing PyInstaller-3.6. Oh wow, that's crazy oudated; it was release Jan 2020 The latest version is PyInstaller-6.7.0 it looks like this PyInstalller-5.12.0 was released on 2023-06-08, but the next release (5.13.0) explicilty states "Add support for Python 3.12" Looks like I'm missing a section to download windows-specific wheels for pyinstaller from the Here's the 3TOFU script |
|
Here's TOFU 1/3 (Tor, exit in Germany) |
|
Here's TOFU 2/3 (VPN, exit in France) |
|
Ah, fuck. I just realized that I've been downloading pyinstaller-6.7.0 (the latest version) though I said I'd use pyinstaller-6.5.0 to be consistent with others. Probably I should have done both |
|
Here's TOFU 1/3 (Tor, exit in the Netherlands) |
|
Here's TOFU 2/3 (VPN, exit in India) |
|
Here's TOFU 3/3 (ISP, exit in Ecuador) |
|
Ok, the hashes match on all 3TOFUs. I'll try upgrading pyinstaller in Windows now. |
maltfield
added a commit
to BusKill/buskill-app-deps
that referenced
this issue
Jun 10, 2024
see reasoning and 3TOFU here * BusKill/buskill-app#78 (comment)
maltfield
added a commit
that referenced
this issue
Jun 10, 2024
for more info, see: * #78 (comment)
|
w00t! The latest windows build completed successfully |
|
Well, I tried to download and run the latest build of BusKill for windows on my Windows 10 VM, but I got an error
Here's the relevant code that it's complaining about
Curiously, it appears this block was only added for running the app in a Windows 10 VM :/ |
|
Apparently the Update: Oh, right, it's because this is a new addition that I made in the |
maltfield
added a commit
that referenced
this issue
Jun 10, 2024
This commit only imports grp on *nix systems, since it isn't available on windows systems, and it will throw an error * #78 (comment) Traceback (most recent call last:): File "main .py", line 22 in <module> import platform, os, grp ModuleNotFoundError: No module named 'grp'
|
Apparently the |
maltfield
added a commit
that referenced
this issue
Jun 13, 2024
this commit makes the 'import pwd' only happen on *nix systems, since it's not available on windows * #78
|
I was successfully able to open the most-recent Windows build in my win10 VM I quickly encountered a bug that triggered the app to crash...but that's an item for another ticket. The builds are working. |
|
I was successfully able to open the most-recent MacOS bulid in our macOS 10.15 mac mini in the cloud. After brief testing, it worked fine (I was unable to get it to crash). |
|
I was successfully able to open the most-recent Linux build in Debian 12. After brief testing, it worked fine (I was unable to get it to crash). |
|
Builds are fixed. This ticket is complete 🎉 |
This was referenced Jul 26, 2024
|
Update: to provide more background on what I did with the "3TOFU" process above in this ticket, I wrote this article: 3TOFU: Verifying Unsigned Releases
|
|
Update I've documented the painful process to download Brew dependencies (for 3TOFU) from GitHub Packages (as opposed to Bintray) in this article: Manually Downloading Container Images (Docker, Github Packages)
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
Currently it's not possible to build the BusKill app due to gpg signing issues with the python-gnupg dependency
To Reproduce
I've tested this on Linux, but it should be an issue on all platforms
Steps to reproduce the behavior:
Execute the following on Debian 11
You'll get the following error
Expected behavior
The build should be able to verify the authenticity of the
python_gnupgpackage and proceed with building the BusKill appThe text was updated successfully, but these errors were encountered: