Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Format and lint notebooks #817

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Format and lint notebooks #817

wants to merge 4 commits into from

Conversation

itcarroll
Copy link
Collaborator

@itcarroll itcarroll commented Sep 18, 2024 •
edited
Loading

Closes #788

Re-enable the format and lint tool for notebook (.ipynb) files everywhere in the repo (which turns out to be in notebooks, docs/howto, and mostly docs/tutorials).

The only lint rules I found to be not great for notebooks are T201 and T203, which are prohibitions on print and pprint in production code. I ignored them for "*.ipynb" in [tool.ruff.lint.per-file-ignores].

I found two packages missing from [project.optional-dependencies.docs] that are needed for the notebooks to run, and added them with a minimum dependency taken from the version pip would install. uv add --extra docs dask cftime.

I did not attempt to fix some remaining errors in the notebooks that are beyond the scope of linting. They did not all run when I started and still don't all run (but not because of missing packages!).

Pull Request (PR) draft checklist - click to expand
  • Please review our
    contributing documentation
    before getting started.
  • Ensure an issue exists representing the problem being solved in this PR.
  • Populate a descriptive title. For example, instead of "Updated README.md", use a
    title such as "Add testing details to the contributor section of the README".
  • Populate the body of the pull request with:
  • Update CHANGELOG.md with details about your change in a section titled
    ## Unreleased. If such a section does not exist, please create one. Follow
    Common Changelog for your additions.
  • Update the documentation and/or the README.md with details of changes to the
    earthaccess interface, if any. Consider new environment variables, function names,
    decorators, etc.

Click the "Ready for review" button at the bottom of the "Conversation" tab in GitHub
once these requirements are fulfilled. Don't worry if you see any test failures in
GitHub at this point!

Pull Request (PR) merge checklist - click to expand

Please do your best to complete these requirements! If you need help with any of these
requirements, you can ping the @nsidc/earthaccess-support team in a comment and we
will help you out!

  • Add unit tests for any new features.
  • Apply formatting and linting autofixes. You can add a GitHub comment in this Pull
    Request containing "pre-commit.ci autofix" to automate this.
  • Ensure all automated PR checks (seen at the bottom of the "conversation" tab) pass.
  • Get at least one approving review.

📚 Documentation preview 📚: https://earthaccess--817.org.readthedocs.build/en/817/

@codecov-commenter
Copy link

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (main@028b8fa). Learn more about missing BASE report.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main     #817   +/-   ##
=======================================
  Coverage        ?   68.00%           
=======================================
  Files           ?       22           
  Lines           ?     1466           
  Branches        ?        0           
=======================================
  Hits            ?      997           
  Misses          ?      469           
  Partials        ?        0

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@itcarroll
Copy link
Collaborator Author

itcarroll commented Sep 18, 2024 •
edited
Loading 

uv-lock..................................................................Failed
- hook id: uv-lock
- exit code: 2

@MattF-NSIDC (use mfisher87) I'm unable to find an explanation for what to do about pre-commit's uv-lock check, which is no doubt failing because I've added packages to pyproject.toml. Can you explain how to safely update uv.lock?

@mfisher87
Copy link
Member

Hey @itcarroll for future reference, I don't log in to @MattF-NSIDC often so I don't see the notifications sent there :)

Did the failing check not also update uv.lock when it failed? This was the output for me when I added flask to the dependencies array in pyproject.toml:

$ pre-commit run -a uv-lock
uv-lock..................................................................Failed
- hook id: uv-lock
- files were modified by this hook

warning: `VIRTUAL_ENV=/home/robatt/.cache/pre-commit/repo16nnfz_r/py_env-python3.12` does not match the project environment path `.venv` and will be ignored
Using Python 3.12.5 interpreter at: /home/robatt/.local/share/miniforge3/envs/earthaccess/bin/python3
Resolved 222 packages in 1.12s
Added blinker v1.8.2
Added flask v3.0.3
Added itsdangerous v2.2.0
Added werkzeug v3.0.4

$ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   pyproject.toml
	modified:   uv.lock

@mfisher87
Copy link
Member

Oh, and thanks so much for taking on this important piece of technical debt!! 🙇

@itcarroll
Copy link
Collaborator Author

Understand now that the uv-lock hook is throwing an error, so that's why it's not updating the uv.lock file.

The error I could not make any sense of was

(earthaccess) $ pre-commit run uv-lock
warning: `VIRTUAL_ENV=/Users/icarroll/.cache/pre-commit/repo48a95whq/py_env-python3` does not match the project environment path `.venv` and will be ignored
error: Failed to build: `earthaccess @ file:///Users/icarroll/tmp/earthaccess`
  Caused by: Failed to deserialize cache entry
  Caused by: wrong msgpack marker FixArray(2)

I tried reinstalling everything. What eventually made the error go away was updating the rev on the uv-pre-commit repo to "0.4.12". Is that okay @mfisher87?

@itcarroll itcarroll marked this pull request as ready for review September 18, 2024 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mfisher87 mfisher87 Awaiting requested review from mfisher87

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Lint and format Jupyter Notebooks
3 participants
@itcarroll @codecov-commenter @mfisher87