Error 500 after adding place

[mamarguerat]

After successfully adding a place, with name, type, enclosed place and lat-long coordinates, I get an error 500 for any api POST.

It works after a restart, but I can only create 1 place and then it's broken again.

My Gramps Web is installed in a docker container. This issue didn't come with the latest version, it was there before, but still there.

Gramps 5.2.0
Gramps Web API 2.0.0
Gramps Web Frontend 24.3.0
locale: en
multi-tree: false
task queue: true

[DavidMStraub]

Can you please check the Web API logs? They will show an error if you encounter 500 in the frontend.

[mamarguerat]

[2024-03-18 11:44:55 +0000] [20] [ERROR] Error handling request /api/places/
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 135, in handle
    self.handle_request(listener, req, client, addr)
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 178, in handle_request
    respiter = self.wsgi(environ, resp.start_response)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1488, in __call__
    return self.wsgi_app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1466, in wsgi_app
    response = self.handle_exception(e)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/auth.py", line 44, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/views.py", line 110, in view
    return current_app.ensure_sync(self.dispatch_request)(**kwargs)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/views.py", line 191, in dispatch_request
    return current_app.ensure_sync(meth)(**kwargs)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/base.py", line 448, in post
    with DbTxn("Add objects", db_handle) as trans:
  File "/usr/local/lib/python3.11/dist-packages/gramps/gen/db/txn.py", line 83, in __exit__
    self.db.transaction_commit(self)
  File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/db/dbapi/dbapi.py", line 309, in transaction_commit
    self.dbapi.commit()
  File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/db/dbapi/sqlite.py", line 165, in commit
    self.__connection.commit()
sqlite3.OperationalError: database is locked

This is right after successfully adding a place. I cannot edit nor add anything. Is this the log you were requesting?

[DavidMStraub]

Yep. Please try adding IGNORE_DB_LOCK=True to your config file.

[DavidMStraub]

(GRAMPSWEB_IGNORE_DB_LOCK: true if you are using YAML docker environment variables.)

[mamarguerat]

Thanks for your help. It worked, but only for 1 more database edit. Now I can add 2 places before getting the 500 error, and the same log traceback.

[DavidMStraub]

Ok, something seems to be wrong either with your setup or with your database. I assume this is not the only object in your database?

Have you ried running check & repair?

[mamarguerat]

I just tried to sync the database with gramps, run a check & repair (1 empty event has been deleted) and sync back the database to gramps web. The issue is still there.
It's not the only object in the database, there is about 240 people, 50 medias, 30 places, 400 events.

[DavidMStraub]

Please check if there is a lock file in the database directory and delete it manually. Not sure what is wrong there.

[mamarguerat]

I couldn't find any lock file in the database directory.

I found the following error in the log

ERROR:root:Error parsing list of recent DBs from file /root/.gramps/recent-files-gramps.xml: no element found: line 1, column 0.
This might indicate a damage to your files.
If you're sure there is no problem with other files, delete it, and restart Gramps.

The docker compose file I use for configuration

version: "3.7"

services:
  grampsweb: &grampsweb
    image: ghcr.io/gramps-project/grampsweb:latest
    restart: always
    ports:
      - "5000:5000"  # host:docker
    environment:
      GRAMPSWEB_TREE: "Family tree"  # will create a new tree if not exists
      GRAMPSWEB_CELERY_CONFIG__broker_url: "redis://grampsweb_redis:6379/0"
      GRAMPSWEB_CELERY_CONFIG__result_backend: "redis://grampsweb_redis:6379/0"
      GRAMPSWEB_RATELIMIT_STORAGE_URI: redis://grampsweb_redis:6379/1
      GRAMPSWEB_IGNORE_DB_LOCK: true 
    depends_on:
      - grampsweb_redis
    volumes:
      - /volume1/docker/grampsweb/users:/app/users  # persist user database
      - gramps_index:/app/indexdir  # persist search index
      - gramps_thumb_cache:/app/thumbnail_cache  # persist thumbnails
      - gramps_cache:/app/cache  # persist export and report caches
      - gramps_secret:/app/secret  # persist flask secret
      - /volume1/docker/grampsweb/db:/root/.gramps/grampsdb  # persist Gramps database
      - /volume1/docker/grampsweb/media:/app/media  # persist media files
      - gramps_tmp:/tmp

  grampsweb_celery:
    <<: *grampsweb  # YAML merge key copying the entire grampsweb service config
    ports: []
    container_name: grampsweb_celery
    depends_on:
      - grampsweb_redis
    command: celery -A gramps_webapi.celery worker --loglevel=INFO

  grampsweb_redis:
    image: redis:alpine
    container_name: grampsweb_redis
    restart: always

volumes:
  gramps_users:
  gramps_index:
  gramps_thumb_cache:
  gramps_cache:
  gramps_secret:
  gramps_db:
  gramps_media:
  gramps_tmp:

[DavidMStraub]

How much RAM does your server have?

[mamarguerat]

It has 20GB, shared with other dockers and apps. The memory usage of the whole system is about 40%.

[DavidMStraub]

Ok it's an SQLite lock error, not a Gramps one, so there must be two processes trying to write simultaneously to the database. Are you accessing the database with anything else, e.g. syncing in parallel?

Can you try manually unlocking the database using the command line? Perhaps there is some process that crashes before releasing the lock.

[mamarguerat]

I'm not accessing the database with another process. The only thing that access the file is a backup once a day. The backup does not access the database directly, it just copies the database file.
How can I unlock the database with the command line ?

[DavidMStraub]

Is this error still happening? If you can't confirm, I'll go ahead and close this issue. There have been improvements in v2.3.1 which might well have solved this.

[mamarguerat]

Hi, I tried with v2.3.1 and the error is still happening. The first write to the database after boot can complete successfully. The second and next ones are reporting an error 500 response to the HTTP put message.

[DavidMStraub]

Thanks for checking