Docker persistent volume: All files get deleted and Octoprint is unhappy

Get Help Docker
1

I am using octoprint since years with Docker on a RPI 3. The volume which is specified in the docker-compose.yaml references a local directory /home/pi/docker_volume.

Octoprint images themselves are managed via "watchtower", i.e. whenever a new version is available as :latest, this will be pulled and the container restarted with the new image. The current version in use is 1.10.2.

I never printed much, but that worked like a charm since years...

Now, since about one week, when I want to print some more stuff, it happens almost every day, that Octoprint is angry and throws this error after some hours when I left it unattended (not during prints though), but when I check the frontend the next morning for example.

image
image1197×205 33 KB

Log file showing a lot of file not found errors:
octoprint.log (9.3 KB)

Investigating I found that the docker volume has basically been reset to the initial state:

root@octopi:/home/pi/docker_volume/octoprint# ls -laR
.:
total 20
drwxr-xr-x 5 root root 4096 Jul 17 08:22 .
drwxr-xr-x 4 pi   pi   4096 Jul 17 08:38 ..
drwxr-xr-x 3 root root 4096 Jul 17 08:22 data
drwxr-xr-x 3 root root 4096 Jul 17 08:22 generated
drwxr-xr-x 2 root root 4096 Jul 17 08:22 translations

./data:
total 16
drwxr-xr-x 3 root root 4096 Jul 17 08:22 .
drwxr-xr-x 5 root root 4096 Jul 17 08:22 ..
drwxr-xr-x 2 root root 4096 Jul 17 08:22 file_check
-rw-r--r-- 1 root root  119 Jul 17 08:22 preemptive_cache_config.yaml

./data/file_check:
total 8
drwxr-xr-x 2 root root 4096 Jul 17 08:22 .
drwxr-xr-x 3 root root 4096 Jul 17 08:22 ..

./generated:
total 12
drwxr-xr-x 3 root root 4096 Jul 17 08:22 .
drwxr-xr-x 5 root root 4096 Jul 17 08:22 ..
drwxr-xr-x 2 root root 4096 Jul 17 08:22 webassets

./generated/webassets:
total 3212
drwxr-xr-x 2 root root    4096 Jul 17 08:22 .
drwxr-xr-x 3 root root    4096 Jul 17 08:22 ..
-rw-r--r-- 1 root root   66934 Jul 17 08:22 packed_client.js
-rw-r--r-- 1 root root   11300 Jul 17 08:22 packed_client.js.gz
-rw-r--r-- 1 root root  110663 Jul 17 08:22 packed_core.css
-rw-r--r-- 1 root root   14145 Jul 17 08:22 packed_core.css.gz
-rw-r--r-- 1 root root  686345 Jul 17 08:22 packed_core.js
-rw-r--r-- 1 root root  159053 Jul 17 08:22 packed_core.js.gz
-rw-r--r-- 1 root root  361353 Jul 17 08:22 packed_libs.css
-rw-r--r-- 1 root root   56592 Jul 17 08:22 packed_libs.css.gz
-rw-r--r-- 1 root root 1681862 Jul 17 08:22 packed_libs.js
-rw-r--r-- 1 root root     314 Jul 17 08:22 packed_plugins.css
-rw-r--r-- 1 root root     216 Jul 17 08:22 packed_plugins.css.gz
-rw-r--r-- 1 root root   93313 Jul 17 08:22 packed_plugins.js
-rw-r--r-- 1 root root   10406 Jul 17 08:22 packed_plugins.js.gz

./translations:
total 8
drwxr-xr-x 2 root root 4096 Jul 17 08:22 .
drwxr-xr-x 5 root root 4096 Jul 17 08:22 ..

Doing a full docker-compose down && docker-compose up presents me with a fresh octoprint where I need to configure everything from scratch. And everything works again for some hours, when suddenly the problem occurs again.

I've set an inotifymonitor on the directory and for sure then saw a lot of deletes happening:

root@octopi:/home/pi/container/octoprint# grep DELETE inotifyout.log
/home/pi/docker_volume/octoprint/printerProfiles/ DELETE _default.profile
/home/pi/docker_volume/octoprint/printerProfiles/ DELETE_SELF
/home/pi/docker_volume/octoprint/ DELETE,ISDIR printerProfiles
/home/pi/docker_volume/octoprint/plugins/ DELETE_SELF
/home/pi/docker_volume/octoprint/ DELETE,ISDIR plugins
/home/pi/docker_volume/octoprint/uploads/ DELETE .metadata.json
/home/pi/docker_volume/octoprint/uploads/ DELETE AI3M_WifiBase.gcode
/home/pi/docker_volume/octoprint/uploads/ DELETE_SELF
/home/pi/docker_volume/octoprint/ DELETE,ISDIR uploads
/home/pi/docker_volume/octoprint/scripts/ DELETE_SELF
/home/pi/docker_volume/octoprint/ DELETE,ISDIR scripts
/home/pi/docker_volume/octoprint/virtualSd/ DELETE_SELF
/home/pi/docker_volume/octoprint/ DELETE,ISDIR virtualSd
/home/pi/docker_volume/octoprint/translations/ DELETE_SELF
/home/pi/docker_volume/octoprint/ DELETE,ISDIR translations
/home/pi/docker_volume/octoprint/ DELETE config.backup
/home/pi/docker_volume/octoprint/watched/ DELETE_SELF
/home/pi/docker_volume/octoprint/ DELETE,ISDIR watched
/home/pi/docker_volume/octoprint/data/achievements/ DELETE 2024.json
/home/pi/docker_volume/octoprint/data/achievements/ DELETE data.json
/home/pi/docker_volume/octoprint/data/achievements/ DELETE_SELF
/home/pi/docker_volume/octoprint/data/ DELETE,ISDIR achievements
/home/pi/docker_volume/octoprint/data/softwareupdate/ DELETE versioncache.yaml
/home/pi/docker_volume/octoprint/data/softwareupdate/ DELETE_SELF
/home/pi/docker_volume/octoprint/data/ DELETE,ISDIR softwareupdate
/home/pi/docker_volume/octoprint/data/ DELETE preemptive_cache_config.yaml
/home/pi/docker_volume/octoprint/data/pluginmanager/ DELETE plugins.json
/home/pi/docker_volume/octoprint/data/pluginmanager/ DELETE notices.json
/home/pi/docker_volume/octoprint/data/pluginmanager/ DELETE_SELF
/home/pi/docker_volume/octoprint/data/ DELETE,ISDIR pluginmanager
/home/pi/docker_volume/octoprint/data/announcements/ DELETE _releases.cache
/home/pi/docker_volume/octoprint/data/announcements/ DELETE _blog.cache
/home/pi/docker_volume/octoprint/data/announcements/ DELETE _important.cache
/home/pi/docker_volume/octoprint/data/announcements/ DELETE _plugins.cache
/home/pi/docker_volume/octoprint/data/announcements/ DELETE _octopi.cache
/home/pi/docker_volume/octoprint/data/announcements/ DELETE_SELF
/home/pi/docker_volume/octoprint/data/ DELETE,ISDIR announcements
/home/pi/docker_volume/octoprint/data/backup/ DELETE_SELF
/home/pi/docker_volume/octoprint/data/ DELETE,ISDIR backup
/home/pi/docker_volume/octoprint/data/appkeys/ DELETE_SELF
/home/pi/docker_volume/octoprint/data/ DELETE,ISDIR appkeys
/home/pi/docker_volume/octoprint/data/file_check/ DELETE_SELF
/home/pi/docker_volume/octoprint/data/ DELETE,ISDIR file_check
/home/pi/docker_volume/octoprint/data/ DELETE print_recovery_data.yaml
/home/pi/docker_volume/octoprint/data/ DELETE_SELF
/home/pi/docker_volume/octoprint/ DELETE,ISDIR data
/home/pi/docker_volume/octoprint/logs/ DELETE plugin_firmwareupdater_console.log
/home/pi/docker_volume/octoprint/logs/ DELETE plugin_pluginmanager_console.log
/home/pi/docker_volume/octoprint/logs/ DELETE octoprint.log
/home/pi/docker_volume/octoprint/logs/ DELETE tornado.log
/home/pi/docker_volume/octoprint/logs/ DELETE auth.log
/home/pi/docker_volume/octoprint/logs/ DELETE plugin_softwareupdate_console.log
/home/pi/docker_volume/octoprint/logs/ DELETE serial.log
/home/pi/docker_volume/octoprint/ DELETE,ISDIR logs
/home/pi/docker_volume/octoprint/ DELETE config.yaml
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE packed_libs.js
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE packed_plugins.js.gz
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE packed_plugins.js
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE packed_plugins.css.gz
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE packed_libs.css.gz
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE packed_libs.js.gz
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE packed_client.js
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE packed_core.css
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE packed_plugins.css
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE packed_core.js.gz
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE packed_core.css.gz
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE packed_client.js.gz
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE packed_core.js
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE packed_libs.css
/home/pi/docker_volume/octoprint/generated/webassets/ DELETE_SELF
/home/pi/docker_volume/octoprint/generated/ DELETE,ISDIR webassets
/home/pi/docker_volume/octoprint/generated/ DELETE_SELF
/home/pi/docker_volume/octoprint/ DELETE,ISDIR generated
/home/pi/docker_volume/octoprint/ DELETE users.yaml
/home/pi/docker_volume/octoprint/timelapse/tmp/ DELETE_SELF
/home/pi/docker_volume/octoprint/timelapse/ DELETE,ISDIR tmp
/home/pi/docker_volume/octoprint/timelapse/ DELETE_SELF
/home/pi/docker_volume/octoprint/ DELETE,ISDIR timelapse
/home/pi/docker_volume/octoprint/slicingProfiles/ DELETE_SELF
/home/pi/docker_volume/octoprint/ DELETE,ISDIR slicingProfiles
/home/pi/docker_volume/ DELETE,ISDIR octoprint

So it seems "something" is deleting everything in the docker volume???

This is my docker-compose.yaml

version: '2.4'

services:
  octoprint:
    image: octoprint/octoprint:latest
    restart: unless-stopped
    container_name: octoprint
    ports:
      - 80:80
    # Allow octoprint to always access our ttyUSB character device type 188
    # https://community.octoprint.org/t/container-doesnt-start-without-a-usb-tty-present-but-i-use-psu-control/49999
    # https://github.com/OctoPrint/octoprint-docker/blob/e76eaafc58af42d984452899897b20ad2d86293d/71-octoprint.docker.rules
    # --> /etc/udev/rules.d/71-octoprint.docker.rules
    device_cgroup_rules:
      - "c 188:* rw"
    privileged: true
    #devices:
    # use `python -m serial.tools.miniterm` to see what the name is of the printer, this requires pyserial
    # No longer needed with the device_cgroup_rules and udev rules above
    # see /etc/udev/...
    #  - /dev/ttyUSB0:/dev/ttyUSB0
    #  - /dev/video0:/dev/video0
    volumes:
      - /home/pi/docker_volume:/octoprint
    # uncomment the lines below to ensure camera streaming is enabled when
    # you add a video device
    #environment:
    #  - ENABLE_MJPG_STREAMER=true

  ####
  # uncomment if you wish to edit the configuration files of octoprint
  # refer to docs on configuration editing for more information
  ####

  #config-editor:
  #  image: linuxserver/code-server
  #  ports:
  #    - 8443:8443
  #  depends_on:
  #    - octoprint
  #  restart: unless-stopped
  #  environment:
  #    - PUID=0
  #    - PGID=0
  #    - TZ=America/Chicago
  #  volumes:
  #    - octoprint:/octoprint

volumes:
  octoprint:

Any idea what might be happening here? :open_mouth: