Long docker exec commands time out #30

New issue

Open

opened 2021-06-01 09:40:18 +02:00 by Jef Roosens · 0 comments

Jef Roosens commented

2021-06-01 09:40:18 +02:00

Owner

There's a weird phenomenon on one of my servers where the backup file fluctuates greatly in size:

8.8M    2021-05-22_00-05-40.sql.gz
13M     2021-05-22_12-05-20.sql.gz
2.5M    2021-05-23_00-05-02.sql.gz
1.8M    2021-05-23_12-05-14.sql.gz
720K    2021-05-24_00-05-33.sql.gz
0       2021-05-24_12-05-00.sql.gz
13M     2021-05-25_12-05-42.sql.gz
13M     2021-05-26_00-05-31.sql.gz
13M     2021-05-26_12-05-20.sql.gz
976K    2021-05-29_12-05-12.sql.gz
6.7M    2021-05-30_00-08-03.sql.gz
14M     2021-05-30_12-06-37.sql.gz
7.7M    2021-05-31_00-06-59.sql.gz
2.3M    2021-05-31_12-05-07.sql.gz

My theory right now is that this is being caused by the docker exec command timing out. These backups are for a Miniflux instance, which regularly locks the entire database because it's polling RSS feeds. This makes pg_dump painfully slow during the duration of this process. I think that subprocess.run therefore times out this command, resulting in these incomplete backups.

This is an important issue to address, and should be fixed as quickly as possible.

There's a weird phenomenon on one of my servers where the backup file fluctuates greatly in size: ``` 8.8M 2021-05-22_00-05-40.sql.gz 13M 2021-05-22_12-05-20.sql.gz 2.5M 2021-05-23_00-05-02.sql.gz 1.8M 2021-05-23_12-05-14.sql.gz 720K 2021-05-24_00-05-33.sql.gz 0 2021-05-24_12-05-00.sql.gz 13M 2021-05-25_12-05-42.sql.gz 13M 2021-05-26_00-05-31.sql.gz 13M 2021-05-26_12-05-20.sql.gz 976K 2021-05-29_12-05-12.sql.gz 6.7M 2021-05-30_00-08-03.sql.gz 14M 2021-05-30_12-06-37.sql.gz 7.7M 2021-05-31_00-06-59.sql.gz 2.3M 2021-05-31_12-05-07.sql.gz ``` My theory right now is that this is being caused by the `docker exec` command timing out. These backups are for a Miniflux instance, which regularly locks the entire database because it's polling RSS feeds. This makes `pg_dump` painfully slow during the duration of this process. I think that `subprocess.run` therefore times out this command, resulting in these incomplete backups. This is an important issue to address, and should be fixed as quickly as possible.