Long docker exec commands time out #30

New Issue

Jef Roosens · 2021-06-01T09:40:18+02:00

Jef Roosens commented

2021-06-01 09:40:18 +02:00

There's a weird phenomenon on one of my servers where the backup file fluctuates greatly in size:

8.8M    2021-05-22_00-05-40.sql.gz
13M     2021-05-22_12-05-20.sql.gz
2.5M    2021-05-23_00-05-02.sql.gz
1.8M    2021-05-23_12-05-14.sql.gz
720K    2021-05-24_00-05-33.sql.gz
0       2021-05-24_12-05-00.sql.gz
13M     2021-05-25_12-05-42.sql.gz
13M     2021-05-26_00-05-31.sql.gz
13M     2021-05-26_12-05-20.sql.gz
976K    2021-05-29_12-05-12.sql.gz
6.7M    2021-05-30_00-08-03.sql.gz
14M     2021-05-30_12-06-37.sql.gz
7.7M    2021-05-31_00-06-59.sql.gz
2.3M    2021-05-31_12-05-07.sql.gz

My theory right now is that this is being caused by the docker exec command timing out. These backups are for a Miniflux instance, which regularly locks the entire database because it's polling RSS feeds. This makes pg_dump painfully slow during the duration of this process. I think that subprocess.run therefore times out this command, resulting in these incomplete backups.

This is an important issue to address, and should be fixed as quickly as possible.

There's a weird phenomenon on one of my servers where the backup file fluctuates greatly in size: ``` 8.8M 2021-05-22_00-05-40.sql.gz 13M 2021-05-22_12-05-20.sql.gz 2.5M 2021-05-23_00-05-02.sql.gz 1.8M 2021-05-23_12-05-14.sql.gz 720K 2021-05-24_00-05-33.sql.gz 0 2021-05-24_12-05-00.sql.gz 13M 2021-05-25_12-05-42.sql.gz 13M 2021-05-26_00-05-31.sql.gz 13M 2021-05-26_12-05-20.sql.gz 976K 2021-05-29_12-05-12.sql.gz 6.7M 2021-05-30_00-08-03.sql.gz 14M 2021-05-30_12-06-37.sql.gz 7.7M 2021-05-31_00-06-59.sql.gz 2.3M 2021-05-31_12-05-07.sql.gz ``` My theory right now is that this is being caused by the `docker exec` command timing out. These backups are for a Miniflux instance, which regularly locks the entire database because it's polling RSS feeds. This makes `pg_dump` painfully slow during the duration of this process. I think that `subprocess.run` therefore times out this command, resulting in these incomplete backups. This is an important issue to address, and should be fixed as quickly as possible.

Jef Roosens added this to the 1.0.0 milestone 2021-06-01 09:40:18 +02:00

Jef Roosens added the

bug

label 2021-06-01 09:40:18 +02:00

Jef Roosens self-assigned this 2021-06-01 09:40:18 +02:00

Jef Roosens added this to the 1.0.0 project 2021-06-01 09:40:18 +02:00

Sign in to join this conversation.