Stop using shell redirection for container-based specs #12

New Issue

Jef Roosens · 2021-04-27T10:58:06+02:00

Jef Roosens commented

2021-04-27 10:58:06 +02:00

The current workings of the container-based spec uses docker exec which then expects a command to output data to stdout. I have however experienced issues with creating binary files using this method (e.g. a pg_dump archive).

There is, however, a better alternative:

We can use subprocess's built-in methods for handling stdout streaming.

The current workings of the container-based spec uses `docker exec` which then expects a command to output data to stdout. I have however experienced issues with creating binary files using this method (e.g. a `pg_dump` archive). There is, however, a better alternative: We can use subprocess's built-in methods for handling stdout streaming.

Jef Roosens added this to the 1.0.0 milestone 2021-04-27 10:58:06 +02:00

Jef Roosens added the

bug

enhancement

labels 2021-04-27 10:58:06 +02:00

Jef Roosens self-assigned this 2021-04-27 10:58:06 +02:00

Jef Roosens added this to the 1.0.0 project 2021-04-27 10:58:07 +02:00

Jef Roosens commented

2021-06-01 11:49:07 +02:00

Poster

A better method would be to use subprocess's built-in methods for handling stdout. We could stream the command's output as follows:

with subprocess.Popen(
    "sleep 2 && echo yeet && sleep 2 && echo yeetus",
    shell=True,
    stdout=subprocess.PIPE
) as cmd, open("test.txt", "ab") as f:
    for line in cmd.stdout:
        f.write(line)

I haven't tested however wether our not this also works for random binary data.

A better method would be to use subprocess's built-in methods for handling stdout. We could stream the command's output as follows: ```python with subprocess.Popen( "sleep 2 && echo yeet && sleep 2 && echo yeetus", shell=True, stdout=subprocess.PIPE ) as cmd, open("test.txt", "ab") as f: for line in cmd.stdout: f.write(line) ``` I haven't tested however wether our not this also works for random binary data.

Jef Roosens changed title from ~~Stop using stdout for container-based specs~~ to Stop using shell redirection for container-based specs

2021-06-01 11:49:16 +02:00

Sign in to join this conversation.