Skip to content

Docker container process remain to run after the timeout of thrust-pcsat-wrapper #49

@coeff-aij

Description

@coeff-aij

Description

When a timeout occurs during the invocation of thrust-pcsat-wrapper(e.g., while verifying complex iterator examples ), Thrust kills the wrapper process. However, the Docker container launched by the wrapper is not stopped and continues running in the background.

Attempted Fix

I attempted to modify the wrapper script to trap signals and forward them to the Docker container to ensure a clean shutdown. Here is the script I tried:

#!/bin/bash

COAR_IMAGE=${COAR_IMAGE:-ghcr.io/hiroshi-unno/coar:main}
CONTAINER_NAME="thrust-solver"

cleanup() {
    rc=$?
    echo "signal received."
    if docker ps -q -f name="^${CONTAINER_NAME}$" >/dev/null; then
        echo "Forwarding ${RECEIVED_SIG:-TERM} to container $CONTAINER_NAME"
        sig=${RECEIVED_SIG:-TERM}
        docker stop -s "$sig" "$CONTAINER_NAME" >/dev/null
    fi
    exit $rc
}
trap cleanup EXIT INT TERM HUP

smt2=$(mktemp -p . --suffix .smt2)
trap "rm -f $smt2" EXIT
cp "$1" "$smt2"
out=$(
docker run --rm -v "$PWD:/mnt" -w /root/coar --name "$CONTAINER_NAME" "$COAR_IMAGE" \
    main.exe -c ./config/solver/pcsat_tbq_ar.json -p pcsp "/mnt/$smt2"
)
exit_code=$?
echo "${out%,*}"
exit "$exit_code"

The script above did not work. It seems that when Thrust times out, it issues a SIGKILL immediately to the child process. Since SIGKILL cannot be trapped or caught, the cleanup function in the wrapper script is never executed, leaving the Docker container running.

Temporary Workaround

As a temporary measure, I have injected the timeout command directly into the execution line within the wrapper to ensure the internal process dies even if the wrapper is killed abruptly:

# Added 'timeout 30' inside the docker run command
out=$(docker run ... "$COAR_IMAGE" timeout 30 main.exe -c ...)

Since the process_control crate doesn't seem to support sending signals other than SIGKILL, we might need to add a new dependency like nix or libc to send SIGTERM.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions