Jack P
12/04/2023, 12:49 AMPREFECT_EXPERIMENTAL_ENABLED_ENHANCED_CANCELLATION
Prefect setting be used in the ENTRYPOINT
command, prefect worker start
command or does it need to be set somewhere else?
# ---HELM_WORKER stage---
FROM python_base as helm_worker
WORKDIR /app
RUN prefect config set PREFECT_EXPERIMENTAL_ENABLE_ENHANCED_CANCELLATION=True
RUN prefect block register --module prefect_gcp
ENTRYPOINT prefect worker start -p helm-$IMAGE_TAG --type cloud-run-v2
Marvin
12/04/2023, 12:49 AMPREFECT_EXPERIMENTAL_ENABLE_ENHANCED_CANCELLATION
setting is being set in the Dockerfile using the prefect config set
command. This means that it will be available to all Prefect commands run within this Docker container, including the prefect worker start
command in the ENTRYPOINT
.
However, please note that the prefect config set
command writes to the user configuration file, which is located by default at ~/.prefect/config.toml
. If the Docker container does not preserve this file across runs (which is often the case), the setting may not be preserved.
To ensure that the setting is always available, you might consider setting it as an environment variable in the Dockerfile, like so:
Dockerfile
ENV PREFECT__EXPERIMENTAL__ENABLE_ENHANCED_CANCELLATION=True
This will ensure that the setting is always available to Prefect commands run within the Docker container, regardless of whether the user configuration file is preserved.Jack P
12/04/2023, 12:51 AMPREFECT__EXPERIMENTAL__ENABLE_ENHANCED_CANCELLATION
setting is not being used as True? 😬
00:32:05.316 | INFO | prefect.worker.cloud-run-v2.cloudrunworkerv2 f44392e9-50bf-4891-8a41-ba430d0aee75 - Skipping cancellation because flow run '0b6ef0fe-2dd6-460e-8c3b-938e3367737c' is using enhanced cancellation. A dedicated runner will handle cancellation.
Marvin
12/04/2023, 12:51 AMPREFECT__EXPERIMENTAL__ENABLE_ENHANCED_CANCELLATION
setting is being used and is set to True
.
When the PREFECT__EXPERIMENTAL__ENABLE_ENHANCED_CANCELLATION
setting is enabled, Prefect uses a dedicated runner to handle task cancellation. This is why the log message says "A dedicated runner will handle cancellation."
So, in this case, the log message is indicating that the normal cancellation process is being skipped because the enhanced cancellation feature is enabled and a dedicated runner is handling the cancellation instead.Jack P
12/04/2023, 3:10 AMprefect flow-run execute