James Gatter
09/10/2024, 9:07 PMprefect.exceptions.MissingResult: State data is missing. Typically, this occurs when result persistence is disabled and the state has been retrieved from the API.
My code:
import argparse
from prefect import flow, task, get_run_logger
import prefect.deployments
import utils
DATETIME = utils.get_current_timestamp()
FLOW_RUN_NAME = "il-workflow-{plate_name}-" + DATETIME
@task
def run_curation(
plate_name: str, bfx_run_label: str, bio_exp_id: int, dev: bool = False
):
flow_run = prefect.deployments.run_deployment(
name=f'ild-curation/ild-curation-{"dev" if dev else "prod"}',
parameters={
"plate_name": plate_name,
"bfx_run_label": bfx_run_label,
"bio_exp_id": bio_exp_id,
"quilt_s3_bucket": f'<s3://compbio-curated>-{"dev" if dev else "prod"}',
},
)
flow_run.state.result()
@flow(flow_run_name=FLOW_RUN_NAME)
def ild_workflow(
plate_name: str, bfx_run_label: str, bio_exp_id: int, dev: bool = False
):
logger = get_run_logger()
logger.info(f'Running {"dev" if dev else "prod"} ILD workflow for {plate_name=}...')
logger.info("Launching curation...")
run_curation(plate_name, bfx_run_label, bio_exp_id, dev)
Marvin
09/10/2024, 9:07 PMWhy am I getting the following error after my subflow deployment successfully completes?
prefect.exceptions.MissingResult: State data is missing. Typically, this occurs when result persistence is disabled and the state has been retrieved from the API.
My code:
```import argparse
from prefect import flow, task, get_run_logger
import prefect.deployments
import utils
DATETIME = utils.get_current_timestamp()
FLOW_RUN_NAME = "il-workflow-{plate_name}-" + DATETIME
@task
def run_curation(
plate_name: str, bfx_run_label: str, bio_exp_id: int, dev: bool = False
):
flow_run = prefect.deployments.run_deployment(
name=f'ild-curation/ild-curation-{"dev" if dev else "prod"}',
parameters={
"plate_name": plate_name,
"bfx_run_label": bfx_run_label,
"bio_exp_id": bio_exp_id,
"quilt_s3_bucket": f's3://compbio-curated-{"dev" if dev else "prod"}',
},
)
flow_run.state.result()
@flow(flow_run_name=FLOW_RUN_NAME)
def ild_workflow(
plate_name: str, bfx_run_label: str, bio_exp_id: int, dev: bool = False
):
logger = get_run_logger()
logger.info(f'Running {"dev" if dev else "prodKevin Grismore
09/10/2024, 9:08 PMJames Gatter
09/10/2024, 9:09 PMJames Gatter
09/10/2024, 9:10 PMKevin Grismore
09/10/2024, 9:10 PMflow_run.state.result()
, prefect is going to try to fetch a result file containing the return value of the flow functionJames Gatter
09/10/2024, 9:28 PM❯ prefect config set PREFECT_DEFAULT_RESULT_STORAGE_BLOCK='cyflows-result-storage'
Set 'PREFECT_DEFAULT_RESULT_STORAGE_BLOCK' to 'cyflows-result-storage'.
Updated profile 'default'.
❯ prefect config set PREFECT_RESULTS_PERSIST_BY_DEFAULT=true
Set 'PREFECT_RESULTS_PERSIST_BY_DEFAULT' to 'true'.
Updated profile 'default'.
I just reran and it seems the same thing happened. Is there anything else I need to do to make sure my flow runs pick this up?Kevin Grismore
09/10/2024, 9:29 PMKevin Grismore
09/10/2024, 9:29 PMJames Gatter
09/10/2024, 9:29 PMJames Gatter
09/10/2024, 9:29 PMJames Gatter
09/10/2024, 9:29 PMJames Gatter
09/11/2024, 1:37 PM{
"PREFECT_RESULTS_PERSIST_BY_DEFAULT": true,
"PREFECT_DEFAULT_RESULT_STORAGE_BLOCK": "cyflows-result-storage"
}
Sometimes I get this
Flow run could not be submitted to infrastructure: An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Log driver awslogs option 'awslogs-stream-prefix' must match pattern [^:*]*
other times I get this regardless of the image:
Flow run infrastructure exited with non-zero status code:
Exited with non 0 code. (Error Code: 1)
This may be caused by attempting to run an image with a misspecified platform or architecture.
When I clear the env setting it goes back to functioning with the exception of not having any results stored
Very strange. I'm going to try to set it on the deployment-level instead.James Gatter
09/11/2024, 2:05 PM{
"cpu": 256,
"env": {
"PREFECT_RESULTS_PERSIST_BY_DEFAULT": true,
"PREFECT_DEFAULT_RESULT_STORAGE_BLOCK": "cyflows-result-storage"
},
"image": "<http://840419303237.dkr.ecr.us-east-1.amazonaws.com/intervention-library:dev|840419303237.dkr.ecr.us-east-1.amazonaws.com/intervention-library:dev>",
"memory": 512,
"ephemeral_storage": 21
}
James Gatter
09/11/2024, 2:17 PM