<@ULVA73B9P> Why am I getting the following error ...
# ask-marvin
j
@Marvin Why am I getting the following error after my subflow deployment successfully completes?
Copy code
prefect.exceptions.MissingResult: State data is missing. Typically, this occurs when result persistence is disabled and the state has been retrieved from the API.
My code:
Copy code
import argparse
from prefect import flow, task, get_run_logger
import prefect.deployments
import utils

DATETIME = utils.get_current_timestamp()
FLOW_RUN_NAME = "il-workflow-{plate_name}-" + DATETIME


@task
def run_curation(
    plate_name: str, bfx_run_label: str, bio_exp_id: int, dev: bool = False
):
    flow_run = prefect.deployments.run_deployment(
        name=f'ild-curation/ild-curation-{"dev" if dev else "prod"}',
        parameters={
            "plate_name": plate_name,
            "bfx_run_label": bfx_run_label,
            "bio_exp_id": bio_exp_id,
            "quilt_s3_bucket": f'<s3://compbio-curated>-{"dev" if dev else "prod"}',
        },
    )
    flow_run.state.result()


@flow(flow_run_name=FLOW_RUN_NAME)
def ild_workflow(
    plate_name: str, bfx_run_label: str, bio_exp_id: int, dev: bool = False
):
    logger = get_run_logger()
    logger.info(f'Running {"dev" if dev else "prod"} ILD workflow for {plate_name=}...')

    logger.info("Launching curation...")
    run_curation(plate_name, bfx_run_label, bio_exp_id, dev)
m
Your message was too long by 39 tokens - please shorten it and try again. For reference, here's your message at the allowed limit:
Why am I getting the following error after my subflow deployment successfully completes?
Copy code
prefect.exceptions.MissingResult: State data is missing. Typically, this occurs when result persistence is disabled and the state has been retrieved from the API.
My code: ```import argparse from prefect import flow, task, get_run_logger import prefect.deployments import utils DATETIME = utils.get_current_timestamp() FLOW_RUN_NAME = "il-workflow-{plate_name}-" + DATETIME @task def run_curation( plate_name: str, bfx_run_label: str, bio_exp_id: int, dev: bool = False ): flow_run = prefect.deployments.run_deployment( name=f'ild-curation/ild-curation-{"dev" if dev else "prod"}', parameters={ "plate_name": plate_name, "bfx_run_label": bfx_run_label, "bio_exp_id": bio_exp_id, "quilt_s3_bucket": f's3://compbio-curated-{"dev" if dev else "prod"}', }, ) flow_run.state.result() @flow(flow_run_name=FLOW_RUN_NAME) def ild_workflow( plate_name: str, bfx_run_label: str, bio_exp_id: int, dev: bool = False ): logger = get_run_logger() logger.info(f'Running {"dev" if dev else "prod
k
does the subflow you're running as a deployment have result persistence enabled?
j
I don't think so, not aware of this concept!
k
yep! if you're accessing
flow_run.state.result()
, prefect is going to try to fetch a result file containing the return value of the flow function
👍 1
j
Copy code
❯ prefect config set PREFECT_DEFAULT_RESULT_STORAGE_BLOCK='cyflows-result-storage'
Set 'PREFECT_DEFAULT_RESULT_STORAGE_BLOCK' to 'cyflows-result-storage'.
Updated profile 'default'.
❯ prefect config set PREFECT_RESULTS_PERSIST_BY_DEFAULT=true
Set 'PREFECT_RESULTS_PERSIST_BY_DEFAULT' to 'true'.
Updated profile 'default'.
I just reran and it seems the same thing happened. Is there anything else I need to do to make sure my flow runs pick this up?
k
are those deployments running locally? or in ECS? if in a remote env, you'll need to set those as env vars on the work pool or deployment
prefect config is local only
j
ECS fargate! push pool
Ahh
makes sense
I've tested going back and forth between having the env set on the Fargate push work pool:
Copy code
{
  "PREFECT_RESULTS_PERSIST_BY_DEFAULT": true,
  "PREFECT_DEFAULT_RESULT_STORAGE_BLOCK": "cyflows-result-storage"
}
Sometimes I get this
Copy code
Flow run could not be submitted to infrastructure: An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Log driver awslogs option 'awslogs-stream-prefix' must match pattern [^:*]*
other times I get this regardless of the image:
Copy code
Flow run infrastructure exited with non-zero status code:
 Exited with non 0 code. (Error Code: 1)
This may be caused by attempting to run an image with a misspecified platform or architecture.
When I clear the env setting it goes back to functioning with the exception of not having any results stored Very strange. I'm going to try to set it on the deployment-level instead.
Same thing when specified in the deployment settings:
Copy code
{
  "cpu": 256,
  "env": {
    "PREFECT_RESULTS_PERSIST_BY_DEFAULT": true,
    "PREFECT_DEFAULT_RESULT_STORAGE_BLOCK": "cyflows-result-storage"
  },
  "image": "<http://840419303237.dkr.ecr.us-east-1.amazonaws.com/intervention-library:dev|840419303237.dkr.ecr.us-east-1.amazonaws.com/intervention-library:dev>",
  "memory": 512,
  "ephemeral_storage": 21
}
Sent an email to support
👍 1