Dylan
06/01/2022, 9:33 PMDylan
06/01/2022, 9:34 PMDylan
06/01/2022, 9:34 PMwith Flow(
name=FLOW_NAME,
storage=Local(
add_default_labels=False,
stored_as_script=True,
path="flows/order_list_ingest.py",
),
run_config=KubernetesRun(
image=f"{config.ECR_REGISTRY}/{config.PROJECT}:{config.IMAGE_TAG}",
labels=[config.ENVIRONMENT],
job_template_path="templates/base-flow-template.yaml",
service_account_name=config.S3_BUCKET,
),
executor=LocalExecutor(),
) as flow:
client = OrderApi()
raw_data = get_order_list(client, **seed_params())
save(raw_data, PREFIX)
Dylan
06/01/2022, 9:34 PMDylan
06/01/2022, 9:35 PMDylan
06/01/2022, 9:35 PMprefect register --project parcel-planet-ingest -p flows/order_list_ingest.py
Dylan
06/01/2022, 9:36 PMDylan
06/01/2022, 9:42 PMAnna Geller
Dylan
06/01/2022, 9:42 PMDylan
06/01/2022, 9:43 PMDylan
06/01/2022, 9:43 PMDylan
06/01/2022, 9:46 PMAnna Geller
Anna Geller
Kevin Kho
Dylan
06/01/2022, 10:32 PMDylan
06/01/2022, 10:33 PMKevin Kho
Dylan
06/01/2022, 10:37 PMget_order_list
in the flow when I meant to call api_get_order_list
, there isn’t a get_order_list
. So I wonder if the __pycache__
came into play at all. Could obvi be a rabbit hole for no good reason but it was definitely strange to see code being executed, whose function name didn’t quite exist.Dylan
06/01/2022, 10:37 PMDylan
06/01/2022, 10:37 PMKevin Kho
flow.run()
or Cloud flow? Could happen if there was no re-registration?Dylan
06/01/2022, 10:47 PM.run()
on anything in the flow itself.Dylan
06/01/2022, 10:47 PMDylan
06/01/2022, 10:47 PMKevin Kho
Anna Geller