Hello :slightly_smiling_face: I have a problem wit...
# prefect-community
r
Hello 🙂 I have a problem with a newly created flow. It is a relative straightforward one and pretty easy but it only works locally (flow.run()) and not on the cloud 😞 The logs don't show any information, just reference task fail. The task is using a custom class for processing and loading the data like this:
Copy code
@task(log_stdout=True)
def etl(
    connector,
    exec_file,
    environment: Optional[str] = None
):
    master_data = MasterDataSet(connector, exec_file)
    master_data.get_dataset(environment)
    master_data.load_dataset()
My flow looks like this:
Copy code
.... as flow:
    source = SOME_SOURCE
    destination = SOME_DESTINATION

    file_list = ['SOME/PATH/TO/FILE', 'ANOTHER/PATH/TO/FILE']

    master_dataset = etl.map(
        connector=unmapped(source),
        exec_file=file_list
    )
The cloud logs look like this:
Copy code
Task 'etl_master_data': Starting task run...
Task 'etl_master_data': Finished task run for task with final state: 'Mapped'
Flow run FAILED: some reference tasks failed.
There is no failed reference task. Does anybody has any idea why this is not working on the cloud but locally? It is configured like all the other flows, even the data handling by class is already implemented for other flows. I really have absolutely no clue why 😄
👀 1
1
it seems adding a return like this, is doing the trick:
return master_data.load_dataset()
I still don't understand completely. My usual separate load tasks never have a return
b
Hey Rainer, yesterday we had an issue with Cloud that caused delays updating task run states. The behavior you're outlining here seems similar to what other users reported. This issue has since been resolved. When was this flow run? Is it able to run successfully now?
r
Thanks for getting back to me 🙂 The first time I have tried was around Wednesday around 3:46 PM CEST. Now it is working since yesterday 1:25 PM CEST after I added the return to my task. Was the issue you're mentioning only affecting new flows? All of the existing ones worked like a charm.
b
Hey Rainer! Glad everything is working now. To my knowledge it was affecting all flows that utilized mapped tasks.