Carlos Cueto06/08/2022, 4:15 PM
This is the flow definition:
Failed to load and execute Flow's environment: ValueError('No flows found in file.')
if __name__ == '__main__':
with Flow('Scouter-Solr-Script') as flow:
snowflake_user = PrefectSecret('snowflake_usr')
snowflake_pwd = PrefectSecret('snowflake_pwd')
flow.run_config = LocalRun(labels=['SVRNAME1'])
flow.storage = Git(repo="Prefect-Flows", flow_path="Python/Scouter/snowflake_to_solr.py", git_clone_url_secret_name="azure_devops_clone_url")
I'm assuming it has to do with the
part on top of the Flow class definition, but I don't know how to go about fixing this. I need that for multiprocessing that happens within the main task of the flow.
if __name__ == '__main__'
Carlos Cueto06/08/2022, 4:45 PM
but now it leads to an error with the multiprocessing that goes on inside the
task defined inside the flow: _pickle.PicklingError: Can't pickle <function convertToRecords at 0x000001E0C09137F0>: attribute lookup convertToRecords on main failed
. Could you move that inside the task instead?
Carlos Cueto06/08/2022, 5:07 PM
Carlos Cueto06/08/2022, 5:40 PM
decorator, with another sub-function inside of it called
. It was previously outside the task function.
Carlos Cueto06/09/2022, 1:38 PM
Faheem Khan07/14/2022, 8:29 AM