Miguel
03/23/2021, 5:25 PM[2021-03-23 17:16:53+0000] ERROR - prefect.my-flow | Unexpected error occured in FlowRunner: ModuleNotFoundError("No module named 'prefect'")
This is what I'm doing:
executor = DaskExecutor("my-dask-cluster-ip")
flow.run(executor=executor)
If I use other executors (LocalExecutor or LocalDaskExecutor) it runs fine.
Any idea on what I can do to address this issue?Charles Leung
03/23/2021, 9:34 PMINTERNAL_SERVER_ERROR: Variable "$input" got invalid value null at
"input.states[0].task_run_id"; Expected non-nullable type UUID! not to be null.
The GraphQL query was:
mutation($input: set_task_run_states_input!) {
set_task_run_states(input: $input) {
states {
message
status
id
}
}
}
The passed variables were:
{"input": {"states": [{"state": {"message": "Trigger was \"all_successful\" but some of the upstream tasks failed.", "_result": {"location": null, "__version__": "0.14.11", "type": "Result"}, "cached_inputs": {}, "context": {"tags": []}, "__version__": "0.14.11", "type": "TriggerFailed"}, "task_run_id": null, "version": null}]}}
what is the issue? I'm not sure how to debug this
Michael Hadorn
03/24/2021, 7:43 AMClemens
03/24/2021, 9:55 AMFinished task run for task with final state: 'Running'
after that the flows state is completely off and it get’s hard to understand what happens. But already that log seems wrong. Any clue about why and when a task might finish with a running state?Michael Hadorn
03/24/2021, 11:10 AMJosh Greenhalgh
03/24/2021, 1:44 PMpostgresql:
postgresqlPassword: ***
nodeSelector:
<http://cloud.google.com/gke-nodepool|cloud.google.com/gke-nodepool>: some-node-pool
Josh Greenhalgh
03/24/2021, 2:37 PMcore-0.14.12
and I am getting this error;
{'type': ['Unsupported value: Module']}
Has module storage been removed?Charles Leung
03/24/2021, 9:54 PMMilton
03/24/2021, 10:30 PMDominique
03/25/2021, 2:51 AMJillian Kozyra
03/25/2021, 7:32 AMModuleNotFoundError: No module named 'data_flows'
when I try to register my flow locally using docker storage (specifically it’s failing the cloudpickle_deserialization_check
healthcheck). we’re using fully qualified imports in our flows (e.g. from data_flows.utils.data_extraction tasks import thing
) to satisfy mypy, so i’m hoping there’s a way to get this working as well. i think the config matches the tutorial, and i’ve tried different values for the python path without any luck. Here’s our Docker setup:
Docker(
registry_url=ecr_registry_url,
python_dependencies=python_dependencies,
files={
"/Users/jilliankozyra/projects/projectname/data_flows/utils/__init__.py": "/data_flows/utils/__init__.py",
"/Users/jilliankozyra/projects/projectname/data_flows/utils/install_dependencies.py": "/data_flows/utils/install_dependencies.py",
"/Users/jilliankozyra/projects/projectname/data_flows/utils/data_extraction_tasks.py": "/data_flows/utils/data_extraction_tasks.py"
},
env_vars={
"PYTHONPATH": "$PYTHONPATH:data_flows/"
},
)
Milton
03/25/2021, 3:24 PMCharles Leung
03/25/2021, 4:37 PMAdam Lewis
03/26/2021, 7:29 PMCharles Leung
03/26/2021, 7:43 PMDana Merrick
03/26/2021, 9:18 PMRunNamespacedJob
task?Michael Hadorn
03/29/2021, 12:57 PMGopinath Jaganmohan
03/30/2021, 2:28 AMNathan Walker
03/30/2021, 1:00 PMDiego Alonso Roque Montoya
03/30/2021, 3:38 PMDiego Alonso Roque Montoya
03/30/2021, 6:26 PMRoss
03/31/2021, 9:16 AMprefect.engine.flow_runner.FlowRunner
or prefect.engine.cloud.flow_runner.CloudFlowRunner
with a prefect server backend? I am wanting to run a flow and get task results so wanting to use the arg return_tasks=[task1,task2]
. But I don't think I can do this with prefect.Client().create_flow_run(version_group_id="redacted")
?Tomás Emilio Silva Ebensperger
03/31/2021, 1:38 PMJonathan Wright
03/31/2021, 5:26 PMprefect server create-tenant -n data-eng
Tomás Emilio Silva Ebensperger
04/02/2021, 12:51 PMSidd
04/02/2021, 5:06 PMeric lumsden
04/05/2021, 12:11 AMpip install prefect[aws]
zsh: no matches found: prefect[aws]
Luca Sorgiacomo
04/05/2021, 7:32 AMasync def
 and await
in defining and within tasks?Luca Sorgiacomo
04/05/2021, 7:33 AMeric lumsden
04/05/2021, 5:30 PMValueError: Failed to infer default networkConfiguration, please explicitly configure using `--run-task-kwargs`