Hey everyone,
Does anyone know why I get this err...
# prefect-serverf
Fabrice Toussaint
05/06/2021, 7:32 AMHey everyone,
Does anyone know why I get this errors (sometimes) with scheduled tasks?
Copy code
Failed to retrieve task state with error: ClientError([{'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 5}], 'path': ['get_or_create_task_run_info'], 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: '}}}])
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/prefect/engine/cloud/task_runner.py", line 154, in initialize_run
task_run_info = self.client.get_task_run_info(
File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 1399, in get_task_run_info
result = self.graphql(mutation) # type: Any
File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 319, in graphql
raise ClientError(result["errors"])
prefect.utilities.exceptions.ClientError: [{'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 5}], 'path': ['get_or_create_task_run_info'], 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: '}}}]k
Kevin Kho
05/06/2021, 1:34 PM
Hey @Fabrice Toussaint, is your flow very large?
f
Fabrice Toussaint
05/06/2021, 3:20 PMEh no, it seems as if this happens when the cluster scales down (during the night) but I am not 100% sure
k
Kevin Kho
05/06/2021, 3:52 PM
The cluster that holds Prefect server scales down?
Kevin Kho
05/06/2021, 3:53 PM
I’ll ask the team for you. You’re aware Prefect Cloud has 10k task runs for free each month right?
Kevin Kho
05/06/2021, 5:41 PM
This is a sign that flow registration failed
f
Fabrice Toussaint
05/06/2021, 5:48 PM@Kevin Kho not the cluster that holds the prefect server but the one that holds the jobs
k
Kevin Kho
05/06/2021, 5:49 PM
Oh I see. Yes this is a sign that flow registration failed. Either something might be off with the Python code or it didn’t register sucessfully.
f
Fabrice Toussaint
05/06/2021, 5:55 PMThe flow is registered beforehand but on a schedule, so maybe there is something off then?
Fabrice Toussaint
05/06/2021, 5:56 PMI will check tomorrow if it is still occuring
k
Kevin Kho
05/06/2021, 5:56 PM
Does it work sometimes but fail sometimes? I guess it’s night time where you are? Have a good night 🙂
Kevin Kho
05/06/2021, 5:56 PM
Will respond when you get back
f
Fabrice Toussaint
05/06/2021, 5:56 PMThank you! 😀
Fabrice Toussaint
05/07/2021, 7:48 AM@Kevin Kho all three flows which I had scheduled in the night failed
Fabrice Toussaint
05/07/2021, 7:48 AM Copy code
Failed to retrieve task state with error: ClientError([{'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 5}], 'path': ['get_or_create_task_run_info'], 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: '}}}])
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/prefect/engine/cloud/task_runner.py", line 154, in initialize_run
task_run_info = self.client.get_task_run_info(
File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 1399, in get_task_run_info
result = self.graphql(mutation) # type: Any
File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 319, in graphql
raise ClientError(result["errors"])
prefect.utilities.exceptions.ClientError: [{'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 5}], 'path': ['get_or_create_task_run_info'], 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: '}}}]Fabrice Toussaint
05/07/2021, 7:49 AMOther flows just keep running fine
Fabrice Toussaint
05/07/2021, 7:55 AM Copy code
0 0 */1 * *Fabrice Toussaint
05/07/2021, 9:01 AMNow it is on every flow with a schedule, I have one without where it does not happen
k
Kevin Kho
05/07/2021, 1:44 PM
Could you show how you attach the schedule to the flow?
f
Fabrice Toussaint
05/12/2021, 12:34 PM@Kevin Kho it seems to only happen to flows with a daily CRON schedule:
Copy code
clocks.CronClock('0 0 */1 * *')k
Kevin Kho
05/12/2021, 4:16 PM
Do you use
flow.schedule = ….2 Views