Igor Bondartsov
03/23/2021, 2:45 PMemre
03/23/2021, 3:01 PMHttpGetTask
(or another task for the HTTP request family)? Lately, I find myself GET
ting a lot of results from random endpoints, and thought it could save boilerplate code on my end.Javier Domingo Cansino
03/23/2021, 5:25 PMJonathan Wright
03/23/2021, 5:47 PMGleb Erokhin
03/23/2021, 7:15 PMDavid Elliott
03/23/2021, 8:09 PM400 Client Error:
... "input.states[0].task_run_id"; Expected non-nullable type UUID! not to be null.
on some of the tasks when I run the flow. I'll put the full stack trace in the 🧵. It's happening on maybe 1 in every 20 tasks or so. The task then gets put into state 'ClientFailed' (and the UI can't see them) and all downstream dependents of these tasks then get set to state 'Pending'.
I've tried many dask workers, then just 1 dask worker (for simplicity), same issue. Can't replicate it with the smaller (196 task) flow. I'm wondering if there's some kind of rate limiting going on whereby there are so many concurrent tasks running simultaneously (there are a tonne all trying to be ran at the same time) that some of them are getting a generic error from cloud or something?
I would try adding a task concurrency limit to see if this helps with the above hypothesis, but the UI says it's not included in our plan (even though we're an enterprise tenant). Is it possible to set task concurrency at the flow level?
Also, the UI can't load the schematic of the big flow, though that's less of an immediate concern. Thanks in advance for any advice!Kelly Huang
03/23/2021, 9:11 PMJillian Kozyra
03/23/2021, 11:49 PMfrom prefect import Flow, Parameter, context, task, unmapped
, mypy complains but flows work
if we do from prefect.src import Flow, Parameter, context, task, unmapped
, mypy is happy but python complains: ModuleNotFoundError: No module named 'prefect.src
Reece Hart
03/24/2021, 4:05 AMMichael Wedekindt
03/24/2021, 8:26 AMVarun Joshi
03/24/2021, 11:04 AMJacob Blanco
03/24/2021, 11:17 AMJeffery Newburn
03/24/2021, 2:35 PMAaron Richter
03/24/2021, 2:52 PMIrfan Habib
03/24/2021, 4:16 PMWill Milner
03/24/2021, 5:45 PMfor x in range(3):
task = some_task(x)
final_task = another_task(upstream_tasks=task)
I see 3 tasks get created in the loop, but for the final task it only has 1 upstream task, instead of all the tasks created in the loopCharles Liu
03/24/2021, 6:56 PMCharles Liu
03/24/2021, 7:20 PMAdam Lewis
03/24/2021, 8:01 PMNathan Walker
03/24/2021, 8:07 PMAlex Papanicolaou
03/24/2021, 10:57 PMmatta
03/25/2021, 12:28 AM김응진
03/25/2021, 7:25 AMShin'ichiro Suzuki
03/25/2021, 8:07 AMVarun Joshi
03/25/2021, 8:33 AMFailed to load and execute Flow's environment: AttributeError("'str' object has no attribute 'keys'")
Any inputs will be much appreciated.Dave Hirschfeld
03/25/2021, 8:51 AMciaran
03/25/2021, 11:28 AMciaran
03/25/2021, 12:04 PM0.14.3
are not aligned to the commands I'm able to run...
$ prefect version
0.14.3
$ prefect register -h
Usage: prefect register [OPTIONS] COMMAND [ARGS]...
Register flows
Usage:
$ prefect register [OBJECT]
Arguments:
flow Register flows with a backend API
Examples:
$ prefect register flow --file my_flow.py --name My-Flow
Options:
-h, --help Show this message and exit.
I'd expect to see
Register one or more flows into a project.
Options:
--project TEXT The name of the Prefect project to register this flow in.
Required.
-p, --path TEXT A path to a file or a directory containing the flow(s) to
register. May be passed multiple times to specify
multiple paths.
-m, --module TEXT A python module name containing the flow(s) to register.
May be passed multiple times to specify multiple modules.
...
As defined in the docs at https://docs.prefect.io/api/latest/cli/register.htmlciaran
03/25/2021, 1:48 PMRyan Abernathey
03/25/2021, 1:52 PM