Emma Rizzi
08/26/2021, 8:52 AMHenning Holgersen
08/28/2021, 1:34 PMdavzucky
09/17/2021, 12:56 AMRichard Pelgrim
09/24/2021, 2:14 PMAnna Geller (old account)
09/29/2021, 7:10 PMAldo Escobar
11/04/2021, 8:55 PMKhuyen Tran
11/09/2021, 5:30 PMpsimakis
11/13/2021, 6:02 PMAnna Geller
11/18/2021, 4:01 PMChris Arderne
11/23/2021, 11:15 AMJarek Lukow
11/26/2021, 2:01 PMhttps://www.youtube.com/watch?v=apCDT3_DmFk&t=438sā¾
Constantino Schillebeeckx
12/06/2021, 7:26 PMMathijs Miermans
12/13/2021, 10:57 PMexecution_name
is not unique. We're trying to set execution_name
to a uuid4 at run time, but that is probably happing at registration time. How can task arguments be computed dynamically at run time?Ari Bajo
12/14/2021, 10:26 PMAlyssa Mazzina
12/21/2021, 7:24 PMAram Panasenco
12/29/2021, 9:24 PMpython register.py -p my_project -m module.that.contains.flow.variable
Source of register.py:
from types import ModuleType
from typing import Union
import click
import pdoc
from prefect import Client
from prefect.utilities.graphql import with_args
def docfilter(docobject):
print(docobject)
@click.command()
@click.option(
"--project",
"-p",
help="The name of the Prefect project to register this flow in. Required.",
required=True,
)
@click.option(
"--module",
"-m",
help="A python module name containing the flow to register. Required.",
required=True,
)
def register_flow(project: str, module: Union[ModuleType, str], **kwargs):
pdoc3_module = pdoc.Module(module=module)
flow = pdoc3_module.obj.flow
flow_id = flow.register(project_name=project, **kwargs)
client = Client()
flow_group_id = client.graphql(
{
"query": {
with_args(
"flow",
{
"where": {
"id": {
"_eq": flow_id,
},
},
}
): {
"flow_group_id"
}
}
}
)["data"]["flow"][0]["flow_group_id"]
docfilter = lambda doc: doc.name in [t.name for t in flow.tasks]
pdoc3_module = pdoc.Module(module=module, docfilter=docfilter)
module_markdown = pdoc3_module.text()
# Do some post-processing to keep Prefect engine from misinterpreting
# the markdown.
module_markdown = module_markdown.replace(": ",":\n").replace("\n:",":").\
replace("\n ","\n")
ret = client.graphql(
{
"mutation": {
with_args(
"set_flow_group_description",
{
"input": {
"flow_group_id": flow_group_id,
"description": module_markdown,
},
}
): {
"success"
}
}
}
)
if not ret["data"]["set_flow_group_description"]["success"]:
raise RuntimeError("Failed to set flow group README")
if __name__ == "__main__":
register_flow()
Christoph Deil
01/04/2022, 9:15 PMflow.run()
with the default serial executor happens or where the algorithm is that linearises the task graph to decide who runs after who. And also how schedulers work, i.e. if they have at the very core some polling for loop in-process or if the scheduler runs in some other thread or process. For now Iām not planning to look into Dask, just trying to understand how the serial execution works under the hood.
Does this information exist in some tutorial form? Or alternatively could you please point me to the few relevant parts in the code or tests to quickly understand how that works?Alyssa Mazzina
01/04/2022, 11:07 PMScott Treloar
01/05/2022, 10:25 AMJoshua S
01/10/2022, 3:14 PMAlyssa Mazzina
01/18/2022, 11:29 PMAlyssa Mazzina
01/18/2022, 11:30 PMZach Schumacher
01/19/2022, 8:02 PMKirk Quinbar
01/24/2022, 5:38 PMBen Welsh
01/29/2022, 9:38 PMBen Welsh
01/29/2022, 9:38 PMKevin Kho
02/10/2022, 8:48 PMKhuyen Tran
02/19/2022, 3:14 PMEvgeniya Sukhodolskaya
03/04/2022, 3:46 PMEvgeniya Sukhodolskaya
03/04/2022, 3:46 PMLaunching human-in-the-loop process on Toloka using Prefectā¾
Anna Geller
03/04/2022, 4:17 PM