Suchindra
02/16/2021, 11:24 PMlogbook
instead of logging
from stdlib. How can I configure it to capture logs from within prefect tasks?itay livni
02/16/2021, 11:30 PMStartFlowRun
in a task with a loop signal? My concern is that with StartFlowRun
the enviroment (including modules) will be "re-initialized" on every loop. But maybe that will happen with recursion also? -- In short is the effort of trying to build a recursive Flow worth the effort from a speed perspective? (Currently running into different issues). Thanksmatta
02/17/2021, 4:11 AMSven Teresniak
02/17/2021, 9:40 AM--skip-if-exists
because this is the default behaviour. but when you're creating a tenant a 2nd time (same name and slug) you will get an exception and return value 1 from the prefect cli. it seems create-tenant
needs a --skip-if-exists
and create project
notHarshal Rane
02/17/2021, 9:51 AMHarshal Rane
02/17/2021, 9:51 AMFailed to load and execute Flow's environment: ModuleNotFoundError("No module named '/root/'")
Attached are logs from Docker agent
Has anyone faced this issue before?Michael Hadorn
02/17/2021, 10:04 AMJonathan Wright
02/17/2021, 10:51 AMMilly gupta
02/17/2021, 1:34 PMJosselin Girault
02/17/2021, 2:37 PMMatthew Blau
02/17/2021, 2:40 PMPuja Thacker
02/17/2021, 3:29 PMNikul
02/17/2021, 4:35 PMAjith Kumara Beragala Acharige Lal
02/17/2021, 5:57 PMHelm
, the Prefect server also deployed to the kubernetes via Helm, now need to try-out a flow on that Prefect-server, does anyone has example/page?Tobias Heintz
02/17/2021, 8:26 PMFina Silva-Santisteban
02/18/2021, 1:27 AMprefect ecs agent
in a terminal, and then use the prefect dashboard to trigger that run. I’d like to build a standalone app, so have a custom GUI that lets a user input flow parameters and trigger a flow run. I’m imagining that I’ll have to create a prefect ecs agent using the prefect api instead of running it in a separate terminal but I’m not sure how to go about that. Pls let me know of any deployment recipes that fit my use case! 🙏Puja Thacker
02/18/2021, 7:45 AMPuja Thacker
02/18/2021, 7:46 AMJulian
02/18/2021, 10:31 AMcrazy_monkey
, but they don't provide any actual value. Since it is quite nice (from ui perspective) to have meaningful names, we rename these flowruns during runtime like {flow_name}_{system_time}
or {flow_name}_{parameters}
. However, if a flowrun fails before executing the rename task, it won't have this "nice" name but the orignal (meaning-less) name. Also, renaming inside the flow will also overwrite flowrun names which were provided when manually starting a flowrun with name, e.g. from ui.Milly gupta
02/18/2021, 11:47 AMSiva Kumar
02/18/2021, 2:57 PMJoël Luijmes
02/18/2021, 2:59 PMtasks = dynamic_list_of_tasks()
windowed_tasks = fixed_window(tasks, window_size=5)
def process_item(item):
x = task_1()
task_2(x)
def process_window(tasks):
apply_map(process_item, tasks)
apply_map(process_window, windowed_tasks)
Jean-Marc LE FEVRE
02/18/2021, 3:04 PMJean-Marc LE FEVRE
02/18/2021, 3:05 PMKarolína Bzdušek
02/18/2021, 5:43 PMflow.storage = GitHub(repo="cividi/gemeinde_workflows",path="/workflows/residential_density.py",access_token_secret="GITHUB_ACCESS_TOKEN")
"GITHUB_ACCESS_TOKEN"
is stored in secret in Cloud.
However, when I run flow through UI, I am still getting this and not sure why.
Note: repository is private and own by our team, I am using token generated under my account.
What I am missing? Thanks for your help in advance.Craig Wright
02/18/2021, 8:08 PMFivetranSyncTask
. If anyone is even vaguely interested in more deterministic Fivetran scheduling, please reach out! https://prefect-community.slack.com/archives/CL09KTZPX/p1613678832100000Sean Talia
02/18/2021, 9:22 PMBatchSubmit
task before? I'm just starting to play around with it and am curious as to exactly what actions i'm going to need to include in my iam policy allow list for the host on which the task will execute (besides obviously the batch:SubmitJob
action)matta
02/18/2021, 10:43 PMAjith Kumara Beragala Acharige Lal
02/19/2021, 1:31 AMprefect auth create-token -n my-runner-token -s RUNNER
i get this errormatta
02/19/2021, 4:31 AMDaskKubernetesEnvironment
for flows in there, and it launches even when I don't specify an image (which is awesome!). Just out of curiosity, what exactly am I launching? Is it the same image as the container of the JupyterHub instance?