Tim Wright
04/29/2022, 1:43 PMWilliam Durksen
04/29/2022, 2:19 PMConstantino Schillebeeckx
04/29/2022, 2:24 PMFilterTask
with my own filter func, does the task still filter out the default NoResult
None
and exceptions?Florian Guily
04/29/2022, 2:58 PMGreg Wyne
04/29/2022, 5:38 PMek
04/29/2022, 5:39 PMMadison Schott
04/29/2022, 5:42 PMJonathan Seery
04/29/2022, 8:08 PMGeoffrey Keating
04/29/2022, 8:55 PMWalter Cavinaw
04/29/2022, 9:34 PMTim Wright
04/30/2022, 12:24 AM__init__
methodRaviraja Ganta
04/30/2022, 11:10 AMprefect.context.get("logger")
is coming in the UI. How to surface my custom logger method logs also into the UI?Raviraja Ganta
04/30/2022, 8:09 PMmain-project
|
src
|
| __init__.py
| module1.py
| module2.py
|
configs
|
| config.yaml
|
flows
|
| sample_flow.py
module1,2 uses configs from config.yaml file and sample_flow import the code from modules, declares them as tasks and wraps them in a flow. I have some other custom inbuilt dependencies as well. Some of the modules needs to run on GPU. I want to run this on AWS EC2.
• What is the best way to package the code?
• What is the way to run agent that uses GPU on EC2?Zach Schumacher
05/01/2022, 3:26 PMCarlos Paiva
05/01/2022, 5:49 PMCole Murray
05/01/2022, 9:52 PM• the centralized nature of the Airflow scheduler provides a single point of failure for the systemIn a typical master/worker deployment, you have a single-point-of-failure schedule (yes you can run it highly available with locking / other mechanisms) responsible for reading from a schedule DB, to invoke tasks into a queue to be processed by workers. Can someone clarify how Prefect server solves this issue? From the docs, it seems Prefect server is also a SPOF in this architecture. Based on code here: https://github.com/PrefectHQ/server/blob/master/src/prefect_server/services/towel/scheduler.py#L21, we would not be able to run several instances of the server simultaneously, as there is no locking taking place against the DB, and would cause double execution
Eddie Atkinson
05/02/2022, 5:58 AMDaskExecutor
and then use a temporary cluster as described here: https://docs.prefect.io/orchestration/flow_config/executors.html#using-a-local-cluster
Just want to make sure I’m not missing a more obvious solution before I move to a cluster.Efraim Globus
05/02/2022, 8:44 AMDockerRun
, only LocalRun
is supported ”
what does it mean?
with Flow("iris_batch", storage=Local(path="/app/batch/flow.py",stored_as_script=True),
run_config=DockerRun(image="test:latest")
) as batch_prediction_flow:
What i am missing?
Amy help will be appreciated:)
Thank you!Alexander van Eck
05/02/2022, 11:21 AMHoratiu Bota
05/02/2022, 11:35 AM<address>:4200/graphql
in the browser - i've followed the troubleshooting guide, but no successNaga Sravika Bodapati
05/02/2022, 11:36 AMRon Levi
05/02/2022, 11:52 AMBob Colner
05/02/2022, 12:24 PMstate_handlers
functionality in Orion/Prefect2.0. Is is on the roadmap? (or I’m I missing something..) thanksMatthew Seligson
05/02/2022, 12:41 PMMars
05/02/2022, 3:16 PMPREFECT__CONTEXT__SECRETS__GITHUB_ACCESS_TOKEN
variable in the k8s manifest created by prefect agent kubernetes install
? Do I have to use the --env
switch or can I provide the value by editing the generated deployment container’s env:
list directly?Daniel Tavares
05/02/2022, 4:28 PMSlackbot
05/02/2022, 4:42 PMChris Reuter
05/02/2022, 4:54 PMLeon Kozlowski
05/02/2022, 5:53 PMPoosh Adhikary
05/02/2022, 7:08 PMnum_workers
for LocalDaskExecutor based on the number of cores available on the container?Poosh Adhikary
05/02/2022, 7:08 PMnum_workers
for LocalDaskExecutor based on the number of cores available on the container?Kevin Kho
05/02/2022, 7:09 PMDockerRun
?Poosh Adhikary
05/02/2022, 7:12 PMDockerRun
.Kevin Kho
05/02/2022, 7:12 PMLocalDaskExecutor()
should infer the number of coresPoosh Adhikary
05/02/2022, 7:14 PMnum_of_cores/2
.Kevin Kho
05/02/2022, 7:15 PMLocalDaskExecutor
, but I think you can with DaskExecutor
using LocalCluster
like thisPoosh Adhikary
05/02/2022, 7:19 PMLocalDaskExecutor
as well. Thank you Kevin!