Guoying Qi
06/23/2022, 9:38 PMWhen will Prefect 2.0 be released?
The Prefect 2.0 beta period is expected to last for at least the second quarter of 2022.
Tilé
06/23/2022, 10:26 PMPrefectSecret
class.
The problem is that there is a secret I have to pass on every task. It works properly, but when I try to visualize my flow I get this secret shown as input to every task, which basically ruins the whole diagram and makes it hard to read. So
1. Is there any other way to pass my secret to the tasks when running locally or using the agent from the cloud, or
2. Is there any way to remove the secret box from the visualized flow?
ThanksJosh
06/23/2022, 11:21 PMJeff Kehler
06/24/2022, 4:57 AMDarren Fleetwood
06/24/2022, 6:07 AMPREFECT__LOGGING__EXTRA_LOGGERS= "['ray']"
As well as this within the flow:
prefect.config.logging.extra_loggers = ['ray']
Plus a few other things, none of which have worked. This is the case for all logging levels (error, info, etc.)
What’s the proper way of doing this?
Thanks!Balveer Singh
06/24/2022, 6:47 AMZheng Cheng
06/24/2022, 9:16 AMGuoying Qi
06/24/2022, 11:06 AM$ prefect -v
2.0b7
$ prefect orion kubernetes-manifest
Error: No such command 'kubernetes-manifest'.
$ prefect orion --help
Commands:
database Commands for interacting with the database.
start Start an Orion server
The documents needs to be updated?
https://orion-docs.prefect.io/tutorials/kubernetes-flow-runner/Joshua Greenhalgh
06/24/2022, 11:29 AMMaverick Humbert
06/24/2022, 11:55 AMBalveer Singh
06/24/2022, 12:36 PMJelle Vegter
06/24/2022, 1:15 PMSander
06/24/2022, 1:38 PMBenjamin Bonhomme
06/24/2022, 2:19 PMredsquare
06/24/2022, 5:28 PMredsquare
06/24/2022, 5:29 PMredsquare
06/24/2022, 5:32 PMkomal azram
06/25/2022, 8:30 AMimport prefect
from prefect import task, Flow
@task
def claims_func():
airbyte_server_host = "localhost",
airbyte_server_port = 8000,
airbyte_api_version = "v1",
connection_id = conn_id,
with Flow("fhir-flow") as flow:
claims_func()
flow.run()
As per my understanding when when I run this flow it should trigger the connection and automatically sync data in from gcp->snowflake. I don't get any error but no data is synced.Guoying Qi
06/25/2022, 3:58 PMprefect config set PREFECT_API_URL="<http://prefect.example.com/api>"
the flow run states being stored in the database successfully, you can query them in the database table.
But nothing shown in the UI, because all the API calls from the UI page still go to local server: http://127.0.0.1:4200/api/Zheng Cheng
06/25/2022, 4:01 PMZheng Cheng
06/25/2022, 4:30 PMprefect orion start
, mainly that this web UI doesn’t require login or other form of authentication. I can’t just deploy and allow anyone on the internet can visit that.yu zeng
06/26/2022, 12:21 PMJean-Paul Berthelot
06/26/2022, 1:04 PMRayTaskRunner
versus DaskTaskRunner
,
I have a requirement to develop a Prefect Workflow for a high volume ingestion workflow. The architecture I am looking at is based on Data Mesh.
I have a local environment and worked through several POCs similar to the workflow described here.
Could I please be directed to relevent reading materials or examples?Dekel R
06/26/2022, 2:07 PMJames Phoenix
06/26/2022, 4:09 PMJames Phoenix
06/26/2022, 4:09 PMJames Phoenix
06/26/2022, 4:10 PMJames Phoenix
06/26/2022, 4:10 PMJames Phoenix
06/26/2022, 4:37 PMibrahem
06/26/2022, 6:37 PM