Oscar Björhn
11/01/2022, 3:04 PMRob Freedy
11/01/2022, 3:10 PMCharleigh Liang
11/01/2022, 3:10 PMStephen Herron
11/01/2022, 3:11 PMredsquare
11/01/2022, 3:11 PMClaire Herdeman
11/01/2022, 3:28 PMNate
11/01/2022, 3:30 PMJohn Mizerany
11/01/2022, 3:33 PMCharleigh Liang
11/01/2022, 5:41 PMuser
11/01/2022, 5:42 PMKalise Richmond
11/03/2022, 9:22 PMBianca Hoch
11/04/2022, 9:14 PMRicardo Gaspar
11/08/2022, 4:35 PMopenlineage
integration?
For ref:
• https://openlineage.io/
• https://github.com/OpenLineage/OpenLineage/issues/81
• there was one for Prefect v1 https://github.com/OpenLineage/OpenLineage/pull/293
CC @limx0 @Anna GellerTibs
11/16/2022, 10:43 AMTibs
11/17/2022, 1:31 PMTibs
11/22/2022, 4:50 PMJames Tisato
11/27/2022, 11:28 PMprefect
package is available on conda-forge
but the integration packages are not. I raised https://github.com/conda-forge/prefect-feedstock/issues/179 about this but I wasn't sure if that was the right place for it so it would get noticed. Is there any guidance on whether you can make the Prefect-maintained integration packages available via conda as well as pip?Tibs
11/28/2022, 4:19 PMKhuyen Tran
11/29/2022, 4:19 PMChris Reuter
11/29/2022, 11:21 PMMichael Kahan
11/30/2022, 10:04 PMprefect-airbyte
to trigger syncs on Airbyte Cloud? Or is it only for open source hosted instances?James Tisato
12/03/2022, 2:02 AMTibs
12/04/2022, 9:43 AMGio
12/20/2022, 4:52 PMprefect_sqlalchemy
? I have not been able to successfully create a transaction using it.Stephen Herron
01/04/2023, 2:09 PMTibs
01/05/2023, 1:16 PMTibs
01/10/2023, 7:17 AMWarning! Failed to load collection 'prefect_aws_ecs': ImportError: cannot import name 'AwsCredentials' from partially initialized module 'prefect_aws' (most likely due to a circular import)
My import statement:
from prefect_aws.credentials import AwsCredentials
from prefect_aws.s3 import S3Bucket
from prefect_aws.ecs import ECSTask
prefect-aws version 0.1.8Dylan Lim
01/17/2023, 10:10 AMprefect-sqlalchemy
and have set up a SqlAlchemyConnector
block. However, I notice that there is an ‘Optional’ fetch size that defaults to 1 and impacts my queries eg. fetch_many
. Is there a way to prevent this fetch size from being enforced, especially if I don’t know how many rows I am retrieving beforehand?Tibs
02/01/2023, 2:17 PMMeng Si
02/14/2023, 8:58 PMMeng Si
02/14/2023, 8:58 PMRyan Peden
02/14/2023, 9:30 PMprefect-great-expectations
to run Great Expectations validations: https://prefecthq.github.io/prefect-great-expectations/
See here for a look at the parameters you can pass to the run_checkpoint_validation Task.
I don't think we have a longer example available right now, but keep in mind Prefect 2 tasks are often simpler to work with than Prefect 1 tasks. With Prefect 1, you needed to first create an instance of the RunGreatExpectationsValidation
task before calling it in a flow.
With Prefect 2, you just install the prefect-great-expectations
collection, import the run_checkpoint_validation
, and run it like you'd run any other Python function inside your flow.Meng Si
02/14/2023, 9:51 PM