Anyone willing to build an integration with Hex?? They’ve just announced v1 of their brand new public APIs. I’m definitely available to help!

Hi! I’m really curious if anyone in this group would be affected by the breaking change presented in https://github.com/PrefectHQ/prefect/pull/6926 tldr; are you calling

State.result()

in an async context? (does not apply to

Future.result()

)

Hey folks 👋 Cross posting from the Soda community --> https://soda-community.slack.com/archives/C02PC0WPRNC/p1664464535579609 Spoiler: Soda released a new Prefect collection that you can use to run data quality checks on your data! Thanks to Soda to provide me the opportunity to collaborate with them on this new collection! 🚀 🚀

Just published a new collection that allows monitoring the prefect server using opentelemetry. This is an initial version that publishes trace only at the moment on the server. https://pypi.org/project/prefect-opentelemetry/ github project https://github.com/davzucky/prefect-opentelemetry The next steps are • Add metrics • Enable Agent to support OTEL • Create a block for OTEL • Create a custom task that would generate traces on the run (Would be nice to be able o add plugins) on flow or task to do that. Please share your feedback

👋 You may have noticed I’m working on result configuration. There’s a bit more wiring up to do but the documentation is in a near-final form. Check out the preview at https://deploy-preview-6908--prefect-orion.netlify.app/concepts/results/ — feel free to share any feedback!

Happy Hacktoberfest! 🍻 Cross-posting that OpenMetadata and Airbyte are celebrating by giving 💰 and swag for contributing connectors. Specifically for OpenMetadata, if you are interested in helping contribute, QA or build examples/tutorials for the integration, feel free to ping me. Thanks to all contributors! 💙

Hi there, I have created a redis collection and am looking for the steps needed to become a part of the official prefect packages: https://github.com/C4IROcean/prefect-redis @alex - @Anna Geller mentioned you could help with this? https://prefect-community.slack.com/archives/CL09KU1K7/p1665169933380519?thread_ts=1665131857.359879&cid=CL09KU1K7

Hey folks 👋 Recently I started working on a new integration. I’ve started from the

prefect-collection-template

as usual. From time to time, I’m seeing the failures on the Github Action named “_Template Synchronization_”. The reported error is

Run tibdex/github-app-token@v1
  with:
    github_api_url: <https://api.github.com>
    repository: AlessandroLollo/prefect-sifflet
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.9.15/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.9.15/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.15/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.15/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.15/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.15/x64/lib
Error: Error: Input required and not supplied: app_id

Anyone knows how to solve it? Thanks! 🙌

Been working on an integration between Sifflet and Prefect. Not released yet, but you can find the code here. Feedbacks more than welcome as usual! 🙌 cc @Wajdi

Hey everyone 👋 I'm working on a general purpose integration between Google Sheets and Prefect. I've a first version ready and would like to publish it. I've read from the docs that I have to open a PR to the Prefect repo with a new yaml file under the

collections/catalog

path. My question is: how can I publish it on PyPi? Is there any automation? Something related to Github action? Thanks! 😄

Hello, what's the status on the prefect-twilio project? Has not been any activity for the past 7 months...and PYPI does not have a package which can be installed via pip

Hey folks 👋 I’m working on a new integration between Prefect and Huggingface 🚀 Here’s the code! Right now it only support Inference API, but I’m planning to extend it to support also Inference Endpoints

Hi, Maybe not smart questions but I didn't see them quick scrolling in the channel... how does one start when wanting to contribute? Is there a list of on-going projects where one can do PRs? Is there a list with prioritized projects based on demand of feature requests or something like that? Is it more useful contributing with a new Block, or with something else? Thanks!

Hey folks 👋 I’m getting these warnings while running tests on a new collection I’m working on

../../../.pyenv/versions/3.10.9/envs/prefect-huggingface/lib/python3.10/site-packages/prefect/testing/standard_test_suites/task_runners.py:155
  /Users/alessandro.lollo/.pyenv/versions/3.10.9/envs/prefect-huggingface/lib/python3.10/site-packages/prefect/testing/standard_test_suites/task_runners.py:155: PytestUnknownMarkWarning: Unknown pytest.mark.flaky - is this a typo?  You can register custom marks to avoid this warning - for details, see <https://docs.pytest.org/en/stable/how-to/mark.html>
    @pytest.mark.flaky(max_runs=4)  # Threads do not consistently yield

Not sure what’s the issue here, anyone experiencing this?

tests/test_tasks.py::test_get_inference_result_fails
  /Users/alessandro.lollo/.pyenv/versions/3.10.9/envs/prefect-huggingface/lib/python3.10/site-packages/prefect/orion/database/migrations/versions/sqlite/2022_04_25_135207_b75d279ba985_replace_version_with_checksum.py:102: RemovedIn20Warning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings.  Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: <https://sqlalche.me/e/b8d9>)
    meta_data = sa.MetaData(bind=connection)

tests/test_tasks.py::test_get_inference_result_fails
  /Users/alessandro.lollo/.pyenv/versions/3.10.9/lib/python3.10/contextlib.py:142: SAWarning: Skipped unsupported reflection of expression-based index ix_flow_run__coalesce_start_time_expected_start_time_desc
    next(self.gen)

tests/test_tasks.py::test_get_inference_result_fails
  /Users/alessandro.lollo/.pyenv/versions/3.10.9/lib/python3.10/contextlib.py:142: SAWarning: Skipped unsupported reflection of expression-based index ix_flow_run__coalesce_start_time_expected_start_time_asc
    next(self.gen)

-- Docs: <https://docs.pytest.org/en/stable/how-to/capture-warnings.html>

These seems to be related to SQLAlchemy 2.0. However, in my environment I have

prefect==2.7.11

and

sqlalchemy==1.4.46

Anyone experiencing the same? Is this something I should be worried about? 😅

Hey folks 👋, I am excited to announce that the integration between Prefect and Sifflet has been released: https://github.com/Siffletapp/prefect-sifflet . Sifflet is a full data stack observability; this integration is the first of many steps toward prefect data pipeline observability. I want to take a moment and thank @ale for leading this integration; it was a real plaisir working with him. We look forward to continuing our collaboration with you and other members of the open-source community.

Hello! Using actions, is it possible (or future possible) for a successful flow run can result in running all flows with a certain tag?

Hello all, nice to e-meet you. I think it's the first time I'm using the prefect Slack, so... forgive me if this isn't the right channel to post this: I had previously worked along Andrew Huang to implement the

upload_from_dataframe

on GcsBucket. But through a user report on DataTalksClub Slack, we recently noticed an issue with the documentation generated from the method's docstring. I've documented on the issue here: https://github.com/PrefectHQ/prefect-gcp/issues/163 And submitted the PR with the fix here: https://github.com/PrefectHQ/prefect-gcp/pull/162 I was wondering if there's anything else I gotta do, like tagging ppl to review the code on GH, or something. Thank you in advance.

Guys, there's something I want to discuss. When I originally worked on the

upload_from_dataframe

, we went with

.parquet.snappy

and

.parquet.gz

for the compressed parquet files. However, comma, ergo, vis-a-vis, it came to my attention that when you're using tools like Tad and others to visualize tabular data, they EXPECT the file extension to end with

.parquet

instead (Like it works if I rename

file.parquet.snappy

to

file.snappy.parquet

, or

file.parquet.gz

to

file.gz.parquet

. I also noticed that Spark and Flink are actually saving compressed parquets as

.snappy.parquet

or

.gz.parquet

instead.

Hi, before propose a PR I opened an issue (https://github.com/PrefectHQ/prefect/issues/9290), any advise about it ?

Hello, can anyone offer me some insight on why the Prefect team has dropped the GraphQL API? What did you learn about GraphQL that made you drop it?

Is there a github project for the prefect website?

Hey folks. Just put in my first PR - https://github.com/PrefectHQ/prefect/pull/9691 for this beginner-friendly issue ( thanks for compiling that list btw ). Hopefully first of many to come 🙌 Looking for suggestions / improvements / general advise from you folks. Thanks 😄

Hi ! Is there existing work about triggering flow runs from webhooks, message brokers, ect... I only found this topic mentionning Kafka In my particular use case I would be interested in developing a block to launch flow runs on through a pub/sub (memphis) and I don't have any idea of where to look for starter, thanks 🙂

Hello, we are trying to run multiple flows in parallel to submit long running jobs, we get this error:

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
[SQL: INSERT INTO configuration ("key", value, id, created, updated) VALUES (:key, :value, :id, :created, :updated)]
[parameters: {'key': 'ENCRYPTION_KEY', 'value': '{"fernet_key": ""}', 'id': 'd3166c4a-7c74-4b65-abf3-c12870049dfb', 'created': '2023-06-28 00:40:27.657444', 'updated': '2023-06-28 00:40:27.657459'}]

docker run -p 4200:4200  prefecthq/prefect:2-python3.9 prefect server start

Crashes with a SQLIte error, is there a reason for it? Is this not a public image that’s supposed to work out of the box?

This sounds like a pretty bad problem, I tried all images going back to 2.8 and still I have the problem

hi is there anyone from prefect who could look into this issue/PR? https://github.com/PrefectHQ/prefect-monte-carlo/issues/28 @Taylor Curran

Hello team, I am looking to contribute to the Prefect-GCP repo, specifically I want to upgrade Cloud Run API version to V2, since there are significantly better features in it. Do you know the process for contributing to this? Do I just open Issue (https://github.com/PrefectHQ/prefect-gcp/issues/199), and then just open a PR for review when I have something working/passing tests and someone will help me get it merged/if it fits in right? Thank you/sorry for newb questions 🦜

Hi, I have this MR failing unit testing with the folowing error message:

botocore.exceptions.NoCredentialsError: Unable to locate credentials

I think it's because of this line referencing S3, but cannot say for sure. I though I'd reach out the community before requesting another review.