• Laura Lorenz

    Laura Lorenz

    1 year ago
    My connection dropped! Be right back
  • Laura Lorenz

    Laura Lorenz

    1 year ago
    It was nice to see everyone again!! 😃
  • Josh

    Josh

    1 year ago
    Not sure if this is the right channel for submitting PRs. But I have one to add chunking to GCS to enable handling of larger files or slower internet connections. Not sure how to best add tests for this in CI https://github.com/PrefectHQ/prefect/pull/3968
    Josh
    nicholas
    +1
    6 replies
    Copy to Clipboard
  • Jack Sundberg

    Jack Sundberg

    1 year ago
    Hey everyone! I just opened a pull request and could use some help with updating a few CLI tests -- if anyone is able to help out, I would greatly appreciate it! https://github.com/PrefectHQ/prefect/pull/4146
    Jack Sundberg
    Michael Adkins
    5 replies
    Copy to Clipboard
  • davzucky

    davzucky

    1 year ago
    Hey, I'm writing a set of new tasks to interact with MongoDB. looking at other database provider I can see they are no consistency and I want to have the community input1. Password : Some task like postgresql or mysql are passing the password as a param to the task, whereas Redis for example is getting the password out of the Prefect secret. What is the recommended way of doing it ? 2. db connection string vs individual params : With Mongo I use usually only a single parameter connection string which looking like mongo://username:passwordf@db0.example.com:27017,db1.example.com:27017,db2.example.com:27017/?replicaSet=myRepl for example. However all the db tasks are using distinct params. Any recommendation ? Anything else I need to be aware about ?
    davzucky
    1 replies
    Copy to Clipboard
  • ale

    ale

    1 year ago
    Hey folks 🙂 Recently I've integrated SodaSQL with Prefect using a custom Task. Here you can find the docs https://docs.soda.io/soda-sql/documentation/orchestrate_scans.html This is just a basic implementation though. I'm thinking about to implement Soda tasks into Prefect core tasks and I'd like to understand if you have any idea or use case that you want like to address with Prefect regarding Data Quality monitoring
  • ale

    ale

    1 year ago
    Hey folks 😒imple_smile: I’ve opened this PR https://github.com/PrefectHQ/prefect/pull/4308 Tests are failing due to missing header files for
    pyodbc
    Can someone help me fix it? Thanks!
    ale
    Michael Adkins
    2 replies
    Copy to Clipboard
  • j

    Jonathan Wright

    1 year ago
    I am doing some final testing of an extension to the Bitbucket storage class which adds support for Bitbucket cloud. When I register a flow using my new storage class a new
    workspace
    variable is not showing in the UI. I have also shown that when the flow runs, in an environment with the same version of the storage class, the
    workspace
    variable is None. I assumed my storage class would be pickled and stored with the flow and I wouldn’t need to make any code changes outside of this file. What am I missing?https://github.com/PrefectHQ/prefect/blob/master/src/prefect/storage/bitbucket.py
    j
    m
    9 replies
    Copy to Clipboard
  • ale

    ale

    1 year ago
    Hey folks 😒imple_smile: Regarding ShellTask, wouldn’t it be useful to return the logs generated by the shell command as part of the FAIL message when the task fails? Maybe we can add a parameter that keeps the current behaviour as a default to avoid breaking changes. I would find it very useful for return useful details when a DbtShellTask fails… What do you think?
    ale
    Jim Crist-Harif
    +1
    11 replies
    Copy to Clipboard
  • Jacopo Tagliabue

    Jacopo Tagliabue

    1 year ago
    hi folks, I"m Jacopo - I built a very small POC on using the ShellTask as a base class to run Metaflow flows as part of a Prefect DAG (
    <https://prefect-community.slack.com/archives/CL09KTZPX/p1617381793056800>
    ). If there is any interest in having MF tasks as part of Prefect task library, I'd be happy to give it some actual love and fix a bunch of things here and there 😉 Let me know what you think!
    Jacopo Tagliabue
    Dylan
    7 replies
    Copy to Clipboard