https://prefect.io logo
Docs
Join the conversationJoin Slack
Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Powered by Linen
prefect-contributors
  • a

    Alex Cano

    06/08/2020, 4:01 PM
    Did the tests for code formatting & mypy get removed? I just realized after my vscode stopped auto-black formatting my code and it failed CI checks 😅
    l
    c
    • 3
    • 4
  • a

    Alex Cano

    06/08/2020, 9:14 PM
    Hey all, I was hoping to do a more synchronous code review of this PR (allowing the
    CloudFlowRunner
    to handle a
    Queued
    state for flow concurrency limiting): https://github.com/PrefectHQ/prefect/pull/2741. Does anyone have time to pair on it later this week? I think I captured the changes where they were appropriate, but wanted a second pair of eyes & wanted to make sure it’d work with what’s planned on the cloud side.
    l
    j
    • 3
    • 5
  • i

    itay livni

    06/10/2020, 4:36 PM
    Hi - Has anyone run into this issue?https://github.com/PrefectHQ/prefect/issues/2761
    l
    • 2
    • 8
  • d

    Dylan

    06/11/2020, 5:42 PM
    Hey Everyone! I’d like to ask for your help to make Prefect Cloud and Prefect Server even better. Would you take 4 minutes to fill out this feedback survey? https://forms.gle/GArq27SyxBd8v4MCA Thank you!
    ❤️ 2
  • j

    jorwoods

    06/12/2020, 4:06 PM
    Regarding Result locations vs targets: I understand having the Task's
    target
    override a Result's
    location
    as the former is more specific to a Task. Laura mentioned keeping them separate for the purposes of having the task caching be more opt-in behavior, but I don't understand how providing a templated string to
    location
    doesn't also qualify as that behavior. Is there opposition to having a Result's
    exists
    method check for the passed in variable if there is one, and also check for
    self.location
    ?
    j
    j
    • 3
    • 7
  • p

    Paweł Cieśliński

    06/13/2020, 10:14 PM
    Hi everyone! I just added my first PR to the project. I started with a small thing, but I’m going to support Prefect in the near future. I really like the approach and philosophy that you are trying to introduce to the world of workflow engines. As soon as I delved into what you have already created, it stole my heart. 🙂 Have a great weekend everyone!
    💯 2
    🚀 3
    c
    • 2
    • 1
  • d

    David Haines

    06/14/2020, 7:59 PM
    Hi all. Added a PR to support mapped args in control flow functions. Would appreciate any thoughts or feedback (especially if there's an alternative option available I've missed entirely).
    🚀 5
  • m

    matta

    06/15/2020, 11:17 PM
    Made a little connector for Gsheets - not sure why the build is failing? https://github.com/PrefectHQ/prefect/pull/2614
    c
    • 2
    • 6
  • l

    Laura Lorenz (she/her)

    06/17/2020, 4:18 AM
    set the channel topic: Core Cantina 6/19 at 4 pm EDT (meet.google.com/quz-zuea-kix)
    :upvote: 3
  • i

    itay livni

    06/19/2020, 9:41 PM
    @Laura Lorenz (she/her) https://github.com/PrefectHQ/prefect/pull/2531#issuecomment-646870197
  • a

    Alex Cano

    06/20/2020, 5:53 PM
    Does anyone have any thoughts on adding some integration tests between server & core? Currently Jim has an issue opened as of a couple days ago thinking about the same for Cloud (https://github.com/PrefectHQ/prefect/issues/2792). The reason I’m asking is I’m currently going through the process of testing the flow concurrency feature (https://github.com/PrefectHQ/prefect/pull/2382), and can’t think of a good place to implement tests that actually have flows running and communicating with Server. I’m sure there would be some considerations of wanting the same integration tests to be both cloud and server compatible, then allow for server specific tests as well as cloud specific tests.
    l
    • 2
    • 1
  • l

    Laura Lorenz (she/her)

    06/22/2020, 6:20 PM
    Just wanted to give a 👏 shout out 👏 to @Mark McDonald who made a DbtShellTask a while back in the task library and with his help Prefect now has a section in the dbt docs here: https://docs.getdbt.com/docs/running-a-dbt-project/running-dbt-in-production/#using-prefect. Hoping to see more dbt users in the Prefect community now :marvin:
    👏 13
    🙌 4
    🚀 2
    💯 1
    r
    m
    • 3
    • 10
  • l

    Laura Lorenz (she/her)

    06/30/2020, 1:33 PM
    Hi contribs — I have cantina scheduled for Friday but that is likely a holiday for several people. I’m wondering if there’s a quorum, especially of regulars (cc @Alex Cano @itay livni), who can let me know if you prefer it earlier in the week or are ok with Friday! 🐫 Wednesday 7/1 🌮 Thursday 7/2 🍺 Leave it on Friday 7/3
    🐫 3
    🌮 2
    🍺 1
    a
    • 2
    • 6
  • l

    Laura Lorenz (she/her)

    06/30/2020, 2:19 PM
    set the channel topic: Core Cantina Wednesday 7/1 (meet.google.com/quz-zuea-kix)
  • l

    Laura Lorenz (she/her)

    06/30/2020, 2:19 PM
    set the channel topic: Core Cantina Wednesday 7/1 @ 4pm EDT (meet.google.com/quz-zuea-kix)
  • l

    Laura Lorenz (she/her)

    07/07/2020, 1:47 AM
    set the channel topic: Livestream on deploying server 7/10 @ 4pm EDT (

    https://youtu.be/yjORjWHyKhg▾

    ) | Scipy2020 virtual sprints Sat/Sun 7/11 and 7/12 all day!
  • b

    Brett Naul

    07/07/2020, 3:18 AM
    is there a way / is it a bad idea to access an upstream
    Result
    directly from within a task? I'm thinking about how a papermill task might look; since it runs in a subprocess you'd need to load the results there instead of in the main prefect taskrunner process....
    l
    • 2
    • 6
  • l

    Laura Lorenz (she/her)

    07/11/2020, 3:42 PM
    Hey all! Jeremiah posted about it in #announcements but further detail here that I’m hanging out for Scipy 2020 sprints all this weekend and will be streaming some contribution stuff on Youtube along the way! There’s a separate slack domain for scipy2020 and a dedicated Meet hangout (https://meet.google.com/zkm-btxh-zih) for the weekend so feel free to drop in there if you like, or I’m of course always on this slack too so lmk if you’re feeling the Scipy energy this weekend and want to contrib!
    :upvote: 2
  • j

    james.lamb

    07/13/2020, 3:22 PM
    Could
    @here
    be disabled for participants in #prefect-community ? That channel has 1000+ people in it and they did not all need to be aware of this: https://prefect-community.slack.com/archives/CL09KU1K7/p1594653685051900
    🙌 1
    :upvote: 8
    c
    • 2
    • 1
  • l

    Laura Lorenz (she/her)

    07/14/2020, 2:46 PM
    set the channel topic: Core Contributors Cantina Thursday 7/15 @ 5pm EDT (meet.google.com/quz-zuea-kix)
  • l

    Laura Lorenz (she/her)

    07/14/2020, 2:48 PM
    Hi all! Just a heads up ^ I’m going to be afk Friday with potentially bad internet so to hedge against any connection snafus I moved cantina up a day. I will post the updates in this channel too for anyone who can’t make it at this irregular time 🙂 And if you aren’t also following along in #contributors-event-driven-flows we are having a little meetup tomorrow on Zoom, if you want even more Prefect face time!
    🙌 2
  • r

    Robin

    07/15/2020, 8:38 PM
    Dear all, I might be ready to make my first small contribution to prefect. 🙂 As I had similar issues as mentioned in https://github.com/PrefectHQ/prefect/issues/2069, I created an
    AWSSecretsManager
    task, analogous to the
    S3Download
    task, using the
    get_boto_client
    as suggested by @josh. Furthermore, I read the Development Overview, forked the repo and created a new pull request, trying to follow the contribution guidelines. All in all, I really like the way all the required information are represented! 🙂 Also, shoutout to the prefect team for great support on Slack, we greatly appreciate it!
    🚀 9
  • l

    Laura Lorenz (she/her)

    07/16/2020, 10:00 PM
    Hi everyone! Just wanted to throw some notes in here from Cantina if you weren’t able to stop by at our weirdly scheduled one today :) I’m going to start 2 threads for discussion on two of the items under ‘ease of getting started’ for your consideration, because I think they might be of interest to yall :) Other than that in particular I want to ping @David Ojeda that we put in a fix for “Mapped tasks returning Nones” which is in 0.12.4 (https://github.com/PrefectHQ/prefect/issues/2656). If you can let us know if it fixes your problem that would be awesome!
    r
    • 2
    • 1
  • l

    Laura Lorenz (she/her)

    07/16/2020, 10:02 PM
    Thread for your thoughts: destroy Environments 🔥 What do you like/use from Prefect Environments? We are thinking of getting rid of them in favor of bloating the agents more with more configuration. The reason for this is to attempt to make it easier to make a choice on where to deploy. For example, today if you use a Kubernetes Agent you probably are using one of a limited list of environments (such as KubernetesJobEnvironment or DaskKubernetesEnvironment); if you are using the Fargate Agent you are probably only using the Fargate Task Environment; etc. So basically we think they are not as uncoupled as we originally thought and it might be easier for people to use them if we just had single configuration on an Agent for each of these combinations. What do you think? Is there something from the Environment config that you particularly like or use now that you would be worried would be lost if we removed them?
    i
    e
    a
    • 4
    • 6
  • l

    Laura Lorenz (she/her)

    07/16/2020, 10:04 PM
    Thread for your thoughts: Storage config from Cloud/Server We are also considering enabling people to list a flow storage source (like a Github Repo or an S3 bucket) directly in the UI in Cloud/Server; and in so doing basically do half of the ‘register’ step by letting Prefect know where the storage location is. As a user you would store/push your flow files to wherever and then log in to Cloud/Server to provide the location. We still need a Python interpreter to get involved at some point to parse the flow object to get out some of the metadata so technically it is not fully registered until that happens -- we would probably have the agent be responsible for finishing off the registration for anything that got pre-registered via Cloud/Server as just a pure storage location. Do you think that this would be too confusing? Do you prefer controlling registration entirely or do you like the idea of having things ‘pre-registered’ like this?
    e
    a
    • 3
    • 3
  • l

    Laura Lorenz (she/her)

    07/23/2020, 3:48 PM
    set the channel topic: Livestream on Results and Resources 7/24 @ 4edt (

    https://youtu.be/LT1qe8cCJEA▾

    ) | Contributor’s Cantina 7/31 @ 4edt (meet.google.com/quz-zuea-kix)
  • m

    Matt Wong-Kemp

    07/28/2020, 2:20 PM
    Hi there - I've opened https://github.com/PrefectHQ/prefect/pull/3046, would love some advice on tests/docs required or if this is the right approach.
    l
    • 2
    • 4
  • a

    Alex Cano

    07/31/2020, 8:04 PM
    👋 Are we doing the contributor cantina? 🙂
    👍 1
  • i

    itay livni

    07/31/2020, 8:09 PM
    Hi - Can you share the link?
  • a

    Alex Cano

    07/31/2020, 8:09 PM
    7/31 cantina link: meet.google.com/quz-zuea-kix
    ✔️ 1
Powered by Linen
Title
a

Alex Cano

07/31/2020, 8:09 PM
7/31 cantina link: meet.google.com/quz-zuea-kix
✔️ 1
View count: 2