• j

    Joe Schmid

    2 years ago
    Another pretty low-level question: I'm debugging a custom Environment and would like to:1. Run with local agent (that part works fine, can register a local flow & have it run, etc.) 2. See debug logging during the Flow run / Environment execute(). I can enable & see debug output for the local agent, but I think since the flow run happens in a subprocess all I get is:
    [2020-04-30 20:35:05,397] INFO - agent | Found 1 flow run(s) to submit for execution.
    [2020-04-30 20:35:05,397] DEBUG - agent | Updating states for flow run f7912fd5-8ce3-4048-8cfe-965a15d46845
    [2020-04-30 20:35:05,400] DEBUG - agent | Flow run f7912fd5-8ce3-4048-8cfe-965a15d46845 is in a Scheduled state, updating to Submitted
    [2020-04-30 20:35:05,505] INFO - agent | Deploying flow run f7912fd5-8ce3-4048-8cfe-965a15d46845
    [2020-04-30 20:35:05,519] DEBUG - agent | Submitted flow run f7912fd5-8ce3-4048-8cfe-965a15d46845 to process PID 46552
    [2020-04-30 20:35:05,608] DEBUG - agent | Completed flow run submission (id: f7912fd5-8ce3-4048-8cfe-965a15d46845)
    [2020-04-30 20:36:22,277] INFO - agent | Process PID 46552 returned non-zero exit code
    Is there a good way to (a) have the flow runner in process or (b) see log output from the subprocess?
    j
    Chris White
    11 replies
    Copy to Clipboard
  • Troy Köhler

    Troy Köhler

    2 years ago
    Hi there! Would love to ask are there issues you (experienced developers and team of prefect) prefer me to get? I need something for person who just start getting familiar with the project, but probably not just one line issues.
    Troy Köhler
    Jenny
    +1
    9 replies
    Copy to Clipboard
  • Laura Lorenz (she/her)

    Laura Lorenz (she/her)

    2 years ago
    set the channel topic: Core Contributors Cantina Fri 5/8 at 4pm EDT. Hangout link: meet.google.com/quz-zuea-kix
  • Laura Lorenz (she/her)

    Laura Lorenz (she/her)

    2 years ago
    Friendly reminder about Cantina today at 4pm EDT at meet.google.com/quz-zuea-kix — and friendly reminder to wear a floral pattern if you want to match Alex 😉
  • a

    Alex Cano

    2 years ago
    @Jeremiah @Laura Lorenz (she/her) I wanted to touch base again on the flow concurrency PR (https://github.com/PrefectHQ/prefect/pull/2382)… specifically on where to put the concurrency check in the flow state setting pipeline. I thought of 2 points that I wanted to either make sure were non-issues from past experiences or figure out how to deal with. I’m going to attach 2 images in separate posts, one for each scenario. They attempt to show what I’m going for, but I’ve never been great at diagramming! I also wanted to make sure I understood the interaction once a flow is marked as
    Submitted
    by the Agent. After this, from what I can tell each flow handles the transition from
    Submitted
    to
    Running
    by means of the
    CloudFlowRunner
    , so each
    CloudFlowRunner
    is calling the
    set_flow_run_states
    with its own flow run, right? So maybe the conditions below don’t matter? First is since the server is written using async, if we put the concurrency check in the
    api.states.set_flow_run_state
    call, and an API call comes in with N flow runs that are trying to transition into a
    Running
    state and there’s M concurrency slots where M < N, how can we guarantee only M flows will transition into the
    Running
    state? Second, when we’re creating the “Run Queue” in the
    api.runs.get_runs_in_queue
    , we’re specifically returning flow runs in the order of first scheduled. Related to the above, if we’re submitting flow runs to a
    Running
    state, are we guaranteeing that state changes in the flow runs submitted in the
    set_flow_runs_states
    mutation are occurring in order? If there are 3 flow runs and only one concurrency slot, are we just guaranteeing that one of the flow runs in the payload will succeed? Or are we guaranteeing that the first flow run in the payload will succeed?
    a
    Jeremiah
    3 replies
    Copy to Clipboard
  • a

    Alex Cano

    2 years ago
    First Scenario: Async concurrency checks
  • a

    Alex Cano

    2 years ago
    Second Scenario: Out of Order
  • Laura Lorenz (she/her)

    Laura Lorenz (she/her)

    2 years ago
    Hey contributors! FYI I am hosting as Prefect this Saturday 12-4 EDT for Pycon’s Hatchery program “Mentored Sprints for Diverse Beginners”. If you want to stop by to sprint or to help mentor, check out the details at https://us.pycon.org/2020/hatchery/mentoredsprints/. It is designed to be super beginner friendly, and they are also looking for independent and/or git/github mentors if that is at all your jam. It is going to be hosted over discord which should be fun! If you’re so gung ho you want to be an ‘official’ mentor for Prefect for the Sunday event, LMK and I will add you to our roster, I think it gets you some extra discord role so people know they can reach out to you
    Laura Lorenz (she/her)
    Edidiong Etuk
    3 replies
    Copy to Clipboard
  • Laura Lorenz (she/her)

    Laura Lorenz (she/her)

    2 years ago
    set the channel topic: Livestream on Vue.js Server dev 5/15 (

    https://youtu.be/YHqfJwFvTFY

    ); Pycon Mentored sprints 5/17 on Pycon’s discord; Cantina 5/22 (meet.google.com/quz-zuea-kix)
  • i

    itay livni

    2 years ago
    What time is the Cantina today?