• Andrew Lawlor

    Andrew Lawlor

    7 months ago
    does anyone have good patterns for registering flows via ci/cd? ideally id like to register only the flows that changed automatically on a git push, and be able to use different run configs for each, while also handling env vars that may be different in prod/dev. currently i have a script that loops through all my files in my flows directory and registers all flows using extract_flow_from_file. it runs on a push but the way i have it set up it: • always registers all flows, regardless of whether they have changed or not • has to pass the same env values to all flows, regardless of what the flow actually needs • passes the same run config to all flows, not allowing me to customize memory requests per flow any guidance on what to do here?
    Andrew Lawlor
    Kevin Kho
    4 replies
    Copy to Clipboard
  • c

    Christopher

    7 months ago
    I've got an
    ECSAgent
    running (in ECS, using the base prefect image with
    prefect agent ecs start
    ), but the tasks it spins up are in the wrong subnet so I want to customise the task definition. It looks like I can pass in a task definition path but that's a bit troublesome because now I need to store file somewhere accessible to the agent. Is there a way to pass a task definition ARN instead? It looks like I can pass that to
    ECSRun
    but the subnet ID is generated by Terraform so I can't figure out how to get it into the Python...
    c
    Kevin Kho
    13 replies
    Copy to Clipboard
  • c

    Christopher

    7 months ago
    I'm also not totally clear on the relationship between ecsagent and ecsrun. If I didn't use ecsrun, would the flow run on the ECS service that's running the agent directly?
    c
    Kevin Kho
    3 replies
    Copy to Clipboard
  • James Sutton

    James Sutton

    7 months ago
    Hi Prefect Community. I’ve been trialing Prefect to replace Airflow and documented my thoughts to date here, along with a brief setup guide, not sure if this would be useful for others: https://jamesutton.medium.com/my-experience-with-prefect-so-far-and-a-quick-getting-sta[…]bc?source=friends_link&sk=58e6e074ee852ffbef2e46999d43b1c7
    James Sutton
    1 replies
    Copy to Clipboard
  • Daniel Nilsen

    Daniel Nilsen

    7 months ago
    Hi! Im trying to start a run though gql. I am using this mutation where the input is a variable: Anyone know what the type is supposed to be for the input?
    mutation MyMutation($input: any) {
        create_flow_run(
          input: $input
        ) {
          id
        }
      }
    Daniel Nilsen
    Kevin Kho
    11 replies
    Copy to Clipboard
  • f

    Faisal k k

    7 months ago
    Hi I need to add Prefect Cloud API IPs for whitelisting in Security group of EC2 where Agent is running ... Couldn't find IP range... Anyone know where can I find it?
    f
    Kevin Kho
    2 replies
    Copy to Clipboard
  • Michael Hadorn

    Michael Hadorn

    7 months ago
    Question about Orion: generic tasks, flow build without running it. Details in thread.
    Michael Hadorn
    Anna Geller
    9 replies
    Copy to Clipboard
  • w

    William Edwards

    7 months ago
    I'm trying out Prefect. Two questions: • I added and registered a flow with a local server. It has caught my attention that when I re-add the exact same flow, a new 'version' is created. What is the best practice for adding a flow? Am I supposed to add it just once and again for updates? Or always before creating a new flor run? • What is the typical separation of concerns? Should I just call
    create_flow_run
    in the client, or should the client be doing anything else?
    w
    Kevin Kho
    3 replies
    Copy to Clipboard
  • Robert Kowalski

    Robert Kowalski

    7 months ago
    Hi, It is possible to fix MAC address when docker storage is using ? Docker run command have
    docker run --mac-address="70:ca:9b:ce:67:ae" IMAGE
    So I want set constant mac address when define docker storage as:
    storage = Docker(
        env_vars={"PYTHONPATH": "$PYTHONPATH:/pipeline"},
        files={f'{parent_dir}': '/pipeline'},
        image_tag=os.environ.get('IMAGE_TAG'),
        image_name=flow_name,
        stored_as_script=True,
        path='/pipeline/flow.py',
        extra_dockerfile_commands=[]
    )
    Someone has tried to achieve something similar ?
    Robert Kowalski
    Kevin Kho
    3 replies
    Copy to Clipboard
  • Josh

    Josh

    7 months ago
    I’m trying to dynamically name a flow following these instructions. But is there some way to name the flow run with the labels used or the environment variables present in the flow run?https://github.com/PrefectHQ/prefect/discussions/3881
    Josh
    Kevin Kho
    8 replies
    Copy to Clipboard