Benny Warlick

    Benny Warlick

    4 months ago
    Hey all, I'm working on a Prefect 2.0 implementation on docker running on GCP compute engine. One issue is the bash script to get everything up and running. This is what I came up with, which seems to work. I'm wondering if I am overcomplicating this and there is an easier way to do it? The same thing was much simpler with Prefect 1.0 (set api key, register flow, then start agent).
    prefect cloud login --key <MY_KEY> -w <MY_WORKSPACE>
    rand="GCS_"$(cat /dev/urandom | tr -cd 'a-f0-9' | head -c 16)
    output=$(printf '%s\n' 2 <MY_BUCKET> <MY_PROJECT> $rand | prefect storage create)
    storage_id=$(echo $output | grep -oP "(?<=identifier \').+?(?=\')")
    prefect storage set-default $storage_id
    prefect deployment create my_flow.py
    output=$(prefect work-queue create my_queue | grep -oP "(?<=UUID\(\').+?(?=\'\))")
    prefect agent start $output
    Kevin Kho

    Kevin Kho

    4 months ago
    Ah I see. It’s because of a lot of referencing by id like with the agent using a work queue that causes a lot more boilerplate. I think this might be simplified when changes to specify by name happen so you can name everything ahead of time. The storage interface will also change
    Michael Adkins

    Michael Adkins

    4 months ago
    You can specifying work queues by name already in 2.0b3
    Storage is a bit more of a pain, we’re actively refactoring that section and hoping for some significant UX improvements
    Benny Warlick

    Benny Warlick

    4 months ago
    Ok, thanks that's helpful. I tried using local storage instead of GCS but ran into this issue: https://prefect-community.slack.com/archives/CL09KU1K7/p1650504918891289
    Michael Adkins

    Michael Adkins

    4 months ago
    2.0b4 (planned for Thursday) will resolve that bug. There’s a PR at https://github.com/PrefectHQ/prefect/pull/5710 if you wanted that code early.