Thread
#prefect-community
    Jay Sundaram

    Jay Sundaram

    1 year ago
    Hi all, I'm looking for some help with StartFlowRun. Executed the following running on my local machine:
    prefect backend server
    prefect server start
    prefect create project etl-project
    prefect agent local start --label etl-label
    prefect register flow --file simple_flow.py --name a-simple-etl-flow -l etl-label -p etl-project
    In the UI, I can click on QUICK RUN and observe the flow execute. Next , in another simple script named start_flow_run.py:
    from prefect.tasks.prefect.flow_run import StartFlowRun
    
    kickoff_task = StartFlowRun(
        project_name='etl-project',
        flow_name='a-simple-etl-flow'
    )
    which I execute like this:
    python start_flow_run.py
    But nothing happens. The agent doesn't detect it; no activity in the UI. I was expecting my registered flow named 'a-simple-etl-flow' to execute. Please advise. Thanks.
    I just figured it out. I just needed to add kickoff_task.run() to the start_flow_run.py. The updated script looks like this:
    from prefect.tasks.prefect.flow_run import StartFlowRun
    
    kickoff_task = StartFlowRun(
        project_name='etl-project',
        flow_name='a-simple-etl-flow'
    )
    
    kickoff_task.run()
    Might want to update the example in the documentation to indicate that kickoff.run() is required.
    Now, if my 'a-simple-etl-flow' flow is stored in S3 and the agent is running on an AMI (EC2), how should the start_flow_run.py be updated to launch/start that flow? And does the start_flow_run.py script need to be executed on the same AMI that the agent is running on?
    Chris White

    Chris White

    1 year ago
    Hi Jay, this task is not intended to be run this way (although it’s perfectly fine if you do want to use it like this); this task, like any Prefect task, should be added to a Flow. Check out this example: https://docs.prefect.io/core/idioms/flow-to-flow.html And in the future, could you please avoid posting large code blocks and reposting in the main channel from a thread? We try to keep the main channel tidy so that it’s easily scrollable - thank you!! 😄 🙏
    Jay Sundaram

    Jay Sundaram

    1 year ago
    Hi @Chris White, thank you- I'll take a look.
    Hi @Chris White, What is the proper way to execute a flow, programmatically i.e.: not from the UI and not a scheduled job. Looking for a way to execute a flow given an external event. Thank you.
    Chris White

    Chris White

    1 year ago
    Great question - ultimately, you’ll want to use the graphql api to achieve this; there are many ways of configuring a service / webhook to call the API that depend on your setup. Check out these docs for some examples: https://docs.prefect.io/orchestration/concepts/flow_runs.html https://medium.com/the-prefect-blog/event-driven-workflows-with-aws-lambda-2ef9d8cc8f1a
    Jay Sundaram

    Jay Sundaram

    1 year ago
    Thank you for this info. I'll try the graphql api. About the medium article, questions:1. Will this only work with Prefect Cloud and not with Prefect Server? 2. The Prefect Cloud API token should be a RUNNER token, correct?
    Chris White

    Chris White

    1 year ago
    1. The GraphQL route is also available in Server (please note that it is snake_cased though, and not camelCased as in the article), but you do need to ensure that where ever you call it from has network access to your Server API 2. You can call this route from either a RUNNER or a TENANT token 👍
    Jay Sundaram

    Jay Sundaram

    1 year ago
    Would like to try this out on server first. How do we create a RUNNER token for server?
    prefect auth create-token -n runner_token -s RUNNER                                                                 
    Usage: prefect auth [OPTIONS] COMMAND [ARGS]...
    
    Error: Auth commands with server are not currently supported.
    In the UI (I believe the only other way to create RUNNER tokens), the API Token option is grayed out:
    Chris White

    Chris White

    1 year ago
    Tokens are only used for authentication with Cloud, there is no token concept in Server so you don’t need to worry about that step if using Server
    Jay Sundaram

    Jay Sundaram

    1 year ago
    Hi Chris, What is the API endpoint/URL for Server?
    Chris White

    Chris White

    1 year ago
    it depends on how you deployed it, but assuming you’re using the default configuration it should be
    <https://localhost:8080/graphql>
    Jay Sundaram

    Jay Sundaram

    1 year ago
    I get the following: HTTP Error 405: Not Allowed. I tried with the following code.
    Chris White

    Chris White

    1 year ago
    Oh you shouldn’t have to go through all of that - in that example I didn’t have access to the Prefect library because I was using a default AWS environment; you can use the prefect library:
    from prefect import Client
    
    c = Client() # will autodetect your configuration
    
    c.create_flow_run(flow_id)
    Jay Sundaram

    Jay Sundaram

    1 year ago
    This worked like a charm. Thank you.
    Did you mention something about specifying the Version Group ID instead of the Flow ID- or am I remembering incorrectly?
    Chris White

    Chris White

    1 year ago
    Yup you can pass
    version_group_id=XXXX
    as a keyword argument instead of flow id and the backend will trigger a run for the latest version of the flow without you having to update the flow ID each time
    Jay Sundaram

    Jay Sundaram

    1 year ago
    Nice! Thank you.