https://prefect.io logo
g

Gabe Villasana

09/01/2023, 10:26 PM
Hi team, I currently use celery + redis to handle background tasks (orchestrated with celery chains) and am trying to move to Prefect cloud. I managed to create the flow code, though I'm having difficulty figuring out how to setup the deployments and call them from my application code. I can trigger the flow runs, though they are not part of a single deployment (not sure if there's any merit to that right now). My code is packaged in a docker container and deployed with Kubernetes via a PaaS provider (porter)
1
c

Chris White

09/02/2023, 12:22 AM
Hey Gabe! That's great to hear, we're seeing more and more people with that use case and would definitely be interested in your feedback as you set things up. We just released some simpler ways of deploying that I think you'd benefit from: • check out this tutorial and the surrounding sections for creating multiple deployments simultaneously from within the same file • this unpublished guide on dockerizing that setup Once you have those setup, you'll be able to trigger runs via the Cloud REST API. Soon we'll expose a native webserver alongside that pattern so you can trigger fully orchestrated runs from within your cluster without making the round trip to the Cloud API
g

Gabe Villasana

09/02/2023, 12:29 AM
Hey @Chris White, I'm pretty sure I was able to get most of the way there but I'm having difficulty triggering runs with the cloud REST API. Any pointers for that?
I've basically got myself to the point where I can execute flow runs using the CLI, but not sure where in the docs I can find information about the REST API
c

Chris White

09/02/2023, 12:31 AM
Yea gotcha; here is an autogenerated set of reference docs for the REST API: https://app.prefect.cloud/api/docs the route you're looking for is titled
Create Flow Run From Deployment
so you'll need: • an API key to use as a Bearer Token • a correctly configured Cloud API URL that has your account and workspace • a deployment ID to put in the path • any additional fields in the body (typically you'll only need
name
and
parameters
)
g

Gabe Villasana

09/02/2023, 12:33 AM
Ok great, yep I should have all of those things. I will admit though these aren't the easiest docs to parse 😅
Will give the cloud REST API a shot and report back 🫡
c

Chris White

09/02/2023, 12:39 AM
awesome!
g

Gabe Villasana

09/02/2023, 2:51 AM
Hey @Chris White, Ran into some issues where I keep getting a 403 HTTP response, any chance you've seen this before?
Copy code
import requests
import src.constants as constants


def create_flow_run_from_deployment():
    """
    Create a flow run from a deployment using the Prefect Cloud API.

    Args:
        account_id (uuid.UUID): Account ID
        workspace_id (uuid.UUID): Workspace ID
        deployment_id (uuid.UUID): Deployment ID
        api_version (str): API version for the header
        token (str): Authorization token
        ... : Other parameters as specified in the API documentation

    Returns:
        response: API response object
    """

    # Define the API endpoint URL
    url = f"<https://api.prefect.cloud/api/accounts/{constants.PREFECT_ACCOUNT_ID}/workspaces/{constants.PREFECT_WORKSPACE_ID}/deployments/{constants.PREFECT_DEPLOYMENT_ID}/create_flow_run>"

    # Construct headers
    headers = {
        "Authorization": f"Bearer {constants.PREFECT_CLOUD_API_KEY}",
        "Content-Type": "application/json",
    }

    # Make the API call
    response = <http://requests.post|requests.post>(url, headers=headers)

    return response
c

Chris White

09/02/2023, 3:08 AM
Yea, you're really close but you're missing a request body; actually if you're using Python, you could use the Prefect library to make your life easier here (or at least you could use the library as inspiration for your lightweight client): • our version of this: https://github.com/PrefectHQ/prefect/blob/main/src/prefect/client/orchestration.py#L474 • an even simpler utility function: https://github.com/PrefectHQ/prefect/blob/62ac0bea4390d83728e68d67f60d97fc42ec6e60/src/prefect/deployments/deployments.py#L49
g

Gabe Villasana

09/02/2023, 3:29 AM
Oh it's funny I was actually trying to use the
run_deployment
function that you linked but was getting the 403 denied error
🧐 1
^ correction:
ObjectNotFound
error on this line, I don't think the
run_deployment
command is recognizing my deployment ID... Maybe I didn't deploy properly? It's appearing in the Prefect Cloud UI... and I know I'm using the same ID ...
Unless the object not being recognized is the Prefect client... definitely haven not initialized that anywhere... not sure If i should be?
Copy code
prefect.exceptions.PrefectHTTPStatusError: Client error '404 Not Found' for url '<http://ephemeral-prefect/api/deployments/3e47a8cf-6055-404c-a2c3-146751ed24b9>'
Response: {'detail': 'Deployment not found'}
c

Chris White

09/02/2023, 4:02 AM
Oh! You haven’t set your local profile to point to the cloud API
If you create an API key and run prefect cloud login -k API_KEY that should set you up
g

Gabe Villasana

09/02/2023, 12:33 PM
ahhhh yes that was it! I had removed the prefect login from my docker build script. once i added it back, it worked 😄 Thanks a ton! @Chris White
🙌 1
2 Views