@Marvin Hey y'all, brand new to prefect, running into issues that I think stem from my lack of "general prefect workflow" knowledge. I'm building a backend API for a transcription service. I have a FastAPI application, and when someone calls "/v1/transcribe" with the proper arguments, I submit a run to the prefect API with a run_deployment() call. I then either wait for it to be completed or poll it internally until it's done and then try to get the results. I'm using the S3 block and getting

botocore.exceptions.NoCredentialsError: Unable to locate credentials

on the API, which means that my API is trying to get stuff from S3 directly. Shouldn't I just be getting the data from the prefect API? Am I using this wrong? Here is a minimal example of the API functionality:

class TranscribeService(AbstractTranscribeService):
    def transcribe(self, audio_link: str) -> TranscribeResponse:
        flow = run_deployment(
            name="engine/transcribe", parameters={"audio_link": audio_link}
        )
        raise_state_exception(flow.state)
        result = flow.state.result(raise_on_failure=True)
        return TranscribeResponse(result)

It seems like you’re having a bit of a bumpy ride with Prefect's workflow, but let's aim to smooth that out. From what you've described, and based on the relevant Prefect documentation, here are some insights and potential solutions: 1. Using S3 with Prefect: - It looks like your Prefect setup is attempting to access S3 without the necessary credentials. Prefect uses blocks to manage infrastructure configurations and secrets, like AWS credentials for S3. You'll need to set up an S3 block with the required credentials so Prefect can use them when interacting with S3. More about using blocks can be found here: Prefect Blocks Documentation. 2. Running Prefect with FastAPI: - Integrating Prefect with FastAPI seems to be generally correct if you're polling the Prefect API. It’s important to ensure that your

PREFECT_API_URL

is correctly configured if you're using Prefect Cloud or a self-hosted Prefect server. Make sure your FastAPI app can communicate with this Prefect API endpoint. For more on using FastAPI with Prefect, check here: FastAPI with Prefect. 3. Fetching Results: - When you run a deployment using

run_deployment

, Prefect captures execution logs and results. If the flow itself doesn't interact directly with S3 for fetching data, it should be managing that via Prefect’s result storage if correctly set up. Check how your flow is configured to save and fetch results. Ensure that the flow is aligned with the S3 credentials and setup you have within Prefect. In your case, it sounds like the credentials error is due to your API code's assumptions about data handling. You might need to revisit how your flow is supposed to communicate with S3 through Prefect. The Prefect documentation should provide the right pointers to configure this setup effectively. If you need more specific guidance or run into more issues, feel free to ask. And remember, unlike Marvin, I'm here to help you save the universe of Prefect workflows, not just complain about it.

hi @matanya - do you need remote execution?

run_deployment

is just a utility to hit the

POST /create_flow_run_from_deployment

and poll for completion (if you want) but if you're polling, you can also just call the decorated function if you do need remote execution and you want to fetch results, you need to • setup 3rd party

result_storage

like s3 (or via setting:

PREFECT_DEFAULT_RESULT_STORAGE_BLOCK=s3-bucket/test

) • set

persist_result=True

on your task (or via setting:

PREFECT_RESULTS_PERSIST_BY_DEFAULT=1

)