Hey, we are thinking about developping a simple mvp web app that needs to pulls data from an external api and do a basic visualization. Do you think we can do it with prefect as the EL tool ? Even tho there is no real Load part as it will only consist of eventually parse/transform the response of the external API and directly pass it to the front. I was thinking about triggering the EL flow with the front end calling the Prefect API and the response would be the data collected by the flow. Is it possible or do you recommend another way ?

We say anything that you can do in Python, you can do with Prefect. So I don’t know the exact stack here, but yes we have people with this setup. What latency do you need to bring it to the frontend though? Prefect might be heavyweight for something you need immediately, but if the API just kicks off background jobs, then I’ve seen people use Prefect for that.

Hi thanks for your response, the latency would be a few seconds. I was thinking about calling the graphql API from the front end to start a flow with parameters and get the result of this flow as a response but i'm struggling on finding the doc of the graphql API. I saw something about mutations but couldn't find a lot about it.

The GraphQL API is mainly documentation in the Interactive API tab of the UI. You can find an example how to trigger a flow run here though. I don’t know that Prefect is ideal for a few seconds but you can try

Ok i see. But this API doesn't give me the result of the workflow as a response right ?

Not in Prefect 1.0. You need to know where the result is loaded and then load it in yourself. We have a

get_task_run_result

task that does it for you if you provide the flow_id and task slug (you can get the flow id from the request).

Okay so this is definetly not suited for what we want to do ! Thanks !

Yeah I think you want something like temporal.io, but they might not have Python

I want to fetch data from an external api to a front end without storing this external data. (Like getting a dataset preview from an api and display it on our front end). I'm sorry if it is not clear, i'm not familiar with full stack dev i have more a data science background :/

Ah ok Prefect will require some intermediate persistence with the result interface. You may just want to create an API directly?

Yeah that's what i was thinking in the end especially because it is for a POC. I considered Prefect because if the POC is accepted, then using it may be the most reliable solution to not get outscaled.