Hi. poetry add prefect[azure] is giving error. And I am guessing this is the reason, my flows are not working. Without azure they were working fine. What can be done here?

I don’t use poetry myself. Will find someone at Prefect to try to replicte this

ok

We can replicate. Looking into it

ok, i'll give this a try

You can just use install

pip install azure-storage-blob

instead of the whole azure suite because they have so many packages underneath that conflict with each other

this is already added. but the flow is giving this error:

Did you restart the agent after adding it?

it is on a local agent. and yes, i have restarted agent multiple times

That’s very weird. Did you move to conda or using vanilla pip?

i tried using vanilla pip.

can you run the agent with

--show-flow-logs

and

--log-level=DEBUG

to the agent and try running the flow?

I agree that you should install it with

pip install azure-storage-blob

to avoid dependency clashes. how far are you in the onboarding process with 1.0? if this is not working for you in 1.0, you could consider upgrading to Prefect 2.0 already 🙂 we manage dependencies more reliably in 2.0 using Prefect Collections - here is one for Azure

azure-storage-blob is already isntalled

and if you want to use 1.0 for this use case, I'd recommend trying a fresh new conda virtual environment with Prefect and then azure-storage-blob installed. Can you share your Flow object's configuration? do you use Azure for Storage only or for something else?

thx for sharing your Flow - I see the issue here. Since your Flow object is defined within a function, Prefect can't find it

i am using a POST call to run this

to explain why this is happening: by default Azure storage pickles your flow object and retrieves it at runtime from storage

how can I call the tasks if everything is not in a single function.

why would you want to register via a POST request?

we have a requirement, for which we need to create and execute flows (which will create resources in Azure using terraform). these flows need to br created and executed based on an API call

The thing is: every time you register your flow through

flow.register()

or

prefect register

CLI, you are doing it already via an API call. What exactly is the requirement? Is this about the audit trail of who registered a flow and when?

the requirement is - the azure resources have to be created on demand. and they can be created by users who are not well versed in azure. all they have to do is to make an PAI call, and the resources should get created.

I see, an interesting use case. It's up to you how you build it, but I wouldn't create a separate API for it because you can already leverage Prefect's API. Can you actually share more about your use case? (only if you can) it made me curious that some users may request Terraform-created resources - curious what is the actual problem here because it could be that you could solve your issue with a single parametrized flow and those users could just trigger those flows from Prefect UI

can I ping you separately about it?

I think we can continue here - you just don't need to share any personal/sensitive details - what do you think?

that sounds good Anna. let me see if I can get this working using parameterized flow