Hi all! Repeating my question since it didn't get any responses over the weekend. Second time lucky? What are the "local" development best practices? My setup is pretty straightforward I have a flow code and I store my secrets as blocks. Including for instance database DNS (but not only that) But when developing locally (or doing a hermetic test) I want to use my local DB instance (spinned up in a container but I don't think it really matters) so, is there any smart solution for that using blocks? Or I just parametrize the flow and when, say, the environment parameter is set to local, pick up the DB URI from environment var instead or loading it from a secret or something along the lines?

hi @Alex Litvinov so is it correct that you're running against a sqlite/postgres instance locally and would like to spin up a server in a container that can talk to your db? or you want to spin up a worker / flow run (e.g. client) in a container? there's no automatic solution for porting block instance between workspaces, but one could script that rather quickly if thats what you're interested in sorry if im not getting at the point, if im not, could you be more specific about what you want best practices for / what you're interested in doing?

a sqlite/postgres instance locally

it's a vectorDB (milvus in this case) but yea, I don't think it actually matters

or you want to spin up a worker / flow run (e.g. client) in a container?

a flow run

there's no automatic solution for porting block instance between workspaces, but one could script that rather quickly if thats what you're interested in

the prefect workspace is the same ie my prefect cloud account

sorry if im not getting at the point, if im not, could you be more specific about what you want best practices for / what you're interested in doing?

no worries, I think it's me not being extremely clear I'll try to rephrase my question with examples tomorrow Thanks @Nate!

ok sure! ill keep an eye out for the examples

hey! I was trying to figure things out and got lost again. I figure I just show you some code if you don't mind So, the parametrizing the flow with an environment question I kinda solved in a dumb-ish way just adding a parameter to the flow itself https://github.com/aaalexlit/faq-slack-bot/blob/main/ingest/ml/ingest_ml.py#L81 and using it just like this https://github.com/aaalexlit/faq-slack-bot/blob/main/ingest/ml/ingest_ml.py#L64 Also I thought that if I had several environments eg staging and prod, I could just have the secrets blocks containing the environment name and then pass it as a parameter doing approx this:

def connect_to_db(environment: str):
  db_uri = Secret.load(f'db-cloud-uri-{environment}').get()
  //connect to DB

is this a way people do it?

hmm okay - my approach when thinking about the same deploy on diff environments is to have the flow discover what it needs from a single

ENV_VAR

that I would set with a unique value in each environment, but yeah having some consistent naming scheme for blocks would certainly not be a bad idea to make this easier - i have seen people do similar things I'd be happy to help with specific questions you have here, don't have time at the moment to go deep into your use case. i would just say that ideally if i have a dockerized version of something like this, i would want it to work for whatever prefect server instance I'm pointing at, i.e. i can just inject

PREFECT_API_URL

at

docker run

time and it should just work

def connect_to_db():
  env = os.getenv('ENV_VAR', 'local')
  db_uri = Secret.load(f'db-cloud-uri-{env}').get()
  //connect to DB

to have the flow discover what it needs from a single

ENV_VAR

then in code it'd be something like this?

yeah that makes sense to me, i just said

ENV_VAR

as a placeholder name, but structurally thats exactly what i was thinking

i just said

ENV_VAR

as a placeholder name

yes-yes, I got it

yes-yes, I got it

haha yeah i figured, i tend to try and be overly verbose in text-only mediums to avoid ambiguity 🙂

i can just inject

PREFECT_API_URL

at docker run time and it should just work

yea, this does indeed work perfectly but that's not exactly what I want to achieve. My intention is to have a worker of docker type running somewhere Create a scheduled deployment (I mean all the point of creating a deployment here is to have it scheduled not to have to run it manually) that pulls my code, builds my image and runs it (on that docker worker) and also be able to parametrize it Here's where I'm kinda getting lost. I think I'm missing some fundamental link. This is my prefect.yaml Problem 1. Losing a parameter along the way Note: I was having this before I commited the code to git, when it was only in local Steps: 1. with the said prefect.yaml do

prefect deploy --all

2. then using said prefect.yaml do

prefect deployment run 'Flow name/deployment-name' -p parameter=value

Then what I see is 2 flow runs. The first one is

Worker 'DockerWorker bla-bla' submitting flow run 'blu-blu'

that has my parameter=value And the second one (the submitted

blu-blu

) that has no clue about the parameter I set. So, the question is (I guess) how do I pass this parameter to that flow that my deployment submits?

great to see you worked through it! feel free to ask more questions if you come across something

pfff, looks like I celebrated the victory way too soon. so, now that the code is committed, I have the same problem again https://prefect-community.slack.com/archives/CL09KU1K7/p1698259694217669?thread_ts=1698086507.729069&cid=CL09KU1K7 so, I don't understand why when the code is changed locally it works and everything is executed in one flow but when the code gets committed to the repo, there's a flow that spins up the other flow execution (and as a consequence the parameter that's intended to the "execution flow" gets lost.