Anthony Finocchiaro

04/19/2023, 6:28 PM
Hi everyone, I was trying to figure out how to get my blocks that I set up in Prefect cloud when running through a docker container. It seems that I am not able to do this.. I was thinking I needed to be able to do this to get "Secrets" at run time (for passwords, etc). I am able to grab these when running on my Windows environment, but when trying to run in the environment in my docker container, it is giving this error for example: "ValueError: Unable to find block document named imdbpassword for block type secret". This goes for any block that I try to access when running the container, so I thought that it might be something with my Dockerfile. But I am able to print out my PREFECT_API_URL.. I have these commands as well in my Dockerfile, which I am pretty sure signs me in:
RUN az login --service-principal -u ${SP_ACCESS_KEY_ID} -p ${SP_SECRET_ACCESS_KEY} --tenant ${SP_TENANT_ID} \
    && /bin/sh -c "prefect config set PREFECT_API_URL=$(az keyvault secret show --name PREFECT-API-URL --vault-name secretsmanager --query value)" \
    && /bin/sh -c "prefect config set PREFECT_API_KEY=$(az keyvault secret show --name PREFECT-API-KEY --vault-name secretsmanager --query value)" \
    && /bin/sh -c "prefect block register -m prefect_azure" \
    && /bin/sh -c "prefect block register -m prefect_azure.container_instance"

Ryan Peden

04/20/2023, 6:17 AM
message you're getting when trying to load any block might mean your flow is running in ephemeral mode and not connecting to the API. After you build the container, if you run it in interactive mode with a command like
docker run -it my-org/my-image:latest bash
then then once you're in, run
prefect config view
do you see the PREFECT_API_URL you're expecting? If not, there might be an issue in your Dockerfile. The first place I'd check is the
az login
step. If you're not already passing the
variables into your Dockerfile as build args, you might need to.

Anthony Finocchiaro

04/27/2023, 5:09 AM
I was able to determine that this was basically the issue. I needed to make sure that the PREFECT_API_URL that I was grabbing was going to the right workspace. I was grabbing from the wrong keyvault on Azure in my dockerfile. It had to do with this line (now corrected below): && /bin/sh -c "prefect config set PREFECT_API_URL=$(az keyvault secret show --name IMATCH-PREFECT-API-URL --vault-name imatchprefectkeyvault --query value)"