Hello all. I moving my prefect local deployment to a remote machine for production using docker compose, and am struggling very hard to get the remote file system configured correctly. I have structured the new prefect environment to be identical to the one located here https://github.com/rpeden/prefect-docker-compose. I have everything working up to the minio storage setup. I was able to launch the minio server and create a user, a bucket, and generate access keys. I then added a remote file system block to the UI with a settings configuration that looks like so:

{
  "key": "my_key",
  "secret": "my_secret",
  "client_kwargs": {
    "endpoint_url": "<http://localhost:9000>"
  }
}

Once I created the block, it told me to paste this snippet into my flow, so I did.

from prefect.filesystems import RemoteFileSystem

remote_file_system_block = RemoteFileSystem.load("scrape-flow-block")

At this point, I should have everything I need to build a deployment. So, I get into the cli container by running docker-compose run cli and run the following command:

prefect deployment build -sb "remote-file-system/scrape-flow-block" -n "Scrape Deployment" -q "default" "flow.py:scrape_flow"

Where the remote file system block I created is called "scrape-flow-block", "default" is my queue name, flow.py is my flow script and scrape_flow is the folder which flow.py is in. When I run this line, I know that the command is successfully accessing the script for the deployment build because at first, I get errors for missing packaging, such as "ModuleNotFoundError("No module named 'pandas'"). However, once I get these all sorted out, the error I am stuck on is

Script at 'flow.py' encountered an exception: NoCredentialsError('Unable to locate credentials')

I have clearly defined the credentials in the prefect block and loaded the prefect block in the script the exact way the documentation explains. I could understand an incorrect credentials message, but unable to locate them makes no sense. I am confused by this error because 1) Why does the deployment search the script for credentials and not the block, what is the point of passing the block through the deployment line if you have to manually call the block through the script anyways 2) Why does the "load block" command given by the prefect UI to paste into the script not load the block correctly in this instance. It seems like it is search the script for credentials it should already have. Even if this is expected behavior, how does it know what variables in the scripts are the desired credentials. This topic is a bit confusing and not very well documented in the docs, so any help would be greatly appreciated. Thanks.

Hey Kyler! Just wanted to check in since we last spoke

I haven't been able to work on it since our discussion, some other work came up, but I am hoping to get to it on monday and will dm you if I have any issues. I appreciate the help from our meeting I think I should be able to get more info on the issue following the plan we discussed

Sweet! Appreciate the update