pk13055
11/02/2022, 10:09 AMorion
+ agent
container setup locally, and everything is working as expected when I try to run the flows manually. I was able to successfully create and apply a deployment as well (from within the cli
container). However, when trying to run the deployment I am faced with a File not found error
.
Here's the traceback:
Flow could not be retrieved from deployment.
Traceback (most recent call last):
File "<frozen importlib._bootstrap_external>", line 879, in exec_module
File "<frozen importlib._bootstrap_external>", line 1016, in get_code
File "<frozen importlib._bootstrap_external>", line 1073, in get_data
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp_zx6qpglprefect/flows/fetch_etfs.py'
My docker-compose
is as follows:
version: "3.9"
services:
db:
image: timescale/timescaledb:latest-pg14
volumes:
- $PWD/data/db:/var/lib/postgresql/data
- $PWD/config/db:/docker-entrypoint-initdb.d/
healthcheck:
test: [ "CMD-SHELL", "pg_isready" ]
interval: 10s
timeout: 5s
retries: 5
environment:
- POSTGRES_DB=$POSTGRES_DB
- POSTGRES_USER=$POSTGRES_USER
- POSTGRES_PASSWORD=$POSTGRES_PASSWORD
networks:
- db_network
orion:
image: prefecthq/prefect:2.6.3-python3.10
restart: always
volumes:
- $PWD/prefect:/root/.prefect
entrypoint: [ "prefect", "orion", "start" ]
environment:
- PREFECT_ORION_API_HOST=0.0.0.0
- PREFECT_ORION_DATABASE_CONNECTION_URL=$PREFECT_DB_URL
ports:
- 4200:4200
depends_on:
- db
networks:
- prefect_network
- db_network
agent:
build: ./prefect
restart: always
entrypoint: [ "prefect", "agent", "start", "-q", "main_queue" ]
environment:
- PREFECT_API_URL=<http://orion:4200/api>
networks:
- prefect_network
- db_network
cli:
build: ./prefect
entrypoint: "bash"
working_dir: "/root/flows"
volumes:
- "$PWD/prefect:/root"
environment:
- PREFECT_API_URL=<http://orion:4200/api>
networks:
- prefect_network
- db_network
networks:
db_network:
web_network:
prefect_network:
To create and apply the deployment(s):
> docker compose run cli
$ prefect deployment create ./flows/fetch_etf.py:flow_name --name flow_name
$ prefect deployment apply deployment.yaml
Ryan Peden
11/02/2022, 1:35 PMbuild
, though. Does the Dockerfile you use to build these copy the flow code into the container image?pk13055
11/02/2022, 7:48 PMcli
volume to agent
as well, and it's working great now! Thanks