Sam Luen-English
11/30/2020, 9:40 PMFROM python:3.8
RUN python -m pip install sqlalchemy prefect
main.py
from prefect import Flow, task
from prefect.environments.storage import Docker
import sqlalchemy
@task
def some_task():
print(sqlalchemy.__version__)
with Flow("Some Flow") as flow:
some_task()
if __name__ == "__main__":
flow.storage = Docker(
registry_url="<http://docker.io|docker.io>",
image_name="storage_image",
image_tag="latest",
base_image="test-base",
local_image=True,
)
flow.register(project_name="Development")
Test script:
#!/usr/bin/env bash
set -x
docker build . -t test-base
#: Approach 1
python main.py
#: Approach 2
docker run -v /var/run/docker.sock:/var/run/docker.sock -v ~/.docker:/root/.docker -e PREFECT__CLOUD__AUTH_TOKEN="$PREFECT__CLOUD__AUTH_TOKEN" -v $(pwd)/main.py:/main.py test-base bash -c 'python /main.py'
Please could you guide me with the best approach?
Many thanks!Kyle Moon-Wright
11/30/2020, 10:09 PMSam Luen-English
11/30/2020, 10:24 PMKyle Moon-Wright
11/30/2020, 10:51 PMWalt Wells
12/01/2020, 12:06 AMSkip Breidbach
12/01/2020, 12:38 AMSam Luen-English
12/01/2020, 11:19 AMJoachim Zaspel
02/08/2021, 8:26 PMSam Luen-English
02/19/2021, 10:39 AMversion: "3"
services:
prefect:
build: .
image: my-project-base
env_file:
- docker.env
volumes:
- ./project-dir/:/project-dir
- /var/run/docker.sock:/var/run/docker.sock
- ~/.docker:/root/.docker
So my image is built and tagged as "my-project-base"
flow.storage = Docker(
registry_url="my-registry",
image_name="my-project",
base_image="my-project-base",
local_image=True,
)
The above code is contained in the "project-dir" shared with the container, and run from within the container.
I think that is all the logic required. Let me know if you have any issues :)