David
08/08/2024, 1:17 PMMarvin
08/08/2024, 1:17 PMFlow.deploy
method or a prefect.yaml
configuration file.
- Your CI/CD pipeline can build and push Docker images containing your flow code to Docker Hub, then deploy it to Prefect Cloud.
- Store sensitive information like PREFECT_API_URL
and PREFECT_API_KEY
as secrets in your repository for secure and non-interactive deployments.
You can find detailed documentation here:
CI/CD With Prefect
Testing Prefect Flows
- Prefect provides a context manager for unit testing which runs flows and tasks against a temporary local SQLite database.
- Example using `prefect_test_harness`:
from prefect import flow
from prefect.testing.utilities import prefect_test_harness
@flow
def my_favorite_flow():
return 42
def test_my_favorite_flow():
with prefect_test_harness():
assert my_favorite_flow() == 42
- This can also be used with `pytest`:
import pytest
from prefect import flow
from prefect.testing.utilities import prefect_test_harness
@pytest.fixture(autouse=True, scope="session")
def prefect_test_fixture():
with prefect_test_harness():
yield
@flow
def my_favorite_flow():
return 42
def test_my_favorite_flow():
assert my_favorite_flow() == 42
- Scoping the fixture to the session level is recommended for efficiency.
Detailed information on testing can be found here:
Testing Prefect Flows
A relevant code example can also be found below for concurrent tasks and flows:
Pefect Tasks Example
These resources should help with setting up and running CI pipelines while ensuring your Prefect flows are adequately tested.