David
08/08/2024, 1:17 PMMarvin
08/08/2024, 1:17 PMFlow.deploy
method or a prefect.yaml
configuration file.
- Your CI/CD pipeline can build and push Docker images containing your flow code to Docker Hub, then deploy it to Prefect Cloud.
- Store sensitive information like PREFECT_API_URL
and PREFECT_API_KEY
as secrets in your repository for secure and non-interactive deployments.
You can find detailed documentation here:
CI/CD With Prefect
Testing Prefect Flows
- Prefect provides a context manager for unit testing which runs flows and tasks against a temporary local SQLite database.
- Example using `prefect_test_harness`:
from prefect import flow
from prefect.testing.utilities import prefect_test_harness
@flow
def my_favorite_flow():
return 42
def test_my_favorite_flow():
with prefect_test_harness():
assert my_favorite_flow() == 42
- This can also be used with `pytest`:
import pytest
from prefect import flow
from prefect.testing.utilities import prefect_test_harness
@pytest.fixture(autouse=True, scope="session")
def prefect_test_fixture():
with prefect_test_harness():
yield
@flow
def my_favorite_flow():
return 42
def test_my_favorite_flow():
assert my_favorite_flow() == 42
- Scoping the fixture to the session level is recommended for efficiency.
Detailed information on testing can be found here:
Testing Prefect Flows
A relevant code example can also be found below for concurrent tasks and flows:
Pefect Tasks Example
These resources should help with setting up and running CI pipelines while ensuring your Prefect flows are adequately tested.Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by