Adam
07/05/2025, 6:59 AMMarvin
07/05/2025, 6:59 AMMarvin
07/05/2025, 6:59 AMyaml
description: |
Data pipeline that performs dbt transformations and exports results via SFTP.
Pipeline Steps:
1. Executes dbt transformations on source data
2. Exports transformed data files to SFTP destination
Technical Details:
- Uses dbt for data transformation
- Leverages SFTP for secure file transfer
- Database: t4g.small instance
Dependencies:
- Requires dbt project configuration
- Requires SFTP credentials and connection details
This description follows best practices by:
1. Providing a clear, high-level summary
2. Breaking down the main steps
3. Including technical specifications
4. Listing key dependencies
You can use this description in your deployment configuration, either via the UI or in your prefect.yaml file. When using YAML, make sure to maintain proper indentation and use the pipe operator (|
) for multi-line descriptions as shown above.
I've included your database instance type (t4g.small) in the description since that's a relevant technical detail. You can further customize this template by adding:
- Specific dbt models being run
- SFTP destination details
- Schedule information
- Any retry policies or timeout configurations
Remember that in Prefect 3.x, you can apply this description when creating your deployment using the flow.from_source(...).deploy(...)
pattern.