@Marvin is there a way of defining sparse checkout in the prefect.yaml under the pull step?
m
Marvin
07/27/2024, 12:13 AM
It appears there isn't direct documentation or existing GitHub issues specifically about defining sparse checkout in the
prefect.yaml
file under the pull step.
However, you can customize the
git_clone
step within the
pull
section in
prefect.yaml
by specifying the necessary arguments and scripts. For sparse checkout, you might need to perform additional steps within the
pull
step to fetch only specific parts of a repository.
Since the direct capability might not be there, you might end up using custom scripts or commands within the
git_clone
step to handle sparse checkouts manually.
For detailed instructions and up-to-date methods, you may need to consult the Prefect Deployment Guide and possibly adapt with some custom scripting.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.