Kaleb King
06/23/2024, 1:56 AMMarvin
06/23/2024, 1:57 AMpython
from prefect.filesystems import GitHub
block = GitHub(
repository="<https://github.com/my-repo/>",
access_token=<my_access_token> # only required for private repos
)
- Accessing Subfolders: You can specify a subfolder of the repository to access specific parts of it:
python
block.get_directory("folder-in-repo")
### Deployment Use
To use the GitHub storage block in a deployment:
shell
prefect deployment build path/to/flow.py:flow_name --name deployment_name --tag dev -sb github/dev -a
This command-line instruction references the storage block github/dev
which you created earlier.
### Repository Accessibility
- Whole Repository Accessible: When you reference a GitHub repository using a storage block, the whole repository becomes accessible unless you specify particular subdirectories or use .prefectignore
to exclude certain files or directories.
- Exclusion of Files: You can exclude files similarly to .gitignore
by using .prefectignore
.
### References
- Prefect Storage Guide
- Prefect Storage Concepts
If you have further questions or need additional explanations, feel free to ask.Kaleb King
06/23/2024, 3:22 AMprefect build...
command instead of prefect deployment build...
, how can I specify an existing GitHub storage block?Marvin
06/23/2024, 3:22 AMprefect build
command and specify an existing GitHub storage block in Prefect, you'll follow these general steps:
1. Install the Required Libraries:
Make sure to install the relevant library:
sh
pip install prefect-github
2. Register the GitHub Block:
Register the GitHub block with Prefect:
sh
prefect block register -m prefect_github
3. Create and Configure the GitHub Block:
Create a GitHub storage block in the Prefect UI or via code with the repository details:
python
from prefect.filesystems import GitHub
block = GitHub(
repository="<https://github.com/my-repo/>",
access_token=<my_access_token> # only required for private repos
)
block.save("my-github-block-name")
4. Use the Block in the prefect build
Command:
When building the deployment using the prefect build
command, include the GitHub storage block reference:
sh
prefect build path/to/flow.py:flow_name --name deployment_name --tag dev -sb github/my-github-block-name -a
### Example of prefect build
Command
sh
prefect build src/flows/my_flow.py:my_flow_function --name my_deployment --tag dev -sb github/my-github-block-name -a
This command specifies the storage block github/my-github-block-name
which you have created and registered earlier.
### Reference Documentation
- For additional configuration and deeper specifics, you can refer to the Prefect Storage Guide.
Feel free to ask if you have additional questions or require further details.