Gaylord Cherencey
09/09/2021, 5:48 AMFileNotFoundError, No such file or directory
while using Git storage to a private git server. I set Secret as ENV and specify git_token_secret_name
. Does this mean that the agent can connect to the repository but don't find the Flow python file?Kevin Kho
Gaylord Cherencey
09/09/2021, 8:57 PMGaylord Cherencey
09/09/2021, 9:02 PMflow.storage = Git(repo="<my_org>/dt-prefect-demo",
flow_path="/examples/flows-composition/etl.py",
branch_name='PAI-3025',
git_token_secret_name='GITHUB',
repo_host='<my_org_server_address>')
Am I missing something here?Kevin Kho
Gaylord Cherencey
09/09/2021, 9:08 PMKevin Kho
Kevin Kho
GitStorage
class directly here . You can try something like:
test = Git(repo="<my_org>/dt-prefect-demo",
flow_path="/examples/flows-composition/etl.py",
branch_name='PAI-3025',
git_token_secret_name='GITHUB',
repo_host='<my_org_server_address>')
test.get_flow()
Gaylord Cherencey
09/09/2021, 9:19 PMGaylord Cherencey
09/13/2021, 6:03 AMget_flow()
I still have ValueError: Flow is not contained in this Storage
but when I do add_flow
I get the FileNotFoundError
. I fixed this by changing this line in the storage class to '/'.join([temp_repo.temp_dir.name, self.flow_path])
but it doesn't change anything when I push the flow to Prefect Cloud I still end up with the file not found. Any suggestions from here?Zach Angell
/
from flow_path
? Providing an absolute path here will force python to read from that path instead of the temporary directory
os.path.join(os.getcwd(), '/my/dir') # returns 'my/dir'
os.path.join(os.getcwd(), 'my/dir') # returns /current/path/and/then/my/dir
Gaylord Cherencey
09/13/2021, 9:12 PMfrom prefect.storage import Git
from prefect import context
storage = Git(repo="<org>/<repo>",
flow_path="examples/flows-composition/etl.py",
branch_name='<branch_name>',
git_token_secret_name='GITHUB',
repo_host='<githib_hostname>',
stored_as_script=True)
with Flow("ETL", run_config=KubernetesRun(image="prefecthq/prefect:latest-python3.7")) as flow:
e = extract()
t = transform(e)
l = load(t)
flow.storage = storage
flow.register(project_name='project-a')
I was wondering is stored_as_script
parameter mandatory?Zach Angell
Gaylord Cherencey
09/13/2021, 9:16 PM