Wei Mei
05/16/2022, 4:19 PMKevin Kho
Wei Mei
05/16/2022, 5:56 PMKevin Kho
db-flow.py
imports from flow.py
Wei Mei
05/16/2022, 6:20 PMKevin Kho
Guilherme Petris
05/17/2022, 2:21 PMKevin Kho
Guilherme Petris
05/18/2022, 9:04 AMKevin Kho
working_dir
. But if you are using something Docker based, then they need to go inside the image.Guilherme Petris
05/25/2022, 2:17 PM❯ prefect diagnostics
{
"config_overrides": {
"context": {
"secrets": false
}
},
"env_vars": [],
"system_information": {
"platform": "macOS-12.3.1-x86_64-i386-64bit",
"prefect_backend": "cloud",
"prefect_version": "1.2.1",
"python_version": "3.9.10"
}
}
Code:
with Flow("zendesk_tickets_incremental") as flow:
latest_date = latest_date_unix()
incremental = incremental_call(latest_date)
incremental_df = create_df(incremental)
upload = upload_to_snowflake(incremental_df)
flow.storage = GitHub(
repo='X/prefect',
path='/zendesk/scripts/{FLOW_NAME}.py',
access_token_secret= Secret('GITHUB_ACCESS_TOKEN').get() # required with private repositories
)
flow.run_config = LocalRun()
flow.register("zendesk_project_test")
Kevin Kho
Guilherme Petris
05/25/2022, 2:40 PMKevin Kho
Guilherme Petris
05/25/2022, 2:49 PMKevin Kho
prefect agent local start --no-hostname-label
Guilherme Petris
05/25/2022, 3:06 PMState Message:
Failed to load and execute flow run: UnknownObjectException(404, {'message': 'Not Found', 'documentation_url': '<https://docs.github.com/rest/reference/repos#get-a-repository>'}
Kevin Kho
Guilherme Petris
05/25/2022, 3:14 PMaccess_token_secret= 'GITHUB_ACCESS_TOKEN'
Kevin Kho
flow.storage = GitHub(
repo='X/prefect',
path='/zendesk/scripts/{FLOW_NAME}.py',
access_token_secret= Secret('GITHUB_ACCESS_TOKEN').get() # required with private repositories
)
flow.storage.add_flow(flow)
flow.storage.get_flow()
to see if you can pull. You can avoid re-registration this way and test a lot faster I thinkGuilherme Petris
05/25/2022, 3:31 PMgithub.GithubException.UnknownObjectException: 404 {"message": "Not Found", "documentation_url": "<https://docs.github.com/rest/reference/repos#get-repository-content>"}
Kevin Kho