https://prefect.io logo
#prefect-community
Title
# prefect-community
r

rectalogic

07/21/2022, 3:03 PM
so flows work locally but
__file__
is not defined when prefect cloud runs the flow. Any idea what the issue is?
@Phillip Kelly
so it sounds like
__file__
is not useable with GitHub storage? https://github.com/PrefectHQ/prefect/issues/3529 What about other local assets that the flow needs to load that are in the github repo (e.g. local .sql files). Will they even be accessible to the flow, is prefect not checking out the repo when it runs the flow?
what about relative imports from other modules in my github repo? does the agent not checkout the entire github repository when running my flow?
unbelievable, I guess it's not supported at all. I really don't understand the reasoning behind most of the decisions prefect makes.. https://github.com/PrefectHQ/prefect/issues/4574
k

Kevin Kho

07/21/2022, 3:52 PM
Yes the Github Storage only gets the Flow file. We know this not user friendly so it is being re-designed for Prefect 2.0. For Prefect 1.0, you need to package the dependencies in a container.
Stuff like SQL files can be achieved with Git Storage here
r

rectalogic

07/21/2022, 4:10 PM
is there any way to make the Git Storage use
GITHUB_ACCESS_TOKEN
PAT env var? or does it have to use a prefect secret or ssh privkey?
k

Kevin Kho

07/21/2022, 4:14 PM
I think no. You need to use a Secret or the ssh key
r

rectalogic

07/21/2022, 8:03 PM
ok, we switched to Git storage and now
__file__
is correct e.g.
/tmp/tmpxy6wy7bw/cureatr/flows/...
- but we get
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpxy6wy7bw/cureatr/flows/patient_funnel/patient_funnel.sql'
when trying to open a file relative to
__file__
- that path seems correct. Do we have to do something specific to make Git storage clone the entire repo?
k

Kevin Kho

07/21/2022, 8:07 PM
On that, Prefect will clone the repo, load in the SQL, and then delete the files so it has to be read in outside of tasks when the script is ran
r

rectalogic

07/21/2022, 8:07 PM
ugh, ok. You guys sure don't make anything easy...
k

Kevin Kho

07/21/2022, 8:11 PM
Yeah we know Git storage can’t be the same form for sure for 2.0
6 Views