hey guys, I’m facing a kind of an obscure error using running a flow with GitHub storage…it seems like I’m downloading the files successfully, but then the flow fails for not finding a certain file…is there a way to produce more verbose logs so I can know exactly what file it fails on?
z
Zach Angell
09/12/2021, 3:15 PM
Hi @Nadav Nuni - setting log level to DEBUG for the flow run might be helpful.
The most common cause of this error is trying to load other files from within your flow file. For example, something like
Copy code
from prefect import task, Flow
# this will fail because the file is
# not available when the flow is loaded from storage
with open('my_config.toml', 'r') as f:
my_config = f.read()
# ... meaningless boilerplate flow code
@task
def foo():
return my_config
with Flow('foo') as flow:
foo()
Zach Angell
09/12/2021, 3:15 PM
Any chance you're doing something like the above in your flow file?
n
Nadav Nuni
09/13/2021, 8:22 AM
Thanks @Zach Angell!
I’m actually not accessing any files directly…could it be the
KubernetesRun.template_path
or
storage.GitHub.path
?
I’ll try to change the logging level and get more clues
👍 1
z
Zach Angell
09/13/2021, 1:40 PM
Hmm perhaps. If you can share a minimum example of your flow that would help narrow things down
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.