https://prefect.io logo
d

David Kuda

12/04/2020, 9:58 PM
Hello again! When I set flow storage to GitHub, I don’t understand what the desired behaviour is supposed to be. 1. Does that mean that I do not need to keep my files locally? In a way that prefect automatically creates a temporary directory, pulls the scripts from the given repository with the credentials, executes the scripts and then deletes the script again? 2. Or do I still need to keep files locally? Does then prefect pull the tasks automatically? As far as I understand, I do not need to re-register the flow, but I can change the tasks and then push them. Plus I seem to have encountered a bug in the tutorial here: https://docs.prefect.io/core/idioms/file-based.html -> At the very end of the very first example, I ran into an error. I follow everything until I need to register the flow with this command:
Copy code
prefect register flow -f flows/my_flow.py -p MyProject
This results in this error:
Copy code
ValueError: Flow could not be deserialized successfully. Error was: ValidationError({'storage': {'ref': ['Field may not be null.']}})
Best wishes from Berlin!
👀 1
This experiment has evolved into this: https://github.com/pnd-dkuda/prefect_github_flow/blob/main/flows/my_flow.py — I can register the flow with a request to the GraphQL API. But … I can’t run the flow. The UI shows me the tasks, but it’s not able to perform the tasks.
n

nicholas

12/04/2020, 11:31 PM
Hi @David Kuda - that's correct; your agent will pull the appropriate files from GitHub. Intra-task changes (changes to the content of tasks) won't require re-registration of the Flow, since each time it runs the flow will be pulled anew. Changes to the structure of the flow (dependency changes, new tasks, params etc) will require a new registration. Good catch on that bug! I've opened a PR to address that here: https://github.com/PrefectHQ/prefect/pull/3764