Hello all! Is there a way to store arbitrary data on a flow, in such a way that you can get it back from the server (i.e. via

prefect describe flows ...

)? What I'm wanting to do is mark each flow with the git repo and commit from which the flow was registered. I'm thinking of situations where a flow run fails, so you want to see the source code; but if you're in an organization with many people making their own flows in different Git repos, you may not know which repo the flow is from, and so you want a way to easily find the source for a given flow. We've been asking people to manually add a link to the Git repo in every flow's "README" section in the Prefect Cloud UI, but as far as I can tell, you can't populate that README section via Python or commandline.

Doing this via README is actually quite a cool idea! One user from the community contributed a script to do that programmatically - see this Discourse topic. Another option would be to add this as an environment variable to your run configuration (those are displayed in the UI in the flow page):

flow.run_config = UniversalRun(env={"GIT_PATH": "<https://github.com/PrefectHQ/prefect/tree/orion>"})

Alternatively, this could be a bit tedious but you could leverage the parameter task to display such information in the UI, e.g. :

from prefect import Flow, Parameter

with Flow("your-flow") as flow:
    git_parameter = Parameter("git-path", default="<https://github.com/PrefectHQ/prefect/tree/orion>")
    flow.add_task(git_parameter)

The environment variable approach seems to be the easiest

Doing this via README is actually quite a cool idea! One user from the community contributed a script to do that programmatically - see this Discourse topic.

😵 Hitting the GraphQL directly?? That's very fancy!

Another option would be to add this as an environment variable to your run configuration (those are displayed in the UI in the flow page)

Ah ok, that's a much simpler approach, more in line with what I had in mind. Thank you! Edit: Ah, it seems the environment variable trick doesn't appear in

prefect describe flows -n my_flow

. But the parameter does. Out of curiosity, is there any plan (or would you consider) to add support for arbitrary metadata? Just a mapping from strings to strings, whose sole purpose is to live on a flow and be accessible via e.g.

flow.metadata

in Python, or

prefect describe flows -n my_flow | jq .metadata

?

Not in Prefect 1.0 but we could think about some way of attaching additional metadata in Prefect 2.0 - I'll forward this to the Product team

Ideally both UI and commandline.

Sure! If you had to choose one, which one would you consider more important? Probably UI correct?

🤔 Personally, I prefer commandline. And I tend to assume that getting something via an API will usually give me at least as much, if not more, information than a UI. I was fairly surprised the README was only in the UI! I think the most important part is that whatever this thing is (README or env var or parameter or generic metadata), we should be able to write it in an automatic way. That way, we can have a flow-registering script which we can run in a git repo, and have it automatically add the repo's URL and current commit to the flow, in a way that we can find it later.

Thanks for responding, so CLI seems to be your preference

Yes! 🙂

this is quite cool, indeed! well found, and thanks for sharing!