https://prefect.io logo
Title
f

Ferdinand von den Eichen

01/27/2023, 11:51 AM
What is the easiest way to share company internal flows? Is it necessary to publish them to Pypi (a private one?) or are there other way?
c

Christopher Boyd

01/27/2023, 1:45 PM
Usually a code repository like github, or bitbucker, or in a Docker image
f

Ferdinand von den Eichen

01/30/2023, 8:39 AM
Sounds great! Could you point me to documentation on how to load tasks / flows from github dynamically?
c

Christopher Boyd

01/30/2023, 3:18 PM
Hi Ferdinand - our docs here have the various storage options available to load from: https://docs.prefect.io/concepts/storage/ Also, our discourse has some articles: https://discourse.prefect.io/t/prefect-collection-for-interacting-with-github-in-your-flows-prefect-github/1169
f

Ferdinand von den Eichen

01/31/2023, 9:57 AM
Hmm, that’s not really what I have in mind. Let me give some more background: In kubeflow we would have curated components (i.e. “tasks”) that we could share across all our pipelines by loading them at DAG construction time into the pipeline. These components would be loaded directly from GitHub. How would the same flow look like in Prefect? Let’s say you would have a task for submitting a job to AWS Glue and you want to use that in multiple pipelines?