Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hi guys!
I have a small issue with Github storage with my flow.

My flow has grown to a point where I need to split tasks and functions into different files. My flow are stored in a flows folder (with __init.py) and I have submodules into the flows folder where I store my common functions.

When I try to register my flow, I get ModuleNotFoundError: No module named 'flows' when I use from flows.aws.s3 import
I get KeyError: "'__name__' not in globals" when I use from .aws.s3 import

Any idea?

Hi Louis, you have a few different options here:

• Store your shared tasks in a task library (which is installed in your execution environment, and kept separate from your flows). The repo structure would look like:
```$ tree
.
├── Dockerfile
├── flows
│   └── flow1.py
├── setup.py
└── shared_tasks
    ├── __init__.py
    └── s3.py```
You could then use `GitHub` storage to pull your flow source from github, while all your common tasks are pre-installed in the execution environment (usually a docker image). This works best if your shared tasks change slowly (maybe you have some common well-tested tasks that you rarely need to edit), but you make more frequent changes to your flows (since you store them in github, you don't need to rebuild your flow every change).

• The other option is to use `Module` storage, and install all your flows in your execution environment. Since you already have a `__init__.py` in your `flows` directory, you're almost there already. This lets you write your flows as you have been, but would require that you rebuild any images your using every time you change your flow. If that's not a problem, then this is the approach I'd recommend. The repo structure would look like:
```$  tree
.
├── Dockerfile
├── my_prefect_library
│   ├── __init__.py
│   ├── flows
│   │   ├── __init__.py
│   │   └── flow1.py
│   └── s3.py
└── setup.py```
If you import all your flows in `my_prefect_library/flows/__init__.py` , registration with the new CLI in 0.14.13 could be done as:

```$ prefect register --module my_prefect_library.flows --project my-project-name```

<@U011EKN35PT>! Thanks a lot for the detailed answers! I decided to go with option A, works like a charm!