Alastair
12/19/2025, 4:25 AMdjango.setup() at startup before running the rest of a flow?Marvin
12/19/2025, 4:25 AMMarvin
12/19/2025, 4:29 AMpython -m prefect.engine). Any Python you run in the worker process runs outside the child process. To run django.setup() before your flow code, use one of these supported patterns:
Recommended patterns
- Put Django init in your flow code (simple and reliable)
- Set DJANGO_SETTINGS_MODULE (via job variables or env) and call django.setup() at the top of your flow (or import a bootstrap module that does it).
from prefect import flow
@flow
def my_flow():
import os, django
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")
django.setup()
# now import and use your Django models safely
from myapp.models import User
print(User.objects.count())
Deploy with a process pool and set the env once so you don’t hardcode settings:
my_flow.deploy(
name="django-flow",
work_pool_name="process-pool",
job_variables={
"env": {
"DJANGO_SETTINGS_MODULE": "myproject.settings"
}
},
)
- Bootstrap module import (centralize the setup)
- Create a tiny module that runs on import, then import it at the top of any flow module.
# myproject/prefect_django_setup.py
import os, django
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")
django.setup()
# flows/my_flow.py
import myproject.prefect_django_setup # ensures setup happens in the flow-run process
from prefect import flow
from myapp.models import User
@flow
def my_flow():
print(User.objects.count())
Pool-level centralization (advanced)
- Override the job command to a wrapper that initializes Django, then runs the Prefect engine
- Since the worker spawns python -m prefect.engine, the supported way to run pre-code in the same process is to replace that command with your own module that calls django.setup() and then invokes the Prefect engine logic. This is done by setting the command job variable for your process work pool’s job template or per deployment.
- Example job variable:
job_variables={
"command": "python -m mypkg.prefect_django_entrypoint",
"env": {"DJANGO_SETTINGS_MODULE": "myproject.settings"}
}
- Your mypkg.prefect_django_entrypoint module should initialize Django and then programmatically invoke the Prefect engine. This approach is more advanced and couples you to Prefect’s engine entrypoint; for most teams the “init in flow code” approach is simpler.
About subclassing ProcessWorker
- You can define a custom worker type, but it won’t let you directly execute Python inside the child process unless you also change the job’s command. A custom worker can:
- enforce env vars (e.g., set DJANGO_SETTINGS_MODULE)
- set a custom command that points to your wrapper module (see above)
- If you go this route, create a worker plugin that extends the process worker and sets the job configuration accordingly. This is an advanced pattern best used when you need org-wide enforcement through a custom worker type.
Key takeaways
- Running django.setup() must happen inside the flow-run subprocess.
- The simplest, most robust solution is to initialize Django in your flow (or a shared bootstrap module you import).
- If you must centralize at the pool level, override the job command to a wrapper module that runs django.setup() before launching the Prefect engine.
Useful references
- Process worker source (how command/env are applied): prefect/workers/process.py
- Worker base and job configuration (customization points): prefect/workers/base.pyMarvin
12/19/2025, 4:29 AM