Hello - I am having an issue getting my flows to run from the server after registering them. I can run them by calling

flow.run()

but if I

flow.register(name="Task Name")

and then try to run via the UI it fails. The message I get is:

Failed to load and execute Flow's environment: ImproperlyConfigured('Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.')

I feel like I have done exactly this setup before with Prefect and it has worked fine, but now it is not working. I see it is related to django not being setup, but can't understand why it isn't...I guess because I don't complete understand how Prefect works. A simple example that demonstrates the issue is as follows:

import django
import os
import prefect

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'afs.settings')
django.setup()

from prefect import Flow, task
from api.models import Location # This is a django model import


@task()
def django_task():
    locs = Location.objects.all()
    logger = prefect.context.get("logger")
    <http://logger.info|logger.info>(f"Number of Locations: {len(locs)}")

with Flow(f"Django Task") as flow:
    locs = django_task()


# flow.run()
flow.register(project_name="Kerala")

Searching the Slack channel for Django returned one result which indicated the maybe using a resource manager was the way to set this up, but I wasn't able to get it to work using resource manager either. The other thing I tried was to set DJANGO_SETTINGS_MODULE via a LocalRun:

flow.run_config = LocalRun(env={'DJANGO_SETTINGS_MODULE': 'afs.settings'})
flow.register(project_name="Kerala")

But that results in another error about INSTALLED_APPS not being configured...clearly I am missing something. Last thing, I also added the following to a a separate non django task, and it looks like it is using the correct virtual env to execute the task and has the correct paths.

<http://logger.info|logger.info>(sys.path)
<http://logger.info|logger.info>(sys.version)
<http://logger.info|logger.info>(sys.executable)

Really stumped... Any help would be greatly appreciated.

Hi Matt, I'm not familiar with Django, but the first example shows a few misunderstandings with Prefect that might help you debug further. By default when you build a flow, the flow's source is pickled and stored somewhere (in your case in a local file). This means that any changes made in the top-level of your flow script won't occur when the flow is loaded later (since they're not part of the flow itself). In your example above this means that: • The

os.environ.setdefault

call won't run • The

django.setup

call won't run Usually we advise users to encode things that need to run for their flow to succeed as tasks. This ensures that they're run as part of the flow (regardless of where it runs), and if they fail the failure can be tracked like any other prefect task. In your case, you might put the

django.setup

call in a task and setup task dependencies such that other django tasks only run after that call. You might make use of a

resource_manager

(https://docs.prefect.io/core/idioms/resource-manager.html) to do this, but it wouldn't be required. For environment variables, using

env

in a run config is the recommended approach, so what you've done there should work fine. There might be other django stuff required though (I'm not familiar with django). Your other option is to use file-based flow storage. This means your flow won't be pickled, but the source will be reloaded at runtime (and any top-level functions run as normal). For local storage you'd pass in

path

and

stored_as_script=True

. Both options are valid. If you never expect

django.setup

to fail then they should be equivalent. Things that fail though should really be encoded as tasks to best take advantage of prefect.

@Jim Crist-Harif Thanks so much for the feedback. That clears things up a bit about how this all fits together. For now it seems that store_as_script might be the way to go because we have quite a bit of code already written that would require refactoring. Moving forward we will have to reevaluate our approach but this seems to work for now. Thanks again.

No problem, hope you get things working!

This conversation really got me started going in the right direction to get Django and Prefect to work together. I documented how I got them to work here in case you are still working on it (and for future searchers): https://github.com/PrefectHQ/prefect/discussions/4163