Hey all, i’m running into a

ModuleNotFoundError

from a flow trying to import a custom .py file. Code is being executed in a gcp cloud run job + using github storage and i’m also using dask to parallelize this particular task. Oddly enough, this error (same import, the import occurs from within the task, maybe this is not a good idea?) does not occur in any of the previous tasks (they do not use dask to parallelize). So the

ModuleNotFoundError

happens to only occur within the task that harnesses Dask. Has anyone ran into a similar issue or has advice on this? Thanks! some flow code as context:

@task(
    task_run_name="get-most-recent-product-date/{type}",
    description="Get the date of current most recent product.",
)
def get_most_recent_product_date(store: str) -> str:
    from src.flows.ner.sql.queries import retrieve_most_recent_ner_product_date_select_store as most_recent_query #error with this query import
    ...


@flow(
    task_runner=DaskTaskRunner,
    persist_result=True,
    result_storage=...,
    name=...,
    description=...,
)
def run_ner_on_product_data(stores: Sequence[str]):
    ...
    ...
    ...
    most_recent_dates = get_most_recent_product_date.map(store=stores) # error with this task

So the import above occurs in other tasks but the other tasks do not get called via

.map()

and the error does not occur for those.

Does it say which module isn’t found?

@Alexander Azzam oops sorry:

ModuleNotFoundError: No module named 'src'

So i just tested moving these imports to the top of the file as opposed to individually in tasks- it resolved the issue.

Sweet!