Apoorva Desai
04/14/2022, 12:20 AMApoorva Desai
04/14/2022, 12:39 AMAnna Geller
Apoorva Desai
04/14/2022, 3:33 AMsrc
on my docker container and yet prefect doesn't seem to be able to do it.
[2022-04-14T03:30:34.389Z] from src.prefect_state_handlers import FlowStateHandler
[2022-04-14T03:30:34.389Z] ModuleNotFoundError: No module named 'src'
Kevin Kho
Apoorva Desai
04/14/2022, 3:34 AMKevin Kho
ADD . .
would not copy source in right? Does it actually when you exec in?Apoorva Desai
04/14/2022, 3:35 AMApoorva Desai
04/14/2022, 3:35 AMpip freeze
during flow execution (without import src
) and see this src @ file:///root/prefect-docker
Apoorva Desai
04/14/2022, 3:37 AMls
during my flow execution and see all of those files listed thereKevin Kho
setup.py
?Apoorva Desai
04/14/2022, 3:39 AMfrom setuptools import setup, find_packages
with open('requirements.txt') as f:
requirements = f.read().splitlines()
setup(
name="src",
version='0.1.0',
description='Python library for use across repositories and prefect flows',
packages=find_packages(),
install_requires=requirements
)
Kevin Kho
Apoorva Desai
04/14/2022, 3:42 AMKevin Kho
Apoorva Desai
04/14/2022, 3:45 AMKevin Kho
cd somewhere else
in the container and try importing src
to make sure it’s really in the python path after the pip install because this might work if you just happen to be in the right directory alsoApoorva Desai
04/14/2022, 3:48 AMKevin Kho
Apoorva Desai
04/14/2022, 3:53 AMApoorva Desai
04/14/2022, 3:55 AMprefect backend server
prefect register --project data -p flows/
Apoorva Desai
04/14/2022, 3:55 AMApoorva Desai
04/14/2022, 3:55 AMflow.run_config = KubernetesRun(job_template=K8S_JOB_TEMPLATE, image = "******.<http://dkr.ecr.us-east-1.amazonaws.com/oh-data-utils:PR-1.16.b7ed902%22|dkr.ecr.us-east-1.amazonaws.com/oh-data-utils:PR-1.16.b7ed902">)
Apoorva Desai
04/14/2022, 3:56 AMKevin Kho
Kevin Kho
Apoorva Desai
04/14/2022, 4:06 AMwith Flow("everysundayweekly-flow", state_handlers=[statehandler.slack_failure_notification]) as flow:
So importing it in a task is causing the flow to not understand where it's coming from. How would you do this?Kevin Kho
Apoorva Desai
04/14/2022, 4:10 AMApoorva Desai
04/14/2022, 4:14 AMKevin Kho
pip install -e .
on your local machine so that it’s importable by the python installation. Or am I missing something?Kevin Kho
import src
with Flow("everysundayweekly-flow", state_handlers=[src.statehandler.slack_failure_notification]) as flow:
Apoorva Desai
04/14/2022, 4:38 AMKevin Kho
Kevin Kho
Apoorva Desai
04/14/2022, 4:46 AM