Greg Roche
01/19/2021, 2:45 PMLocalDaskExecutor
flow, using a LocalAgent running inside a Docker image? TypeError: start() missing 1 required positional argument: 'self'
Edit: solved, I wasn't initialising the LocalDaskExecutor.
flow.executor = LocalDaskExecutor # wrong
flow.executor = LocalDaskExecutor() # this works
[2021-01-19 11:30:59+0000] INFO - prefect.CloudFlowRunner | Beginning Flow run for 'hello-world'
[2021-01-19 11:30:59+0000] ERROR - prefect.CloudFlowRunner | Unexpected error: TypeError("start() missing 1 required positional argument: 'self'")
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/prefect/engine/runner.py", line 48, in inner
new_state = method(self, state, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/prefect/engine/flow_runner.py", line 418, in get_flow_run_state
with self.check_for_cancellation(), executor.start():
File "/usr/local/lib/python3.8/contextlib.py", line 240, in helper
return _GeneratorContextManager(func, args, kwds)
File "/usr/local/lib/python3.8/contextlib.py", line 83, in __init__
self.gen = func(*args, **kwds)
TypeError: start() missing 1 required positional argument: 'self'
Here's an example flow:
from prefect import Flow, task
from prefect.executors import LocalDaskExecutor
from prefect.storage import S3
from prefect.utilities.logging import get_logger
logger = get_logger()
@task
def say_hello():
<http://logger.info|logger.info>("Hello, world!")
with Flow("hello-world", storage=S3(bucket="my-s3-bucket")) as flow:
hello = say_hello()
flow.register(project_name="test") # flow runs successfully when triggered by Prefect Server
flow.executor = LocalDaskExecutor
flow.register(project_name="test") # TypeError: start() missing 1 required positional argument: 'self'
I've tried stripping back the Dockerfile to just the essentials, here's a minimal setup which still causes the problem above:
FROM prefecthq/prefect:latest-python3.8
WORKDIR /my_project
COPY ./my_project_etl ./my_project_etl
COPY requirements.txt .
COPY setup.py .
COPY agents/start_agent.sh .
COPY agents/aws_config /root/.aws/config
# needed to build psycopg2
RUN apt-get update
RUN apt-get install -y libpq-dev
RUN pip install -e .
ENTRYPOINT ["/my_project/start_agent.sh"]
start_agent.sh is just this:
#!/bin/bash
prefect backend server
prefect agent local start --show-flow-logs --api <http://xxx.xxx.xxx.xxx:4200/graphql> --label docker-test
josh
01/19/2021, 2:56 PMflow.executor = LocalDaskExecutor()
Greg Roche
01/19/2021, 3:00 PM