). I'm now trying to set up an agent on my local machine which would be able to communicate with the server on EC2, so I can execute and monitor the flow from the UI. When I try to run
on my local machine, it fails with
prefect agent start local --api <http://my.ec2.prefect.ip:4200/graphql>
. In the server UI I don't see any options to create tokens and if I try to run
No agent API token provided
it fails with
prefect auth create-token
, I guess because generating tokens is a Cloud-only feature. So my question is: how would I be able to set up a relationship between my local machine, where the flow would be running, and the server, without using Prefect Cloud? If that's not possible, would I be able to have the flow present on the same machine as the server and just use
Error: Auth commands with server are not currently supported
in my flow script to have it managed by the server, or are there more steps involved?
which I point prefect via the env var:
My config.toml looks like (obviously with host name actually filled in):
[server] [server.ui] graphql_url = "<host-name>:4200/graphql"
and visit host:8080 I get the front page of the ui asking to connect to host:4200/graphql. Ideally it would be good if this could still be connected automatically using the
prefect server start
Any advice would be greatly appreciated. Please let me know if you need anymore details I might have missed
Any idea what I am missing?
Traceback (most recent call last): File "urllib3/connectionpool.py", line 677, in urlopen File "urllib3/connectionpool.py", line 392, in _make_request File "http/client.py", line 1252, in request File "http/client.py", line 1298, in _send_request File "http/client.py", line 1247, in endheaders File "http/client.py", line 1026, in _send_output File "http/client.py", line 966, in send File "docker/transport/unixconn.py", line 43, in connect PermissionError: [Errno 13] Permission denied
Operating System: Ubuntu 18.04.5 LTS Kernel: Linux 5.4.0-42-generic Architecture: x86-64 Using pyenv with python 3.7.9 Thanks in advance!
ERROR: Can't find a suitable configuration file in this directory or any parent. Are you in the right directory? Supported filenames: docker-compose.yml, docker-compose.yaml ............ ............ subprocess.CalledProcessError: Command '['docker-compose', 'down']' returned non-zero exit status 1.
# Dockerfile FROM prefecthq/prefect:0.13.5-python3.8
Everything is working as expected when using (i) the local docker-compose server and UI, (ii) a minimal flow
docker build -t prefect-base .
and (iii) a docker agent running via
# register.py import prefect from prefect import task, Flow from prefect.environments.storage import Docker @task def hello_task(): logger = prefect.context.get("logger") <http://logger.info|logger.info>("test message") flow = Flow("prefect-base", tasks=[hello_task]) base_image = "prefect-base:latest" flow.storage = Docker(base_image=base_image, local_image=True) flow.register(project_name="test")
However, when I change to:
prefect agent start docker --no-pull -v
I'm getting an error (upon triggering a run in the UI) of the form:
prefect agent start docker --no-pull -v --show-flow-logs
[2020-09-09 13:21:44,229] DEBUG - agent | Updating states for flow run 457ba6ff-9a8e-4117-b563-166720de3ba9 [2020-09-09 13:21:44,230] DEBUG - agent | Flow run 457ba6ff-9a8e-4117-b563-166720de3ba9 is in a Scheduled state, updating to Submitted [2020-09-09 13:21:44,262] INFO - agent | Deploying flow run 457ba6ff-9a8e-4117-b563-166720de3ba9 [2020-09-09 13:21:44,263] DEBUG - agent | Creating Docker container prefect-base:2020-09-09t13-21-12-637383-00-00 [2020-09-09 13:21:44,315] DEBUG - agent | Starting Docker container with ID 4e2404b2148c82b43ec9185d2c553ec1dc8b6ad8a653267c54664f7eae28d77c [2020-09-09 13:21:44,483] DEBUG - agent | Querying for flow runs [2020-09-09 13:21:44,501] DEBUG - agent | No flow runs found [2020-09-09 13:21:44,501] DEBUG - agent | Next query for flow runs in 0.5 seconds [2020-09-09 13:21:44,523] ERROR - agent | Logging platform error for flow run 457ba6ff-9a8e-4117-b563-166720de3ba9 [2020-09-09 13:21:44,568] ERROR - agent | Error while deploying flow: AttributeError("Can't pickle local object 'Agent.run_heartbeat_thread.<locals>.run'") [2020-09-09 13:21:44,568] DEBUG - agent | Completed flow run submission (id: 457ba6ff-9a8e-4117-b563-166720de3ba9)