Matt Allen
09/02/2020, 7:02 PMCharles Leung
09/02/2020, 7:49 PMCharles Leung
09/02/2020, 8:38 PMGreg Roche
09/04/2020, 3:57 PMserver
). I'm now trying to set up an agent on my local machine which would be able to communicate with the server on EC2, so I can execute and monitor the flow from the UI.
When I try to run prefect agent start local --api <http://my.ec2.prefect.ip:4200/graphql>
on my local machine, it fails with No agent API token provided
. In the server UI I don't see any options to create tokens and if I try to run prefect auth create-token
it fails with Error: Auth commands with server are not currently supported
, I guess because generating tokens is a Cloud-only feature.
So my question is: how would I be able to set up a relationship between my local machine, where the flow would be running, and the server, without using Prefect Cloud? If that's not possible, would I be able to have the flow present on the same machine as the server and just use flow.register(project_name="project_name_in_prefect_server")
in my flow script to have it managed by the server, or are there more steps involved?Charles Leung
09/04/2020, 9:55 PMHarry Nguyen
09/08/2020, 4:44 AMElliot Oram
09/08/2020, 11:07 AMconfig.toml
which I point prefect via the env var: PREFECT__USER_CONFIG_PATH="/home/ubuntu/config.toml"
My config.toml looks like (obviously with host name actually filled in):
[server]
[server.ui]
graphql_url = "<host-name>:4200/graphql"
When I prefect server start
and visit host:8080 I get the front page of the ui asking to connect to host:4200/graphql.
Ideally it would be good if this could still be connected automatically using the config.toml
Any advice would be greatly appreciated. Please let me know if you need anymore details I might have missedGeorg Zangl
09/08/2020, 5:37 PMTraceback (most recent call last):
File "urllib3/connectionpool.py", line 677, in urlopen
File "urllib3/connectionpool.py", line 392, in _make_request
File "http/client.py", line 1252, in request
File "http/client.py", line 1298, in _send_request
File "http/client.py", line 1247, in endheaders
File "http/client.py", line 1026, in _send_output
File "http/client.py", line 966, in send
File "docker/transport/unixconn.py", line 43, in connect
PermissionError: [Errno 13] Permission denied
Any idea what I am missing?Jonathan Loscalzo
09/08/2020, 8:14 PMERROR:
Can't find a suitable configuration file in this directory or any
parent. Are you in the right directory?
Supported filenames: docker-compose.yml, docker-compose.yaml
............
............
subprocess.CalledProcessError: Command '['docker-compose', 'down']' returned non-zero exit status 1.
Operating System: Ubuntu 18.04.5 LTS
Kernel: Linux 5.4.0-42-generic
Architecture: x86-64
Using pyenv with python 3.7.9
Thanks in advance!ale
09/09/2020, 10:44 AMmasonlr
09/09/2020, 1:42 PM# Dockerfile
FROM prefecthq/prefect:0.13.5-python3.8
docker build -t prefect-base .
Everything is working as expected when using (i) the local docker-compose server and UI, (ii) a minimal flow
# register.py
import prefect
from prefect import task, Flow
from prefect.environments.storage import Docker
@task
def hello_task():
logger = prefect.context.get("logger")
<http://logger.info|logger.info>("test message")
flow = Flow("prefect-base", tasks=[hello_task])
base_image = "prefect-base:latest"
flow.storage = Docker(base_image=base_image, local_image=True)
flow.register(project_name="test")
and (iii) a docker agent running via
prefect agent start docker --no-pull -v
However, when I change to:
prefect agent start docker --no-pull -v --show-flow-logs
I'm getting an error (upon triggering a run in the UI) of the form:
[2020-09-09 13:21:44,229] DEBUG - agent | Updating states for flow run 457ba6ff-9a8e-4117-b563-166720de3ba9
[2020-09-09 13:21:44,230] DEBUG - agent | Flow run 457ba6ff-9a8e-4117-b563-166720de3ba9 is in a Scheduled state, updating to Submitted
[2020-09-09 13:21:44,262] INFO - agent | Deploying flow run 457ba6ff-9a8e-4117-b563-166720de3ba9
[2020-09-09 13:21:44,263] DEBUG - agent | Creating Docker container prefect-base:2020-09-09t13-21-12-637383-00-00
[2020-09-09 13:21:44,315] DEBUG - agent | Starting Docker container with ID 4e2404b2148c82b43ec9185d2c553ec1dc8b6ad8a653267c54664f7eae28d77c
[2020-09-09 13:21:44,483] DEBUG - agent | Querying for flow runs
[2020-09-09 13:21:44,501] DEBUG - agent | No flow runs found
[2020-09-09 13:21:44,501] DEBUG - agent | Next query for flow runs in 0.5 seconds
[2020-09-09 13:21:44,523] ERROR - agent | Logging platform error for flow run 457ba6ff-9a8e-4117-b563-166720de3ba9
[2020-09-09 13:21:44,568] ERROR - agent | Error while deploying flow: AttributeError("Can't pickle local object 'Agent.run_heartbeat_thread.<locals>.run'")
[2020-09-09 13:21:44,568] DEBUG - agent | Completed flow run submission (id: 457ba6ff-9a8e-4117-b563-166720de3ba9)
ale
09/09/2020, 5:06 PMMarvin
09/09/2020, 5:06 PMsark
09/10/2020, 8:10 AMprefect server start
and registered some flows but when i run them they never start (always “late”)
i have also started agents with prefect agent start
and prefect agent start docker
but the agents just keep polling and fail to find any flows
how can i check that the server sees the agent or the agents see the servers?Greg Desmarais
09/10/2020, 6:36 PMMarvin
09/10/2020, 6:36 PMAnish Chhaparwal
09/10/2020, 7:14 PMMichael J Hall
09/10/2020, 11:55 PMMichael J Hall
09/10/2020, 11:55 PMMichael J Hall
09/10/2020, 11:56 PMMichael J Hall
09/10/2020, 11:57 PMRizky Eko Putra
09/11/2020, 1:27 PM[server]
endpoint = "http://<IP>:4200/graphql"
[server.ui]
graphql_url = "http://<IP>:4200/graphql"
but when I open the UI,
the value of this on Prefect Server GraphQL endpoint
always http://localhost:4200/graphql
have tried to change the API URL via UI, to http://<IP>:4200/graphql
, this will affect only for that window browser, when I open on another incognito browser or another computer, that value reset again to http://localhost:4200/graphql...
is there any mistake with the config that I used?
thanks beforeBivu Raj
09/11/2020, 2:11 PMMarvin
09/11/2020, 2:11 PMBivu Raj
09/11/2020, 2:12 PMMarvin
09/11/2020, 2:12 PMBivu Raj
09/11/2020, 2:12 PMKIRYL BUCHA
09/11/2020, 6:45 PMMarvin
09/11/2020, 6:45 PMAshton
09/12/2020, 5:54 PMAshton
09/12/2020, 5:54 PM