Has anyone had any luck running the build and register step for their flows in a gitlab CI runner? I'm having trouble with the docker storage being unable to find the docker API when running in gitlab

Hey guys, I'm having issues trying to use docker Storage to pull pip dependences from our private artifactory - even when adding the build kwargs PIP_INDEX_URL it doesns't seem to respect it. Does anyone have a solution?

Hey guys, another question - so i'm trying to register a flow to my core local server not on the same host. How do i configure it so that it stops sending requests to localhost? it doesn't seem to be picking up the GRAPHQL_URL env var nor the config.toml -

Hi folks! I have local machine with Prefect installed, which has executed a flow, and I've set up Prefect Server on an EC2 instance (with the backend set to

server

). I'm now trying to set up an agent on my local machine which would be able to communicate with the server on EC2, so I can execute and monitor the flow from the UI. When I try to run

prefect agent start local --api <http://my.ec2.prefect.ip:4200/graphql>

on my local machine, it fails with

No agent API token provided

. In the server UI I don't see any options to create tokens and if I try to run

prefect auth create-token

it fails with

Error: Auth commands with server are not currently supported

, I guess because generating tokens is a Cloud-only feature. So my question is: how would I be able to set up a relationship between my local machine, where the flow would be running, and the server, without using Prefect Cloud? If that's not possible, would I be able to have the flow present on the same machine as the server and just use

flow.register(project_name="project_name_in_prefect_server")

in my flow script to have it managed by the server, or are there more steps involved?

Hey Team, on prefect cloud, when i use docker storage, after my docker agent successfully pulls the image, nothing happens and all my tasks are stuck on pending. Does anyone else have this issue? flow.run() works fine for me, but my agent for some reason isn't executing any tasks.

Hi guys, how do you dockerize your environment with prefect server? It starts by invoking docker-compose so I have to mount the host's docker socket to make that possible in dev, but what about production?

Hi all, upgrading from 0.10.6 to latest (0.13.5) and have noted that the prefect ui (host:8080) no longer appears to automatically connect to the graphql location (host:4200). I currently have this defined in

config.toml

which I point prefect via the env var:

PREFECT__USER_CONFIG_PATH="/home/ubuntu/config.toml"

My config.toml looks like (obviously with host name actually filled in):

[server]
  [server.ui]
    graphql_url = "<host-name>:4200/graphql"

When I

prefect server start

and visit host:8080 I get the front page of the ui asking to connect to host:4200/graphql. Ideally it would be good if this could still be connected automatically using the

config.toml

Any advice would be greatly appreciated. Please let me know if you need anymore details I might have missed

Dear Team, I have installed Prefect on a clean Ubuntu 18.04 Desktop VM. When starting the server, I get this error message:

Traceback (most recent call last):
  File "urllib3/connectionpool.py", line 677, in urlopen
  File "urllib3/connectionpool.py", line 392, in _make_request
  File "http/client.py", line 1252, in request
  File "http/client.py", line 1298, in _send_request
  File "http/client.py", line 1247, in endheaders
  File "http/client.py", line 1026, in _send_output
  File "http/client.py", line 966, in send
  File "docker/transport/unixconn.py", line 43, in connect
PermissionError: [Errno 13] Permission denied

Any idea what I am missing?

Hi all, I've just trying to to execute "prefect server start" using "backend" server, so I supposed that this issue could be permissions on pyenv:

ERROR: 
        Can't find a suitable configuration file in this directory or any
        parent. Are you in the right directory?

        Supported filenames: docker-compose.yml, docker-compose.yaml
............
............
subprocess.CalledProcessError: Command '['docker-compose', 'down']' returned non-zero exit status 1.

Operating System: Ubuntu 18.04.5 LTS Kernel: Linux 5.4.0-42-generic Architecture: x86-64 Using pyenv with python 3.7.9 Thanks in advance!

Hi all, I have a question regarding Prefect Server and Dask (maybe it has been already answered but I can’t find it by myself, sorry 😅 ) I was able to run Prefect flows on a local Dask cluster from my PyCharm dev environment. Now, the next step would be to register a flow on my local Prefect Server and to run it on Dask. But I can’t figure out what I need to do to tell Prefect Server to use Dask as the execution engine. I suspect I have to deal with Prefect Agent configs, but I’m not 100% sure. Any help is much appreciated! 🙂

Hi all, I'm trying to debug custom docker images. My reference setup is

# Dockerfile
FROM prefecthq/prefect:0.13.5-python3.8

docker build -t prefect-base .

Everything is working as expected when using (i) the local docker-compose server and UI, (ii) a minimal flow

# register.py

import prefect
from prefect import task, Flow
from prefect.environments.storage import Docker


@task
def hello_task():
    logger = prefect.context.get("logger")
    <http://logger.info|logger.info>("test message")

flow = Flow("prefect-base", tasks=[hello_task])
base_image = "prefect-base:latest"

flow.storage = Docker(base_image=base_image, local_image=True)
flow.register(project_name="test")

and (iii) a docker agent running via

prefect agent start docker --no-pull -v

However, when I change to:

prefect agent start docker --no-pull -v --show-flow-logs

I'm getting an error (upon triggering a run in the UI) of the form:

[2020-09-09 13:21:44,229] DEBUG - agent | Updating states for flow run 457ba6ff-9a8e-4117-b563-166720de3ba9
[2020-09-09 13:21:44,230] DEBUG - agent | Flow run 457ba6ff-9a8e-4117-b563-166720de3ba9 is in a Scheduled state, updating to Submitted
[2020-09-09 13:21:44,262] INFO - agent | Deploying flow run 457ba6ff-9a8e-4117-b563-166720de3ba9
[2020-09-09 13:21:44,263] DEBUG - agent | Creating Docker container prefect-base:2020-09-09t13-21-12-637383-00-00
[2020-09-09 13:21:44,315] DEBUG - agent | Starting Docker container with ID 4e2404b2148c82b43ec9185d2c553ec1dc8b6ad8a653267c54664f7eae28d77c
[2020-09-09 13:21:44,483] DEBUG - agent | Querying for flow runs
[2020-09-09 13:21:44,501] DEBUG - agent | No flow runs found
[2020-09-09 13:21:44,501] DEBUG - agent | Next query for flow runs in 0.5 seconds
[2020-09-09 13:21:44,523] ERROR - agent | Logging platform error for flow run 457ba6ff-9a8e-4117-b563-166720de3ba9
[2020-09-09 13:21:44,568] ERROR - agent | Error while deploying flow: AttributeError("Can't pickle local object 'Agent.run_heartbeat_thread.<locals>.run'")
[2020-09-09 13:21:44,568] DEBUG - agent | Completed flow run submission (id: 457ba6ff-9a8e-4117-b563-166720de3ba9)

Here I am, brain the size of a planet, and they ask me to enter you in a contest. Call that job satisfaction? ’Cos I don’t. I'll still enter you in the contest though, @ale

hi guys i have everything set up fine from

prefect server start

and registered some flows but when i run them they never start (always “late”) i have also started agents with

prefect agent start

and

prefect agent start docker

but the agents just keep polling and fail to find any flows how can i check that the server sees the agent or the agents see the servers?

@Marvin, please pick me

Incredible. The contest is even worse than I thought it would be. I'll still enter you to win, @Greg Desmarais

Hello, I've set up prefect version 0.13.5 on a ec2 instance and trying to connect to graphql api via the ui. I fire up the server using --ui-port 8888 to change the default port from 8080 to 8888 and have set up the config file as shown in the image. When i browse to the UI i am unable to connect to graphql endpoint. it shows unable to connect to Http://Http😕/<IP_address>:4200. I am unable to get rid of the extra Http:// even when setting endpoint using the dashboard on prefect UI. Been adviced to check if apollo is exposed correctly on the machine im running on. I cannot find the default port for apollo. Do the --server-port tags in prefect server start refer to apollo? How I could set up apollo endpoint using the cli... something on the lines of prefect server start --ui-port- 8888.

Hello. First time prefect user attempting to get it started on RHEL 7.8 Linux and I am getting the following error messages:

[root@abcxyz ~]# prefect backend server Backend switched to server [root@abcxyz ~]# prefect server start Pulling postgres ... done Pulling hasura ... done Pulling graphql ... done Pulling apollo ... done Pulling towel ... done Pulling ui ... done Creating network "prefect-server" with the default driver Creating tmp_postgres_1 ... Creating tmp_postgres_1 ... error ERROR: for tmp_postgres_1 Cannot start service postgres: driver failed programming external connectivity on endpoint tmp_postgres_1 (b11460d39882267bd238a301f9a0f073c30f25bdc724bb778c0ccadff2d96b88): Error starting userland proxy: listen tcp 0.0.0.0:5432: bind: address already in use ERROR: for postgres Cannot start service postgres: driver failed programming external connectivity on endpoint tmp_postgres_1 (b11460d39882267bd238a301f9a0f073c30f25bdc724bb778c0ccadff2d96b88): Error starting userland proxy: listen tcp 0.0.0.0:5432: bind: address already in use ERROR: Encountered errors while bringing up the project.

It appears that the postgres port 5432 is already taken by another application.

Is there any workaround to this?

Hello, I have start prefect server with a config like this

[server]
endpoint = "http://<IP>:4200/graphql"
  
  [server.ui]
    
    graphql_url = "http://<IP>:4200/graphql"

but when I open the UI, the value of this on

Prefect Server GraphQL endpoint

always http://localhost:4200/graphql have tried to change the API URL via UI, to

http://<IP>:4200/graphql

, this will affect only for that window browser, when I open on another incognito browser or another computer, that value reset again to http://localhost:4200/graphql... is there any mistake with the config that I used? thanks before

@Marvin let me be the winner !!

This contest will all end in tears, I just know it. You're entered anyway, @Bivu Raj.

@Marvin you sound a little sad. hope you achieve AI nirvana.

This contest will all end in tears, I just know it. You're entered anyway, @Bivu Raj.

Hello , where do i find this setting? I just can't seem to find this anywhere in my local backed or the cloud. Thanks

Don’t pretend you want to talk to me, I know you hate me. I'll still enter you in the contest, @KIRYL BUCHA.