• Greg Roche

    Greg Roche

    2 years ago
    Hi folks! I have local machine with Prefect installed, which has executed a flow, and I've set up Prefect Server on an EC2 instance (with the backend set to
    server
    ). I'm now trying to set up an agent on my local machine which would be able to communicate with the server on EC2, so I can execute and monitor the flow from the UI. When I try to run
    prefect agent start local --api <http://my.ec2.prefect.ip:4200/graphql>
    on my local machine, it fails with
    No agent API token provided
    . In the server UI I don't see any options to create tokens and if I try to run
    prefect auth create-token
    it fails with
    Error: Auth commands with server are not currently supported
    , I guess because generating tokens is a Cloud-only feature. So my question is: how would I be able to set up a relationship between my local machine, where the flow would be running, and the server, without using Prefect Cloud? If that's not possible, would I be able to have the flow present on the same machine as the server and just use
    flow.register(project_name="project_name_in_prefect_server")
    in my flow script to have it managed by the server, or are there more steps involved?
    Greg Roche
    j
    6 replies
    Copy to Clipboard
  • c

    Charles Leung

    2 years ago
    Hey Team, on prefect cloud, when i use docker storage, after my docker agent successfully pulls the image, nothing happens and all my tasks are stuck on pending. Does anyone else have this issue? flow.run() works fine for me, but my agent for some reason isn't executing any tasks.
    c
    nicholas
    5 replies
    Copy to Clipboard
  • Harry Nguyen

    Harry Nguyen

    2 years ago
    Hi guys, how do you dockerize your environment with prefect server? It starts by invoking docker-compose so I have to mount the host's docker socket to make that possible in dev, but what about production?
    Harry Nguyen
    nicholas
    3 replies
    Copy to Clipboard
  • e

    Elliot Oram

    2 years ago
    Hi all, upgrading from 0.10.6 to latest (0.13.5) and have noted that the prefect ui (host:8080) no longer appears to automatically connect to the graphql location (host:4200). I currently have this defined in
    config.toml
    which I point prefect via the env var:
    PREFECT__USER_CONFIG_PATH="/home/ubuntu/config.toml"
    My config.toml looks like (obviously with host name actually filled in):
    [server]
      [server.ui]
        graphql_url = "<host-name>:4200/graphql"
    When I
    prefect server start
    and visit host:8080 I get the front page of the ui asking to connect to host:4200/graphql. Ideally it would be good if this could still be connected automatically using the
    config.toml
    Any advice would be greatly appreciated. Please let me know if you need anymore details I might have missed
    e
    nicholas
    2 replies
    Copy to Clipboard
  • Georg Zangl

    Georg Zangl

    2 years ago
    Dear Team, I have installed Prefect on a clean Ubuntu 18.04 Desktop VM. When starting the server, I get this error message:
    Traceback (most recent call last):
      File "urllib3/connectionpool.py", line 677, in urlopen
      File "urllib3/connectionpool.py", line 392, in _make_request
      File "http/client.py", line 1252, in request
      File "http/client.py", line 1298, in _send_request
      File "http/client.py", line 1247, in endheaders
      File "http/client.py", line 1026, in _send_output
      File "http/client.py", line 966, in send
      File "docker/transport/unixconn.py", line 43, in connect
    PermissionError: [Errno 13] Permission denied
    Any idea what I am missing?
    Georg Zangl
    2 replies
    Copy to Clipboard
  • Jonathan Loscalzo

    Jonathan Loscalzo

    2 years ago
    Hi all, I've just trying to to execute "prefect server start" using "backend" server, so I supposed that this issue could be permissions on pyenv:
    ERROR: 
            Can't find a suitable configuration file in this directory or any
            parent. Are you in the right directory?
    
            Supported filenames: docker-compose.yml, docker-compose.yaml
    ............
    ............
    subprocess.CalledProcessError: Command '['docker-compose', 'down']' returned non-zero exit status 1.
    Operating System: Ubuntu 18.04.5 LTS Kernel: Linux 5.4.0-42-generic Architecture: x86-64 Using pyenv with python 3.7.9 Thanks in advance!
    Jonathan Loscalzo
    Jim Crist-Harif
    +2
    38 replies
    Copy to Clipboard
  • ale

    ale

    2 years ago
    Hi all, I have a question regarding Prefect Server and Dask (maybe it has been already answered but I can’t find it by myself, sorry 😅 ) I was able to run Prefect flows on a local Dask cluster from my PyCharm dev environment. Now, the next step would be to register a flow on my local Prefect Server and to run it on Dask. But I can’t figure out what I need to do to tell Prefect Server to use Dask as the execution engine. I suspect I have to deal with Prefect Agent configs, but I’m not 100% sure. Any help is much appreciated! 🙂
    ale
    1 replies
    Copy to Clipboard
  • m

    masonlr

    2 years ago
    Hi all, I'm trying to debug custom docker images. My reference setup is
    # Dockerfile
    FROM prefecthq/prefect:0.13.5-python3.8
    docker build -t prefect-base .
    Everything is working as expected when using (i) the local docker-compose server and UI, (ii) a minimal flow
    # register.py
    
    import prefect
    from prefect import task, Flow
    from prefect.environments.storage import Docker
    
    
    @task
    def hello_task():
        logger = prefect.context.get("logger")
        <http://logger.info|logger.info>("test message")
    
    flow = Flow("prefect-base", tasks=[hello_task])
    base_image = "prefect-base:latest"
    
    flow.storage = Docker(base_image=base_image, local_image=True)
    flow.register(project_name="test")
    and (iii) a docker agent running via
    prefect agent start docker --no-pull -v
    However, when I change to:
    prefect agent start docker --no-pull -v --show-flow-logs
    I'm getting an error (upon triggering a run in the UI) of the form:
    [2020-09-09 13:21:44,229] DEBUG - agent | Updating states for flow run 457ba6ff-9a8e-4117-b563-166720de3ba9
    [2020-09-09 13:21:44,230] DEBUG - agent | Flow run 457ba6ff-9a8e-4117-b563-166720de3ba9 is in a Scheduled state, updating to Submitted
    [2020-09-09 13:21:44,262] INFO - agent | Deploying flow run 457ba6ff-9a8e-4117-b563-166720de3ba9
    [2020-09-09 13:21:44,263] DEBUG - agent | Creating Docker container prefect-base:2020-09-09t13-21-12-637383-00-00
    [2020-09-09 13:21:44,315] DEBUG - agent | Starting Docker container with ID 4e2404b2148c82b43ec9185d2c553ec1dc8b6ad8a653267c54664f7eae28d77c
    [2020-09-09 13:21:44,483] DEBUG - agent | Querying for flow runs
    [2020-09-09 13:21:44,501] DEBUG - agent | No flow runs found
    [2020-09-09 13:21:44,501] DEBUG - agent | Next query for flow runs in 0.5 seconds
    [2020-09-09 13:21:44,523] ERROR - agent | Logging platform error for flow run 457ba6ff-9a8e-4117-b563-166720de3ba9
    [2020-09-09 13:21:44,568] ERROR - agent | Error while deploying flow: AttributeError("Can't pickle local object 'Agent.run_heartbeat_thread.<locals>.run'")
    [2020-09-09 13:21:44,568] DEBUG - agent | Completed flow run submission (id: 457ba6ff-9a8e-4117-b563-166720de3ba9)
    m
    j
    7 replies
    Copy to Clipboard
  • ale

    ale

    2 years ago
    @Marvin
  • Marvin

    Marvin

    2 years ago
    Here I am, brain the size of a planet, and they ask me to enter you in a contest. Call that job satisfaction? ’Cos I don’t. I'll still enter you in the contest though, @ale