• c

    Chohang Ng

    1 year ago
    A colleague of mine asked me if the prefect.io will still execute the flows when the computer is not login. I'm not sure about the answer but I assume a yes, so the agent would still be able to monitor upcoming flows as long as the power is on right/ He is comparing with the concept of window task scheduler where the task execution is optional on whether the computer is login or not.
    c
    1 replies
    Copy to Clipboard
  • c

    Chris Bowen

    1 year ago
    Hello, I'm very new to Prefect. I have it running on a RHL server, but I'm running into a couple behaviors I'm not able to find answers for via googling- hoping to get some advice. First thing: when I initially did the tutorial/installation and ran
    prefect server start
    , the task submitted and didn't run interactively. I'm not sure if I inadvertently changed a setting, but for the past day when I start the server, it locks up the terminal and runs as an interactive job. I don't see a setting anywhere to modify it and run it as a daemon or comparable solution. Any ideas? I realize this might b e a Linux issue more than a Prefect one. Second thing: I can't get an agent to start. I'm getting a timeout every time I try to fire it up. I can access the UI from a browser on port 8081 outside of the RHL VM that the server is running on. I can also access the utility that runs on port 4200 from a browser. For reference, there is also an Airflow instance running on this RHL server, so I've been using this command to start prefect:
    prefect server start --postgres-port=5433 --ui-port=8081
    . I haven't found documentation, troubleshooting guides, or anybody asking online about the agent timing out. I tried running the start agent job as the root user and it still times out, so I don't think it's permission, but I'm happy to check anything. I did run
    prefect backend server
    . Here's the agent command:
    prefect agent local start
    I did try messing around with the various hostname flags for this command, I believe I tried every port displayed in the docker list below. I think the agent should be coming up on 4200? Here's the docker info for the Prefect containers:
    d343f6b8e7a5   prefecthq/ui:core-0.14.20       "/docker-entrypoint.…"   8 minutes ago   Up 8 minutes (healthy)   80/tcp, 0.0.0.0:8081->8080/tcp, :::8081->8080/tcp   tmp_ui_1
    9c993c53daed   prefecthq/apollo:core-0.14.20   "tini -g -- bash -c …"   8 minutes ago   Up 8 minutes (healthy)   0.0.0.0:4200->4200/tcp, :::4200->4200/tcp           tmp_apollo_1
    f17c041a6ac8   prefecthq/server:core-0.14.20   "tini -g -- python s…"   8 minutes ago   Up 8 minutes                                                                 tmp_towel_1
    65cd55a58004   prefecthq/server:core-0.14.20   "tini -g -- bash -c …"   8 minutes ago   Up 8 minutes (healthy)   0.0.0.0:4201->4201/tcp, :::4201->4201/tcp           tmp_graphql_1
    9a251fd9322f   hasura/graphql-engine:v1.3.3    "graphql-engine serve"   8 minutes ago   Up 8 minutes (healthy)   0.0.0.0:3000->3000/tcp, :::3000->3000/tcp           tmp_hasura_1
    fd22ad83e1a5   postgres:11                     "docker-entrypoint.s…"   8 minutes ago   Up 8 minutes (healthy)   0.0.0.0:5433->5432/tcp, :::5433->5432/tcp           tmp_postgres_1
    Here's the command I'm executing for the agent:
    $ prefect agent local start
    /usr/local/lib/python3.8/site-packages/prefect/tasks/__init__.py:8: UserWarning: SQLite tasks require sqlite3 to be installed
      import prefect.tasks.database
    [2021-05-28 16:32:21,594] INFO - agent | Registering agent...
    And here's the error:
    File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 200, in connect
        conn = self._new_conn()
      File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn
        raise ConnectTimeoutError(
    urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7f1a862b83d0>, 'Connection to localhost timed out. (connect timeout=15)')
    ...
    requests.exceptions.ConnectTimeout: HTTPConnectionPool(host='localhost', port=4200): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f1a862b83d0>, 'Connection to localhost timed out. (connect timeout=15)'))
    Here's the installation guide I followed: https://docs.prefect.io/core/getting_started/installation.html Any help or advice is much appreciated, thank you!
    c
    Kevin Kho
    +2
    20 replies
    Copy to Clipboard
  • Aurélien Vallée

    Aurélien Vallée

    1 year ago
    Im having issues when trying to delete a flow with a self-hosted prefect server. If I check "delete all versions" of the flow, then I get the error message "We could not delete your flow. Please try again. If this problem continues, contact help@prefect.io"
    Aurélien Vallée
    Jenny
    5 replies
    Copy to Clipboard
  • Aurélien Vallée

    Aurélien Vallée

    1 year ago
    and nothing appears in the log to explain what went wrong
  • m

    Marko Jamedzija

    1 year ago
    Hi there! I’m trying to run a
    KubernetesRun
    so that it runs jobs in a different namespace than the agent is in. I tried setting the namespace in job template yaml, but it doesn’t work. Any suggestions?
    m
    Kevin Kho
    +1
    10 replies
    Copy to Clipboard
  • Damien Ramunno-Johnson

    Damien Ramunno-Johnson

    1 year ago
    I am currently having an issue running a lot of short tasks at once
    UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
    Flow run is no longer in a running state; the current state is: <Failed: "UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)">
    Is this limit on the cloud api side?
    Damien Ramunno-Johnson
    nicholas
    4 replies
    Copy to Clipboard
  • d

    Dana Merrick

    1 year ago
    is there a way to manually remove an agent? we have an agent that is old and i cant figure out where it is running and i'd like to just ban it. this is a k8s agent btw
    d
    nicholas
    8 replies
    Copy to Clipboard
  • s

    Stef van Dijk

    1 year ago
    Hello! I'm using Prefect Cloud and I like its features. I'm struggling with some security issues at the moment. Is it possible for an agent to be restricted to execute only flows that use local storage? Let's say I have a machine that should never execute harmful flows. Can I make sure that when someone accidentally gets my Prefect credentials, that person can never make the agent on my machine execute a flow that he just registered using my credentials? I want to make this restriction at agent level, because otherwise it seems to me that a hacker could always register a flow living in some github repo and make the agent execute it. Restricting the agent to only execute flows with local storage would prevent this from happening. I hope my question is clear, thanks in advance!
    s
    nicholas
    2 replies
    Copy to Clipboard
  • s

    Snehotosh

    1 year ago
    Can anyone please clarify the type of open source license of Prefect server? Is there any licensing impact using in production?
    s
    Amanda Wee
    2 replies
    Copy to Clipboard
  • Mckelle

    Mckelle

    1 year ago
    Hi all. I've been working on setting up Prefect Server in GCP and wanted to see if anyone had successfully secured it under Identity Aware Proxy? I've set it up on a VM and have created a load balancer that directs traffic from my domain to the port 8080 on the VM to access the UI. Initially, when connecting graphql (4200) to the UI, I tried to just set it to the VM IP address:4200, but that was denied because it is HTTP and the domain is https so chrome throws an error saying it's not secure. To fix that error, I then made a second backend service on the Load Balancer pointing to port 4200 and created a subdomain for that port. I set the subdomain as my Prefect Server Graphql endpoint in the UI and that resolved the first issue. However, the issue with this is that as soon as I turn on Identity Aware Proxy for both the UI and Graphql backend service, I get a CORS error.
    Access to manifest at '<https://accounts.google.com/o/oauth2/v2/auth?…>' (redirected from '<http://maindomain.com|maindomain.com>') from origin '<http://maindomain.com|maindomain.com>' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
    Does anyone have any experience with this or anything similar?
    Mckelle
    nicholas
    +1
    4 replies
    Copy to Clipboard