• b

    Ben Welsh

    7 months ago
    My next challenge trying to configure my Prefect
    LocalDaskExecutor
    to run mapped tasks in parallel inside a single GKE k8s container with what, if I understand the jargon correctly, is known as vertical scaling. Are there any resources that show the optimal way to do that?
    b
    Anna Geller
    11 replies
    Copy to Clipboard
  • b

    Bo

    7 months ago
    Hello everyone! I am trying to set up an ecs agent on fargate, but I don't have a "default" vpc set up, so I need to specify the network config, as I was getting this error:
    ERROR - agent | Failed to infer default networkConfiguration, please explicitly configure using
    --run-task-kwargs`` . However when I add the yaml path to the command, I start to receive this error:
    Error: No such option: --run-task-kwargs s3://*****/options.yaml
    Any thoughts?
    b
    Kevin Kho
    5 replies
    Copy to Clipboard
  • Dotan Asselmann

    Dotan Asselmann

    7 months ago
    Hi everyone. Is there an equivalent for celery metrics exporter in prefect server?https://github.com/danihodovic/celery-exporter
    Dotan Asselmann
    Kevin Kho
    3 replies
    Copy to Clipboard
  • Tony Liberato

    Tony Liberato

    7 months ago
    Good afternoon. I have been making great progress on our inital prefect workflow model. However, I ran into a snag I have no idea how to fix. I have a typical download/unzip/save to s3 workflow that I am trying to send to a generic docker agent (with Flow("<whatever>", storage = Docker()) as flow😃. When I try to register, I get a healthcheck error: no module named bs4 / Flow uses module that is not importable. How do I get the flow to use bs4 on the docker agent?
    Tony Liberato
    Kevin Kho
    8 replies
    Copy to Clipboard
  • Jamie Miller

    Jamie Miller

    7 months ago
    Hey everyone. If I want to have my flow set up in a 'flows' folder, and then have that flow import 2 tasks from a different file, how would I go about that? What kind of import statement would I need to add at the top of my file?
    Jamie Miller
    Kevin Kho
    2 replies
    Copy to Clipboard
  • d

    David Mannion

    7 months ago
    Hi all, I'm looking to get started with Prefect. I've installed Prefect Server on an EC2 instance in a private subnet which I can access via a bastion server. I've successfully started the server and can see it running. I'm trying to access the ui by using a putty tunnel to forward a port on my laptop, via the bastion server, to the <machine with prefect>:8080. I configured firefox to use SOCKS5 with localhost and the port forwarded by putty. Unfortunately I'm not getting the ui to load up. It's a difficult one to debug as there's lots of components involved. Has anyone any experience doing something like this? Any tips on prefect configuration that might be pertinent?
    d
    Kevin Kho
    +1
    15 replies
    Copy to Clipboard
  • Sylvain Hazard

    Sylvain Hazard

    7 months ago
    Hey there ! Starting to try out GraphQL (first time 😬) to clean up old flow runs in order to reduce the size of Postgre Database. I manage to get flow run ids before a given date as well as deleting a single flow run from its id using
    delete_flow_run
    but is there a way to delete multiple flow runs at once ? Thanks !
    Sylvain Hazard
    Anna Geller
    +1
    7 replies
    Copy to Clipboard
  • frojo

    frojo

    7 months ago
    Hi everyone! I'm new to Prefect (just a couple of months). I've been using it as provided for some local tests without an issue. I was trying to replicate this tests on a remote machine (ssh mediated) but I have runned into some issues. I'm trying to spin up a local prefect server (just testing some flows) on a remote machine. Some data on versions prefect diagnostics:
    {
    "config_overrides": {},
    "env_vars": [],
    "system_information": {
    "platform": "Linux-5.4.0-96-generic-x86_64-with-glibc2.27",
    "prefect_backend": "server",
    "prefect_version": "0.15.13",
    "python_version": "3.8.12"
    }
    }
    I was setting the local server as option and started as usual:
    prefect backend server
    prefect server start
    No special options. Images are pulled and all the services started. However some errors (regarding graphql mainly) start to apper on logs.
    graphql_1   | Error applying Hasura metadata from /prefect-server/services/hasura/migrations/metadata.yaml
    graphql_1   | 
    graphql_1   | Could not upgrade the database!
    graphql_1   | Error: Failed to apply Hasura metadata! Error: 503 Server Error: Service Unavailable for url: <http://hasura:3000/v1/query>
    apollo_1    | Checking GraphQL service at <http://graphql:4201/health> ...
    tmp_graphql_1 exited with code 1
    (I have the full trace until stopping them). I also checked (while waiting the spin up - just displaying the failing one)
    > docker ps 
    7036b92f538d   prefecthq/server:core-0.15.13   "tini -g -- bash -c …"   About a minute ago   Restarting (1) 1 second ago                                      tmp_graphql_
    which is continously restarting. I have tryied: • previuos Prefect version (0.15.0) with the same effect. • the same env on my local machine with success (so I suspect from the remote configuration, but I could not figure what is missing) Thanks!
    frojo
    Anna Geller
    +1
    15 replies
    Copy to Clipboard
  • m

    Max Watermolen

    7 months ago
    Howdy, Oh man I am struggling to figure out why all my tasks are failing with
    Failed to load and execute Flow's environment: ModuleNotFoundError("No module named '/home/magnox/'")
    Env: • Local Agent + Local Storage (Running in docker [python:3.10] ... yeah yeah lol) • External Postgres 12 • Prefect Server in Kubernetes (Official HELM) • Django Integrated Code: The Source is located in
    /code/
    We have added
    /code
    to the path and Python path on execution. Notes: • Sometimes a single container will work fine (using the same exact image and everything)(Cant seem to figure out why) • I have tried with and without Dask • I tried to symlink the /code -> /home/magnox to no avail
    m
    Kevin Kho
    23 replies
    Copy to Clipboard
  • William Clark

    William Clark

    7 months ago
    Hello, do we have the ability to delete a tenant?
    William Clark
    Kevin Kho
    9 replies
    Copy to Clipboard