• Vrinda Kailash

    Vrinda Kailash

    2 months ago
    hi all We are using an Automation for a DAG to create a new flow-run after a flow-run has “Finished”, We are facing an issue where multiple flow-runs are created at the same time
    Vrinda Kailash
    Anna Geller
    14 replies
    Copy to Clipboard
  • f

    FuETL

    2 months ago
    Guys do prefect have priority flows? Example i have 4 flows: a, b, c, d and i want to prioritize a and c to agent run first is this possible?
    f
    1 replies
    Copy to Clipboard
  • j

    Joshua Massover

    2 months ago
    I'm having trouble getting prefect agent logs for the debug level to show up. I've set
    PREFECT__LOGGING__LEVEL=DEBUG
    as an environment variable, confirmed it's in the environment, and my agent logs are not showing anything but INFO. Is there anything else that I should need to set?
    j
    2 replies
    Copy to Clipboard
  • Thomas Opsomer

    Thomas Opsomer

    2 months ago
    Hello 🙂 I'm trying to build a Graphql query to compute the number of running tasks for a particular task. I tried with the
    task_run_state_aggregate
    but I don't know how to filter on a specific task 😕 If anyone can help 🙏
    Thomas Opsomer
    Anna Geller
    2 replies
    Copy to Clipboard
  • j

    Joshua Massover

    2 months ago
    I'm running into a case where the prefect cloud UI shows jobs as submitted. I have queried the graphql api by the flow id to confirm it is submitted (i presume that's where the ui gets it's info from anyway). The agent however can finds no ready flows. I am running with debug mode to see that it's polling and never finding anything:
    [2022-07-14 17:49:44,859] DEBUG - agent | Querying for ready flow runs...
    DEBUG:agent:Querying for ready flow runs...
    [2022-07-14 17:49:44,959] DEBUG - agent | No ready flow runs found.
    DEBUG:agent:No ready flow runs found.
    DEBUG:agent:Sleeping flow run poller for 10.0 seconds...
    [2022-07-14 17:49:44,960] DEBUG - agent | Sleeping flow run poller for 10.0 seconds...
    [2022-07-14 17:49:54,960] DEBUG - agent | Querying for ready flow runs...
    DEBUG:agent:Querying for ready flow runs...
    [2022-07-14 17:49:55,046] DEBUG - agent | No ready flow runs found.
    DEBUG:agent:No ready flow runs found.
    [2022-07-14 17:49:55,046] DEBUG - agent | Sleeping flow run poller for 10.0 seconds...
    DEBUG:agent:Sleeping flow run poller for 10.0 seconds...
    Anyone have an suggestions on how to continue to debug?
    j
    Anna Geller
    3 replies
    Copy to Clipboard
  • Matan Drory

    Matan Drory

    2 months ago
    Does anyone have any tips working with a random number generator in prefect. I want my tasks to be able to recreate the result and also make sure they don’t get the same value when concurrency is involved. One of the things I though about is generating a set of random numbers based on a seed and passing them along to specific tasks. That way concurrency isn’t an issue. Just feel it’s not a very elegant solution. I though about putting the RNG in the context but then concurrency might be an issue
    Matan Drory
    Anna Geller
    3 replies
    Copy to Clipboard
  • Minh Mai

    Minh Mai

    2 months ago
    Does anyone know on top of their head why a script with
    flow.run()
    will fail once I register it to the UI? I'm current using the local server
    prefect backend server
    prefect server start
    prefect agent local start
    the error im getting is
    Failed to load and execute flow run: FlowStorageError('An error occurred while unpickling the flow:\n ModuleNotFoundError("No module named \'utils\'")\nThis may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')
    Minh Mai
    Anna Geller
    2 replies
    Copy to Clipboard
  • Sergey Goncharov

    Sergey Goncharov

    2 months ago
    Hi I'm trying to prepare a parametrised flow for Prefect v1. It has almost the same code as https://github.com/anna-geller/packaging-prefect-flows/blob/master/flows_task_library/s3_kubernetes_run_RunNamespacedJob_and_get_logs.py , but I only changed some functions to make them ready to work with parameters. I cannot get why does not it work now: function
    CreateAndRunJob
    marked as succeeded but prefect does not wait for the end of the execution and then function
    get_our_pod_name
    fails because it cannot find Job which is not yet created. I though to use threading or queue from python here, but I think it's not the right way with Prefect. Could you please advice here? I believe I do something wrong here.
    Sergey Goncharov
    Anna Geller
    8 replies
    Copy to Clipboard
  • c

    Cab Maddux

    2 months ago
    Hi! I'm trying to use the Orion python API
    prefect.packaging.docker.DockerPackager
    class and have it push to Google Container Registry. It looks like the current
    package()
    method is unable to push to GCR because the
    prefect.docker.push_image
    here ignores any path provided in the registry url. So it looks to me like if I provide a
    registry_url
    like
    <https://gcr.io/my-project-name>
    (where
    my-project-name
    is a GCP project) for a flow named
    my-flow
    rather than pushing an image to
    <http://gcr.io/my-project-name/my-flow|gcr.io/my-project-name/my-flow>
    the current implementation will try to push to
    <http://gcr.io/my-flow|gcr.io/my-flow>
    (which fails because I don't have a project named
    my-flow
    ). Wondering if I'm missing something here or maybe this should go into an issue?
    c
    Anna Geller
    3 replies
    Copy to Clipboard
  • Alex Fok

    Alex Fok

    2 months ago
    hello everyone, i have been trying to set up the prefect server. When I run prefect server start, I encountered this error. I have already installed docker-compose.
    Alex Fok
    Anna Geller
    5 replies
    Copy to Clipboard