• Zach Schumacher

    Zach Schumacher

    4 months ago
    hi - we started seeing a new log from prefect manifesting in some failed k8s jobs. Any idea what could be going on here?
    Zach Schumacher
    Kevin Kho
    +1
    8 replies
    Copy to Clipboard
  • m

    Matt Alhonte

    4 months ago
    Where in
    ECSRun
    would I put Dask security args? Like
    tls_ca_file
    and stuff?
    m
    Kevin Kho
    7 replies
    Copy to Clipboard
  • a

    Anthony L

    4 months ago
    Hello team! I'm looking use Prefect at my company. While testing, I noticed that
    random
    generates the same values between runs. I'm using Prefect Cloud 1.0 and running a local agent (with Prefect 1.2.1 library installed). Here's my flow:
    import prefect
    from prefect import task, Flow
    from random import random
    
    @task
    def task1():
        logger = prefect.context.get("logger")
        random_val = random()
        <http://logger.info|logger.info>(f"{random_val=}")
        if random_val < 0.5:
            raise Exception()
    
    
    with Flow("random-bug-flow") as flow:
        task1()
    
    flow.register(project_name="ant-flows")
    I manually scheduled a run of this flow in Prefect Cloud and I see the random value in the logs. However, when I schedule another run (and any run thereafter), it generates the same "random" value. Is this expected?
    a
    Kevin Kho
    +1
    12 replies
    Copy to Clipboard
  • Gaurav

    Gaurav

    4 months ago
    Hello Prefect team! Hope you all are doing well 🙂 We are currently using Prefect 1.0 to launch tasks on AWS ECS using a local agent installed on an EC2 machine. The reason we are using a local agent is because we plan to do the same on other clouds (namely Azure and GCP) as well, and since they dont have their specific native agents (like ECSAgent for ECS), the local agent seemed more universal/generic approach for a multi cloud environment. We had some questions regarding the above however: 1. Using the local agent with the Prefect Cloud, is there a hard limitation or some throttling/delays in starting flows that can occur if we schedule a lot of tasks concurrently. For example running 250 flows (and the tasks therein) concurrently. Can a local agent support this? 2. I noticed Prefect 2.0 coming up soon, however upon reading the docs there, things like ECSAgent etc do not seem to exist there, and I was wondering what the migration process from 1.0 to 2.0 would look like. Basically - we would like to run our flows in a multi cloud setup (1 agent per cloud, tagged with respective labels). Thanks!
    Gaurav
    Kevin Kho
    +1
    5 replies
    Copy to Clipboard
  • Artem Vysotsky

    Artem Vysotsky

    4 months ago
    Hi community, I’m trying to find a documentation that will show an example (if possible) of programmatic cron job scheduling. I.e. I would imagine, that there will be a way to create a parametrized schedule via server api. Use case: Imagine a user interface where user could create a recurring job, specify job schedule and some additional parameters for the job to be executed (i.e. an ETL pipeline with sources and destinations configured via params)
    Artem Vysotsky
    Kevin Kho
    2 replies
    Copy to Clipboard
  • Cole Murray

    Cole Murray

    4 months ago
    Following up on https://prefect-community.slack.com/archives/CL09KU1K7/p1651543946765889 I wanted to confirm this was not an issue in Orion. In Orion, we've decoupled the one to many relationship between a Flow and Clocks (technically Schedule has the clocks) that was present with the introduction of a DeploymentSpec, which has a 1-1 with a schedule. Rather than iterating clocks here:https://github.com/PrefectHQ/server/blob/master/src/prefect_server/api/flows.py#L688 https://github.com/PrefectHQ/prefect/blob/cb0f80c6fc743cdae9f2af5b493ed454bc53d07c/src/prefect/schedules/schedules.py#L108 We are iterating deployments here:https://github.com/PrefectHQ/prefect/blob/orion/src/prefect/orion/services/scheduler.py#L83, which have a 1-1 between schedules. To solve the similar case of one flow, parameterized N many times & different schedules, we should be able to create N many DeploymentSpecs for the given flow without scaling issues, correct? cc: @Kevin Kho
    Cole Murray
    1 replies
    Copy to Clipboard
  • b

    Bernard Greyling

    4 months ago
    Hello We've been using prefect internally for about two years now. Congrats on the cadence of innovation. I'ts been a pleasure using your tools. At this stage we have two environments setup, one for ETL runs and another for internal reporting mechanisms. We are on the standard plan and would like to expand the number of users. I've tried twice now to get access to the sales team via the main prefect portal. Haven't heard anything from you guys in over two weeks, Can someone please assist me in having the relevant conversation ?
    b
    1 replies
    Copy to Clipboard
  • davzucky

    davzucky

    4 months ago
    I need a script that restart a lot of flow run. I'm able to get the flow runs, however I'm not sure if the mutation to restart them. We had about 600 failed run because of infra that I need to restart
    davzucky
    Anna Geller
    5 replies
    Copy to Clipboard
  • Suresh R

    Suresh R

    4 months ago
    Hi! I am getting below error sometimes when i try to update a Prefect secret, i suspect multiple flows updating same secret at same time causing this issue, still i am not sure. Can someone help.
    prefect.exceptions.ClientError: [{'path': ['secret_value'], 'message': 'An unknown error occurred.', 'extensions': {'code': 'INTERNAL_SERVER_ERROR'}}]
    Suresh R
    Anna Geller
    +1
    4 replies
    Copy to Clipboard
  • Domenico Di Gangi

    Domenico Di Gangi

    4 months ago
    Hi all, I am using prefect 2.0 more and more these days and loving it! Is there a way to quickly see execution's time of a flow run from the UIs listings? I guess I could filter runs, but it would be nice to have a faster way to select a run based on its execution time. It seems that the visualization has enough space available in the runs' rows and maybe timestamp could easily fit on it:
    Domenico Di Gangi
    Anna Geller
    +2
    12 replies
    Copy to Clipboard