https://prefect.io logo
Docs
Join the conversationJoin Slack
Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Powered by Linen
prefect-community
  • l

    Leon Kozlowski

    10/07/2021, 8:00 PM
    In order to integrate slack (through the
    state_handler
    ) with prefect cloud, does the webhook URL have to be stored in Prefect Secrets, or can it just be available as an environment variable (my use case would be loading an eternal secret to an eks cluster)
    k
    7 replies · 2 participants
  • c

    Constantino Schillebeeckx

    10/07/2021, 8:15 PM
    I'm trying to send out notifications when a flow fails by defining a callback setting the flow's
    on_failure
    to it. The callback has (according to the docs) the following signature
    def f(obj: Union[Task, Flow], state: State) -> None:
    My quest is this: how to identify the failed task from the (Flow) obj. I see I can iterate over the flows' tasks, I'm just not sure how to identify the task's state
    k
    3 replies · 2 participants
  • k

    kevin

    10/08/2021, 12:52 AM
    Hey guys, I'm just curious if there's been any changes recently to authenticating and POSTing to Prefect Cloud? I've recently been getting some AuthenticationErrors indicating Forbidden and haven't recently changed auth tokens
    c
    2 replies · 2 participants
  • k

    Klemen Strojan

    10/08/2021, 6:06 AM
    Hello Community! We are getting this error quite often in the last weeks:
    KeyError: 'The secret <some_secret> was not found.  Please ensure that it was set correctly in your tenant: <https://docs.prefect.io/orchestration/concepts/secrets.html>'
    Now this happens in a mapped task, only for one or two of those mapped tasks, others work fine. The secret exists and is clearly usable. Re-running the flow usually helps, however we are not enjoying that re-running part so much. Any advice?
    m
    a
    +1
    3 replies · 4 participants
  • m

    Marko Herkaliuk

    10/08/2021, 8:15 AM
    Hello, at 17:22, Oct 7 2021 (UTC-5) we have
    requests.exceptions.HTTPError: 404 Client Error: Not Found for url: <https://api.prefect.io/>
    when flow try create other flow. Was it a common problem or a problem on our side?
    k
    6 replies · 2 participants
  • a

    Aldo M

    10/08/2021, 9:26 AM
    Hello, Following the announcement for Orion, If we create our workflows based on Prefect Core, won’t it be a pain to move it to Orion ? As for now we are going to Prefect Server and launch it on our infra
    :upvote: 1
    a
    a
    3 replies · 3 participants
  • j

    Joseph Oladokun

    10/08/2021, 12:10 PM
    Hello, I’m having this error, all the solutions seems outdated
    TypeError: cannot pickle '_thread.lock' object
    a
    k
    6 replies · 3 participants
  • j

    Joseph Oladokun

    10/08/2021, 12:10 PM
    Any help will be appreciated
  • z

    Zheng Xie

    10/08/2021, 1:25 PM
    the users of the workflow can upload a file from their own laptop or their own box folder, where the workflow has no prior knowledge where the files can be. How to handle this in Prefect?
    k
    3 replies · 2 participants
  • m

    Max Kureykin

    10/08/2021, 1:31 PM
    Hi all! Is there any way not to fail the entire flow that consists only of 1
    map
    task and some
    mapped
    runs are failed? Thx in advance)
    k
    5 replies · 2 participants
  • m

    Marwan Sarieddine

    10/08/2021, 2:02 PM
    Hi folks, I believe we are facing an issue with manual_only triggered tasks. Mainly the task seems to be nullifying the parameters passed to it
    k
    17 replies · 2 participants
  • g

    García Gimenez Ignacio - 6to 2da

    10/08/2021, 3:10 PM
    Hi all! Im having an issue trying to create a tenant, it throws the following exception: "HTTPConnectionPool(host='localhost', port=4200): Max retries exceeded with url....". I'm running my own docker compose file. Apollo's server is running fine, it connects to the UI. But it seems like wont connect when trying to create the tenant with the command "prefect server create-tenant --name default --slug default". Is there any way to change the default ip "localhost" to a custom one?
    k
    n
    19 replies · 3 participants
  • m

    Michael Pavese

    10/08/2021, 5:23 PM
    I am trying to run a flow that I registered with the CLI and is using Gitlab storage. I am getting the following error in my logs:
    k
    2 replies · 2 participants
  • m

    Michael Pavese

    10/08/2021, 5:23 PM
  • t

    Tadej Svetina

    10/08/2021, 5:26 PM
    I am trying to run a simple task, but I am getting
    Unexpected error: TypeError("cannot pickle '_io.BufferedReader' object")
    k
    m
    11 replies · 3 participants
  • m

    Michael Pavese

    10/08/2021, 5:26 PM
    Where do I need to install "gitlab extra". When I run this on my local machine, it's fine.
    k
    2 replies · 2 participants
  • m

    Matthew Seligson

    10/08/2021, 6:21 PM
    How can we use the GraphQL API to find which user manually restarted a flow run? Or separately, a task run?
    k
    26 replies · 2 participants
  • v

    Vincent

    10/08/2021, 6:28 PM
    Hi- I was wondering if there was a prefect cloud interruption today. My flows exited with 404 and Protocol Errors.
    Failed to send heartbeat with exception: HTTPError('404 Client Error: Not Found for url: <https://api.prefect.io/graphql>')
    Traceback (most recent call last):
      File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/cli/heartbeat.py", line 68, in flow_run
        client.update_flow_run_heartbeat(id)
      File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 1610, in update_flow_run_heartbeat
        self.graphql(
      File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 548, in graphql
        result = <http://self.post|self.post>(
      File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 451, in post
        response = self._request(
      File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 737, in _request
        response = self._send_request(
      File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 647, in _send_request
        response.raise_for_status()
      File "/opt/conda/envs/dev/lib/python3.8/site-packages/requests/models.py", line 943, in raise_for_status
        raise HTTPError(http_error_msg, response=self)
    requests.exceptions.HTTPError: 404 Client Error: Not Found for url: <https://api.prefect.io/graphql>
    k
    10 replies · 2 participants
  • c

    Chris Reuter

    10/08/2021, 7:57 PM
    Hi everybody 👋! The Prefect livestream where @Laura Lorenz will take Orion for a spin is live in 5 minutes! Sign up on Meetup to attend, or go straight to the

    Youtube stream▾

    . Hope to see you there!
    :party-parrot: 2
    🚀 4
    📺 3
    :marvin: 3
    s
    j
    +1
    5 replies · 4 participants
  • m

    Matt Alhonte

    10/08/2021, 10:58 PM
    Hrm, so I'm running a big Flow of Flows - some of them should be kicked off in parallel I think, but they're all running one at a time.
    m
    11 replies · 2 participants
  • u

    张强

    10/09/2021, 8:18 AM
    hi
    👋 5
    c
    a
    2 replies · 3 participants
  • u

    张强

    10/09/2021, 2:43 PM
    Is there a base class like BaseFlow and BaseTask to set default parameters, such as
    max_retries
    , or set default
    state_handlers
    k
    j
    6 replies · 3 participants
  • u

    张强

    10/10/2021, 1:21 AM
    Two undependent
    map
    tasks are not parallel。 For the code, see reply.
    times
    and
    times2
    are parallel, but then
    sleep
    and
    sleep2
    did not,Is there a limit to the implementation of
    map
    ? There are application scenarios, such as a task operating different types of devices, each type of device has two serial tasks, but the number of devices is determined by parameters, such as (Of course, this is pure python serial.):
    ret = []
    for device in params.get('device_type1_list'):
        result = operate_device_type1_step2(operate_device_type1_step1(device))
        ret.append(result)
    for device in params.get('device_type2_list'):
        result = operate_device_type2_step2(operate_device_type2_step1(device))
        ret.append(result)
    # do something for ret
    handle(ret)
    Here, each device can be parallel, but the
    steps
    are serial. So is there a solution to this situation?
    k
    7 replies · 2 participants
  • o

    Oliver Mannion

    10/10/2021, 3:53 AM
    Hiya, Orion looks very slick and clean! I was wondering: 1. v1 has the concept of projects to group flows, does Orion have something similar? 2. How are flows associated with specific prefect agents (ie: labels in v1)?
    j
    m
    3 replies · 3 participants
  • t

    Toprak Nihat Deniz Öztürk

    10/10/2021, 9:51 AM
    Hello everbody, I need to divide dataframe(df) into dataframes -> load them into gpu and works on them. I can write a single task that does all but I prefer to keep them seperated and I already have a subflow that does the job. I wonder if I can call flow in a loop.
    a
    2 replies · 2 participants
  • t

    Tadej Svetina

    10/10/2021, 11:55 AM
    If anyone is trying to add BasicAuth to their server instance (through a reverse proxy such as nginx or traefik), here's how you can deal with some issues related to UI-server communication, and authentication of client https://prefect-community.slack.com/archives/C014Z8DPDSR/p1633866651298300?thread_ts=1632763954.014600&amp;cid=C014Z8DPDSR https://prefect-community.slack.com/archives/C014Z8DPDSR/p1633866798298600?thread_ts=1632763954.014600&amp;cid=C014Z8DPDSR
    a
    1 reply · 2 participants
  • f

    Fina Silva-Santisteban

    10/11/2021, 1:48 AM
    Hi everyone! I’m following the steps outlined in the docs to test a flow’s composition: https://docs.prefect.io/core/idioms/testing-flows.html#testing-flow-composition I’ve noticed that
    flow.tasks
    returns not just tasks but parameters as well! How would you go about checking those? I’ve tried the following without success:
    from prefect import Parameter
    self.assertTrue(Parameter('parameter_name') in flow.tasks)
    # returns False
    e
    a
    +1
    15 replies · 4 participants
  • v

    Vamsi Reddy

    10/11/2021, 3:52 PM
    Hi Everyone!! I have been experimenting with prefect lately and wanted to know if there is anything similar to s3 sensors in prefect. I want to trigger my flow or make my tasks wait/run until there is a file/files being uploaded in the s3 buckets
    k
    c
    +1
    5 replies · 4 participants
  • a

    Andy Waugh

    10/11/2021, 4:17 PM
    Hi there 👋 I’m playing with control flow logic using
    case
    and seeing something unexpected and I’m wondering if it is expected to you or if it is a bug. Equally if I’m doing something weird and there’s a better way, please let me know!
    m
    k
    6 replies · 3 participants
  • h

    Hui Zheng

    10/11/2021, 5:29 PM
    Hello, Just want to report that this morning we had one flow scheduled run missing
    pearl-anteater
    . It is not triggered properly by prefect cloud for some reason. The remaining runs were fine. want to submit to your back-end investigation to avoid further issue.
    n
    2 replies · 2 participants
Powered by Linen
Title
h

Hui Zheng

10/11/2021, 5:29 PM
Hello, Just want to report that this morning we had one flow scheduled run missing
pearl-anteater
. It is not triggered properly by prefect cloud for some reason. The remaining runs were fine. want to submit to your back-end investigation to avoid further issue.
https://cloud.prefect.io/semios/flow-run/1d732c22-5ea8-4660-8484-d3b41df58991
n

nicholas

10/11/2021, 5:30 PM
Hi @Hui Zheng - thanks for letting us know; I’ll let the team know so we can investigate and will report back with any findings.
View count: 3