https://prefect.io logo
Docs
Join the conversationJoin Slack
Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-australia
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Powered by Linen
prefect-community
  • c

    CA Lee

    06/06/2021, 11:03 AM
    Hello, has anyone encountered issues logging into Prefect from the CLI? Sorry, this isn't a Prefect-related issue, but I'd appreciate if anyone with Docker experience could point me in the right direction --- Prefect version:
    0.14.21
    I am using VS Code, dev container, and running
    pip install prefect[aws]
    in the build step . After the container is done, I am unable to do `prefect auth login -t {token}`as it leads to the error below:
    vscode ➜ /workspaces/dbt-bigquery (main ✗) $ prefect auth login -t $PREFECT_AUTH_TOKEN
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/pathlib.py", line 1287, in mkdir
        self._accessor.mkdir(self, mode)
    FileNotFoundError: [Errno 2] No such file or directory: '/home/vscode/.prefect/client/https-api.prefect.io-graphql'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/local/bin/prefect", line 8, in <module>
        sys.exit(cli())
      File "/usr/local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
        return self.main(*args, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/click/core.py", line 782, in main
        rv = self.invoke(ctx)
      File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/usr/local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
        return callback(*args, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/prefect/cli/auth.py", line 83, in login
        success_login = client.login_to_tenant(
      File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 654, in login_to_tenant
        self._save_local_settings(settings)
      File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 524, in _save_local_settings
        self._local_settings_path.parent.mkdir(exist_ok=True, parents=True)
      File "/usr/local/lib/python3.8/pathlib.py", line 1291, in mkdir
        self.parent.mkdir(parents=True, exist_ok=True)
      File "/usr/local/lib/python3.8/pathlib.py", line 1287, in mkdir
        self._accessor.mkdir(self, mode)
    PermissionError: [Errno 13] Permission denied: '/home/vscode/.prefect/client'
    2 replies · 1 participant
  • j

    Jacob Blanco

    06/07/2021, 6:29 AM
    Any detailed recommendations on doing full blown unit-testing with Prefect flows? I’m looking for some best practices and how to deal with removing certain tasks (running queries against a database).
    k
    1 reply · 2 participants
  • t

    Thomas Hoeck

    06/07/2021, 7:34 AM
    Hi, Sometimes have a problem where my flows suddenly can't connect to Prefect. Have others seen this? There is obvilously still some connection since the run was able to be logged. This resulted in the flow never failing and just keep running (so I only discovered it after the weekend, where it had run for 3 days).
    j
    2 replies · 2 participants
  • s

    Satheesh K

    06/07/2021, 10:14 AM
    Hi, Can someone please help me to suppress INFO level logs while running the flow, I tried
    run_config
    while building the flow, but it seems not working.
    ✅ 1
    e
    g
    10 replies · 3 participants
  • j

    João Amorim

    06/07/2021, 12:35 PM
    hi, in the last two days i have a issue on some scripts running by prefect, i think is some related to prefect running more than one process when starts ... it seems to run three times, instead of having only one process running. All successful flows start two process: where one fails and the other go to the end and finish all ok. This occurs with a script that executes a gdal command and fails because of permission when it goes to overwrite the file (maybe because concurrent process try read/write the same file?)
  • j

    João Amorim

    06/07/2021, 12:48 PM
    Other issue, is in a flow that runs another flows, he is triggering a fail in some of the sub flows, but in prefect ui, there are no fails in the histories of these subflows... First occurs:
    prefect.utilities.exceptions.ClientError: [{'path': ['create_flow_run'], 'message': 'Uniqueness violation.', 'extensions': {'code': 'INTERNAL_SERVER_ERROR'}}]
    and after some time, other error shows:
    prefect.utilities.exceptions.ClientError: [{'path': ['set_task_run_states'], 'message': 'State update failed for task run ID B: provided a running state but associated flow run A is not in a running state.', 'extensions': {'code': 'INTERNAL_SERVER_ERROR'}}]
    does anyone have any guesses as to what is causing this? 😅
    k
    c
    86 replies · 3 participants
  • m

    Matthew Neary

    06/07/2021, 6:16 PM
    Hey everybody, I'm working on a CircleCI pipeline for Prefect running on EKS/ECR I was wondering fi there's a way to run
    prefect build
    without pushing the image into ECR as a test for CI. I'd like to have some verification that the containers will be build correctly before registering them, and definitely don't want to be pushing up an image every time there's a PR opened. Thanks for any feedback.
    k
    3 replies · 2 participants
  • c

    Chris Vrooman

    06/07/2021, 6:18 PM
    Anyone aware of any problems with Prefect Cloud today? We’ve been getting this on all of our flows. Even on flows that have not been recently deployed. Message: [{‘path’: [‘set_flow_run_states’], ‘message’: “‘<=’ not supported between instances of ‘int’ and ‘str’“, ‘extensions’: {‘code’: ‘INTERNAL_SERVER_ERROR’}}]
    k
    14 replies · 2 participants
  • c

    ct

    06/07/2021, 6:57 PM
    Hi. I'm trialing prefect, looks good so far and hybrid model in particular is almost a great fit for us. But I'm concerned about data leakage via logs and exceptions as listed here: https://docs.prefect.io/orchestration/faq/dataflow.html. Is there any code out there that'd prevent these from going to the cloud, but still allow them to be accessed locally?
    j
    c
    5 replies · 3 participants
  • c

    CM

    06/07/2021, 8:10 PM
    Hi, I am new to prefect but very excited to try it out. I am setting up my use case but running into some issues. Are we not meant to have tasks call other tasks? Should tasks be only called from within a flow directly? I am getting a 'Could not infer an active Flow context' exception
    k
    3 replies · 2 participants
  • c

    Claire Herdeman

    06/07/2021, 9:25 PM
    Hi all, I've got a question regrading usage of the BatchSubmit task. Im using it in a flow to launch a series of AWS Batch jobs and it has been working as expected as I test and develop locally. However when I try to register the flow, it is still actually launching the jobs rather than registering the flow. My setup is in the thread
    k
    5 replies · 2 participants
  • f

    Felipe Saldana

    06/07/2021, 10:47 PM
    Hey prefect, I need to write unit tests for my flows and was wondering if y'all had any guidance or examples of some patterns used internally or by existing customers?
    k
    7 replies · 2 participants
  • v

    Volker Hilsenstein

    06/08/2021, 7:14 AM
    Dear all, as a newcomer evaluating Prefect I am struggeling a bit to understand exactly when I may need Prefect server and/or Prefect cloud. Our main application will involve be processing array data (dask/numpy arrays, mostly representing images in the life sciences) from high-throughput imaging experiments. We hope that dask will help us • distribute the processing across HPC nodes (Slurm cluster) • help us to save intermediate results and restart processing runs from intermediate results if the processing has been cancelled or terminated due to an error. From my initial reading I thought that we would have to at least set up a Prefect server to handle distributed computing, but from initial experiments it appears that simply using a
    DaskExecutor
    can handle the distribution across nodes even if no Prefect server is running. Is this correct or am I overlooking something important here? We don't necessarily need all the Web UI things such as the nicely formatted logs and status and the ability to trigger/run registered workflows from the web UI by clicking a button.
    k
    2 replies · 2 participants
  • t

    Talha

    06/08/2021, 11:49 AM
    I am running prefect UI on my machine and I have one task which should run on another machine (OS) in my office. Is it possible to use agent to run a specific task on a specific machine
    m
    2 replies · 2 participants
  • t

    Thomas Weatherston

    06/08/2021, 12:45 PM
    Hey All, I have a task that makes a request to an API and I only want to retry the task if I receive a certain status code from the response (eg. 500). Status codes like 404 I'd like to fail the task and anything else return the result. I'd still like to have the max retries and retry delay but I can't seem to figure out how to fail the task without it retrying! Any ideas?
    k
    j
    +2
    9 replies · 5 participants
  • j

    Joseph Loss

    06/08/2021, 1:49 PM
    is anyone able to run a prefect docker agent when docker is installed with process-isolation? We don't have WSL2 available on our windows servers
  • j

    Joseph Loss

    06/08/2021, 1:50 PM
    ah nvrm I see. Will work with IT to activate hyper-v in our bios settings
    👍 1
  • b

    Banjo Obayomi

    06/08/2021, 2:23 PM
    DataOpsDC will be having a Prefect meetup tonight https://www.meetup.com/DataOpsDC/events/278379544/
    🚀 6
    👍 6
    ‼️ 3
  • s

    Sean Talia

    06/08/2021, 3:37 PM
    I'm reading through the documentation on triggers and it doesn't seem to me like there's a functionality for the specific use case I'm trying to handle. If I have a task T with 3 upstream dependencies – call them A, B, and C – and I want T to run regardless of the outcome of C (which may occasionally fail or time out), is there a way to specify that? In other words, I don't want T to run if A or B don't succeed, but if C doesn't succeed for whatever reason, that's okay
    ➕ 1
    k
    j
    +1
    13 replies · 4 participants
  • m

    Marwan Sarieddine

    06/08/2021, 4:13 PM
    Hi folks, a question about lazarus processes and running an dask-kuberenetes executor via run configs …
    j
    17 replies · 2 participants
  • f

    Florian K. (He/Him)

    06/08/2021, 4:55 PM
    Hi All! I am curious what modeling tools and diagrams y'all are using, if any, to plan and design your workflows. Is UML still a thing amongst professionals (are you adhering to UML notation) or are you a fan of other diagram styles? I am still exploring all the intricacies of Prefect Flows, Tasks, StateHandlers and am wondering how to best visualize these objects and interactions. BTW: I myself have not deliberately used a specific diagram type in since I got out of high school. I rather used mix of diagram types and notation styles that I thought would serve the needs of the task at hand. However, I believe there is value in upping the game a bit, and why not starting with a Prefect workflow 🙂 Looking forward to your thoughts! Cheers, F
    j
    k
    +1
    4 replies · 4 participants
  • v

    Varun Joshi

    06/08/2021, 5:17 PM
    Can I trigger flow when one flow ends?
    k
    j
    +1
    6 replies · 4 participants
  • h

    Hugo Kitano

    06/08/2021, 11:16 PM
    Hi, I’m running a simple flow where get_logs is decorated with an always_run trigger, in case the container fails. In this case, the container fails, and get_logs is correctly run. But why is ping_slack_channel, which is set up with a default “all_successful” trigger. running as well?
    k
    11 replies · 2 participants
  • p

    Peter Roelants

    06/09/2021, 7:26 AM
    Hi Prefect, Is there some guidance on how to mock flows runs for testing? More specifically, I want to write a test to test my flows end-to-end, however my flow contains subflows started via
    StartFlowRun
    . This would require having access to a Prefect server when running my tests. I want to avoid running a Prefect Server since this will tremendously slow down and complicate my unit tests. So I was wondering if there are ways to easily mock flow runs without needing access to a Prefect server?
    k
    6 replies · 2 participants
  • b

    Ben Collier

    06/09/2021, 7:51 AM
    Hi all - if I set a task to manual trigger only, is there a way of triggering it programmatically? I know that sounds counterintuitive, but we have a task which we want to pause and wait on a particular task until a manual process takes place on another system. We then want to trigger the workflow to continue from that system. It would be more elegant than having the task sleeping and polling. Can we use a mutation query against the cloud API to do it?
    j
    7 replies · 2 participants
  • d

    Domantas

    06/09/2021, 8:23 AM
    Hello Prefect, I'm having a problem with
    DaskExecutor
    - after the flow run is finished, DASK workers are not freeing up used RAM resources. There are some references in Dask documentation about memory management(https://distributed.dask.org/en/latest/memory.html) but it is hard to understand how to apply these methods into Prefect code context. Does anybody have dealt with a Prefect Dask memory management problem and knows any solution to it?
    k
    s
    +3
    29 replies · 6 participants
  • j

    Jeremy Tee

    06/09/2021, 10:03 AM
    Hello people, am I misunderstanding what 
    prefect.context.flow_run_version
     is? When I use it, I am expecting it to be based off the prefect UI flow version. However, it just shows the number 
    3
     regardless of what version my flow is
    j
    j
    +1
    7 replies · 4 participants
  • c

    cj

    06/09/2021, 1:58 PM
    Hey folks - I'm having an issue setting up Slack notifications within my local environment. More information in 🧵
    k
    13 replies · 2 participants
  • z

    Zach Schumacher

    06/09/2021, 2:09 PM
    running into an error I haven’t seen before when registering flows. Can someone point me in the right direction?
    k
    m
    14 replies · 3 participants
  • a

    Adam Lewis

    06/09/2021, 2:22 PM
    Hi all, I'm creating a dask-kubernetes cluster within a task (not using DaskExecutor). I'd like to pass the cluster around to other tasks, but it's not cloud pickleable. As a workaround, I'm passing the
    cluster.scheduler_address
    around, and I'm able to create client to the cluster by running
    client = Client(scheduler_address)
    . I want to be sure I shut down the cluster in an ending task which always runs as I've seen the scheduler stay up sometimes when an error is thrown during the flow, but I can't find a way to do that directly from the client object. I believe I can only shut down the cluster with the original cluster object which I'm not sure how to recover. Have y'all seen this and if so, do you have any suggestions for workarounds?
    k
    2 replies · 2 participants
Powered by Linen
Title
a

Adam Lewis

06/09/2021, 2:22 PM
Hi all, I'm creating a dask-kubernetes cluster within a task (not using DaskExecutor). I'd like to pass the cluster around to other tasks, but it's not cloud pickleable. As a workaround, I'm passing the
cluster.scheduler_address
around, and I'm able to create client to the cluster by running
client = Client(scheduler_address)
. I want to be sure I shut down the cluster in an ending task which always runs as I've seen the scheduler stay up sometimes when an error is thrown during the flow, but I can't find a way to do that directly from the client object. I believe I can only shut down the cluster with the original cluster object which I'm not sure how to recover. Have y'all seen this and if so, do you have any suggestions for workarounds?
k

Kevin Kho

06/09/2021, 2:26 PM
Hey @Adam Lewis, not 100% but does the
client.shutdown()
work for your use case?
a

Adam Lewis

06/09/2021, 2:27 PM
Oh my 🤦, yes, by looking at the documentation again, it does look like it will work. Thank you for the help 🎉
👍 1
View count: 2