• Fabian Brück

    Fabian Brück

    1 year ago
    Hi! I just tried to run prefect agentless on AWS ECS and s3 storage. For some reason boto3 cannot find any AWS credentials and it fails downloading flows from S3. When I run it locally with credentials as env variables it reports the token is expired. (which isn't true) Is that caused by being executed in a subprocess? Thanks 🙂
    prefect run --name hello-world --execute       
    Looking up flow metadata... Done
    Creating run for flow 'hello-world'... Done└── Name: malachite-lobster└── UUID: 381ad17b-e2d8-4e8b-84aa-ab880d2f0458└── Labels: ['agentless-run-149b4f62']
    └── Parameters: {}
    └── Context: {}
    └── URL: <https://cloud.prefect.io/data-and-bi/flow-run/381ad17b-e2d8-4e8b-84aa-ab880d2f0458>
    Executing flow run...
    └── 01:11:26 | INFO    | Creating subprocess to execute flow run...
    └── 01:11:32 | INFO    | Downloading flow from <s3://bucket/hello_world.py>
    └── 01:11:32 | ERROR   | Error downloading Flow from S3: An error occurred (ExpiredToken) when calling the GetObject operation: The provided token has expired.
    An error occurred (ExpiredToken) when calling the GetObject operation: The provided token has expired.
    Fabian Brück
    Kevin Kho
    +1
    13 replies
    Copy to Clipboard
  • Wilson Bilkovich

    Wilson Bilkovich

    1 year ago
    Anybody seen this error before? I’m getting it in the prefect job pods that get spun up:
    RuntimeError: Unable to find any timezone configuration
    Wilson Bilkovich
    Kevin Kho
    5 replies
    Copy to Clipboard
  • Zac Chien

    Zac Chien

    1 year ago
    Hi, I’ve found there are lots of unnamed tasks such as List, Dict, case(True) occur when I call StartFlowRun with parameter, is there any suggestions to make schematic more readable?
    Zac Chien
    1 replies
    Copy to Clipboard
  • Wilson Bilkovich

    Wilson Bilkovich

    1 year ago
    My understanding is that the Agent deployment is what needs to have local secrets configured, in order to use for example GitHub storage?
    Wilson Bilkovich
    Kevin Kho
    40 replies
    Copy to Clipboard
  • Fabian

    Fabian

    1 year ago
    I have a daily flow "daily_exporter" (which exports data from another system) in the morning at e.g. 6:00am. Beginning at 7:00am there are many other scheduled flows relying on the exported data of "daily_exporter". They all check if the exported data file is available, retry for a while, and if the export is not available after several retries they are configured to fail. Unfortunately, sometimes the initial "daily_exporter" fails (due to low memory or whatever reason). This means I have many failed flows at 09:00am. To resolve the situation it seems, that I have to manually go through each flow run in the UI and restart each failed flow individually. This manual process is very time-consuming and annoying. Does anyone has any practical hint to improve this? Is there any way to just click on a single button to restart ALL failed flows?
    Fabian
    1 replies
    Copy to Clipboard
  • CA Lee

    CA Lee

    1 year ago
    Hi, is there any way to register a single Prefect flow to run different functions on different schedules? (details in thread)
    CA Lee
    Bouke Krom
    +1
    10 replies
    Copy to Clipboard
  • Bouke Krom

    Bouke Krom

    1 year ago
    Good morning! We are trying to run a single flow with many (about 30) different parameters at the same schedule: all Monday morning 9am. This morning exactly 10 runs were performed, the others were silently skipped. We run Prefect server with a single LocalAgent. It seems like we're running into some default 'max 10' setting. The agent displays as having 10 'submittable runs' -- maybe this can be increased but no hits for
    submittable
    in the docs... Any pointers?
    Bouke Krom
    Wilson Bilkovich
    +4
    44 replies
    Copy to Clipboard
  • j

    Jelle Vegter

    1 year ago
    I'm looking to trigger a script on the 4th day before the end of the month. Can a setup like this work where I check if the result of a task is True to run the rest or do I need a different approach? Thanks!
    j
    Bouke Krom
    +4
    19 replies
    Copy to Clipboard
  • YD

    YD

    1 year ago
  • Tony Yun

    Tony Yun

    1 year ago
    Hi, I tried to search but don’t seems find related answer. I want to change the auto-generated container like names “happy-jelly” stuff. But I don’t want to overcomplicate things like to
    create_flow_run
    in code. What’s the simpliest thing to change by giving the flow run a customized run-id? Assume I only need to trigger the flow run using schedules or UI.
    Tony Yun
    Kevin Kho
    14 replies
    Copy to Clipboard