Hi. I'm curious about testing

pypy 3.9

since it has JIT. I'm trying to install prefect as the minimal dependency but there are some problems that will probably be outside of the 'supported things' -- prefect has

orjson

as a dependency which is not pypy-supported. I wonder how big of a role that dependency plays in prefect and whether there's some way to hack a bit to make it use some other lib?

Hey, community! How do I set up task caching for k8s jobs? not really clear what setting

PREFECT_LOCAL_STORAGE_PATH

does in this context, I guess points to pod’s filesystem and persistent storage/bucket has to be mounted to a path

in prefect 0.15.13 is there a good way to check whether a task and its output is serializable? I know you can use

is_serializable

to check if entire flows can be serialized but that doesnt allow you to narrow down which tasks are responsible for your serialization issues. what is the best way to debug in this situation?

A question on how to use async tasks: Context

class Result:
  pass

@task
async def my_task_1(arg: str) -> Result:
  ...

@task
async def my_task_2(arg: Result) -> None:
  ...

Question Doc says I can do

await my_task_1('arg')

in a flow, although mypy says

my_task_1('arg')

is not awaitable Is following a best practice?

task_result = await my_task.submit('arg')
result = await task_result.result()

await my_task_2.submit(result)

I am suddenly finding that mapped tasks are not reflecting that the child tasks have completed. I'm not positive that it's not because I introduced some kind of bug while developing, but I've done my best to validate that it's not something I did. Has anyone else seen this? Is something just going wonky with the cloud service at the moment? For example, the flow in the image has finished successfully, with all child tasks having completed. But it still shows 0/13.

I seem to be running into an issue where sub 5 minute flows aren't being scheduled. If I re-register the flow it'll schedule the next 10 runs but once they have been completed now more are scheduled. I've tried using cron clocks and interval clocks and manually creating a schedule in the UI but keep seeing the same result. I'm using Prefect Cloud 1.0 I have two agents both deployed on AWS ECS.

Anyone notices issues with Prefect Cloud today? Our flows show all tasks as "pending", but we can see it logs that they get executed. This happens for all the flows, there wasn't any updates or deployments at our organization since quite a few days. Yesterday all was working fine, quite a few flows completed successfully. The issue seems to be happening since a few hours 🤔

Hello Prefect team, Since yesterday we're experiencing many issues: • restarting flow/task doesn't work • tasks with manual approval don't start after being approved • manually changing task state doesn't work

Hi all, I am trying to figure out whether there is an option for running orion server in url subpath like my-domain.com/prefect. Is there any documentation for this case?

Hi - I am seeing lots of these errors over last few days;

Exception raised while calling state handlers: ClientError([{'path': ['secret_value'], 'message': 'Unable to complete operation. An internal API error occurred.', 'extensions': {'code': 'API_ERROR'}}])
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/prefect/client/secrets.py", line 140, in get
    value = secrets[self.name]
KeyError: 'SLACK_WEBHOOK_URL'

Hi there, I have a question to the Prefect team on the hybrid model but also want to raise awareness on a potential security/data protection issue for other users. According to the Prefect website, code and data should never leave a customers infrastructure. The "exception" here are of course the log entries which can contain any kind of data. In Prefect 1, this was ok for us as we sanitized our own log entries and the logs generated by Prefect itself did not seem to contain any sensitive data. However, in Prefect 2, the log entries generated by Prefect may contain all arguments passed to a function! So far I only found the example below where it happens when the passed argument names do not match what the task expects. However, are there more of these cases were data can leak into the logs? Is this really the intended behavior?

from prefect import task, flow

@task
def task1(arg1):
    pass

@flow
def main_flow():
    task1(arg1_wrong=1)

main_flow()

Logs on Prefect Cloud:

Hi Prefect team. Apart from the issues mentioned before by Krzysztof and Thomas, we also see that some flows are not actually executed. It seems so from the parent flow, with a duration of 30 seconds, and the next dependent flows are triggered, but we cannot find the flow run when accessing the child flow directly. Besides, the UI shows both a “success” and a “failed” status

same here...

We are seeing even weirder behavior now, one flow was running one of its tasks twice (in parallel). Fortunately we have duplicate email detection on our notification gateway, but this is really concerning.

I'm trying to change PREFECT_API_URL by running

prefect config set PREFECT_API_URL=<https://example.com/api>

. However when I try to run the server it defaults back to http://127.0.0.1:4200/api

Hey I created a secret block on prefect cloud ui,

secret-block = Secret.load(“api-key”)

I want to use this block in the body of my flow how can I go about this? Should I just pass

secret-block

? Do I need to use the get method anywhere?

Hello, I see there is way to limit concurrency for a task using tags. But what about flows? How to make sure the same flow (or the same deployment) does not run concurrently?

Is there any nice way to modify the arguments given in the

@task

decorator after that task has been defined? Say I use a task from a collection that and want to use a custom

cache_key_fn

. It looks like i could do it by

from copy import copy
from some_collection import some_task
some_task_cache = copy(some_task)
some_task_cache.cache_key_fn = my_cache_fn

but that seems a bit hacky. The actual problem is the other way around - I want to provide a collection for users, but they would want to modify the task decorator arguments.

@Anna Geller I’m still working on implementing the aws-lambda-serverless-flow template example. On a closer look at the repo, I can see that you didn’t build deployment with prefect(no deployment scripts). Was this intentional or an omission? My attempt to build and apply deployment led to this error:`File "/home/ec2-user/.servy/lib/python3.9/site-packages/botocore/handlers.py", line 278, in validate_bucket_name` raise ParamValidationError(report=error_msg) botocore.exceptions.ParamValidationError: Parameter validation failed: Invalid bucket name “s3:“: Bucket name must match the regex “^[a-zA-Z0-9.\-_]{1,255}$” or be an ARN matching the regex “^arn:(aws).:(s3|s3-object-lambda):[a-z\-0-9]:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*😒3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$”

Hello, recently (within the past week) I have been unable to delete my flow runs (old and new) in Prefect 2 Cloud. I receive a

Failed to delete flow run

message in the UI. Is this a known issue? Or any thought as to why this occurs?

Hi All... I am trying to pull detailed prefect logs using the graphql, however i could not find the table/column name in graphql which holds the detailed logs. Any help would be really appreciated

Hi All, I am working on building profiler(to measure the CPU, Memory and Network IO utilizations) to attach it to any prefect2 task. Any ideas about where I can start from, does ORION collect those kind of data, so we can grab it, or need to create something from scratch?

Hello all, love the project, one question: I find myself really wanting a feature flag to turn task level result caching on and off. My use case seems common enough: if I update my ETL code, I want to invalidate caching so that new results on existing input data are produced. Keying the cache on input argument hash doesn’t solve this use case; the input data is identical, it is my code that has changed. Time based cache expiration doesn’t fit this use case; I don’t know when my code will be updated, it might be tomorrow, it might be in a year. It seems like prefect 1.0 had some options in the API to serve this use case. What am I misunderstanding about prefect 2.0, which as far as I understand has no clear way to address this?

Hi all, In Prefect 2, is it possible to schedule a deployment flow with custom input parameters for each schedule in UI/CLI? Or do I need to create a deployment for each set of parameters? Thanks!

is there a way to associate arbitrary data to orion objects? i.e. git repo, sha, contact team, etc may all be useful from a discoverability perspective on all flows/deployments

Hi, I'm using Prefect v1 and I'm having trouble with configuring timeouts for

ShellTasks

. Setting timeout like it says in the documentation doesn't have any effect, the task doesn't get terminated after 10 seconds in this case. Is that a bug in Prefect or is it an issue on my part and I'm somehow misconfiguring

ShellTask

? I really appreciate any help on this and thank you in advance!

Hi Prefect, We’re still in our journey to evaluate prefect, one question that comes to mind How do we run a flow that takes input parameters ? I mean without deploying to a agent, simply by running it with the command line

Hi all - I am working through the Getting Started tutorial for Deployments using Prefect 2.4.5 via a Miniconda environment on Windows:

name: prefect-service
channels:
  - defaults
dependencies:
  - python=3.9.13
  - pip=22.1.2
  - pip:
    - prefect==2.4.5

I am hitting an error after running

prefect deployment apply log_flow-deployment.yaml

:

Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x00000226A897BEE0>
... redacted for readability ...
RuntimeError: Event loop is closed

Any idea what is happening here? Please let me know if I can provide any additional info.

Hi everyone. We're trying to update our prefect v1 from 1.2.2 to 1.4.0. Since we operate the infrastructure on our own, we use the official helm chart to deploy everything. That part works just fine. Our services make sure that they work with the correct version of prefect when they are about to start after an upgrade in the k8s cluster. This check makes sure that Prefect Server version matches the used prefect library. We use following graphl query to fetch the API version:

query { api { core_version } }

We ran into a problem with version 1.4.0 when prefect API reports version "1.3.0+13.g4142953a5". Is that intended?