Hey, cloud 2.0 user here. After upgrading an agent to prefect 2.10.2 one of my flow runs which runs once per day failed. After manually retriggering the the run, it worked perfectly. From the logs I can't see any issue with the flow itself. The log message which precedes the error was

2023-04-12T00:01:35.802260259Z 00:01:35.800 | WARNING | Task run 'get_details_from_db-2' - Task run '17c77b72-df31-4899-836d-23eb934becc5' received abort during orchestration: The enclosing flow must be running to begin task execution. Task run is in PENDING state.

I'm not sure whether this is related to version 2.10.2 - I've just never seen a run failing due to this specific message. The full log output please see in the thread.

Hi all, I think I have found a bug 🐛. I have opened a ticket. https://github.com/PrefectHQ/prefect/issues/9193

Hi people, I have a problem with connecting an agent to an https-Endpoint. Following Scenario. I Have an Host A and and Host B. Both in the same network. Host A is my Master and Host B my Slave. I try to connect an Agent from Host A to Host B with the PREFECT_API_URL-Config https://xxxxxxxxxxx/api but get the following error ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131) If i curl from Slave to Master all works finde and the certificate is fetched from the System-Truststore. But the agent seemse to not use these truststore and can't verify the certificate. Any Solutions?

Hi. Is it possible to transfer a workspace ownership to another account? I'm about to start a posistion another company, and currently i am the workspace owner. Can i transfer it to another mail?

Anybody getting any issues like this? I have no idea what's causing this and I can find any logs besides this line. Running on ecs,

Crash detected! Execution was interrupted by an unexpected exception: KeyError: 1267

1267

is random (not sure what it is yet)

question: in kubernetes, kafka consumer launching pipelines using celery executors with task saying "hello world" should be almost immediate right? no spinning up of new containers. why does it take several seconds to spin up a pipeline that does nothing? what's your experience to bring it near real time?

Hi everyone. I’m evaluating Prefect for our company. I’m running a Prefect agent in a Docker container on Azure Container Instances. When I leave the agent idle overnight, the next day I find it ‘stuck’ with a log full of stack traces similar to this:

Failed the last 3 attempts.  Please check your environment and configuration.
Examples of recent errors:

Traceback (most recent call last):
  File "/opt/conda/envs/prefect/lib/python3.10/site-packages/h2/connection.py", 
line 224, in process_input
    func, target_state = self._transitions[(self.state, input_)]
KeyError: (<ConnectionState.CLOSED: 3>, <ConnectionInputs.SEND_HEADERS: 0>)

During handling of the above exception, another exception occurred:

The agent doesn’t accept any flows to execute. The only thing I can do is to restart the agent container. Does anyone know what the issue might be, and how to solve it or work around it?

Good morning. Wondering if anyone has come across this issue when using

prefect-dask

. We seem to be getting a big number of what looks like the ephemeral cluster shutting down early (Stack trace in the thread) which then leads into the flow crashing. Any help greatly appreciated

Hi, Just upgraded to 2.10.2. Console does not show flowrun log while flowrun is in progress. It will show once flowrun is completed. Is this new behavior?

Do folks typically organize their tasks in separate files and then import them into their flows as needed? For example, there are some general tasks that I would most likely use in multiple flows so they do not need to be defined and attached to a particular flow. Curious about this as I see a lot of examples of tasks always being defined within the flow.

One of the tasks in a flow I'm working on is to download a CSV file with Selenium and use the file for further processing. It works well in the local environment, but when we deploy to Prefect Cloud and try to access the file, it does not exist. Selenium doesn't give an error, so I assumed the file was downloaded. It seems Prefect Cloud uses the

/tmp/

directory as its working directory. Does this directory delete files that enter there or the file never entered the directory. I would appreciate any help.

Is it possible to get results of a Prefect flow after it has been completed via API request (I know the flow_run_id)?

Hi, I am getting this error when more than 2 flow runs concurrently "Flow run infrastructure exited with non-zero status code -9". When I set concurrency limit = 2 it is fine, but when concurrency limit >2, only 2 are running, and the rest crashed status with error code -9. In the past it was due to memory shortage but in this case I have plenty of memory. Prefect 2.10.3

Prefect-Ray spins up workers before the inputs for the worker tasks are ready / before the task's dependencies are complete: Hi folks, we're using Prefect and Ray to orchestrate distributed workflows. When we generate a future

x = task_1.submit()

, if we pass that future as an input to another task,

y = task_2.submit(x=x)

, Ray will spin up a worker for

task_2

with the requested resources, and this worker will wait on

x.result()

. This is obviously undesirable! If we had

x1 = task1(); x2 = task2(x1=x1); x3=task3(x2=x2); ...

, all the later tasks in the pipeline would be wasting worker resources while waiting for the earlier workers to complete. We would like the worker for a task to only launch when the task inputs become ready. Is this known behavior / does anyone know how to solve this? Here's a working example:

from prefect import flow, task
from prefect_ray import RayTaskRunner
from prefect_ray.context import remote_options
import time


@task
def task_a():
    time.sleep(100)
    return 'a'


@task
def task_b():
    return 'b'


@flow(
    task_runner=RayTaskRunner(
        address="<ray://ray-cluster-kuberay-head-svc:10001>"
    )
)
def multi_test():
    with remote_options(num_cpus=1, num_gpus=1):
        x = task_a.submit()
    for i in range(10):
        with remote_options(num_cpus=1, num_gpus=1):
            task_b.submit(wait_for=[x])


if __name__ == '__main__':
    print(multi_test())

When run, we immediately get 11 workers created, per the image. CC @Muxin Fang @Haotian Li

Hi there, I've been noticing that all of my flows don't have the dependency arrows in the flow graph. Is there a specific way that I should be structuring my code so that I do get these dependency arrows? Using prefect UI version 2.10.3. Sorry for the dumb question, still pretty new to Prefect.

Hello, is it possible to run two tasks at the same time?

Hello, would anyone know why it takes several seconds for the pipeline to start on a simple task running in pre-warmed dask executors? Any advice on how to bring it to near real time?

hello, I struggle with the new projects feature and couldn't find any detailed example for dockerized deployments, nor any list of parameters needed/accepted in prefect.yaml and deployment.yaml. I have deployed a flow with

prefect deployment build

and I am now trying to migrate to projects to create flows as docker images, but

prefect deploy

fails, details in thread

Hi,

How to use state handlers to trigger a email notification upon failure of a flow ? I am looing for some documentation on this. An example could also help. thanks

Hi, My data source provides updated data through an webhook. Is it possible for Prefect 2 to trigger a flow with an incoming webhook or do I have to deploy a separate web application to handle those incoming webhooks and trigger a flow using the Prefect API ?

@Jake Kaplan Hi Jake, users are seeing the 502 Bad Gateway error in Prefect Cloud again. Just letting you know.

Hello chaps. I’ve got a c# azure function. Can I use this to invoke the prefect cloud to schedule a job? Or rather invoke a job?

Hi, I have Concurrency limit set to 5 in work-queue. I can see 2 flowrun are in running state and 3 are in pending state while I expect 5 flowrun in running state. What can be reason for this? Prefect 2.8.7

Hey community, how's it going 👋? For those of you subscribed to our status page, you should be seeing an alert that the Prefect team is investigating high API Latency. We will keep you all updated as the investigation continues:

🤖 Status Bot: [Investigating] Investigating high API latency - We are investigating sporadic periods of high latency and error rates for some Cloud 2 APIs beginning on April 13 around midnight UTC (April 12th, 5pm PT/8pm ET). We are investigating the problem and will provide updates as we learn more. (https://prefect.status.io)

Hey guys, is possible to create or enable user to create flows using a no-code approach? I want to give my user more control of the flows and i'm wondering if we have a solution for create flows via UI workflows

Hello. I have some unit tests to exercise some code I’ve got around

Deployment.build_from_flow

. Things have been working fine until I tried adding in

work_pool

configuration. Not the tests fail with

except HTTPStatusError as exc:
>           raise PrefectHTTPStatusError.from_httpx_error(exc) from exc.__cause__
E           prefect.exceptions.PrefectHTTPStatusError: Client error '404 Not Found' for url '<http://ephemeral-prefect/api/deployments/>'
E           Response: {'detail': 'Work pool "<work-pool-name>" not found.'}
E           For more information check: <https://httpstatuses.com/404>

This works fine if I’m using

work_queue

, or leave them both out altogether. Any suggestions?

Hello, is there a way to keep a cron schedule while only having it run on weekdays, or will I have to convert my schedule to an

RRule

if I want that level of control? I know in prefect 1 it was possible to still have flexibility with cron schedules

Hey community, another update from our team on the API latency report from earlier.:

🤖 Status Bot: [Monitoring] Investigating high API latency - The issue appears to be related to a routine upgrade of our Kubernetes cluster, and we have rolled back to the previously running version. Our internal metrics show a marked improvement in the p99 latency of our services, and we are still working to identify the underlying cause. We are continuing to monitor the situation, however, we believe that user-impacting issues should have cleared. ✅ (https://prefect.status.io)

Hi, in Prefect V2 when doing a custom run from a deployment, is it possible to pass a JSON object with custom parameters from the UI? This was possible in V1 but I'm not seeing a way to do that in V2, it seems to only allow setting parameters one by one. We have flows with lots of parameters, so would prefer to pass entire JSON objects instead of manually setting them one by one.