I'm suddenly having issues of prefect saying that it can't log to Orion without a flow run id..but it's right as the flow starts, so there is definitely a flow run id. A flow that was working yesterday is failing like this all of a sudden. Any ideas?

Hi, I need some help with the prefect docker deployment please see my issue in the thread.

Are there some arguments to

prefect deployment

that are not covered in the docs? I just spent 30 or so minutes trying to figure out how to provide parameters to the deployment.yaml file without having to manually edit it (which is all I found in the docs). Turns out it is simple, you can use

--param

, but that wasn't in the docs anywhere as far as I can see.

Hi 👋 We’ve just started using Task Run Concurrency and I think we are currently in a stuck state because of it. We defined a limit of 5 tasks, and the UI reports that there are 5 tasks running, but the list is empty and we can’t understand why:

Hey Community! I am using this code for retrieving the last execution of a flow by its ID:

from prefect.orion.schemas.sorting import FlowRunSort
from prefect.orion.schemas.filters import FlowFilter, FlowFilterId,FlowRunFilterStartTime

flow_run = await client.read_flow_runs(
                            limit=1, 
                            sort=FlowRunSort.START_TIME_DESC, 
                            flow_filter=FlowFilter(
                                id=FlowFilterId(any_=[flow_id])
                            )

Any leads on how I can make sure the result I get is for today's last execution? I can't find anything in the documentation or maybe I am missing something here.

Hi all, how do you define

import_paths

in Orion like how you would do so in Prefect 1.0? My flows need to import functions from another folder in my source code and it’s unable to find it when I run the agent.

Hi. Is the Deployment ID Immutable?

Is there an equivalent of Prefect 1.0's

prefect.context.get("scheduled_start_time")

in Prefect 2.0?

FlowRunContext.start_time

seems to return the actual start time of the flow instead of the scheduled start time. If there isn't, is there a way to get the scheduled start time of a flow during runtime?

Hi, we are using prefect 1.2.4 and our flows all fails building now without any change in our code. It seems the error comes from the

pip install prefect

generated automatically while building the flows :

Getting requirements to build wheel did not run successfully

error stack in thread!

Hi. I am registered custom block. Then I try import it:

from __prefect_loader__ import CustomBlockName

But I get an error: ModuleNotFoundError: No module named '__prefect_loader__' How can I fix it?

hey there, we recently updated to

prefect 2.7.11

and I'm now getting an error when trying to deploy from the same

deployment.py

files I've been using all along. It seems to have a problem when executing the

apply

command in the file. Any ideas what could be going on? Thanks for your help! I leave the full message here:

Traceback (most recent call last):
  File "/home/jupyter-santiago/repos/via_ds_amm_gtfs_real_time_tools/valhalla_map_matching/deployments/python/tfl_deployment.py", line 26, in <module>
    deployment.apply()
  File "/home/jupyter-santiago/miniconda3/envs/prefect-gtfs/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 230, in coroutine_wrapper
    return run_async_in_new_loop(async_fn, *args, **kwargs)
  File "/home/jupyter-santiago/miniconda3/envs/prefect-gtfs/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 181, in run_async_in_new_loop
    return anyio.run(partial(__fn, *args, **kwargs))
  File "/home/jupyter-santiago/miniconda3/envs/prefect-gtfs/lib/python3.10/site-packages/anyio/_core/_eventloop.py", line 70, in run
    return asynclib.run(func, *args, **backend_options)
  File "/home/jupyter-santiago/miniconda3/envs/prefect-gtfs/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 292, in run
    return native_run(wrapper(), debug=debug)
  File "/home/jupyter-santiago/miniconda3/envs/prefect-gtfs/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/jupyter-santiago/miniconda3/envs/prefect-gtfs/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/home/jupyter-santiago/miniconda3/envs/prefect-gtfs/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
    return await func(*args)
  File "/home/jupyter-santiago/miniconda3/envs/prefect-gtfs/lib/python3.10/site-packages/prefect/deployments.py", line 651, in apply
    deployment_id = await client.create_deployment(
  File "/home/jupyter-santiago/miniconda3/envs/prefect-gtfs/lib/python3.10/site-packages/prefect/client/orion.py", line 1347, in create_deployment
    response = await <http://self._client.post|self._client.post>(
  File "/home/jupyter-santiago/miniconda3/envs/prefect-gtfs/lib/python3.10/site-packages/httpx/_client.py", line 1848, in post
    return await self.request(
  File "/home/jupyter-santiago/miniconda3/envs/prefect-gtfs/lib/python3.10/site-packages/httpx/_client.py", line 1533, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/home/jupyter-santiago/miniconda3/envs/prefect-gtfs/lib/python3.10/site-packages/prefect/client/base.py", line 253, in send
    response.raise_for_status()
  File "/home/jupyter-santiago/miniconda3/envs/prefect-gtfs/lib/python3.10/site-packages/httpx/_models.py", line 749, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '409 Conflict' for url '<http://10.90.64.221:4200/api/deployments/>'

Hi team! I am trying to add a docker-container block to the local interface. When running

prefect block register -m prefect_docker

I only get the docker host and docker registry details blocks in the UI. How do i get the docker container block?

Hi there! Just a question about cache with maps. Description in the thread:

Heya! [Moved here from #prefect-getting-started] I'm struggling to find resources / reference for scheduling run container tasks with Prefect. The prefect docs on Running flows with Docker does not really fit my case as it relies on python code present, but in my case all I need it to do is run a pre-build image. Currently I have cron entry to do this

0 * * * * docker compose -f /some/path/docker-compose.yml run --rm my_service my_command

the docker-compose.yml

version: '3.8'
services:

  my_service:
    build: app/.
    env_file:
      - app/.env
    logging:
      driver: journald

How can I tell prefect to create a deployment to • run a container with a pre-defined image (I want to build it separately.. the docker-compose currently takes care of that) • load .env file into the container's envs • run specified

my_cmd

(entry-point's args) All I really need is some sort of reference material of building such deployment 🤷

We are on Prefect 2.7.11 and occasionally seeing this error:

Task run '40d8d867-e54c-47dc-b098-a37e17d5dff0' received abort during orchestration: Error validating state: DBAPIError("(sqlalchemy.dialects.postgresql.asyncpg.Error) <class 'asyncpg.exceptions.QueryCanceledError'>: canceling statement due to statement timeout") Task run is in RUNNING state.

It occurs randomly during task executions and results in the flow getting terminated in a Failed state. Happened twice over last weekend with the flow being scheduled to run every 15 minutes. What is causing this error, and is there something that can be done on our side?

What is the reason that this code throws

RuntimeError: Tasks cannot be run outside of a flow

? The

parse

function is a task.

with ThreadPoolExecutor() as executor
   res = [executor.submit(parse, root, n) for n in range(0, table_count, 5)]

data = [r.result() for r in as_completed(res)]

Is there any way to get the name/id/etc. of the task that failed in a flow? Idea being to be able to put in a slack message which task in the flow failed.

Hello I have a really weird error. When I run 3-5 flows in the same moment, there isn’t any problem. But when I try to run over 50 flows, I get the error :

concurrent.futures._base.CancelledError

I see that my cluster hasn’t any problem to create over 50 pods simultaneously. Queue concurrency limit is 500. Stack trace in thread. PS: Does it rely that during transition phase to Prefect2 I used free cloud account? (if yes where can I can see limits)

Hi, I am using dockers for prefect and my flows are stored in MinIO. /root/flows is my working directory where the flow i want to run do exist. Here is my code

from prefect.deployments import Deployment
import requests

deploy = Deployment.build_from_flow(flow=start_inferencing, name="api deployment", work_queue_name="testing")
id_ = deploy.apply()
print("This is Deployment ID", id_)

def create_flow_run(deployment_id: str, base_url="<http://IP:4200/api>"):
    print(f"{base_url}/deployments/{deployment_id}/create_flow_run")
    return <http://requests.post|requests.post>(
        url=f"{base_url}/deployments/{deployment_id}/create_flow_run",
        json={
            "name": "trigger using api",
            "state": {"type": "SCHEDULED"},
            "parameters": {"my_param": "bef5f437-49b7-43ec-aac8-a8b4747cb67b", "file_path": "root/flows/tmp_folder"},
        },
    )


r = create_flow_run(id_)
print(r.status_code)
print(r.text)

When i try to open base url i.e. http://IP:4200 it open but this code throws me a error saying deployment not found, but i guess the whole point of this code is to create a deployment and then run the flow.

404
{"detail":"Deployment not found"}

can anyone please help, what i am doing wrong.

Hi all, I'm seeing stats not being updated on cloud.prefect.io (1 flow as running, however the pod is already marked Completed)

Prefect V2 question - I am trying to use try/except/finally, but I'm having trouble combining both "do not run until all of the tasks in the Try block have ran" and "run at the end, no matter what". I'm using

wait_for

, because without that the tasks in the Finally block ran immediately. But with

wait_for

, the tasks in the Finally block do not run if something has failed.

Hey y'all this error is getting more frequent, any idea how to fix this?

State message: Flow run encountered an exception. Traceback (most recent call last):
 File "/usr/local/lib/python3.8/site-packages/httpcore/backends/asyncio.py", line 34, in read
   return await self._stream.receive(max_bytes=max_bytes)
 File "/usr/local/lib/python3.8/site-packages/anyio/streams/tls.py", line 195, in receive
   data = await self._call_sslobject_method(self._ssl_object.read, max_bytes)
 File "/usr/local/lib/python3.8/site-packages/anyio/streams/tls.py", line 137, in _call_sslobject_method
   data = await self.transport_stream.receive()
 File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 1265, in receive
   await self._protocol.read_event.wait()
 File "/usr/local/lib/python3.8/asyncio/locks.py", line 309, in wait
   await fut
asyncio.exceptions.CancelledError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "/usr/local/lib/python3.8/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
   yield
 File "/usr/local/lib/python3.8/site-packages/httpcore/backends/asyncio.py", line 36, in read
   return b""
 File "/usr/local/lib/python3.8/site-packages/anyio/_core/_tasks.py", line 118, in __exit__
   raise TimeoutError
TimeoutError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "/usr/local/lib/python3.8/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
   yield
 File "/usr/local/lib/python3.8/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
   resp = await self._pool.handle_async_request(req)
 File "/usr/local/lib/python3.8/site-packages/httpcore/_async/connection_pool.py", line 253, in handle_async_request
   raise exc
 File "/usr/local/lib/python3.8/site-packages/httpcore/_async/connection_pool.py", line 237, in handle_async_request
   response = await connection.handle_async_request(request)
 File "/usr/local/lib/python3.8/site-packages/httpcore/_async/connection.py", line 90, in handle_async_request
   return await self._connection.handle_async_request(request)
 File "/usr/local/lib/python3.8/site-packages/httpcore/_async/http11.py", line 112, in handle_async_request
   raise exc
 File "/usr/local/lib/python3.8/site-packages/httpcore/_async/http11.py", line 91, in handle_async_request
   ) = await self._receive_response_headers(**kwargs)
 File "/usr/local/lib/python3.8/site-packages/httpcore/_async/http11.py", line 155, in _receive_response_headers
   event = await self._receive_event(timeout=timeout)
 File "/usr/local/lib/python3.8/site-packages/httpcore/_async/http11.py", line 191, in _receive_event
   data = await self._network_stream.read(
 File "/usr/local/lib/python3.8/site-packages/httpcore/backends/asyncio.py", line 36, in read
   return b""
 File "/usr/local/lib/python3.8/contextlib.py", line 131, in __exit__
   self.gen.throw(type, value, traceback)
 File "/usr/local/lib/python3.8/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
   raise to_exc(exc)
httpcore.ReadTimeout
The above exception was the direct cause of the following exception:
httpx.ReadTimeout

Hey 😄 Quick question What are the

parent_task_id

keys of the

FlowRun

object pointing to ? It doesn't seem to point to a FlowRun uuid or a Flow uuid 🤔 ?

Will late runs delete if I delete the work queue they are attached to?

Hello community, Wishing you all a great day ahead! I am new to github actions. Right now, I am looking forward to migrate ECS clusters in cloudformation stacks from Bamboo to Github actions. I need your assistance.

Hi, are there any default concurrency limits for flows? For me it seems only 1 flow is able to run concurrently - I am using prefect cloud and 1 agent

Hi everyone. When I inspected usage data from Prefect Cloud I was able to see usage for only 26 days. What is the retention period for usage data? Thanks.

When setting up automations, are automations triggered by the union or intersection of flow run tags? e.g., given automation with flow run tags

production

,

auto-scheduled

, should we be expecting it to run for any flow tagged

production

, or only those both

auto-scheduled

and

production

? It appears to be that either tag will trigger it, which is not what i was aiming for

Hi, my orion server threw this error

TimeoutError FlowRunNotifications took 5.071013 seconds to run, which is longer than its loop interval of 4 seconds.

Why exactly is this happening and how can this be avoided?

Hi, I am running a Flow, which runs a subflow. When I cancel the Flow, it is stuck in "Cancelling" state, and the subflow is actually still running. The subdlow just polls some API and uses

asyncio.sleep

in between. How can I make it so that the subflow gets cancelled too?