Mihai H
11/29/2022, 1:32 PMKelvin Garcia
11/29/2022, 1:51 PMon_failure
function handler either from the flow
variable or the state
variable are passed to that function?Mihai H
11/29/2022, 2:41 PMMihai H
11/29/2022, 2:42 PMMihai H
11/29/2022, 2:42 PMSlackbot
11/29/2022, 2:42 PMFuETL
11/29/2022, 2:45 PMcreate_flow_run(flow_id=flow_id, parameters=parameters)
But sometimes when theres nothing to run or running, the flow is scheduled to run in like 20~30 min, even when i'm not providing the scheduled_start_time parameter, how can i make my flows always get reschedule immediately? (I was think that this is the default behaviour), maybe is the amount of agent that i'm running? Thanks.Sunjay
11/29/2022, 3:04 PMMatt Delacour
11/29/2022, 3:34 PMJean-Michel Provencher
11/29/2022, 4:40 PMPatrick Tan
11/29/2022, 4:59 PMKalise Richmond
11/29/2022, 6:03 PMMac
11/29/2022, 10:01 PMRegisterTaskDefinition operation: Too many concurrent attempts to create a new revision of the specified family.
errorsBradley Hurley
11/30/2022, 4:17 AMGithub-flavored Markdown
, but I don’t think <details>
are actually supported. I searched for existing GitHub issues, but wasn’t able to find anything.Tim Galvin
11/30/2022, 8:51 AMPREFECT_LOGGING_EXTRA_LOGGERS
mechanism? I am running a DaskTaskExecutor with a SLURMCluster
backend to create the set of dask-workers
.
I can see that the loggers in modules I am using in my prefect2 pipeline are beingreported in the slurm
stderr output that is written to disk through the sbatch --error log.err
argument. These messages are following the prefect log handler configurtion ("Time | State | _ name _ - Message"), but they are not being saved to the orion database / presented by the orion UI.Vadym Dytyniak
11/30/2022, 9:43 AMeddy davies
11/30/2022, 10:34 AMroady
11/30/2022, 10:44 AMChristopher
11/30/2022, 2:29 PMVadym Dytyniak
11/30/2022, 2:46 PMl1 = task1.map([1, 2, 3])
l2 = task2.map(l1)
task3.map(l1=l1, l2=l2)
Prefect does not guarantee that in task3
list l1
elements will be passed with corresponding l2
elements?Anna Geller
@flow(log_prints=True)
Nate Roberts
11/30/2022, 6:28 PMAshley Felber
11/30/2022, 7:06 PMKevin McCraney
11/30/2022, 8:15 PMis_successful()
method in an assert in our testing, and to move that to Prefect 2, we were thinking we might call get_run_context()
to get the flow run state and return some value (likely 'Completed' from the state_name
parameter). We've observed that calling get_run_context()
within a flow's context causes the flow to hang. Has anyone observed this, or does anyone have any suggestions for remedy?Simon Macklin
11/30/2022, 8:20 PMSean Conroy
11/30/2022, 9:34 PMis_successful()
method from Prefect 1 for Prefect 2?
In other words, is testing the state name = 'Completed' the best way to check for success?
myflow_state = my_flow(return_state=True)
# Test if state is successfull:
if not myflow_state.name == 'Completed':
raise Exception("myflow failed")
Michael K
11/30/2022, 10:00 PMprefect-airbyte
to trigger syncs on Airbyte Cloud? Or is it only for open source hosted instances?Boggdan Barrientos
11/30/2022, 10:07 PMrun_deployment
for execute two deployments that has to be completed the next subflow.
I'm having issues in the execution of the seconds subflows, the agent dies after one hour of waiting.
How can I apply this logic successfully? I need to execute n deployments and when the n deployments has been completed, execute another flow.
Image 1: The deployments runs properly.
Image 2: The tasks that runs the deployments never ends.wonsun
12/01/2022, 4:09 AMKhyaati Jindal
12/01/2022, 5:01 AMThe above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/ubuntu/5c-ed-sumary-telegram-bot/env/lib/python3.10/site-packages/prefect/agent.py", line 154, in get_and_submit_flow_runs
queue_runs = await self.client.get_runs_in_work_queue(
File "/home/ubuntu/5c-ed-sumary-telegram-bot/env/lib/python3.10/site-packages/prefect/client/orion.py", line 763, in get_runs_in_work_queue
response = await <http://self._client.post|self._client.post>(
File "/home/ubuntu/5c-ed-sumary-telegram-bot/env/lib/python3.10/site-packages/httpx/_client.py", line 1848, in post
return await self.request(
File "/home/ubuntu/5c-ed-sumary-telegram-bot/env/lib/python3.10/site-packages/httpx/_client.py", line 1533, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
File "/home/ubuntu/5c-ed-sumary-telegram-bot/env/lib/python3.10/site-packages/prefect/client/base.py", line 160, in send
await super().send(*args, **kwargs)
File "/home/ubuntu/5c-ed-sumary-telegram-bot/env/lib/python3.10/site-packages/httpx/_client.py", line 1620, in send
response = await self._send_handling_auth(
File "/home/ubuntu/5c-ed-sumary-telegram-bot/env/lib/python3.10/site-packages/httpx/_client.py", line 1648, in _send_handling_auth
File "/home/ubuntu/5c-ed-sumary-telegram-bot/env/lib/python3.10/site-packages/httpx/_client.py", line 1685, in _send_handling_redirects
response = await self._send_single_request(request)
File "/home/ubuntu/5c-ed-sumary-telegram-bot/env/lib/python3.10/site-packages/httpx/_client.py", line 1722, in _send_single_request
response = await transport.handle_async_request(request)
File "/home/ubuntu/5c-ed-sumary-telegram-bot/env/lib/python3.10/site-packages/httpx/_transports/default.py", line 352, in handle_async_request
with map_httpcore_exceptions():
File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
self.gen.throw(typ, value, traceback)
File "/home/ubuntu/5c-ed-sumary-telegram-bot/env/lib/python3.10/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.LocalProtocolError: Invalid input ConnectionInputs.SEND_HEADERS in state ConnectionState.CLOSED
Any idea why this is happening ? I have deployed lot of deployments before, and never had such a issue