Leon Kozlowski
10/07/2021, 8:00 PMstate_handler
) with prefect cloud, does the webhook URL have to be stored in Prefect Secrets, or can it just be available as an environment variable (my use case would be loading an eternal secret to an eks cluster)Constantino Schillebeeckx
10/07/2021, 8:15 PMon_failure
to it. The callback has (according to the docs) the following signature
def f(obj: Union[Task, Flow], state: State) -> None:
My quest is this: how to identify the failed task from the (Flow) obj. I see I can iterate over the flows' tasks, I'm just not sure how to identify the task's statekevin
10/08/2021, 12:52 AMKlemen Strojan
10/08/2021, 6:06 AMKeyError: 'The secret <some_secret> was not found. Please ensure that it was set correctly in your tenant: <https://docs.prefect.io/orchestration/concepts/secrets.html>'
Now this happens in a mapped task, only for one or two of those mapped tasks, others work fine. The secret exists and is clearly usable. Re-running the flow usually helps, however we are not enjoying that re-running part so much.
Any advice?Marko Herkaliuk
10/08/2021, 8:15 AMrequests.exceptions.HTTPError: 404 Client Error: Not Found for url: <https://api.prefect.io/>
when flow try create other flow. Was it a common problem or a problem on our side?Aldo M
10/08/2021, 9:26 AMJoseph Oladokun
10/08/2021, 12:10 PMTypeError: cannot pickle '_thread.lock' object
Joseph Oladokun
10/08/2021, 12:10 PMZheng Xie
10/08/2021, 1:25 PMMax Kureykin
10/08/2021, 1:31 PMmap
task and some mapped
runs are failed?
Thx in advance)Marwan Sarieddine
10/08/2021, 2:02 PMGarcía Gimenez Ignacio - 6to 2da
10/08/2021, 3:10 PMMichael Pavese
10/08/2021, 5:23 PMMichael Pavese
10/08/2021, 5:23 PMTadej Svetina
10/08/2021, 5:26 PMUnexpected error: TypeError("cannot pickle '_io.BufferedReader' object")
Michael Pavese
10/08/2021, 5:26 PMMatthew Seligson
10/08/2021, 6:21 PMVincent
10/08/2021, 6:28 PMFailed to send heartbeat with exception: HTTPError('404 Client Error: Not Found for url: <https://api.prefect.io/graphql>')
Traceback (most recent call last):
File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/cli/heartbeat.py", line 68, in flow_run
client.update_flow_run_heartbeat(id)
File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 1610, in update_flow_run_heartbeat
self.graphql(
File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 548, in graphql
result = <http://self.post|self.post>(
File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 451, in post
response = self._request(
File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 737, in _request
response = self._send_request(
File "/opt/conda/envs/dev/lib/python3.8/site-packages/prefect/client/client.py", line 647, in _send_request
response.raise_for_status()
File "/opt/conda/envs/dev/lib/python3.8/site-packages/requests/models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: <https://api.prefect.io/graphql>
Chris Reuter
10/08/2021, 7:57 PMYoutube stream▾
Matt Alhonte
10/08/2021, 10:58 PM张强
10/09/2021, 8:18 AM张强
10/09/2021, 2:43 PMmax_retries
, or set default state_handlers
张强
10/10/2021, 1:21 AMmap
tasks are not parallel。
For the code, see reply.
times
and times2
are parallel, but then sleep
and sleep2
did not,Is there a limit to the implementation of map
?
There are application scenarios, such as a task operating different types of devices, each type of device has two serial tasks, but the number of devices is determined by parameters, such as (Of course, this is pure python serial.):
ret = []
for device in params.get('device_type1_list'):
result = operate_device_type1_step2(operate_device_type1_step1(device))
ret.append(result)
for device in params.get('device_type2_list'):
result = operate_device_type2_step2(operate_device_type2_step1(device))
ret.append(result)
# do something for ret
handle(ret)
Here, each device can be parallel, but the steps
are serial.
So is there a solution to this situation?Oliver Mannion
10/10/2021, 3:53 AMToprak Nihat Deniz Öztürk
10/10/2021, 9:51 AMTadej Svetina
10/10/2021, 11:55 AMFina Silva-Santisteban
10/11/2021, 1:48 AMflow.tasks
returns not just tasks but parameters as well! How would you go about checking those? I’ve tried the following without success:
from prefect import Parameter
self.assertTrue(Parameter('parameter_name') in flow.tasks)
# returns False
Vamsi Reddy
10/11/2021, 3:52 PMAndy Waugh
10/11/2021, 4:17 PMcase
and seeing something unexpected and I’m wondering if it is expected to you or if it is a bug. Equally if I’m doing something weird and there’s a better way, please let me know!Hui Zheng
10/11/2021, 5:29 PMpearl-anteater
. It is not triggered properly by prefect cloud for some reason. The remaining runs were fine. want to submit to your back-end investigation to avoid further issue.Hui Zheng
10/11/2021, 5:29 PMpearl-anteater
. It is not triggered properly by prefect cloud for some reason. The remaining runs were fine. want to submit to your back-end investigation to avoid further issue.nicholas
10/11/2021, 5:30 PM