Amy
03/11/2024, 11:55 PMmassumo
03/12/2024, 9:08 AMRJD
03/12/2024, 2:09 PMMax Eggers
03/12/2024, 2:49 PMModuleNotFoundError: No module named 'prefect.testing'
when I use prefect_test_harness
. I am not seeing a specific prefect-testing package that I could use, am I missing it? Should I take prefect
as a test dependency?Abhishek Singh
03/12/2024, 3:19 PMAdam Gold
03/12/2024, 4:29 PMidempotency_key
but not sure that it fits the purpose - maybe I’m missing something?David Beck
03/12/2024, 5:54 PMPending
state without any logging or failures. I've tried debugging this by isolating out a few things:
1. Checked my work pool is able to successfully run flows---it can.
2. Tested our code on the previous Agent/Worker pattern--it runs successfully.
3. Checked my docker image to make sure that the files are copied correctly to the opt/prefect/flows
directory as specified in the prefect.yaml
--it is.
I'm really at a loss as to what I need to do to figure this out further. It seems like it could be a docker issue, but I have no obvious way to investigate it further as the k8s pods that are spun up give no substantive logging or any really. If anyone has a suggestion as to what I could investigate further, I'd appreciate itCarlos Cueto
03/12/2024, 7:17 PMGrace
03/12/2024, 7:38 PMset_task_run_state
using the client; however, I've had no luck accessing the task UUID from the scope of the flow (when I use get_run_context, or read_flow_run, information about the tasks are not populated, maybe because they are still active?). Is this possible?Sodaman
03/12/2024, 10:38 PMfrom prefect import flow, task, serve
import time
@flow(log_prints=True)
def hello_world(name: str = "world", goodbye: bool = False):
x = 1
s = str(x)
print(f"Hello {name} {s} from Prefect! :hugging_face:")
x += 1
if goodbye:
print(f"Goodbye {name}!")
@flow(log_prints=True)
def slow():
time.sleep(100)
print("ok")
if __name__ == "__main__":
a_deploy = hello_world.to_deployment(name="hellow")
b_deploy = slow.to_deployment(name="slow")
serve(a_deploy, b_deploy)
Abhishek Mitra
03/13/2024, 6:48 AMUnable to kill process The process is running on a different host
Why this is happening? What could be the cause of this issue?
Any help appriciated
ThanksJoe D
03/13/2024, 7:35 PMrunning
state - with the last log being days ago. On our ECS cluster I can see that they are no longer running. Does anyone have suggestions for troubleshooting this disconnect between prefect dash/ECS status?Sean Shillo
03/13/2024, 10:37 PMAvinash Santhanagopalan
03/14/2024, 3:57 PMflow.with_options
? I have an async flow refresh_data_for_plugin
that I used to patch like this
patch("refresh_data_for_plugin", new_callable=AsyncMock)
But after adding refresh_data_for_plugin.with_options
I am not able to patch it in a similar way
patch("refresh_data_for_plugin.with_options", new_callable=AsyncMock)
I’m getting TypeError: 'coroutine' object is not callable
Any help appreciated!Tomas Moreno
03/14/2024, 4:41 PMbotocore.errorfactory.InvalidParameterException: An error occurred (InvalidParameterException) when calling the RunTask operation: subnets can have at most 16 items.
I checked the config for my ecs worker service and confirmed it only has 5 subnets not 16. anyone know what else I should look into?Alex Papanicolaou
03/14/2024, 7:07 PMJoey Mukherjee
03/14/2024, 9:34 PMMatthew Bell
03/14/2024, 10:13 PMCharles Leung
03/14/2024, 11:23 PMSean Shillo
03/14/2024, 11:48 PMAlan
03/15/2024, 4:53 PMFile "/opt/anaconda/lib/python3.8/site-packages/httpcore/_backends/anyio.py", line 69, in start_tls
ssl_stream = await anyio.streams.tls.TLSStream.wrap(
File "/opt/anaconda/lib/python3.8/site-packages/anyio/streams/tls.py", line 123, in wrap
await wrapper._call_sslobject_method(ssl_object.do_handshake)
File "/opt/anaconda/lib/python3.8/site-packages/anyio/streams/tls.py", line 131, in _call_sslobject_method
result = func(*args)
File "/opt/anaconda/lib/python3.8/ssl.py", line 944, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)
Daphne
03/15/2024, 5:29 PM{'exception_message': 'Invalid request received.', 'exception_detail': [{'loc': ['body', 'paused'], 'msg': 'extra fields not permitted', 'type': 'value_error.extra'}
I double checked to ensure every parameter declared in the flow function is used somewhere in the function; I'm not sure what else this error message could be referring to. Is this a known issue?Andy Dyer
03/17/2024, 2:12 AMAkhil Jain
03/18/2024, 6:52 AMaviv
03/18/2024, 9:38 AM@flow(task_runner=DaskTaskRunner())
def myflow():
download_data = download_task() # return data in batches
adapted_data = adapt_task.map(download_data) # adapt each batch data
create_json = create_json_task.map(adapted_data) # create json from each batch
merged_data = merge_task(create_json) # merge all batches into one file
all tasks are working fine but when it get to merge task it gives me this error:
prefect.exceptions.UnfinishedRun: Run is in PENDING state, its result is not available.
and when I debug it it doesn’t enter the merge task
have anyone encountered something similar or have any suggestions?Joey Mukherjee
03/18/2024, 3:14 PMFile "/home/jmukherjee/.local/lib/python3.10/site-packages/prefect/cli/flow.py", line 34, in ls
flows = await client.read_flows(
File "/home/jmukherjee/.local/lib/python3.10/site-packages/prefect/client/orchestration.py", line 489, in read_flows
return pydantic.parse_obj_as(List[Flow], response.json())
File "/usr/local/lib/python3.10/dist-packages/httpx/_models.py", line 762, in json
return jsonlib.loads(self.content, **kwargs)
File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)"
I think it is connecting since I can put a print in and get Response [200 OK]. But it is an empty response or something. Any ideas on how I can troubleshoot this?
Fixed! We were running Cosmos in front of Prefect and it was having authentication on which conflicted with prefect. Anyway, turning that off fixed it!Tomas Moreno
03/18/2024, 7:59 PMprefect_aws.workers.ecs_worker.TaskFailedToStart: ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve ecr registry auth: service call has been retried 3 time(s): RequestError: send request failed caused by: Post "<https://api.ecr.us-west-2.amazonaws.com/>": dial tcp 34.223.25.158:443: i/o timeout. Please check your task network configuration.
Jean-Michel Provencher
03/18/2024, 9:02 PMTraceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/prefect/deployments/steps/core.py", line 154, in run_steps
step_output = await run_step(step, upstream_outputs)
File "/usr/local/lib/python3.9/site-packages/prefect/deployments/steps/core.py", line 125, in run_step
result = await from_async.call_soon_in_new_thread(
File "/usr/local/lib/python3.9/site-packages/prefect/_internal/concurrency/calls.py", line 326, in aresult
return await asyncio.wrap_future(self.future)
File "/usr/local/lib/python3.9/site-packages/prefect/_internal/concurrency/calls.py", line 388, in _run_async
result = await coro
File "/usr/local/lib/python3.9/site-packages/prefect/deployments/steps/pull.py", line 195, in pull_with_block
await storage.pull_code()
File "/usr/local/lib/python3.9/site-packages/prefect/runner/storage.py", line 542, in pull_code
await self._block.get_directory(local_path=str(self.destination))
File "/usr/local/lib/python3.9/site-packages/prefect_aws/s3.py", line 508, in get_directory
All my tentatives following the documentation leads to that.Nicolas Ouporov
03/19/2024, 2:39 AMLinenBot
03/19/2024, 7:16 AMjoseph njogu arcride
joined #prefect-community.