https://prefect.io logo
g

Geese Howard

09/13/2023, 4:58 AM
Hi all, I am testing prefect 2 + GCP combination: • Prefect cloud • Using Google Kubernetes Engine to deploy prefect-agent • Google Storage Buckets as storage I am using the following command line script to deploy my flow:
Copy code
prefect deployment build  \
    hello.py:hello  \
    --name "dev: hello cpu" \
    --pool gke \
    --work-queue dev \
    --storage-block gcs-bucket/prefect-flows \
    --path prefect_practice  \
    --infra-block kubernetes-job/cpu-job-test  \
    --timezone Asia/Seoul  \
    --cron '*/1 * * * *'  \
    --apply
Using the deployment code above, 1. The flow code is successfuly stored in gcs-bucket/prefect-flows/prefect_practice folder (something I checked for more than ten times) 2. The agent successfully retrieves flow deployment, and creates new kubernetes pod to run the hello.py:hello flow. 3. But the newly created pod fails running the flow printing out the following error:
Copy code
/usr/local/lib/python3.9/runpy.py:127: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
 warn(RuntimeWarning(msg))
02:57:05.303 | INFO  | Flow run 'smoky-crab' - Downloading flow code from storage at 'prefect_practice'
02:57:20.536 | ERROR  | Flow run 'smoky-crab' - Flow could not be retrieved from deployment.
Traceback (most recent call last):
 File "<frozen importlib._bootstrap_external>", line 846, in exec_module
 File "<frozen importlib._bootstrap_external>", line 982, in get_code
 File "<frozen importlib._bootstrap_external>", line 1039, in get_data
FileNotFoundError: [Errno 2] No such file or directory: 'hello.py'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
 File "/usr/local/lib/python3.9/site-packages/prefect/engine.py", line 394, in retrieve_flow_then_begin_flow_run
  flow = await load_flow_from_flow_run(flow_run, client=client)
 File "/usr/local/lib/python3.9/site-packages/prefect/client/utilities.py", line 51, in with_injected_client
  return await fn(*args, **kwargs)
 File "/usr/local/lib/python3.9/site-packages/prefect/deployments/deployments.py", line 232, in load_flow_from_flow_run
  flow = await run_sync_in_worker_thread(load_flow_from_entrypoint, str(import_path))
 File "/usr/local/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 91, in run_sync_in_worker_thread
  return await anyio.to_thread.run_sync(
 File "/usr/local/lib/python3.9/site-packages/anyio/to_thread.py", line 33, in run_sync
  return await get_asynclib().run_sync_in_worker_thread(
 File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
  return await future
 File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 807, in run
  result = context.run(func, *args)
 File "/usr/local/lib/python3.9/site-packages/prefect/flows.py", line 1164, in load_flow_from_entrypoint
  flow = import_object(entrypoint)
 File "/usr/local/lib/python3.9/site-packages/prefect/utilities/importtools.py", line 201, in import_object
  module = load_script_as_module(script_path)
 File "/usr/local/lib/python3.9/site-packages/prefect/utilities/importtools.py", line 164, in load_script_as_module
  raise ScriptError(user_exc=exc, path=path) from exc
prefect.exceptions.ScriptError: Script at 'hello.py' encountered an exception: FileNotFoundError(2, 'No such file or directory')
I have been trying to fix this issue for a while, and could not find satisfactory answer from the community. Does anyone suffered from similar error or know how to deal with this situation?