Emma Rizzi
03/30/2023, 3:24 PMprefect deployment build flow.py:check -n k8sjob -sb s3-bucket/my-s3 -i kubernetes-job --override env.EXTRA_PIP_PACKAGES="s3fs prefect-aws" -a
The build ends with success, and I can see the flow.py being uploaded in my bucket. When I launch the flow, it fails to retrieve the code, stack trace in thread!Downloading flow code from storage at ''
04:47:41 PM
prefect.flow_runs
Flow could not be retrieved from deployment.
Traceback (most recent call last):
File "<frozen importlib._bootstrap_external>", line 839, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1032, in get_data
FileNotFoundError: [Errno 2] No such file or directory: 'flow.py'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 274, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "/usr/local/lib/python3.8/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
return await fn(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/prefect/deployments.py", line 187, in load_flow_from_flow_run
flow = await run_sync_in_worker_thread(load_flow_from_entrypoint, str(import_path))
File "/usr/local/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 91, in run_sync_in_worker_thread
return await anyio.to_thread.run_sync(
File "/usr/local/lib/python3.8/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.8/site-packages/prefect/flows.py", line 809, in load_flow_from_entrypoint
flow = import_object(entrypoint)
File "/usr/local/lib/python3.8/site-packages/prefect/utilities/importtools.py", line 197, in import_object
module = load_script_as_module(script_path)
File "/usr/local/lib/python3.8/site-packages/prefect/utilities/importtools.py", line 160, in load_script_as_module
raise ScriptError(user_exc=exc, path=path) from exc
prefect.exceptions.ScriptError: Script at 'flow.py' encountered an exception: FileNotFoundError(2, 'No such file or directory')
04:47:42 PM
prefect.flow_runs
Downloading flow code from storage at ''
04:48:20 PM
prefect.flow_runs
Flow could not be retrieved from deployment.
Traceback (most recent call last):
File "<frozen importlib._bootstrap_external>", line 839, in exec_module
File "<frozen importlib._bootstrap_external>", line 975, in get_code
File "<frozen importlib._bootstrap_external>", line 1032, in get_data
FileNotFoundError: [Errno 2] No such file or directory: 'flow.py'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/prefect/engine.py", line 274, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "/usr/local/lib/python3.8/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
return await fn(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/prefect/deployments.py", line 187, in load_flow_from_flow_run
flow = await run_sync_in_worker_thread(load_flow_from_entrypoint, str(import_path))
File "/usr/local/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 91, in run_sync_in_worker_thread
return await anyio.to_thread.run_sync(
File "/usr/local/lib/python3.8/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.8/site-packages/prefect/flows.py", line 809, in load_flow_from_entrypoint
flow = import_object(entrypoint)
File "/usr/local/lib/python3.8/site-packages/prefect/utilities/importtools.py", line 197, in import_object
module = load_script_as_module(script_path)
File "/usr/local/lib/python3.8/site-packages/prefect/utilities/importtools.py", line 160, in load_script_as_module
raise ScriptError(user_exc=exc, path=path) from exc
prefect.exceptions.ScriptError: Script at 'flow.py' encountered an exception: FileNotFoundError(2, 'No such file or directory')
from prefect import flow, get_run_logger
from platform import node, platform
@flow
def check():
logger = get_run_logger()
<http://logger.info|logger.info>(f"Network: {node()}. ✅")
<http://logger.info|logger.info>(f"Instance: {platform()}. ✅")
if __name__ == "__main__":
check()
###
### A complete description of a Prefect Deployment for flow 'check'
###
name: k8sjob
description: null
version: e433c7d8474dc025c33ba4ec3a690abe
# The work queue that will handle this deployment's runs
work_queue_name: default
work_pool_name: default-agent-pool
tags: []
parameters: {}
schedule: null
is_schedule_active: true
infra_overrides:
env.EXTRA_PIP_PACKAGES: s3fs prefect-aws
infrastructure:
type: kubernetes-job
env: {}
labels: {}
name: null
command: null
image: prefecthq/prefect:2.8.7-python3.8
namespace: default
service_account_name: null
image_pull_policy: null
cluster_config: null
job:
apiVersion: batch/v1
kind: Job
metadata:
labels: {}
spec:
template:
spec:
parallelism: 1
completions: 1
restartPolicy: Never
containers:
- name: prefect-job
env: []
customizations: []
job_watch_timeout_seconds: null
pod_watch_timeout_seconds: 60
stream_output: true
finished_job_ttl: null
block_type_slug: kubernetes-job
_block_type_slug: kubernetes-job
###
### DO NOT EDIT BELOW THIS LINE
###
flow_name: check
manifest_path: null
storage:
bucket_name: prefect
minio_credentials: null
aws_credentials: null
basepath: ''
endpoint_url: null
credentials:
aws_access_key_id: ###
aws_secret_access_key: '**********'
aws_session_token: null
profile_name: null
region_name: eu-west-1
aws_client_parameters:
api_version: null
use_ssl: true
verify: true
verify_cert_path: null
endpoint_url: ###
config: null
_block_document_id: ###
_block_document_name: s3-credentials
_is_anonymous: false
block_type_slug: aws-credentials
bucket_folder: prefect
_block_document_id: ###
_block_document_name: my-s3
_is_anonymous: false
block_type_slug: s3-bucket
_block_type_slug: s3-bucket
path: ''
entrypoint: flow.py:check
parameter_openapi_schema:
title: Parameters
type: object
properties: {}
required: null
definitions: null
timestamp: '2023-03-30T14:45:30.993032+00:00'
redsquare
03/30/2023, 3:35 PMChristopher Boyd
03/30/2023, 3:36 PMEmma Rizzi
03/30/2023, 3:53 PM