Elliott Wilson
01/17/2023, 12:40 PM/data/prefect
and it seems that it can not pull the flow from the deployment when using a path.
I have also created a new repo which has the deployment and the flow in the root directory and this works with the same PAT token.
I am not sure if its an issue with the nested path or permissions on the file. Does any one have any ideas? I have included the deployment and stack trace, in the thread.###
### A complete description of a Prefect Deployment for flow 'mixpanel-to-s3'
###
name: mixpanel_to_s3
description: This flow extracts mixpanel data from the api and sends it to an s3 bucket.
version: 9f231da05569c0499e2e5b59d36049fa
# The work queue that will handle this deployment's runs
work_queue_name: default
work_pool_name: null
work_pool_queue_name: null
tags: []
parameters: {}
schedule:
cron: 0 2 * * *
timezone: Europe/London
day_or: true
infra_overrides: {}
###
### DO NOT EDIT BELOW THIS LINE
###
flow_name: mixpanel-to-s3
manifest_path: null
infrastructure:
type: docker-container
env:
EXTRA_PIP_PACKAGES: mixpanel_utils==2.2.5 prefect_aws==0.2.1 s3fs
labels: {}
name: null
command: null
image: prefecthq/prefect:2.7.8-python3.9
image_pull_policy: null
image_registry: null
networks: []
network_mode: null
auto_remove: true
volumes: []
stream_output: true
memswap_limit: null
mem_limit: null
privileged: false
_block_document_id:
_block_document_name: mixpanel-s3-docker-container
_is_anonymous: false
block_type_slug: docker-container
_block_type_slug: docker-container
storage:
repository: <https://github.com/gaia-family/monorepo.git>
reference: main
access_token: '**********'
_block_document_id:
_block_document_name: github-monorepo
_is_anonymous: false
block_type_slug: github
_block_type_slug: github
path: /data/prefect
entrypoint: mixpanel_to_s3.py:mixpanel_to_s3
parameter_openapi_schema:
title: Parameters
type: object
properties: {}
required: null
definitions: null
timestamp: '2023-01-17T12:24:47.396515+00:00'
WARNING: You are using pip version 21.3.1; however, version 22.3.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
/usr/local/lib/python3.9/runpy.py:127: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
warn(RuntimeWarning(msg))
12:28:50.491 | INFO | Flow run 'enlightened-jerboa' - Downloading flow code from storage at 'monorepo/data/prefect'
12:28:53.081 | ERROR | Flow run 'enlightened-jerboa' - Flow could not be retrieved from deployment.
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/prefect/engine.py", line 266, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "/usr/local/lib/python3.9/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
return await fn(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/prefect/deployments.py", line 174, in load_flow_from_flow_run
await storage_block.get_directory(from_path=deployment.path, local_path=".")
File "/usr/local/lib/python3.9/site-packages/prefect/filesystems.py", line 922, in get_directory
copytree(src=content_source, dst=content_destination, dirs_exist_ok=True)
File "/usr/local/lib/python3.9/shutil.py", line 566, in copytree
with os.scandir(src) as itr:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmphajvvwbjprefect/monorepo/data/prefect'
09:28:54.436 | INFO | prefect.infrastructure.docker-container - Docker container 'enlightened-jerboa' has status 'removing'
09:28:54.445 | INFO | prefect.infrastructure.docker-container - Docker container 'enlightened-jerboa' has status 'removing'
Bianca Hoch
01/23/2023, 3:34 PMElliott Wilson
01/26/2023, 6:01 PM