Grant
09/01/2023, 3:20 PMalex
09/01/2023, 3:23 PMGrant
09/01/2023, 3:27 PMFlow could not be retrieved from deployment.
Traceback (most recent call last):
File "c:\Users\Grant.Arbuckle\Prefect\Data-Pipeline-Flows\.venv\Lib\site-packages\prefect\deployments\steps\core.py", line 124, in run_steps
step_output = await run_step(step, upstream_outputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "c:\Users\Grant.Arbuckle\Prefect\Data-Pipeline-Flows\.venv\Lib\site-packages\prefect\deployments\steps\core.py", line 95, in run_step
result = await from_async.call_soon_in_new_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "c:\Users\Grant.Arbuckle\Prefect\Data-Pipeline-Flows\.venv\Lib\site-packages\prefect\_internal\concurrency\calls.py", line 292, in aresult
return await asyncio.wrap_future(self.future)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "c:\Users\Grant.Arbuckle\Prefect\Data-Pipeline-Flows\.venv\Lib\site-packages\prefect\_internal\concurrency\calls.py", line 316, in _run_sync
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "c:\Users\Grant.Arbuckle\Prefect\Data-Pipeline-Flows\.venv\Lib\site-packages\prefect\deployments\steps\pull.py", line 28, in set_working_directory
os.chdir(directory)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\grant.arbuckle\\OneDrive - Steel Dynamics Inc\\Desktop\\Python Files\\Data-Pipeline-Flows'
alex
09/01/2023, 4:08 PMprefect deploy
was run in a local git repo and there isn’t already a pull step configured in your prefect.yaml
. Can you share your prefect.yaml
file?Grant
09/01/2023, 4:21 PM# Welcome to your prefect.yaml file! You can use this file for storing and managing
# configuration for deploying your flows. We recommend committing this file to source
# control along with your flow code.
# Generic metadata about this project
name: Data-Pipeline-Flows
prefect-version: 2.12.0
# build section allows you to manage and build docker images
build: null
# push section allows you to manage if and how this project is uploaded to remote locations
push: null
# pull section allows you to provide instructions for cloning this project in remote locations
pull:
- prefect.deployments.steps.set_working_directory:
directory: C:\Users\grant.arbuckle\OneDrive - Steel Dynamics Inc\Desktop\Python
Files\Data-Pipeline-Flows
# the deployments section allows you to provide configuration for deploying flows
deployments:
- name: update_qa_salesforce_tables
version: null
tags: []
description: null
entrypoint: salesforce_staging_qa\salesforce_staging_qa_flow.py:update_qa_salesforce_tables
parameters: {}
work_pool:
name: cw_datapro_pool
work_queue_name: null
job_variables: {}
schedule:
cron: 16 * * * *
timezone: EST
day_or: true
# Welcome to your prefect.yaml file! You can use this file for storing and managing
# configuration for deploying your flows. We recommend committing this file to source
# control along with your flow code.
# Generic metadata about this project
name: Data-Pipeline-Flows
prefect-version: 2.12.0
# build section allows you to manage and build docker images
build: null
# push section allows you to manage if and how this project is uploaded to remote locations
push: null
# pull section allows you to provide instructions for cloning this project in remote locations
pull:
- prefect.deployments.steps.git_clone:
repository: <https://github.com/steeldynamics/Data-Pipeline-Flows.git>
branch: All-Flows
# the deployments section allows you to provide configuration for deploying flows
deployments:
- name: update_qa_salesforce_tables
version: null
tags: []
description: null
entrypoint: salesforce_staging_qa\salesforce_staging_qa_flow.py:update_qa_salesforce_tables
parameters: {}
work_pool:
name: cw_datapro_pool
work_queue_name: null
job_variables: {}
schedule:
cron: 16 * * * *
timezone: EST
day_or: true
alex
09/01/2023, 4:43 PMprefect deploy
again after updating your prefect.yaml
? The steps in the pull
section must be saved via the Prefect API for the change to take effect in your deployments.Grant
09/01/2023, 4:49 PMalex
09/01/2023, 4:50 PMprefect deploy
?Grant
09/01/2023, 4:51 PMalex
09/01/2023, 4:55 PMprefect deploy
in the same directory as a prefect.yaml
file.
I notice that you have multiple prefect.yaml
files. We generally recommend having one prefect.yaml
file per project/repository with all the deploy configs in that one file. That would allow you to deploy any flow in the entire project from the project/repo root.Grant
09/01/2023, 4:57 PM