https://prefect.io logo
r

Romain Vincent

07/04/2023, 12:59 PM
Hello folks, I'm trying to take advantage of
prefect.deployments.steps.run_shell_script
to install my project dependancies using poetry (rather than with
pip_install_requirements
). The script seems to run, but nothing happens and nothing is streamed to my ECS logs or the Prefect console. Here is my pull configuration :
Copy code
pull:
- prefect.deployments.steps.git_clone:
    id: clone-step
    repository: "{{ prefect.blocks.secret.repository }}"
    branch: master
- prefect.deployments.steps.run_shell_script:
    directory: "{{ clone-step.directory }}"
    script: "pip install poetry && poetry install"
    stream_output: true
And the subsequent log from my ECS task :
Copy code
04/07/2023 14:45:34 UTC+2	> Running git_clone step...
04/07/2023 14:45:34 UTC+2	> Running run_shell_script step...
04/07/2023 14:45:34 UTC+2	12:45:34.403 | ERROR | Flow run 'screeching-cuttlefish' - Flow could not be retrieved from deployment.
04/07/2023 14:45:34 UTC+2	Traceback (most recent call last):
04/07/2023 14:45:34 UTC+2	File "<frozen importlib._bootstrap_external>", line 883, in exec_module
04/07/2023 14:45:34 UTC+2	File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
04/07/2023 14:45:34 UTC+2	File "/opt/prefect/prefect_testing/prefect_testing/flows/test.py", line 3, in <module>
04/07/2023 14:45:34 UTC+2	import pandas as pd
04/07/2023 14:45:34 UTC+2	(...)
Any idea? Should I use
pip_install_requirements
instead?
1
Ok found it.
&&
was not parsed properly and clone-step.directory refers to the directory where the project is cloned, not the actual project root directory.
Slowly but surely, we're getting there.
🙌 2
y

Yaron Levi

09/25/2023, 7:29 PM
@Romain Vincent Hi… stuck at the same place. What was the value you out in directory instead of “{{ clone-step.directory }}”
r

Romain Vincent

09/26/2023, 8:31 AM
Here is the whole thing :
Copy code
pull:
- prefect.deployments.steps.git_clone:
    id: clone-step
    repository: '{{ prefect.blocks.secret.test-repository }}'
    branch: main
- prefect.deployments.steps.run_shell_script:
    directory: '{{ clone-step.directory }}'
    script: pip install poetry
    stream_output: true
- prefect.deployments.steps.run_shell_script:
    directory: /opt/prefect/prefect_testing/
    script: poetry install
    stream_output: true
1
y

Yaron Levi

09/26/2023, 8:32 AM
@Romain Vincent Thanks! I was missing the “id” prop.
👍 1