:wave: Hi …
Trying the new ECS Push work pools tog...
# ask-communityy
Yaron Levi
09/25/2023, 9:05 PM👋 Hi …
Trying the new ECS Push work pools together with the prefect.yaml file (following https://docs.prefect.io/2.13.2/guides/prefect-deploy/?h=pip_install_requirements#utility-steps)
But can’t seem to make the pip_install_requirements phase work.
✅ 1
Yaron Levi
09/25/2023, 9:05 PMThis is the prefect.yaml file
Yaron Levi
09/25/2023, 9:06 PMYaron Levi
09/25/2023, 9:06 PMThis is the req file that sits in the root folder
Yaron Levi
09/25/2023, 9:06 PMYaron Levi
09/25/2023, 9:08 PMBut when running this simple flow that imports the “decouple” module:
Yaron Levi
09/25/2023, 9:08 PMYaron Levi
09/25/2023, 9:08 PMThe module is not found:
Yaron Levi
09/25/2023, 9:08 PMYaron Levi
09/25/2023, 9:09 PMAny idea how to debug/observe if the pip_install_requirements happens and what are the logs?
Yaron Levi
09/25/2023, 9:09 PMThank you for any help on the issue!
w
Will Raphaelson
09/25/2023, 10:07 PMI see you typing jake, and youre probably more knowledgeable than me on this. i did just try the pip install requirement steps and i didn’t have any trouble. Would it be possible for you to manually specify the working directory? I have a bad feeling about that clone-step.directory variable resolving wrong or something.
There may be some better logging and handling we could do, just trying to unblock you in the short term
Will Raphaelson
09/25/2023, 10:07 PMis it able to install pandas? Is there something funky with this import in particular?
👀 1
j
Jake Kaplan
09/25/2023, 10:16 PMhey! It's likely that your pip install step is not succeeding. Those logs aren't captured by prefect but should be available to view on your ECS task
👏 1
y
Yaron Levi
09/25/2023, 10:18 PM@Jake Kaplan Sure, thanks. Let me check this.
Yaron Levi
09/25/2023, 10:25 PM@Will Raphaelson @Jake Kaplan Tried only the pandas module, still the same:
Yaron Levi
09/25/2023, 10:25 PMYaron Levi
09/25/2023, 10:25 PMalso can’t see any logs in the Task:
Yaron Levi
09/25/2023, 10:26 PMYaron Levi
09/25/2023, 10:26 PMj
Jake Kaplan
09/25/2023, 10:26 PMjust double checking you have
stream_outputy
Yaron Levi
09/25/2023, 10:27 PMchecking
Yaron Levi
09/25/2023, 10:34 PM@Jake Kaplan I’ve added the stream_output:
Yaron Levi
09/25/2023, 10:34 PMYaron Levi
09/25/2023, 10:34 PMbut logs are still empty:
Yaron Levi
09/25/2023, 10:35 PMYaron Levi
09/25/2023, 10:50 PMany other idea how to debug that install requirements.txt step?
j
Jake Kaplan
09/25/2023, 10:56 PMAhh unfortunately you need to configure Cloud watch logs which is a whole thing on it's own.
Can you try one more thing for me?
Jake Kaplan
09/25/2023, 10:56 PMcan you add
id: clone-stepJake Kaplan
09/25/2023, 10:56 PM Copy code
- prefect.deployments.steps.git_clone:
id: clone-stepJake Kaplan
09/25/2023, 10:57 PMyou reference
clone-step.directoryy
Yaron Levi
09/26/2023, 7:04 AM@Jake Kaplan Yes! This was the problem:
Yaron Levi
09/26/2023, 7:04 AMYaron Levi
09/26/2023, 7:04 AMThank you 🙏
🙌 2
2 Views