Miguel Moncada
09/09/2025, 11:43 AMMarvin
09/09/2025, 11:43 AMMarvin
09/09/2025, 11:45 AMjob_variables
argument to Flow.deploy()
. The key is pip_packages
and it expects a list of pip requirement strings.
Example:
from prefect import flow
@flow
def my_flow():
return "hello"
if __name__ == "__main__":
my_flow.deploy(
name="my-managed-deployment",
work_pool_name="managed",
job_variables={
"pip_packages": [
"pandas==2.2.2",
"pyarrow>=15",
"prefect-aws>=0.4.0",
],
},
)
Notes:
- pip_packages
entries can be any valid pip requirement specifier (pinned versions, ranges, extras).
- These packages are installed into the managed runtime before your flow runs.
Docs for more details:
- Managed work pool
- Customize job variablesMiguel Moncada
09/09/2025, 11:53 AMFlow run infrastructure exited with non-zero status code:
Essential container in task exited (Error Code: 1)
{
"pip_packages": [
"dataflows@git+<https://x-access-token:XXXXXX@github.com/XXXXX/XXXXX.git@main#subdirectory=XXXX>"
]
}
Marvin
09/09/2025, 11:53 AMMarvin
09/09/2025, 11:56 AM<
and >
will break pip parsing if copied into your job_variables
.
- Use valid PEP 508 syntax for VCS requirements. It should be name @ git+https://...
(note the space on both sides of @
), or the legacy #egg=
form.
- Make sure your token is URL-encoded (tokens sometimes contain characters that must be percent-encoded) and has access to the repo.
- Prefer pinning to a commit SHA instead of a branch for reliability.
Working examples
- PEP 508 style (recommended):
{
"pip_packages": [
"dataflows @ git+<https://x-access-token:<YOUR_TOKEN>@github.com/OWNER/REPO.git@main#subdirectory=path/to/pkg>"
]
}
- Legacy #egg=
style (also valid):
{
"pip_packages": [
"git+<https://x-access-token:<YOUR_TOKEN>@github.com/OWNER/REPO.git@main#egg=dataflows&subdirectory=path/to/pkg>"
]
}
Notes
- Do not include literal angle brackets in the actual value. Only replace <YOUR_TOKEN>
with your real token (URL-encoded).
- If you’re using a GitHub App installation token, the x-access-token:<TOKEN>
pattern is fine. If you’re using a PAT, https://<USERNAME>:<TOKEN>@github.com/...
also works.
- If you continue to see failures, the most robust alternative is to build a custom image with this dependency pre-installed and pass it to Flow.deploy(image=...)
. This avoids VCS installs at runtime entirely.
If it still fails:
- Could you paste the exact pip_packages
entry you’re using (with the token redacted), making sure it’s without Slack’s < >
markup?
- Is the repo private? If so, confirm the token scopes (read access to that repo).
- If you can, try swapping @main
for a commit SHA to rule out branch-related changes.
Related docs:
- Managed work pool
- Customize job variablesMiguel Moncada
09/09/2025, 11:59 AM