https://prefect.io logo
Title
m

Mohit Singhal

01/19/2023, 6:56 AM
Hi All, We are getting below error when we are creating deployment file using python . We have already installed s3fs and when we are deploying using CLI its is working fine
RuntimeError: File system created with scheme 's3' from base path '<s3://gdm-de-prefect-dev/de_ingest_workfront>' could not be created. You are likely missing a Python module required to use the given storage protocol.
from de_ingest_workfront import de_ingest_workfront
from prefect.blocks.core import Block
from prefect.deployments import Deployment
from prefect.filesystems import S3

storage = S3.load("test")
##In this "test" block we have set path as bucket_name
deployment = Deployment.build_from_flow(
    flow=de_ingest_workfront,
    name="workfront",
    parameters={},
    infra_overrides={
        "image": "docker_image",
        "service_account_name": "test-sa",
        "image_pull_policy": "Always",
        "namespace": "dp",
    },
    # infrastructure={"type": "kubernetes-job"},
    work_queue_name="k8s",
    storage=storage,
    path="de_ingest_workfront",
    skip_upload=True
)

if __name__ == "__main__":
    deployment.apply()
j

James Sopkin

01/19/2023, 4:27 PM
could you add
FILESYSTEM_EXTRAS=s3fs
to your env and try the deployment again? @Mohit Singhal
otherwise please validate that s3fs is installed and that your virtual env is activated