Ivan
07/04/2024, 2:06 PMfrom pathlib import Path
from prefect import flow
from prefect_gcp import GcpCredentials, GcsBucket
import pandas as pd
@flow
def upload_to_gcs():
name = 'ready_df.csv'
file_path = Path(name)
df = pd.read_csv(file_path)
gcp_credentials = GcpCredentials.load("...", validate=False)
gcs_bucket = GcsBucket(
bucket="...",
gcp_credentials=gcp_credentials
)
gcs_bucket.upload_from_dataframe(df=df, to_path=name)
return None
if __name__ == "__main__":
upload_to_gcs.serve(name="...")
To run this I install prefect-gcp, and with it the prefect version that comes is 2.19.7. My project is using 3.x. So if I upgrade to 3.x in the terminal where I just installed prefect-gcp, when running python this_flow.py I get:
from prefect.infrastructure import Infrastructure, InfrastructureResult
ImportError: cannot import name 'Infrastructure' from 'prefect.infrastructure'
and I cannot .serve() it while having version 2.19.7 because I guess .serve was not available then and I get:
prefect.deployments.runner.DeploymentApplyError: Error while applying deployment: Client error '422 Unprocessable Entity' for url '<http://127.0.0.1:4200/api/deployments/>'
Is this a known issue with prefect-gcp and prefect 3.x? Thank youAlexander Azzam
07/04/2024, 2:13 PMpip install prefect-gcp
installs the last 2.x compatible version, since the 3.x compatible version is still a prerelease.
⢠Then upgrading just the prefect dependency to 3.x, while keeping around a prefect-gcp
version that is looking for some old import paths.Ivan
07/04/2024, 2:33 PMpip install prefect-gcp
I get prefect 2.19.7, then I do
pip install -U prefect --pre
and then when I run the flow, I get
from prefect.infrastructure import Infrastructure, InfrastructureResult
ImportError: cannot import name 'Infrastructure' from 'prefect.infrastructure'
Alexander Azzam
07/04/2024, 2:33 PMpip install -U prefect-gcp --pre
might work hereIvan
07/04/2024, 2:36 PMAlexander Azzam
07/04/2024, 2:36 PM