oleander-whirlwind
10/29/2025, 9:14 AMhttps://media3.giphy.com/media/v1.Y2lkPTc5MGI3NjExZmQydzBjOHQ2M3BhdWJ4M3V1MGtoZGxuNmloeGh6b2dvaHhpaHg0eSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/3KC2jD2QcBOSc/giphy.gif▾
Marvin
10/29/2025, 9:14 AMMarvin
10/29/2025, 9:18 AM/localfilestore/myimage.png and expect it to render in the Prefect UI. The UI needs a browser-accessible HTTP(S) URL.
- You don’t have to use boto3 to upload; RemoteFileSystem works great with MinIO via the s3:// protocol. You’ll need fsspec’s S3 plugin (s3fs). To generate a URL for the browser, either make the object public or create a presigned URL (MinIO SDK or boto3).
Details and a working pattern:
What the Image artifact expects
- create_image_artifact(image_url=...) is designed for a publicly accessible URL. Local paths (e.g., /localfilestore/...) and file:// URLs won’t render in the Prefect UI.
- Data URIs (data:image/png;base64,...) are not officially supported and may be blocked by the UI’s sanitizer. Prefer an HTTP(S) URL.
- Docs: Artifacts in Prefect 3.x and source: artifacts.py
Using RemoteFileSystem with MinIO (no boto3 required to upload)
- RemoteFileSystem uses fsspec under the hood. For S3/MinIO, install s3fs:
- pip install s3fs
- Your RemoteFileSystem config is on the right track. Don’t include angle brackets in basepath or endpoint_url though.
Example: upload local image → MinIO via RemoteFileSystem → create image artifact with a browser URL
from datetime import timedelta
import os
from prefect import flow, task
from prefect.filesystems import RemoteFileSystem
from prefect.artifacts import create_image_artifact
# 1) Define and save a RemoteFileSystem block (run once, or do this in code)
rfs = RemoteFileSystem(
basepath="<s3://prefect-test/results/>",
settings={
"key": "um_minio",
"secret": "um_minio",
"client_kwargs": {
"endpoint_url": "<http://host.docker.internal:9900>"
},
"use_ssl": False,
# If your MinIO needs path-style addressing, uncomment:
# "config_kwargs": {"s3": {"addressing_style": "path"}},
},
)
rfs.save("prefect-results-minio-s3", overwrite=True)
@task
def upload_to_minio(local_path: str) -> str:
# 2) Load the block and upload the file bytes
rfs = RemoteFileSystem.load("prefect-results-minio-s3")
bucket = "prefect-test"
key = f"results/{os.path.basename(local_path)}" # e.g. results/myimage.png
remote_path = f"s3://{bucket}/{key}"
with open(local_path, "rb") as f:
data = f.read()
rfs.write_path(remote_path, data)
# 3) Construct a browser URL for the uploaded object
# For a public bucket or if MinIO serves objects over HTTP:
# If your MinIO is using path-style addressing:
endpoint = "<http://host.docker.internal:9900>"
public_url = f"{endpoint}/{bucket}/{key}"
return public_url
@task
def make_artifact(image_url: str):
create_image_artifact(
image_url=image_url,
description="My plot from MinIO",
key="plot",
)
return image_url
@flow
def my_flow(local_path: str = "/localfilestore/myimage.png"):
url = upload_to_minio(local_path)
return make_artifact(url)
if __name__ == "__main__":
print(my_flow())
If your bucket is private
- Either make the object public or generate a presigned URL. You can do this with the MinIO Python SDK (no boto3 required):
from datetime import timedelta
from minio import Minio
def presign_minio_url(endpoint, access_key, secret_key, bucket, key, secure=False, expires=timedelta(hours=1)):
client = Minio(
endpoint.replace("http://", "").replace("https://", ""),
access_key=access_key,
secret_key=secret_key,
secure=secure,
)
return client.presigned_get_object(bucket, key, expires=expires)
Then pass that presigned URL to create_image_artifact.
Notes
- s3fs/fsspec handles the S3-compatible upload side. boto3 is not required for uploads with RemoteFileSystem.Marvin
10/29/2025, 9:18 AMRemoteFileSystem settings are the right place to configure MinIO specifics like endpoint_url, use_ssl, and optional path-style addressing.
References
- Artifacts overview: Artifacts (Prefect 3.x)
- Source for ImageArtifact and friends: prefect/artifacts.py
- Filesystems block source: prefect/filesystems.py
If you can share whether your MinIO bucket is public or private, I can tailor the URL generation step accordingly.