Assuming Prefect 3.x: How artifacts are stored - Artifacts are records in the Prefect orchestration database (Cloud or Server), with their content stored as JSON. They are not automatically written to object storage. - Built-in types include link, markdown, table, progress (updatable), and image (URL reference). - Artifacts can be given a `key`; multiple artifacts may share a key (useful for “latest” vs. history). - Practical guidance: keep artifact payloads small (e.g., summaries, metadata, links). Large blobs belong in external storage, referenced by a link artifact. - Docs: Concepts • How-to Should you store log files as artifacts? - Generally, no. Prefect already stores logs separately (in the logs table) and shows them in the UI and CLI. - If you want to persist or share full log files, the recommended pattern is: 1) Store the log file in external storage (S3/GCS/etc.) 2) Create a Link artifact that points to it 3) Optionally create a small Markdown artifact with a summary Examples - Create a link to a large log file stored externally:

from prefect.artifacts import create_link_artifact

create_link_artifact(
    key="run-logs",
    link="<https://my-bucket.s3.amazonaws.com/logs/flow-run-123.log>",
    link_text="Download full logs"
)

- Create a short log summary as markdown:

from prefect.artifacts import create_markdown_artifact

create_markdown_artifact(
    key="run-summary",
    markdown="# Run Summary\n\n- Errors: 0\n- Warnings: 3\n- Duration: 12m",
    description="Concise summary of this run"
)

- View logs via CLI (preferred for raw logs):

prefect flow-run logs <FLOW_RUN_ID>
# or save to file:
prefect flow-run logs <FLOW_RUN_ID> > flow.log

More on logging: Add logging

Short answer: yes—there are several good “free or nearly free” options to temporarily store logs. Pick based on your security, size, and retention needs. Good options (S3-compatible, easy to automate) - Cloudflare R2 (free tier): ~10 GB storage free and zero egress; S3-compatible API, works with boto3. Great for short-lived objects + lifecycle rules. - Backblaze B2 (free tier): ~10 GB free, ~1 GB/day free egress; also S3-compatible. - AWS S3 (free tier for new accounts): ~5 GB for 12 months; after that, standard charges apply. Supports lifecycle rules and presigned URLs. Other options (quick but less controlled) - Ephemeral paste/file services (e.g., transfer.sh, 0x0.st, dpaste): easy for non-sensitive, short logs; retention is limited and links are public—use with caution. Best practices - Keep buckets private and share access via presigned URLs that expire (e.g., 1–24 hours). - Enable lifecycle rules to auto-delete logs after N days. - Gzip logs before upload to cut costs and speed uploads. - In Prefect, create a Link artifact pointing to the log file so it’s visible in the run UI, but don’t embed the full log content as an artifact. Example: upload log to Cloudflare R2 (S3-compatible) and create a Prefect link artifact

import boto3
from prefect.artifacts import create_link_artifact

# R2 is S3-compatible; use your account’s endpoint and keys
endpoint = "https://<account_id>.<http://r2.cloudflarestorage.com|r2.cloudflarestorage.com>"
region = "auto"  # R2 accepts 'auto' or 'us-east-1' for signing
bucket = "my-temp-logs"
key = "runs/123/flow.log"

s3 = boto3.client(
    "s3",
    endpoint_url=endpoint,
    aws_access_key_id="<R2_ACCESS_KEY_ID>",
    aws_secret_access_key="<R2_SECRET_ACCESS_KEY>",
    region_name=region,
)

# Upload the log file
with open("flow.log", "rb") as f:
    s3.upload_fileobj(f, bucket, key, ExtraArgs={"ContentType": "text/plain"})

# Create a short-lived presigned URL
url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": bucket, "Key": key},
    ExpiresIn=3600,  # 1 hour
)

# Surface it in Prefect UI as a Link artifact
create_link_artifact(
    key="full-logs",
    link=url,
    link_text="Download full logs (expires in 1h)",
    description="Temporary link to full run logs",
)

If you share a bit about: - expected log size (KB/MB/GB), - how long you need to keep them, - whether logs are sensitive, …I can recommend the best fit and a minimal setup (including lifecycle policies and example IaC).

Given 10 MB max per log, 30-day TTL, and non-sensitive data, the simplest low-cost approach is an S3-compatible object store with a 30-day lifecycle rule. Two good options: - Cloudflare R2: generous free tier, zero egress, S3-compatible API. Add a lifecycle rule to auto-delete after 30 days. - Docs: Cloudflare R2 docs - Backblaze B2: free storage tier (S3-compatible), straightforward lifecycle policies. - Docs: B2 S3-compatible API Recommended pattern - Gzip the log (shrinks transfer/storage). - Upload to R2/B2 under a predictable key. - Generate a short-lived presigned URL (e.g., 1–24 hours) for sharing. - Create a Prefect Link artifact to surface the URL in the run UI. - Use a bucket lifecycle rule to auto-delete after 30 days. Example: R2 upload + 30-day lifecycle + Prefect link artifact

import gzip
import io
import boto3
from prefect.artifacts import create_link_artifact

# Configure your R2 S3-compatible client
endpoint = "https://<account_id>.<http://r2.cloudflarestorage.com|r2.cloudflarestorage.com>"
region = "auto"
bucket = "temp-logs"
key = "runs/123/flow.log.gz"

s3 = boto3.client(
    "s3",
    endpoint_url=endpoint,
    aws_access_key_id="<R2_ACCESS_KEY_ID>",
    aws_secret_access_key="<R2_SECRET_ACCESS_KEY>",
    region_name=region,
)

# One-time: ensure a 30-day delete lifecycle rule exists on the bucket
# (run this once per bucket setup)
s3.put_bucket_lifecycle_configuration(
    Bucket=bucket,
    LifecycleConfiguration={
        "Rules": [
            {
                "ID": "delete-after-30-days",
                "Status": "Enabled",
                "Filter": {"Prefix": ""},
                "Expiration": {"Days": 30},
            }
        ]
    },
)

# Gzip and upload
log_text = open("flow.log", "rb").read()
buf = io.BytesIO()
with gzip.GzipFile(fileobj=buf, mode="wb") as gz:
    gz.write(log_text)
buf.seek(0)

s3.upload_fileobj(
    buf,
    bucket,
    key,
    ExtraArgs={
        "ContentType": "text/plain",
        "ContentEncoding": "gzip",
        "CacheControl": "max-age=60",  # optional
    },
)

# Create a short-lived presigned URL (1 hour)
url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": bucket, "Key": key},
    ExpiresIn=3600,
)

# Surface in Prefect as a Link artifact
create_link_artifact(
    key="full-logs",
    link=url,
    link_text="Download full logs (expires in 1h)",
    description="Temporary log storage (auto-deletes in 30 days)",
)

Backblaze B2 notes - Use endpoint like https://s3.us-west-004.backblazeb2.com with your region. - Same boto3 flow works (S3-compatible). - Lifecycle rules can be configured via B2 UI or via the S3 lifecycle API. Why this works well - Free/low-cost for your size and retention. - Automatic cleanup after 30 days. - Prefect UI shows a clean link via an artifact without storing large blobs in the orchestration DB. If you want, I can provide a minimal Terraform snippet for either R2 or B2 to create the bucket + 30-day lifecycle in one step.

Short answer: Prefect does not fetch logs from CloudWatch. In Prefect 3.x, flow/task logs are sent directly from your ECS container to the Prefect API and stored in the Prefect database (Cloud or Server). Those are the logs you see in the Prefect UI and via CLI. Your ECS “awslogs”/CloudWatch configuration collects container stdout/stderr separately; Prefect doesn’t ingest those from CloudWatch. What this means for your setup - With your awslogs config, you’ll have two destinations: - Prefect logs: batched to the Prefect API by the APILogHandler and stored in the Prefect DB (visible in Prefect UI/CLI) - CloudWatch logs: container stdout/stderr via the ECS awslogs driver (visible in CloudWatch) - Disabling Prefect’s API logging (PREFECT_LOGGING_TO_API_ENABLED=false) will stop logs from appearing in Prefect UI; they’ll still go to CloudWatch via ECS. - If the ECS task cannot reach the Prefect API (network/firewall), Prefect logs won’t make it to the UI; CloudWatch will still have stdout/stderr. Common configs - Ensure API logging is on (default is on):

PREFECT_LOGGING_TO_API_ENABLED=true
PREFECT_LOGGING_LEVEL=INFO
# Optional tuning
PREFECT_LOGGING_TO_API_BATCH_INTERVAL=1.0
PREFECT_LOGGING_TO_API_MAX_LOG_SIZE=1000000

- Capture prints as logs:

@flow(log_prints=True)
def my_flow(): ...

- View logs via CLI (from the Prefect DB):

prefect flow-run logs <FLOW_RUN_ID>

If you want CloudWatch-only or both - CloudWatch only: set

PREFECT_LOGGING_TO_API_ENABLED=false

in the task env (logs won’t appear in Prefect UI). - Both (typical): keep Prefect API logging enabled and keep the awslogs driver for infrastructure observability in CloudWatch. Docs - Add logging: Add logging - Artifacts (if you want to link CloudWatch or external logs into the Prefect UI): Artifacts how-to