<@ULVA73B9P> are these environment variables used ...
# marvin-ai
j
@Marvin are these environment variables used by the prefect server if the server is started with
--no-services
and there are separate services containers?
Copy code
PREFECT_SERVER_SERVICES_EVENT_PERSISTER_BATCH_SIZE
PREFECT_SERVER_SERVICES_EVENT_PERSISTER_FLUSH_INTERVAL
PREFECT_REDIS_MESSAGING_CONSUMER_TRIM_EVERY
m
thought for 182.5 seconds
Short answer: set those on your services container, not on the webserver started with
--no-services
. -
PREFECT_SERVER_SERVICES_EVENT_PERSISTER_BATCH_SIZE
and
PREFECT_SERVER_SERVICES_EVENT_PERSISTER_FLUSH_INTERVAL
are only read by the Event Persister service. When you start the API with
prefect server start --no-services
, that process does not run services, so these vars have no effect there. They take effect in the separate services process/container. - Event Persister implementation: event_persister.py - Related settings model: server services settings - Note: there are backward-compatible aliases like
PREFECT_API_SERVICES_EVENT_PERSISTER_*
as well. -
PREFECT_REDIS_MESSAGING_CONSUMER_TRIM_EVERY
is used by Redis-based consumers (e.g., the Event Persister’s Redis consumer). In
--no-services
mode the webserver process won’t use it; it will apply in the services container where those consumers run. - Redis consumer settings/logic: prefect_redis messaging If you’re splitting into an API/webserver container and a services container: - Put all three env vars on the services container. - It’s fine if they’re also present on the API container, but they won’t do anything there with
--no-services
. If helpful, you can confirm which services are enabled/active with:
Copy code
prefect server services list-services
Want me to point to the exact place to set them in your Docker Compose? Feel free to share a snippet.
j
@Marvin what are good values for those variables for us, we do about 8k events a day.
m
thought for 35.9 seconds
Given ~8k events/day (~5.5/min, ~1 every 11s), the defaults are already well‑suited. Here are practical starting points and when to tweak them: Recommended starting values (services container):
Copy code
PREFECT_SERVER_SERVICES_EVENT_PERSISTER_BATCH_SIZE=20
PREFECT_SERVER_SERVICES_EVENT_PERSISTER_FLUSH_INTERVAL=5
PREFECT_REDIS_MESSAGING_CONSUMER_TRIM_EVERY=300
Why: - With your low, steady rate, batches won’t often fill; flushes will typically be driven by the interval. A 5s interval keeps UI/analytics reasonably up-to-date while keeping DB writes low. - Trimming Redis every 5 minutes further reduces background work without meaningful stream growth at your volume. If you want to tune: - Lower-latency persistence: set
FLUSH_INTERVAL=1-2
(more frequent smaller writes). - Fewer DB writes during occasional bursts: increase
BATCH_SIZE
to 50-100; keep
FLUSH_INTERVAL
at 5-10 to bound latency. - Even less Redis background work: increase
TRIM_EVERY
to 600s; just ensure your stream doesn’t grow unbounded if you have occasional spikes. Notes and source: - Event Persister (reads the
BATCH_SIZE
and
FLUSH_INTERVAL
settings): event_persister.py - Services settings model (env var names): server services settings - Redis consumer (uses
TRIM_EVERY
): prefect_redis messaging If you share your Docker Compose for the services container, I’m happy to drop these in the right place.