https://prefect.io logo
Join Slack
Powered by
****Solved* Hi everyone, I'm trying to set up my s...
# ask-community
r
****Solved* Hi everyone, I'm trying to set up my self-hosted Prefect server (running locally, non-containerized) with Redis for event messaging. I can't get the setup to work - I can ping my Redis server and publish events to it from my Python code, but when I run a Prefect workflow I don't see any task runs in the Prefect UI or relevant log entries even in debug mode - I just see 
prefect.server.events.messaging - Publishing event: prefect.task-run.Running with id:
log entries, but no log entries that the event message was delivered or received. I do see this in my Prefect server start logs: 
Loaded collection 'prefect_redis'.
15:32:34.104 | DEBUG   | prefect.prefect_redis.messaging - Consumer group already exists: BUSYGROUP Consumer Group name already exists
My environment variables are set up as:
Copy code
PREFECT_MESSAGING_BROKER="prefect_redis.messaging"
PREFECT_MESSAGING_CACHE="prefect_redis.messaging"
PREFECT_REDIS_MESSAGING_HOST="localhost"
PREFECT_REDIS_MESSAGING_PORT="6379"
PREFECT_REDIS_MESSAGING_DB="0"
Does anyone know what I'm missing to get Redis working for event messaging on my local Prefect server?
b
Curious what you're trying to accomplish here... Do you see anything in the "Event Feed" page in the Prefect UI?
r
***Solved* For some context, I'm trying to run workflows locally with up to hundreds of thousands of tasks, and Nate recommended Redis for the event messaging (thread). Before setting everything up with Docker I wanted to see if I can set up Redis first with my basic local Prefect server for testing, but it doesn't seem to connect to the Prefect server. When I use Redis as my messaging broker I don't see any events in the Event Feed page. As a sidenote, when I run 
prefect config view
I don't see the "PREFECT_REDIS_*" environment variables, is this to be expected? I'd appreciate any help here, thanks!
Fixed In the logs from running the workflow, I see: > 130904.337 | DEBUG | GlobalEventLoopThread | prefect._internal.concurrency - Service prefect.events.worker.EventsWorker object at 0x74caf4590910 handling item Event(occurred=DateTime(2025, 7, 31, 11, 9, 3, 890428, tzinfo=Timezone('UTC')), event='prefect.concurrency-limit.v1.released', resource=Resource(root={'prefect.resource.id': 'prefect.concurrency-limit.v1.e2c9e498-8cff-4be0-a487-5ea765ae3b12', 'prefect.resource.name': 'concurrency-test', 'limit': '101', 'task_run_id': '0198602b-d639-76c8-be3c-4cd5af2d032a'}), related=[], payload={}, id=UUID('0198602b-dc12-7e62-882d-def021796e8f'), follows=UUID('0198602b-d81c-758c-b735-0c28be852611')) > 130904.337 | DEBUG | prefect.events.clients - EventsClient(id=128415031727632): Emitting event id=0198602b-dc12-7e62-882d-def021796e8f. > 130904.338 | DEBUG | prefect.events.clients - Added event id=0198602b-dc12-7e62-882d-def021796e8f to unconfirmed events list. There are now 526 unconfirmed events. > 130904.338 | DEBUG | prefect.events.clients - EventsClient(id=128415031727632): Emit reconnection attempt 0. > 130904.338 | DEBUG | prefect.events.clients - EventsClient(id=128415031727632): Sending event id=0198602b-dc12-7e62-882d-def021796e8f. > 130904.338 | DEBUG | prefect.events.clients - EventsClient(id=128415031727632): Checkpointing event id=0198602b-dc12-7e62-882d-def021796e8f. > 130904.339 | DEBUG | EventsWorkerThread | prefect._internal.concurrency - Running call get() in thread 'EventsWorkerThread' > 130904.339 | DEBUG | EventsWorkerThread | prefect._internal.concurrency - WatcherThreadCancelScope, name='get' RUNNING, runtime=0.00 entered > 130904.339 | DEBUG | EventsWorkerThread | prefect._internal.concurrency - WatcherThreadCancelScope, name='get' COMPLETED, runtime=0.00 exited > 130904.339 | DEBUG | EventsWorkerThread | prefect._internal.concurrency - Finished call get(dropped) In case this is useful
Solved: turned out to be an issue with old Redis version.
b
Interesting. Thanks for the update @Robyn H
10 Views
Open in Slack
PreviousNext
Powered by