Socialness: High-Throughput Activity Feed & Engagement Graph
Engineered a horizontally scalable fan-out activity feed and social graph handling over 15 million daily interactions with sub-50ms latency.
The Challenge
Socialness’s monolithic Postgres database melted during celebrity user posts when millions of follower feeds required updates simultaneously (the "fan-out" problem).
The Solution & Architecture
WDL re-architected the feed system using hybrid fan-out on write and fan-out on read, backed by Redis cluster caches, Apache Kafka, and ScyllaDB for persistent activity storage.
Project Metadata
Key Technical Capabilities Delivered
Architectural components engineered specifically to resolve the client's scaling hurdles.
Hybrid Feed Fan-Out
Active users receive pre-computed timelines in Redis, while high-follower accounts are merged on read, preventing cache thrashing.
Real-Time Push Notifications
Sub-100ms notification dispatches across iOS and Android utilizing distributed worker queues.
Automated Content Moderation
Streaming text and image safety classifiers quarantine toxic interactions before appearing in public feeds.
Technologies Used
Build Fast, Scale Reliably
Ready to achieve similar results? Select a package to start immediately.
Validate Data Strategy
Technical audit and architecture blueprint to validate pipeline viability.
Build Data MVP / Foundation
Production-ready automated pipeline and live analytics lakehouse delivered in just 20 days.