Real-Time Audience Analytics for a Streaming Media Company

The Challenge

All of that telemetry landed in nightly batch jobs feeding a licensed BI tool, so audience numbers arrived next-day at best. Content teams could not see how a new release was performing on launch day, ad-ops could not react to under-delivering campaigns until the flight was half over, and live-event programming decisions were made on gut feel. Meanwhile, per-seat BI licensing costs kept climbing as more teams wanted access, and every new question meant a ticket to a two-person analytics group.

The company had strong product engineers but no streaming-data expertise in-house, and a data-engineering hiring search had dragged on for months. They wanted a senior team that could start immediately and work as part of their organization — not a black-box project.

What We Did

Dillo assembled a four-person augmented data pod — two senior data engineers, an analytics engineer, and a data-platform engineer — matched to the client's stack preferences and interviewed by their head of data. The pod plugged into the client's planning and on-call rotations and was shipping to production in its first week.

• Stood up a Kafka event backbone for player and ad telemetry, replacing the nightly file drops, with schema-validated topics per event family.
• Built a ClickHouse analytics store tuned for high-cardinality audience queries — concurrent viewers, watch time, completion rates, ad fill — over hundreds of millions of daily events.
• Modeled business-facing metrics with dbt, giving the client versioned, tested definitions of "a view," "a viewer," and "an impression" that every team now shares.
• Orchestrated batch and backfill workloads with Airflow, keeping historical data consistent with the real-time path.
• Deployed Superset as the self-service BI layer, with curated dashboards for content, marketing, and ad-ops — retiring most paid BI seats.
• Documented the platform and trained the client's analysts to build their own dashboards and dbt models, so the capability stays in-house.

The Results

• The pipeline reliably processes over 150 million events per day, with headroom for live-event spikes.
• Audience dashboards refresh in under a minute — down from next-day — so content and ad-ops teams act on launch-day and mid-flight data.
• Moving casual consumers to Superset self-service cut BI licensing spend by about 40%.
• The pod was productive in week one, and the analytics group now fields strategy questions instead of report tickets.
• Shared dbt metric definitions ended the recurring "whose numbers are right?" debates between teams.

Figures reflect outcomes reported for this engagement; they are project results, not audited benchmarks.

Tech Stack

Services Used

IT Staff Augmentation — a four-person embedded data pod working inside the client's engineering organization.