Challenging Google Analytics: Building a Scalable, Cost-Effective User Tracking Service

Most teams treat product analytics as a buy decision. Alina Krasavina's talk is a report from a team that made the opposite choice and can show the numbers: an engineering manager at Delivery Hero describes how her group deprecated Google Analytics in favour of an internal user tracking service, and what happened over the following years. Her central thesis is that a deliberately simplistic architecture, combined with carefully chosen KPIs and disciplined testing, let a small team beat a hyperscale third-party product on data quality, cost, and capability.

Delivery Hero is a food delivery company organised as a central office in Germany plus many local brands around the world. The central office builds shared services that the local delivery brands consume, and the tracking platform Krasavina leads — referred to in the talk as Perseus Tracking — is one of those centralised tools. The talk was recorded at InfoQ Dev Summit Munich and published by InfoQ on June 22, 2026; it runs 46 minutes and 32 seconds.

These notes report what Krasavina presented. Where I add background that an intermediate engineer needs but the speaker did not state, the text says so explicitly.

What You Will Learn

• Why a forced migration deadline, GDPR, cost, and product limitations combined to justify replacing Google Analytics rather than upgrading it.
• How a tracking pipeline built from an SDK, one API, Pub/Sub, and BigQuery scaled to ten times its original load without architectural rewrites.
• How to define a tracking KPI (order match rate) against a real source of truth instead of trusting the analytics system to grade itself.
• What parallel-pipeline ("doubled flow") testing costs in practice, including the long tail of users who never update their app.
• Why compile-time schema validation beats asking developers to send clean data.
• The specific reliability and cost fixes the team applied: synchronous request handling, gRPC, data archival, native JSON storage, cheaper node types, client queue monitoring, and event prioritization.

Why Deprecate Google Analytics At All

Krasavina gives four reasons, and the first is the one that unlocked the others. Google was retiring Universal Analytics, the previous generation of Google Analytics, so the team faced a migration effort regardless of what they chose. Given that the work had to happen anyway, moving to an internal tool competed against moving to GA4 on roughly equal footing rather than having to justify itself against doing nothing.

The second reason was product limitations that the team had already hit. They needed real-time data, and Google Analytics delivered data only once or twice per day. That mattered because some of their data is billable — data derived from advertisement activity that money changes hands over — and those use cases cannot wait for a daily export. They had also reached Google Analytics' ceiling on the number of distinct event types that can be defined. In their own system, Krasavina notes, the number of event types is unlimited.

The third reason was GDPR. Google is a third party, and Delivery Hero has data that, for legal reasons, must be stored in its own infrastructure. Bringing tracking in-house removed a class of legal concern and, in Krasavina's words, made the privacy officers happy.

The fourth reason was cost, but it entered as a constraint rather than a goal. The initial rule was simply that the internal service must not cost more than Google Analytics. Only later did cost become an active area of improvement. On top of these, the team gained capabilities Google Analytics could not offer — most importantly the data validation layer described later, which is the foundation of their data quality work.

The Architecture: Small On Purpose

Structurally the system looks like any user tracking service. A mobile SDK and a frontend SDK written in TypeScript collect events on the client and send them to an API. Internal infrastructure streams events to consumers and into the data store, which is BigQuery because the company runs on Google infrastructure. There are two consumer classes: real-time consumers that subscribe directly to Pub/Sub, and everyone else, who query BigQuery.

For readers unfamiliar with the Google Cloud pieces: Pub/Sub is a managed publish-subscribe message queue that decouples producers from consumers and buffers bursts, and BigQuery is a columnar analytical warehouse designed for large scans rather than row-level transactional access. That split is why the two consumer classes exist at all — latency-sensitive consumers cannot afford to wait for the warehouse write.

The initial MVP was almost trivially small: an API, plus two processors reading messages from Pub/Sub — a main processor and a fallback processor for fallback cases. Krasavina is emphatic that this simplicity was the point. It was scalable and, in her words, caused zero problems.

Architecture And Data Flow

flowchart TD
    A[Mobile SDK] --> Q[On-device queue]
    B[Frontend SDK - TypeScript] --> API[Tracking API]
    Q --> WM[Work manager: batched background send]
    WM --> API
    API --> PS[Pub/Sub]
    PS --> P[Processor]
    PS --> FP[Fallback processor]
    PS --> RT[Real-time consumers]
    P --> BQ[(BigQuery)]
    FP --> BQ
    API -.-> GRPC[gRPC write path]
    GRPC --> BQ
    BQ --> AN[Analysts and batch consumers]
    BQ --> LS[Looker Studio dashboards]
    API --> FWD[Event forwarding to marketing third parties]

The dotted gRPC path is a later addition, described below. Around that core, the team has since added many services: a data validation service, more curation jobs, and more SDKs. Krasavina frames every one of these as a response to a specific problem — reliability, data validation, serving producers and consumers — rather than as part of the original design. The tiny API remained the centre.

Choosing KPIs You Can Actually Trust

The headline result from the MVP rollout is that the data quality metric started at 85% under Google Analytics and gained "6% more" after the rollout — the transcript does not make clear whether that is six percentage points or a six percent relative gain — while cost fell 25% and the system absorbed twice the load it had carried under Google Analytics, with zero data-loss incidents during testing and rollout. Krasavina notes that at their scale, and for billable data in particular, that gain is a lot of money. But the more transferable lesson is how the team measured it.

The metric is what they call the order match rate, and it works because they have a genuine source of truth. When a user orders food, the food delivery backend knows that the order was placed and paid for. That backend record cannot be lost by the tracking pipeline, because it is not produced by the tracking pipeline. The team compares orders known to the backend with orders that arrived through user tracking; the ratio is the order match rate.

Krasavina highlights a property that makes this metric unusually well behaved. Raw event volume is confounded by seasonality — Christmas or a public holiday produces a sudden spike in orders for reasons that have nothing to do with the tracking system. Because the order match rate is a ratio against a simultaneously observed ground truth, a demand spike moves both the numerator and the denominator. A drop in the ratio therefore points at the tracking system, not at human behaviour. As she puts it, there is no other data that could be tracked that well.

The cost KPI followed the same philosophy: a simple, normalised cost per message. Normalising by message volume means the metric does not deteriorate just because the business grew, so it can be optimised quarter over quarter.

Her generalised takeaway is that if you choose KPIs carefully, you can track how each change affects them and tell whether your fixes and features are actually working. The corollary, which the rest of the talk repeatedly demonstrates, is that KPIs you cannot attribute are not worth optimising.

Getting from parity to a surplus was not a single trick. Krasavina attributes it to two ongoing streams of work: fixing the SDK so it stops losing data on the device, and building the server-side infrastructure reliably so that data which already arrived is not lost afterwards. Both failure modes were real for them. The rollout began with four brands, and a chart covering the year after rollout shows the internal service first reaching parity with Google Analytics and then overtaking it.

Testing: Load, Parallel Pipelines, And Their Real Costs

Three testing practices carried the migration.

Load testing with real data. The team took their actual peak load and tested at three times that volume. This paid off concretely: on a public holiday in one of their countries, order volume spiked instantly and enormously, and the system survived. Krasavina's advice here is unqualified — please do load testing.

Parallel pipeline testing. To validate removing Google Tag Manager, the team ran a doubled pipeline. Under Google Analytics they had used Google Tag Manager (GTM) — a hosted layer that lets you configure tag firing, data enrichment, and filtering rules without shipping new application code — to enrich events and filter some of them out via configured rules. Deprecating Google Analytics meant deprecating GTM too, so a new app version shipped an SDK that sent every event twice: once through their own pipeline and once through GTM. Comparing the two streams proves the new path loses nothing.

Krasavina is blunt about the price of this technique, and this is the most practically useful warning in the talk. You are doubling load and ingesting twice the data you need, and you pay for that. Worse, the cost is not bounded by the length of the experiment. Because the doubled behaviour ships inside a mobile application version, ending the test requires users to update their app — which does not happen instantly. She reports that at least half a year later, some users were still running the old version with the doubled sending behaviour. Any mobile A/B or migration test must budget for that update tail, not just the test window.

Head-to-head SDK evaluation. The team applied the same comparative method to the build-versus-adopt question, evaluating the open-source Snowplow SDK against their own. In Q&A she confirms the full shortlist was small: GA4, the Snowplow SDK, and a survey of other mobile SDKs, with GA4 and Snowplow as the serious contenders. Her framing is that the comparison came out close to even, so the deciding factor was control. With an open-source dependency you can file a pull request and talk to the maintainer, but if you are not a maintainer, everything becomes slow. If you need features quickly, you end up forking and maintaining your own version anyway — at which point, she argues, you may as well build the thing yourself from the start. This is a judgement about their required feature velocity, not a general argument against open-source SDKs.

Rollout Discipline

The rollout sequence mirrors the testing discipline. The team only deprecated the second, GTM-based pipeline after they had confirmed they were receiving at least as much data through their own path. Later application versions then shipped with their SDK alone. Because the doubled pipeline was still running during the comparison, verification cost them no data.

Two rollout lessons came out of incidents rather than planning:

• Progressive rollout by market size. After a data-loss incident caused by an SDK rollout, the team adopted a staged rollout of both backend changes and the SDK, starting with smaller brands and moving to bigger ones. Testing changes on smaller markets means less blast radius and a cheaper rollback.
• Chaos testing. A GCP outage prompted the team to start deliberately testing what happens when Google infrastructure misbehaves, rather than assuming the managed platform is always available.

Krasavina also stresses observability, and frames it as a mistake they made. The team thought about logging but did not add enough of it, and when data loss occurred they could not debug it easily. Alerting was similarly thin, and later data-loss incidents were also hard to diagnose. Her rule: if you are running an experiment, add logging everywhere, and put monitoring and alerting at the key points of your system.

She is candid that beating Google was a genuine surprise. It began as an experiment, and the assumption was that if Google achieved 85% data capture, there was no way to do better. The result came not from a clever idea but from iterating on a measurable KPI.

Results After A Few Years

Krasavina notes that the cost figure is specific to their case and that further improvement remains possible; she treats cost work as a never-ending story. The data-loss incidents in the final column are the ones described below — they happened after the MVP, not during it.

Data Completeness And Automated Governance

The team's first blind spot was that they had optimised for data accuracy — does every event that was sent arrive in storage? — without ever asking about data completeness, meaning how good the contents of those events actually are. It was not good, and Krasavina calls it a huge data governance problem.

Her example is null values. A field intended to be null could arrive as a space character, an empty string, the literal string null, the number zero, or essentially anything else. The goal was to reach a state where the data does not require normalisation downstream. She is honest that they have not fully arrived: such problems still exist, but are smaller than before.

The first response was to start measuring data completeness. She adds a useful caveat about new metrics generally: when you introduce one and think carefully about it, the initial reading is usually poor — and that is the point, because it gives you something to improve.

The second response was social, and it failed. Telling producers to send null as null rather than the string "null" works somewhat, but not really, because people are human. Metrics dashboards showing that a field should be numeric rather than a string did not change behaviour reliably either.

The third response worked: code-generated event models. Event schemas are stored in a third-party schema registry, and code is generated from them so that application developers get errors and warnings at compile time. A developer writes code, compiles, and immediately sees that a field expected to be null is being sent as the string null. Krasavina's conclusion is direct — an error at compilation time works far better than a metric shown after the fact. Framed generally, this is a shift-left move: the feedback arrives at the moment the mistake is cheapest to fix, in the tool the developer is already using.

The schema registry solved a second problem, which is organisationally specific to Delivery Hero's structure. There are global requirements for events, which the central office needs for company-wide statistics, and local properties needed only by individual brands. Without a shared registry, each side documents its requirements in its own way and someone must manually keep them synchronised. Storing the event models in one place means, as she puts it, you do not need a data governance person watching the global requirements — governance becomes automated.

There is a downside the team hit later. Code generation produces more code as the number of event types grows, and for some markets it is extremely important that the application binary stay as small as possible because users' phones do not have much memory. Shrinking the SDKs, and the generated code in particular, became its own engineering challenge.

Backend Reliability: The Synchronous Trade

The most instructive reliability problem was data loss on pod restarts. Their API pods sometimes ran out of memory and were killed by Kubernetes, and a killed pod lost everything it was holding. Something had to either preserve that data or cause it to be sent again.

Their fix was to make every request synchronous: the API does not acknowledge the client until the event is durably handed off. If the pod dies mid-request, the client receives a 500-class error and resends. The cost was roughly a sevenfold increase in latency, which the team accepted because the mobile SDK sends events in a non-blocking background flow — the user never waits on the tracking request, so the extra latency is invisible to them and buys guaranteed delivery.

Krasavina's own commentary on this is worth keeping: it is a fairly obvious solution, and they still needed to experience a data-loss incident to arrive at it. Sometimes, she says, you figure out obvious things through incidents and losing money.

Two supplementary points an intermediate engineer should note, since the talk does not spell them out: acknowledging only after durable handoff converts an at-most-once delivery path into an at-least-once one, which means duplicates become possible on retry and consumers need to tolerate or deduplicate them. And raising per-request latency sevenfold raises the number of in-flight requests a pod holds at a given throughput, so capacity planning has to be revisited alongside the change.

Separately, the team introduced gRPC and now runs a doubled write flow: not only the Pub/Sub processors, but also a gRPC path writing into BigQuery. gRPC is a binary, HTTP/2-based RPC framework, and BigQuery's streaming write interface is gRPC-based; Krasavina does not detail the exact motivation for the second path in the transcript, so treat its role as a redundancy and ingestion improvement rather than a fully specified design.

Cost Optimization Levers

Cost work became a standing quarterly key result: look at the cost metrics, find something to optimise, repeat. Krasavina lists the levers, and freely calls most of them obvious — the interesting part is that obvious savings go unrealised until someone is accountable for the metric.

• Data archival. Originally the team archived nothing. Data that is not expected to be accessed frequently is now moved to cheaper storage after a period of time. Lower storage tier, lower cost.
• Native JSON storage. They had been storing a JSON field as text. Switching it to a proper JSON type reduced cost by 20%. (Supplementary context, not stated in the talk: BigQuery's native JSON type stores data in a columnar, semi-structured encoding rather than as an opaque string, which both compresses better and lets queries prune to the fields they touch.) She qualifies that this matters if you are a data-intensive application.
• Cheaper compute nodes. The team moved to cheaper node types. In the transcript she contrasts "standard nodes" with "on-demand nodes" as the cheaper option; on Google Cloud the usual cheaper alternative to standard on-demand nodes is Spot or preemptible instances, so the exact node classes here are ambiguous in the source. The principle she states is unambiguous: use everything cheaper that is still reliable.

The Mobile SDK: Queue, Priorities, And Backpressure

The client SDK is not a thin wrapper around an HTTP call. When a developer calls "send event," the event goes into an on-device queue. A work manager then runs in the background, reads a batch of events from the queue, and sends the batch to the API. The design is deliberately non-blocking, which is precisely what made the sevenfold server latency increase tolerable.

Several design details came from operational pain:

• Queue ordering is genuinely unsettled. The team has debated last-in-first-out versus first-in-first-out repeatedly — Krasavina says the discussion resurfaces a couple of times a year and is still unresolved. She floats making it configurable, without committing. This is a rare and useful admission that a reasonable team can fail to find a clear winner. LIFO favours delivering the freshest events when the queue is backed up; FIFO preserves chronological ordering and avoids starving older events.
• Queue monitoring. Data-loss incidents occurred when the team launched very data-intensive A/B tests. The experiments overflowed the client queue, and the observable symptom was a drop in billable data. That prompted explicit monitoring of the on-device queue.
• Event prioritization. The same incidents motivated treating events unequally. The system has several layers of importance; billable data is the most important and everything else is less so. Under pressure, the queue can preserve what carries revenue.
• Exponential backoff on retries, which she describes as an industry standard for this kind of sending rather than an innovation.

Serving Developers, Not Just Analysts

One of Krasavina's closing observations is a product insight rather than a technical one. Originally the team built for product analysts: what data are they receiving, what use cases does it serve? Introducing event modelling and compile-time validation forced them to recognise that developers are also stakeholders and users of the product. The question changed from "what do analysts need?" to "how do we make developers produce better data?" — and answering that second question is what improved data quality.

The validation layer also changed the shape of their support load. Most errors are now caught at development time, so the class of issues reaching the support line is different from before.

She also credits the earlier team, including her earlier self, for the simplistic but scalable architecture, saying it saved a lot of time and that things could have gone considerably worse.

Beyond Analytics: Reuse And Roadmap

Because the platform is generic event ingestion, the team is extending it well past its original scope:

• Event forwarding to third parties. Some incoming data is forwarded to external systems for marketing use cases — Krasavina names Facebook campaigns as an example. The service is not just a store that consumers query; it is also a source feeding marketing activation.
• Application metrics. They are reusing the platform to store application metrics data, the sort of thing Firebase would handle. She notes this is a completely different type of data but that similar storage still serves it.
• Experimentation data. Planned work, motivated by A/B testing being very data-intensive and structurally somewhat different from product events.
• Internal tools. Asked in Q&A whether tracking extends beyond the consumer mobile app and web app, Krasavina notes that the boundary is fuzzy at Delivery Hero — the help page counts as an internal tool, and they also track the application used by deliverers. The same API serves these. An audience member pushed further, observing that internal tools have a much more fixed event set and a different user-journey shape than consumer apps; Krasavina's answer is that for such cases, and for experiment data, the strict validation layer is often unnecessary and they expect to build a pipeline variant that skips it.
• Further mobile SDK refactoring. They still lose some data on the phone. Recovering another one or two percent sounds trivial, but at their scale she expects it to save significant money.

She also frames the general benefits of the in-house solution: full control over the implementation, their own prioritisation rather than waiting on a vendor who has bigger customers and different roadmap pressures, full control over cost and over the levers available to reduce it, and satisfied compliance officers because no data goes to a third party.

Trade-offs And Limitations

The talk is a success story, but the trade-offs are visible throughout and worth separating out.

• You are now the vendor. Full control means owning SDK maintenance across platforms, incident response, schema tooling, cost engineering, and a support line. The gains reported here accrued over several years of sustained investment by a dedicated team; they are not a weekend replacement for a managed product.
• Parallel-pipeline testing is expensive and long-lived. You pay for double ingestion, and on mobile the tail extends far beyond the intended test because users update on their own schedule — at least six months in their case.
• Latency was traded for durability. Making requests synchronous increased latency roughly sevenfold. This was acceptable only because the client sends in the background. A tracking client on a blocking path, or a system with tight end-to-end latency requirements, could not make the same trade.
• At-least-once delivery implies duplicates. Client-side resends after 500 responses mean consumers must tolerate repeated events. (Supplementary: the talk does not describe their deduplication approach.)
• Data completeness is still imperfect. Krasavina explicitly says normalisation problems remain, only smaller than before. Schema-driven generation reduced them rather than eliminating them.
• Code generation inflates binary size. More event types means more generated code, which conflicts directly with the requirement that the app stay small for markets where devices are memory-constrained.
• Queue ordering remains unresolved. LIFO versus FIFO is still being argued internally. Do not read the talk as endorsing either.
• Validation is not universally wanted. In Q&A, Krasavina notes that some newer use cases — experiment data among them — do not need the validation layer, and the team expects to build a separate pipeline that skips it. A governance mechanism that is mandatory everywhere becomes friction somewhere.
• The visualisation layer is a genuine regression. Asked how they replaced the Google Analytics dashboard, she answers that they use Looker Studio and calls it "pretty ugly", saying only that it serves the purpose. Losing a polished analyst-facing UI is a real cost of leaving a mature commercial product.
• The numbers are Delivery Hero's. The 97% capture rate, 3x cost reduction, and 20% JSON storage saving are their reported results in their environment. The order match rate in particular is only available because food ordering provides a paid-transaction source of truth; many products have no equivalent.

Practical Takeaways

• Time a build-versus-buy decision to a forced migration. The strongest part of the business case was that the team had to do migration work regardless, so the internal option only had to beat GA4, not beat doing nothing.
• Define one KPI against an independent source of truth before you start changing anything. Ratios against ground truth are robust to seasonality in a way that raw volume counts are not.
• Normalise your cost metric by unit of work — cost per message, not total spend — so it stays meaningful as the business grows.
• Load test at a multiple of observed peak with real data. Three times peak was what let them survive a public holiday spike.
• Budget the mobile update tail into any dual-write migration. Plan for months of residual double sending after the test formally ends.
• Push data-quality enforcement to compile time. Generated models from a shared schema registry changed behaviour where dashboards and requests did not.
• Store global and local schema requirements in one registry so cross-team governance stops depending on a person keeping documents in sync.
• Acknowledge only after durable handoff, and let clients resend on failure, if your client path can absorb the added latency.
• Roll out progressively by blast radius, smallest markets first, for both backend and client changes.
• Instrument the client queue, not just the server. Their billable data loss originated on the device, under load from an unrelated A/B test.
• Prioritise events by business value so that backpressure degrades the least important data first.
• Revisit the boring cost levers periodically: archival tiers, native column types instead of serialised text, and cheaper compute classes.
• Treat producing developers as first-class users of an internal data platform, alongside the analysts who consume it.

Key Terms

• Universal Analytics (UA) — The previous generation of Google Analytics, retired by Google, whose deprecation forced Delivery Hero to migrate.
• GA4 — Google Analytics 4, the successor product and the alternative the team rejected in favour of building in-house.
• Google Tag Manager (GTM) — A hosted configuration layer for firing, enriching, and filtering analytics tags without shipping application code. Delivery Hero used it for enrichment and rule-based filtering, and deprecated it alongside Google Analytics.
• Order match rate — The team's invented data quality KPI: the proportion of orders known to the ordering backend that also arrive through user tracking.
• Billable data — Tracking data tied to advertisement revenue, where loss translates directly into lost money; the highest priority tier in their SDK.
• Data accuracy vs data completeness — Accuracy is whether sent events reach storage; completeness is whether the contents of those events are well-formed and usable without downstream normalisation.
• Pub/Sub — Google Cloud's managed publish-subscribe messaging service, used here to decouple the ingestion API from processors and real-time consumers.
• BigQuery — Google Cloud's columnar analytical data warehouse, the tracking platform's storage layer.
• Parallel or doubled pipeline — Sending each event through both the old and new paths simultaneously so their outputs can be compared before the old path is removed.
• Progressive rollout — Releasing changes to smaller, lower-risk markets before larger ones to limit blast radius and cheapen rollback.
• Chaos testing — Deliberately injecting infrastructure failures to verify system behaviour; adopted here after a GCP outage.
• Exponential backoff — Retrying failed sends after geometrically increasing delays to avoid amplifying load during an outage.
• Work manager — The background component in the mobile SDK that drains the on-device event queue in batches, keeping event sending off the user-facing path.
• Snowplow — An open-source user tracking SDK and pipeline, evaluated head-to-head against the in-house SDK and rejected on control and velocity grounds.
• Looker Studio — Google's reporting and dashboard tool, used to replace the Google Analytics dashboard.

Reference: Alina Krasavina, Challenging Google Analytics: Building a Scalable, Cost-Effective User Tracking Service, InfoQ Dev Summit Munich, published by InfoQ on June 22, 2026.