Every event has a hidden number of consumers

A REST API has one caller you can usually name. An event on a topic has however many consumers happen to be subscribed - including ones the producing team may not know about, added by a different team six months after the event was first designed. That's the core difference that makes event schema changes riskier than they look: the producer of OrderConfirmed can ship a change, see their own tests pass, and only find out a downstream analytics consumer broke when someone notices a dashboard stopped updating three days later.

This isn't a hypothetical edge case - it's the default failure mode of event-driven systems past a certain size, and it's exactly why "just add a field, what could go wrong" is the sentence that precedes most schema-related incidents.

Backward and forward compatibility, defined precisely

Backward compatible means a consumer built against the old schema can still read messages written with the new schema. Forward compatible means a consumer built against the new schema can still read messages written with the old schema - which matters during any rolling deploy, since producers and consumers are never all upgraded in the same instant. 

// v1 event
{ "orderId": "ord_123", "amount": 4999, "currency": "USD" }

// v2 - SAFE: adding an optional field is backward AND forward compatible.
// Old consumers ignore the field they don't know about. New consumers
// treat its absence (from old messages still in flight) as a default.
{ "orderId": "ord_123", "amount": 4999, "currency": "USD", "discountCode": null }

// v2 - UNSAFE: renaming or removing a field breaks every consumer
// still reading it under the old name, immediately, with no warning
// until something downstream throws a null-pointer or KeyError.
{ "orderId": "ord_123", "total": 4999, "currency": "USD" }

The rules that keep changes safe

  • Only add optional fields, never required ones. A new required field breaks every consumer that hasn't been updated to send or expect it.
  • Never remove or rename a field - deprecate it instead. Stop populating meaningful data in it once all consumers have migrated off, but leave the field present until then.
  • Never change a field's type (string to int, single value to array) - this breaks deserialization outright rather than just producing wrong values.
  • Version the event type itself for genuinely breaking changes - publish OrderConfirmed.v2 as a new event type alongside OrderConfirmed.v1, and dual-publish both until every consumer has migrated, then retire v1 on a announced date.

Schema registries make this enforceable, not just a convention

A schema registry (Confluent Schema Registry is the standard choice for Kafka, but the concept applies broadly) stores every version of every event schema and can reject a producer's attempt to publish a schema that breaks compatibility with prior versions - before the bad message ever reaches a consumer, not after. 

{
  "type": "record",
  "name": "OrderConfirmed",
  "fields": [
    { "name": "orderId", "type": "string" },
    { "name": "amount", "type": "int" },
    { "name": "currency", "type": "string" },
    { "name": "discountCode", "type": ["null", "string"], "default": null }
  ]
}
// compatibility mode: BACKWARD - registry rejects any schema change
// that an old consumer couldn't still read. Enforced at publish time,
// not discovered by a broken consumer in production.

Avro is the most common format used with schema registries because it embeds the schema version with every message and supports these compatibility checks natively. Protobuf and JSON Schema both support similar evolution rules; plain unstructured JSON with no registry gives you none of this - compatibility becomes a matter of discipline and code review rather than something the infrastructure enforces.

Consumer-driven contract testing catches what the registry can't

A schema registry checks structural compatibility - types, required-vs-optional. It can't tell you that a consumer actually depends on currency always being exactly three characters, or that a value of 0 in amount means something specific downstream. Consumer-driven contract tests (Pact is the common tool) let each consumer publish an explicit contract - "here's what I actually read and expect from this event" - which the producer's CI pipeline runs against before every deploy, catching semantic breakage the schema alone would miss.

Key takeaways

  • Events have an unbounded, often-unknown consumer list - schema changes carry more hidden risk than API changes with a known caller.
  • Only add optional fields; never remove, rename, or retype an existing field - deprecate and version instead.
  • A schema registry with backward-compatibility enforcement rejects breaking changes at publish time, before they reach a single consumer.
  • For genuinely breaking changes, dual-publish an old and new event version until every consumer has migrated off the old one.
  • Consumer-driven contract tests catch semantic breakage a schema registry's structural checks can't see.
If your team's current process for changing an event schema is "ping the other teams on Slack and hope someone notices," a schema registry with enforced compatibility checks is a small infrastructure investment against a genuinely common outage cause.