When Business Definitions Change: Keeping Your Semantic Layer and AI in Sync

A semantic layer needs a change process because business definitions evolve. When a metric changes, its SQL is only one part of the update. Agent instructions, saved questions, cached results, evaluation fixtures, and report explanations may still refer to the old meaning.

The objective is to make the change intentional and visible. A consistent answer is not trustworthy if nobody knows which definition it uses.

Start with a definition change, not a code change

Suppose a fictional company defines an active account as any account with a login in the last 30 days. It later decides that the commercial dashboard should count only paying accounts with a qualifying product event during that period.

The first definition returns 400 accounts; the second returns 250 on the same snapshot. This does not demonstrate a loss of 150 customers. It demonstrates that the population changed.

Document the reason, owner, effective date, and treatment of historical periods before changing the implementation. Sometimes the right answer is a new metric name rather than replacing an existing definition used for another purpose.

Inventory the consumers

Search for the metric in semantic models, dashboards, agent tools, prompt examples, scheduled reports, and downstream exports. Include descriptions and synonyms. An agent can select the new calculation while explaining it with an old example embedded in its context.

Record the consumers that matter for the release. Do not assume an API consumer will notice that a description changed in the BI interface. Machine-readable version information and release communication help those consumers update intentionally.

Use a small change record

Field	What to record
Business owner	Person approving the meaning
Previous definition	Population, event, window, and exclusions
Proposed definition	Exact changes to those rules
Historical behavior	Restate history or apply from an effective date
Affected consumers	Reports, agent tools, exports, and tests
Validation	Expected differences and checked examples
Rollback	How code and dependent outputs can be restored

Keep the implementation in version control. The business change record should reference the corresponding model revision, so a disputed answer can be traced to both code and intent.

Compare old and new on the same snapshot

Run both definitions across representative periods and segments. Explain the differences with records that enter or leave the population. In the active-account example, separate unpaid accounts from accounts without qualifying activity; do not treat the total difference as unexplained drift.

Check edge cases: newly paid accounts, cancelled subscriptions, delayed events, and activity at the 30-day boundary. Test the aggregation by different dimensions so the change does not introduce double counting.

Keep both expected results in the test suite while the migration is in progress. The semantic-layer implementation guide shows how small fixtures make these checks concrete.

Update the agent's contract

Expose the new metric name or version, update descriptions and examples, and define how the agent should answer questions that refer to historical reports. If two legitimate definitions remain, require an explicit selection or ask for clarification.

Invalidate or partition caches that include the old result. A deployment can successfully update the calculation while a cached narrative continues to quote the previous number. Include definition version and data snapshot in result provenance.

Run natural-language regression questions too. “How many active customers did we have?” should not silently switch between commercial and product-usage definitions based on phrasing.

Release with evidence

Before replacing a widely used metric, review the differences with its owner and communicate the effective date to affected users. Consider a transition period with both definitions clearly named when continuity matters.

Monitor disputed answers and unexpected segment changes after rollout. A rollback of SQL may be insufficient if an exported report or a downstream process has already consumed the new definition; document the recovery scope.

For related accuracy issues, see why AI gets business numbers wrong. Labs4Change can help establish metric ownership and change controls that support both analytics teams and AI agents.