Feature Flags

1The core idea#

A feature flag is a runtime switch. Your code checks a condition before it runs a path:

if (flags.isEnabled("new_checkout", user)) {
  renderNewCheckout();
} else {
  renderOldCheckout();
}

The load-bearing word is runtime.

• The deploy already happened. Both code paths are in the binary.
• What changes behavior is a value fetched from a flag service, not a new build.
• So deploy means the code is on the servers, and release means users actually get the new behavior.
• You can do those on two different days. I have deployed a feature on a Monday and not released it until Thursday, after the on-call rotation changed to someone who knew the code.

Flags come in four flavors, and interviewers like when you name them.

• Release flags: hide unfinished work. Short lived, deleted after launch.
• Experiment flags: A/B tests. Bucket users, measure, pick a winner.
• Operational flags: kill switches and load shedding. Turn off the recommendation engine when the DB is on fire.
• Permission flags: entitlements. The paid tier gets the export button.

Most flag bugs come from treating all four as the same thing.

• A kill switch you forget to remove is fine. It is supposed to live forever.
• An experiment flag left on for two years is tech debt that eventually breaks.
• The difference is lifecycle: a release flag is supposed to die, an operational flag is supposed to live.
• Store and review them the same way and you lose track of which is which.

2How it works#

The flag value has to reach your running code on every request, and it has to be fast, because you are adding a check to the hot path. This is where the real engineering lives.

The key decision: evaluate in process, not over the network per request.

• The SDK pulls the full flag ruleset once, caches it in memory, and streams updates.
• Every isEnabled call is then a local function on a map you already hold.
• No per-request network dependency means the flag service can have a bad minute without your app having one too.

Worked example: 10,000 rps across 20 instances, 8 flag checks per request.

• Option A, call the flag service per check. 10,000 x 8 = 80,000 network calls per second. At 2ms per call best case, you add 2ms of latency to a request that may have taken 15ms, and you do it 8 times. Your p99 is now hostage to the flag service's p99. One more hop that can fail, 80,000 times a second.
• Option B, in-process evaluation. The SDK holds the ruleset in RAM. An isEnabled call is a hashmap lookup plus a hash for percentage bucketing, roughly 1 to 5 microseconds. At 5 microseconds each, 80,000 evaluations is 0.4ms of CPU spread across 20 machines, so about 20 microseconds per machine per second. It does not register. Updates stream in over about 200ms.

Option B wins and it is not close.

• The point of flags is to reduce risk, so adding a network call that can take your site down defeats the purpose.
• The cost is a few seconds of staleness after a flag change, which is almost always fine.
• Flip a kill switch and waiting 200ms for it to propagate is not the problem. The DB that is on fire is the problem.

Percentage rollouts have to be sticky.

• You do not want a user flipping between old and new checkout on every page.
• The standard trick is a deterministic hash:

bucket = hash(flagKey + ":" + userId) % 100
return bucket < rolloutPercent

• Same user plus same flag always lands in the same bucket.
• Ramp rolloutPercent from 1 to 5 to 25 to 100 and the cohort only grows, it never reshuffles. The user who hashed to bucket 3 is in at 5 percent, stays in at 25, stays in at 100.
• Nobody who saw the new path ever gets yanked back to the old one.

3Trade-offs#

The big architectural choice is where evaluation happens.

Fail open or fail closed?

• If the SDK cannot reach the control plane on boot and has no cached value, what does isEnabled return?
• For a release flag, fail closed: keep the old safe path.
• For a kill switch, fail to the safe state, which usually means the feature is off.
• Pick per flag, with a sane default. This is the single most common follow-up, so have the answer ready before they ask.

Who can flip a flag?

• A flag that changes money flow or data deletion needs an approval step and an audit log.
• A UI color flag does not.
• Treat flags like config changes, because they are, and config changes cause outages. I gate anything touching billing behind a two-person approval for exactly this reason.

Flag debt.

• Every flag is a branch. N flags means up to 2^N code paths, and nobody tests all of them. 20 flags is 1,048,576 combinations.
• Put an expiry on release flags and delete them.
• I have seen a flag default flip years later and resurrect dead code. That is a real outage pattern, not a theoretical one.

4Where it is used#

This is a core practice at basically every large shop, so name names in the interview.

• LaunchDarkly is the canonical commercial product. Streaming SDKs, in-process evaluation, percentage rollouts. The model in section 2 is essentially theirs.
• Statsig, Split, and Flagsmith are common alternatives, with Statsig leaning hard into experimentation and stats.
• Facebook Gatekeeper and internal Google systems pioneered this at scale, gating features to employees, then 1 percent, then the world.
• Unleash is the popular open-source option if you do not want a vendor.
• Netflix uses flags heavily for region-by-region and device-by-device rollouts of streaming features.

You reach for flags when:

• You want to merge to main continuously without releasing (trunk-based development needs flags to hide incomplete work).
• You want progressive rollout to catch problems at 1 percent instead of 100.
• You need a kill switch for a risky dependency.
• You are running experiments.

If a prompt mentions "gradual rollout," "A/B test," "canary," or "kill switch," flags are a legitimate part of your answer.

5Common pitfalls#

• Flags as permanent config. A kill switch living forever is fine. A release flag living forever is debt. Separate the two and expire the release ones.
• Flaky percentage rollouts. A random number instead of a stable hash. Users flip-flop, metrics turn to noise, and you cannot tell if the new path is actually worse.
• No fallback on SDK failure. If your code throws when the flag service is unreachable, you turned a safety tool into a new outage source. Always return a default, never throw.
• Evaluating per request over the network. Covered above. It adds a hard dependency to the hot path and multiplies your QPS against someone else's service.
• Flag explosion in tests. 20 flags is over a million combinations. You will never test them all. Keep the live count small and test the two paths that matter: all-on and all-off.
• Leaking flag state to the client. If the client evaluates flags, a user can read which unlaunched features exist from the payload. For secret launches, evaluate server side and send only the resolved answer. A reporter reading your unreleased product names out of a JSON blob is a bad day.

6How it shows up in interviews#

Interviewers use flags to check whether you understand operational safety, not just feature code. The probes I hear most: how do you roll out to 1 percent and keep it sticky, what happens when the flag service is down, how do you kill a bad feature in seconds without a deploy, and how do you keep flags from becoming permanent mess.

Here is the contrast that separates levels.

Weak answer.

• "I would add an if statement that checks a config value, and I can change the config to turn the feature on or off."
• Not wrong, but it is the dictionary definition. It says nothing about latency, failure modes, stickiness, or lifecycle.
• The interviewer will immediately poke the holes: what config, read how often, what happens when it is unreachable?

Strong answer.

• In-process SDK evaluation, so each check is a microsecond-level local lookup, not a network call, because I do not want the flag service to be a hard dependency on the hot path. At 10,000 rps and 8 checks each that is 80,000 network calls a second, which is 80,000 new ways to fail.
• Rollout percentage comes from a stable hash of flag key plus user id, so a user stays in the same bucket as I ramp 1 to 5 to 25 to 100.
• The SDK caches the last known ruleset and fails to a safe default if the control plane is unreachable: fail closed for release flags, fail to off for kill switches.
• Gate the risky dependency behind an operational flag so we can shed it in 200ms without a deploy, and put an expiry on release flags so they get deleted after launch to avoid 2^N untested code paths.

Why the second answer wins.

• It hits deploy-versus-release, the latency decision with a reason and a number, stickiness, failure behavior, and lifecycle.
• It is mostly the same facts the weak answer had, arranged around trade-offs instead of a definition.

The junior names the switch. The senior names what happens when the switch is unreachable.

One more thing that scores points: say explicitly that flags are how trunk-based development and continuous delivery actually work. You merge unfinished code to main daily, hidden behind a flag, instead of living on a long branch that rots. Connecting flags to the delivery model shows you see the system, not just the switch.

If this clicked, try the related Design Interview Question on designing a progressive rollout and canary release system.