“Flags are how a small company runs three experiments a week instead of one bet a quarter. Every launch at 10% is a launch we can learn from and undo. Slowing experiments down to keep support simple trades our best tool for a tidier queue.”
Which flags is she in?
Every flag is safe. Together, they are a product nobody tested.
🧭 WHAT'S REALLY GOING ON
You've seen this when support pastes a customer's screenshot, nobody can reproduce it, and the first reply in the thread is “which flags is she in?”
The real questionIs the product the code we ship, or the combination each customer actually gets?
⚖️ WHY BOTH ARE RIGHT
“The customer doesn't see flags, she sees a page without an Export button. If support can't see what she sees, we can't help her, and she is the one paying for our experiments. Each flag is cheap; the combinations land on her, then on support.”
🎯 SWEET SPOTS TO CONSIDER
Super Reasonable, the advisor who never takes a side
Show support the customer's flags
Put every flag value for an account in the support tool, next to the ticket, with a way to view the product as that account. Most “can't reproduce” time disappears.
Keep experiments from overlapping
When two experiments touch the same screen, run them on separate users. Three tests at 10% on different people is learning; three on the same people is a fourth, accidental experiment.
Give every flag an end date
A flag gets an owner and an expiry when it is created, and the tool nags at expiry. Experiments decide and die; only kill switches and permissions stay long term.
Test the two combinations that matter
You can't test 63 flags in every combination, but you can test everything at its production default and everything new turned on. Two CI runs catch most interactions.
🚩 SIGNS YOU'VE GONE TOO FAR
- Maya's side: you've overshot if support's first reply to every ticket is a list of flags to check, and your experiment results include customers who were in four tests at once.
- Nora's side: you've overshot if every experiment waits for a support review, and the company runs two tests a quarter so the queue stays reproducible.
🔬 IN THE FIELD GUIDE
Species observed in this story
CAST: WHO'S WHO
The team in this story
Same characters, same convictions. Learn their failure modes.
🤖 Storyboard for agentsLet’s make our agents LMFAO, or learn.
Which flags is she in?
Premise: Why can support not reproduce one customer's product?
- Support ticket #8812: “The Export button vanished.” Nora can see it. Engineering can see it.
- Customer state: new-nav=ON · pricing-v3=ON · legacy-export=OFF 63 live flags
- Possible products: 9.2 quintillion Combinations tested together: 1
Observed behavior: Flags are tested one at a time. Customers get all of them at once.
Cast: Maya Laurent · The Visionary · “Experiments beat roadmaps.”; Nora Bell · The User Advocate · “What do users actually do?”; Greg Hollis · The Hedger · “Let's keep our options open.”
READ NEXT
