Finding the information wasn't the hard part
Travel rewards data changes constantly. Credit card issuers add benefits, modify existing ones, introduce enrollment requirements, change expiration dates, and sometimes discontinue benefits altogether.
Keeping that information current inside Travel RewardsGO initially meant reviewing sources and translating those changes into structured product data manually.
AI presented an obvious opportunity. Instead of manually inspecting every source for every possible change, a system could scan the source material, identify relevant information, compare it with what already exists, and surface what might need to change.
The first versions showed that this part was achievable. They also exposed the more interesting problem.
Finding a statement on a page doesn't necessarily tell you what should happen next.
A source might say that a benefit is available through December. Does that mean the benefit has been discontinued? No. It means it has a scheduled end date.
A benefit mentioned on one page might apply to several cards rather than the single card associated with the source. Another benefit might look new because the wording changed even though an equivalent record already exists.
And sometimes the system can correctly identify a card that Travel RewardsGO simply doesn't have in its catalog yet.
None of those situations are solved by extracting more text. They require interpretation.
Automation earns authority. The ability to automate a decision doesn't automatically mean the system should make it.
Confidence isn't the same as certainty
That changed how I thought about the workflow.
The goal couldn't simply be: Find change → update database.
There needed to be something between discovery and action.
Instead, the system needed to distinguish between information it could reasonably resolve, information that appeared ambiguous, and information it couldn't confidently map to the existing product data.
That distinction became part of the product itself.
A proposed update can be classified based on what the system understands about it. When multiple cards are involved, each card can have its own resolution rather than forcing one answer across the entire result. Potential duplicates can be surfaced before another record is created. Future end dates can be treated differently from benefits that have actually ended.
And when the evidence isn't strong enough, the correct outcome isn't to guess. It's to say: this needs review.
That may sound like a limitation of automation. I think it's the opposite.
Knowing when not to act is part of making an automated system useful.
Human review is part of the system, not a fallback
It's tempting to think about human-in-the-loop systems as temporary scaffolding: people review AI output until the model becomes good enough and the review step can eventually disappear.
That's not how I'm approaching this one.
Some decisions may absolutely earn more automation over time. If a particular type of change can be identified reliably, validated against known rules, and measured across enough examples, there may be little reason for a person to approve every instance forever.
Other decisions have a higher cost of being wrong.
Incorrectly marking an active card benefit as discontinued doesn't just create a bad database record. It can affect what someone believes their card offers and potentially influence a decision they make with their points, miles, or money.
That changes the threshold.
The review queue therefore isn't just an administrative interface sitting behind the AI. It's part of the product architecture.
The system does the repetitive work first: finding information, structuring it, identifying likely matches, checking for duplicates, and explaining what it believes changed.
The person reviewing it handles the part where context and judgment still matter.
The objective isn't maximum automation. It's less manual work without giving up control over data quality.
Automation should earn authority
Building this workflow has reinforced a principle I'm increasingly interested in as AI becomes embedded in more products: Automation earns authority.
Start by letting the system observe and suggest.
Measure where it performs well and where it doesn't.
Make uncertainty visible rather than hiding it behind a confident interface.
Give people an efficient way to verify what the system found.
Then expand what the system is allowed to do when the evidence supports it.
That progression is less dramatic than handing an agent a task and letting it run autonomously. But for products where accuracy matters, it's considerably more useful.
The most important question isn't whether AI can perform the next step. It's whether the system has earned enough confidence for us to let it.
Entelibelle LLC builds intelligent digital products and partners selectively on product strategy, leadership, and applied AI. Start a conversation.