Nox

All work CompleteSMS · B2B SMS provider · Active engagement

The model doesn't get to decide. It gets to make a case.

Applicant fraud detection and 10DLC compliance review, built so every decision can be defended.

22/22 Approved brands matched on the regression set · zero false rejects
The problem

Decisions about someone else's business, made from documents

CompleteSMS offers free SMS trials. Every free trial is also a door for spammers, scammers and competitors, and every wrongly rejected applicant is a customer lost at the moment they were most interested.

Separately, US business SMS runs under 10DLC, a carrier registration regime requiring brands and campaigns to be vetted before they can send. That vetting was manual, document-heavy and slow.

Both problems are the same shape: a decision about someone else's business, made from documents, that has to be right and has to be defensible. An AI that decides and can't explain itself is worse than useless here. "The model said so" is not an audit trail.

The architecture

The opposite of how most AI decisioning gets built

  • A deterministic rules engine makes the decision

    Not the model. The rules are explicit, reproducible and auditable. The same inputs produce the same outcome every time, and you can trace exactly which rule fired.

  • The AI acts as an evidence adjudicator

    It answers narrow, specific questions against supplied evidence and returns structured findings with citations and a confidence score. Those findings enter the rules engine as signals, weighted alongside everything else. The model contributes evidence. It does not hold the verdict.

  • A second AI opinion runs in shadow mode

    It sees every case and records what it would have decided, under an escalate-only policy: it can push a case toward human review, and that is all. It cannot clear something the rules rejected. Its opinions are logged and nothing else, so its judgment can be measured against real outcomes before anyone considers giving it authority.

  • Untrusted input is fenced

    Applicant-supplied fields and third-party search results are isolated in the prompt, and the deterministic rules remain the actual gate, so text inside a company name cannot talk its way to an approval.

  • Failures degrade to human review

    Every response builder is wrapped so an unexpected error produces a correctly-shaped "send this to a human" response, never a raw server error, and never a silent approval.

APPLICANT SIGNALS ADJUDICATOR cited evidence only RULES ENGINE SHADOW logs only PASS REVIEW REJECT escalate only unexpected error → always human review
The model never holds the verdict
What it does

Two stages, plus a fast lane

Stage one — identity and legitimacy

Establishes whether the person and business are real: IP geolocation, domain registration age, email deliverability and reputation, and targeted searches across professional profiles, business registries, general business footprint and domain reputation. Evidence goes to the adjudicator. The rules engine returns pass, reject, or human review.

Stage two — phone and country

Verifies phone and country against a defined list of supported markets, and returns the final decision.

A fast lane for the obvious cases

Clean, strongly corroborated business signups skip the research stage entirely. If a real business email matches a real company domain with good deliverability, an established domain, a supported country and no risk flags, there is nothing for the AI to adjudicate. Those applicants get an instant answer, and the expensive path is reserved for cases that need it.

Fail-safe on missing signals

If a check can't complete, that is not treated as a pass. A domain whose age couldn't be verified because the lookup was down goes to research rather than through the fast lane, because "we couldn't check" and "we checked and it was fine" are different answers.

10DLC compliance review

Brand submissions are ingested with their supporting documentation, routed through four different extraction paths depending on document type, validated for registration details, and put through a deterministic compliance decision engine that returns an approval or an escalation with reason codes and a written rationale.

Results

22 of 22, zero false rejects

Replaying a 28-brand regression set through the rebuilt compliance pipeline: all 22 brands with an expected approval decision matched, with zero false rejects. Six further brands ran as observation cases with no expected verdict. Four document extraction paths were exercised: embedded text layers, real OCR, vision-based OCR, and submissions with no document at all.

Read this precisely

This is a regression test set, not production traffic. It is stated that way here, and it should be stated that way in conversation.

It caught something a person missed

During the build, the document review system flagged a compliance issue that had been missed in manual review by the person who handles their 10DLC applications.

That is the outcome that matters. Not that it was fast, but that it caught something a knowledgeable human reviewer, doing the job properly, had gone past.

Founder's direct observation from the engagement

It doesn't reject real people

The verification system was tested against real CompleteSMS staff, who passed correctly.

False rejects are the expensive failure in applicant screening. A missed spammer costs a little. A rejected genuine customer is gone, and they don't tell you why. Validating against known-legitimate people was the check that mattered.

Founder's direct observation from the engagement

The audit that started it

11 critical and 13 high-severity defects

Before building, I audited four existing production workflows and documented them by severity.

An integrity check that could be bypassed, protected by a forgeable hashcritical
A decision step whose outgoing connections were never wired — approval emails had never sentcritical
An enrichment lookup discarded before scoring, making the call pure costcritical
An AI response parser breaking on the model's own reasoning outputcritical

Each was reported with severity and reproduction before anything was changed. Audit first, approval, then code.

Method

Changing a live decisioning system without breaking the business on it

The client's public integration contract was frozen. Field names, URLs and response shapes could not change, because another vendor's live website depended on them.

So every change was verified to produce byte-identical responses on every path it wasn't meant to affect. Not reviewed. Executed in a mocked runtime and compared.

Changes were applied by scripts that patch components by name with occurrence-count assertions, so a rename upstream fails the build instead of silently patching nothing.

Research path latency~20s
Worst-case target<30s
Clean signupsinstant
Status

Built and validated. Deployment pending.

This is an active engagement. The systems are built and validated, and deployment is pending CompleteSMS going live.

Transferable

Any automated decision that has to be explained afterwards

Underwriting, onboarding, KYC, vendor approval, eligibility, fraud screening, regulatory review. The pattern transfers whole: deterministic rules hold the decision, AI supplies evidence, new model behaviour proves itself in shadow before it is trusted.

Automating a decision you'll have to defend?

Build it so the reasoning survives scrutiny, not just the outcome.

Start a diagnostic More work