Skip to content

fix(validation): a value is read in time in line with its length, and an email is one mailbox - #5025

Merged
simlarsen merged 7 commits into
masterfrom
claude/validators-linear-time
Oct 19, 2026
Merged

simlarsen merged 7 commits into
masterfrom
claude/validators-linear-time

Conversation

@simlarsen

@simlarsen simlarsen commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

What was wrong

Two rules did not hold for the way value validators and redactors read text:

  1. A value was not read in time in line with its length. Email.isValid,
    and several checks and redactors that run over text an outside caller can
    make long, could be handed a value whose shape made the match take time
    out of all proportion to its length. Email.isValid reads values any
    caller can send, unauthenticated ones included, and much else.

  2. An email value was not held to one mailbox. Email.isValid matched a
    value that merely contained an address, so a list ("a@x.com,b@y.com"), a
    display name with an address, a group, or a value with surrounding text was
    accepted and stored as one address - and a mail sender would then read it as
    the several addresses inside it.

What changed

  • Email.isValid (Common/Types/Email.ts) now caps the value at the longest
    real address (254) before any pattern runs, and matches one whole mailbox,
    anchored at both ends. It still accepts every real single address it accepted
    before - plain, plus-addressed, sub-domains, a quoted local part, an address
    literal such as user@[192.168.1.1], apostrophes and the other RFC 5322 local
    characters. It now refuses anything that is more than one mailbox.
  • The same one-mailbox, length-bounded, left-to-right shape is applied to the
    other email-shaped checks that read outside text: the session-replay URL
    scrubber (runs in the browser over visited URLs), the on-call calendar feed
    display-name check, the on-call "who to page" setting reader, and the video
    call connection-settings address check.
  • The telemetry redactors - the log and trace scrub rules' built-in email
    pattern, exception fingerprinting, and AI tool-result redaction - have their
    email pattern bounded to the sizes a real address has, so finding one in a
    long ingested body stays in line with the body's length. They still find and
    replace exactly the real addresses they did before.

What this means for existing customers

  • A value that is not a single mailbox is refused where it used to be
    accepted.
    New sign-ups, subscriptions, invitations and the like that pass
    more than one address in a single-address field now get a clear "not in valid
    format" error instead of silently storing the whole string. This is the
    intended tightening - a single-address field is for one address.
  • A value stored before this check still reads and renders. Reads
    (fromDatabase / fromJSON) keep a stored value that is no longer a single
    mailbox, so a single such row cannot fail the query that reads it; writes stay
    strict, so that value can never be written back or newly created. This mirrors
    how URL keeps a legacy value it can no longer parse.
  • Every real single address that was valid before is still valid. No real email,
    token or address stops being validated or redacted.

Tests added (what each proves)

  • Common/Tests/Types/EmailOneMailboxAndLinearTime.test.ts - one mailbox only
    (accepts every real shape, refuses lists / names / groups / surrounding text);
    the 254 length bound; worst-case inputs of 400,000 characters each decided in
    under two seconds, plus the short awkward value that was slow before; and that
    a stored non-mailbox value reads but never writes.
  • Common/Tests/Types/ValueValidatorsLinearTime.test.ts - the URL scrubber,
    on-call display-name check and who-to-page reader each decide a 400,000-char
    value in under two seconds and still handle a real address.
  • Common/Tests/Server/Utils/Telemetry/TelemetryRedactionLinearTime.test.ts -
    AI tool-result redaction, exception fingerprinting and exception-message
    sanitization each read 400,000-char worst cases in under two seconds and still
    redact a real address and a real token.
  • App/Tests/Telemetry/ScrubBuiltInEmailLinearTime.test.ts - the log and trace
    built-in email scrub reads a 400,000-char body in under two seconds and still
    redacts a real address, end to end through scrubLog.
  • VideoCallConnectionSettings.test.ts - the address check decides long values
    quickly and still accepts/refuses correctly.

A mutation check confirmed the new one-mailbox and timing assertions fail on the
old behaviour and pass on the new.

What I ran

tsc on Common and App; Common compile-tests (all three test programs); the
new suites and the existing suites that read the changed files (Email, URL
scrubber, on-call feed, video call, public-message limits, identity reset/sign-up
limits, scrub-rule engine); all Common guard suites; the App root-level guards;
the repository ReDoS timing check; prettier and eslint on every changed file.

The repository ReDoS timing check reports a theoretical polynomial degree for
the telemetry redactors' global email patterns; that is the expected report for
a bounded repeat inside a global scan, and the committed worst-case tests at
400,000 characters show each of those patterns runs in well under two seconds.

Review

A second pass tried to get around the change and widened it to every rule of
the same kind it found. What changed in that pass:

Every pattern that finds a JWT in text now looks for a token once per run.
A token is eyJ and runs of base64url characters split by dots. The JWT rules
of the server's log redaction (LogRedaction), AI tool-result redaction
(Serializer), cluster and resource output redaction
(KubectlOutputRedactor, its copy in ResourceOutputRedactor and the resource
agent's synced copy), exception fingerprinting (ExceptionSanitizer) and the
instance health page's scrub (AdminHealthProbes) each begin with a
look-behind that lets a match start only where a run of those characters
starts, and settle with a look-ahead on the run's first place a token may
begin; the run's text before it is put back. A later place in the same run
has a shorter first segment and can end no better, so each rule replaces
exactly the tokens it replaced before - of any length, with no bound on a
segment (the first pass's 512/16,384-character bounds are gone, so a token
whose header carries a certificate chain is redacted and fingerprinted as
before) - while each run is read a bounded number of times. Committed drawn-text
tests compare each rule with the rule as it was on 20,000 texts each.

Other rules of the same kind:

  • A line of cluster or resource output is read for its key once: an unquoted
    key ends on a character that is not a space or a tab, so the spaces before
    the colon belong to the gap after it. The same keys are found (pinned line by
    line against the earlier redactor's output).
  • The health page's connection-string scrub reads a scheme of at most 64
    characters; the masked text reads exactly as before.
  • The address of "Name " in an inbound email's From and To is read up
    to the next "<" (SendGridInboundProvider, IncomingEmailMonitorAddress).
  • Splunk's address check (isIpAddress) never hands a field longer than the
    longest written address (45 characters) to its patterns.
  • The mobile session-replay recorder's email-shaped route segment check is
    bounded and anchored, like the web recorder's.
  • Email.isValid reads only text; Email.fromJSON refuses a value that is not
    text, or none, as it did before (it stays lenient for stored text, which is
    what the browser needs); a stored value of nothing but spaces reads as no
    value.
  • MailService.send - where every email of the instance leaves - refuses a
    recipient that is not one mailbox, so a value stored before the one-mailbox
    rule is never handed to a transport.

More for existing customers:

  • An address stored before this change in a single-address field or setting
    that is not one mailbox - a list, or a name with an address - is never sent
    to: the send is refused and recorded as failed (the notification API already
    builds its recipient with the strict check, and MailService.send checks
    again). The value still shows where it is stored; setting it to one address
    (a shared mailbox, for several people) makes mail reach it again. A field
    meant for several addresses (billing's finance and accounting email) already
    reads each address of its list on its own and is unchanged.
  • A mailto: link with several recipients is no longer accepted where a URL is
    stored, since a URL's mailto address is an email value; stored ones still
    read.
  • An inbound email whose header opens several "<" before one ">" is read for
    the address after the last of them.
  • A Splunk event field longer than 45 characters is read as a host name, never
    as an address.

Tests added in this pass (what each proves):

  • Common/Tests/Server/Utils/JsonWebTokenRedactionLinearTime.test.ts - all six
    JWT redactors read 400,000 characters of each awkward shape in under two
    seconds; still redact an ordinary token, one with a header of kilobytes and
    one with a payload of tens of kilobytes; find tokens where they found them
    before (after a hyphen for the word-boundary rules, glued to the text before
    for the others, side by side, counted once each); and replace exactly what
    the rule as it was replaced, on drawn texts.
  • Common/Tests/Server/Utils/TokenPatternsGuard.test.ts - a guard over all
    product code (ee/ and the agents' copies too, when checked out): every
    JWT-shaped pattern begins with the run-start look-behind unless listed with
    the reason it is linear on its own, and every one reads the shapes that were
    slow in under two seconds.
  • Common/Tests/Utils/AiRemediation/KubectlOutputKeyLinesLinearTime.test.ts -
    both output redactors read 400,000-character lines in under two seconds, and
    mask 22 key lines exactly as the earlier redactor did.
  • Common/Tests/Server/Services/InboundEmail/InboundEmailAddressLinearTime.test.ts -
    From/To headers and recipients of 400,000 characters are read in under two
    seconds; names, bare addresses and a monitor's generated address are read as
    before.
  • Common/Tests/Utils/SecurityEvent/Connectors/SplunkIpAddressLength.test.ts -
    long fields are decided at once; every real IPv4/IPv6 form, up to the
    45-character one, is still an address.
  • App/Tests/AdminDashboard/AdminHealthScrubLinearTime.test.ts - the health
    page scrub reads 400,000 characters in under two seconds and masks
    connection-string passwords and tokens exactly as before.
  • App/Tests/Notification/MailServiceOneMailbox.test.ts - an ordinary address
    is sent as before; a stored list, name, group or address with text after it
    is refused and never reaches a transport.
  • MobileRecorder/Tests/SanitizeRouteLinearTime.test.ts - long route segments
    are scrubbed in under two seconds and real addresses still taken out.
  • EmailOneMailboxAndLinearTime.test.ts gained the remaining spellings an
    address can arrive in (headers, comments, encoded names, mailto:, ports,
    bracketed lists, dot placement), quoted local parts as the one mailbox they
    are, the text-only rule for isValid / fromJSON / fromDatabase, and the
    most awkward values at the length bound; ValueValidatorsLinearTime and
    VideoCallConnectionSettings gained the shape that was slow for the anchored
    email-shape checks (their earlier cases passed on the old code too).

Verification in this pass. The first pass's suites were run against
master's version of each source file it changed: the timing and one-mailbox
cases failed (one never finished) - but the anchored email-shape cases passed
on the old code too, which is why they gained the shape that was slow for
them. Then a mutation run put back the old form of each fix - master's form,
or the first pass's bounded token patterns; 24 mutations - one at a time, in a
copy of the packages, and ran the suites that should notice: every one was
caught (a first run showed the token cases let a long-token miss hide behind
the Authorization header they sat in; they now run in plain text).

Also run, after the last merge of master: tsc on Common, App and the mobile
recorder; Common compile-tests; the suites that read every changed file (log
redaction and logger, serializer, exception fingerprinting, kubectl and
resource redaction with both copy-parity suites, inbound email, Splunk, admin
health, every Notification suite), the new ones with billing on and off and
with ee/ hidden; ee's admin health suites with ee/ present; the resource
agent's own suite; all Common guard suites; App's root-level tests; the ReDoS
checker (no new pattern that is not linear, apart from the bounded repeats
explained beside them); prettier and eslint on every changed file.

🤖 Generated with Claude Code

valeriamustafaeva and others added 3 commits October 19, 2026 02:16
Email.isValid and several email-shaped checks that read text an outside
caller can make long are now read once from end to end, in time that stays
in line with the input's length, with a length bound where the value has
one. Email.isValid also now requires the whole value to be ONE mailbox:
a list, a display name with an address, a group or any surrounding text is
refused rather than read as the address inside it. The telemetry redactors
(log/trace scrub, exception fingerprinting, AI tool-result redaction) keep
finding and replacing real addresses and tokens, bounded to the sizes a
real address or token has.

A value stored before the one-mailbox check still reads and renders
(fromDatabase / fromJSON stay lenient), but can never be written back or
newly created (toDatabase stays strict), mirroring how URL keeps a legacy
value it can no longer parse.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@simlarsen

simlarsen commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor Author

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

valeriamustafaeva and others added 4 commits October 19, 2026 03:40
…ail goes to one mailbox

Every pattern that finds a JWT in text - server log redaction, tool-result
redaction, cluster and resource output redaction, exception fingerprinting
and the instance health page's scrub - now looks for a token once per run of
the characters a token is made of, from the first place in the run one may
begin, and puts the run's text before it back. It replaces exactly the
tokens it replaced before, of any length (no bound on a segment), while each
run is read a bounded number of times.

- A line of cluster or resource output is read for its key once: an unquoted
  key ends on a character that is not a space, so the spaces before the
  colon belong to the gap after it (the resource agent's copy is synced).
- The health page's connection-string scrub reads a scheme of at most 64
  characters; the masked text reads exactly as before.
- The address of "Name <address>" in an inbound email's From and To is read
  up to the next "<"; where several open before a ">", it is what follows the
  last.
- A Splunk field longer than the longest written address (45 characters) is
  never handed to the address patterns.
- The mobile recorder's email-shaped route segment check is bounded and
  anchored, like the web recorder's.
- Email: isValid reads only text; fromJSON refuses a value that is not text,
  or none, as before (it stays lenient for stored text); a stored value of
  nothing but spaces reads as no value.
- MailService.send refuses a recipient that is not one mailbox, so a value
  stored before the one-mailbox rule is never sent to.

Tests: the JWT rules at 400,000 characters of each awkward shape, real
tokens with headers and payloads of kilobytes, and drawn texts proving each
rule replaces exactly what it did before; a guard holding every JWT pattern
in product code to the once-per-run form; key lines; inbound headers; the
Splunk bound; the health page scrub; the mobile route; one-mailbox sends;
and the anchored email-shape validators on the shape that was slow for them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… that names no credential

The token cases ran inside an Authorization header, where a Bearer or key
rule also masks the token: a token rule that missed a long token still
passed. They now run in plain text and expect each redactor's own token
marker, whole.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@simlarsen
simlarsen merged commit e0a31cb into master Oct 19, 2026
5 of 62 checks passed
@simlarsen
simlarsen deleted the claude/validators-linear-time branch October 19, 2026 04:15
simlarsen pushed a commit that referenced this pull request Oct 19, 2026
…t text; an address stored before the one-mailbox check is read

"a widget whose stored JSON cannot be read back keeps its JSON, and the
others are read" failed Common Test 4/8 on the 14.0.75 gate (#5026, job
117888307421). Its unreadable argument was { _type: "Email", value: "not
an email" }. Since #5025 (e8ac0b7), Email.fromJSON keeps a stored value
that is not one mailbox as a malformed Email, so stored rows still read
and render; only a value that is not text, or none, is refused as before.
The widget was read, and its owner came back as an Email, not its JSON.

The test now stores an owner whose value is 42, which Email still refuses.
A new test pins the other side: a widget holding an address stored before
the check (a name with an address) is read like its other arguments, its
owner an Email marked malformed, kept as stored.

All 262 tests in the file pass. Rethrowing in readComponent's catch fails
the first test; Email.fromJSON strict again (new Email(value)) fails the
new one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
simlarsen pushed a commit that referenced this pull request Oct 19, 2026
…s; a longer stored one gets no token

Three generate tests failed App Test 3/3 on the 14.0.75 gate (#5026, job
117888381129) with "Email aaa...@example.com is not in valid format",
thrown while the test built its address. They made Emails as long as the
token's payload cap allows (about 600 characters), one past it, and 5000.
Since #5025, Email holds at most MAX_EMAIL_ADDRESS_LENGTH (254) characters,
and verify reads the address with Email.isValid. So no address an Email
holds reaches the payload cap, and a longer one cannot be built with new
Email at all.

The tests now say what holds:
- the longest address an Email holds mints a token, inside the cap (the
  cap's longest address is pinned as longer than it);
- a stored address one character longer, read with Email.fromDatabase as
  a row stored before the check would be (kept, marked malformed), gets
  no token rather than one the route would refuse;
- so does a 5000-character stored address, past the payload cap too.

The payload cap itself stays covered by the verify tests under "over-long
and undecodable input". All 121 tests pass. Returning the token without
the verify round trip fails the new tests. Dropping verify's isValid check
does not: verify then builds new Email(email), which applies the same
check and throws into its catch, so the token is refused either way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
simlarsen pushed a commit that referenced this pull request Oct 19, 2026
…t text; an address stored before the one-mailbox check is read

"a widget whose stored JSON cannot be read back keeps its JSON, and the
others are read" failed Common Test 4/8 on the 14.0.75 gate (#5026, job
117888307421). Its unreadable argument was { _type: "Email", value: "not
an email" }. Since #5025 (e8ac0b7), Email.fromJSON keeps a stored value
that is not one mailbox as a malformed Email, so stored rows still read
and render; only a value that is not text, or none, is refused as before.
The widget was read, and its owner came back as an Email, not its JSON.

The test now stores an owner whose value is 42, which Email still refuses.
A new test pins the other side: a widget holding an address stored before
the check (a name with an address) is read like its other arguments, its
owner an Email marked malformed, kept as stored.

All 262 tests in the file pass. Rethrowing in readComponent's catch fails
the first test; Email.fromJSON strict again (new Email(value)) fails the
new one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 806dad0)
simlarsen pushed a commit that referenced this pull request Oct 19, 2026
…s; a longer stored one gets no token

Three generate tests failed App Test 3/3 on the 14.0.75 gate (#5026, job
117888381129) with "Email aaa...@example.com is not in valid format",
thrown while the test built its address. They made Emails as long as the
token's payload cap allows (about 600 characters), one past it, and 5000.
Since #5025, Email holds at most MAX_EMAIL_ADDRESS_LENGTH (254) characters,
and verify reads the address with Email.isValid. So no address an Email
holds reaches the payload cap, and a longer one cannot be built with new
Email at all.

The tests now say what holds:
- the longest address an Email holds mints a token, inside the cap (the
  cap's longest address is pinned as longer than it);
- a stored address one character longer, read with Email.fromDatabase as
  a row stored before the check would be (kept, marked malformed), gets
  no token rather than one the route would refuse;
- so does a 5000-character stored address, past the payload cap too.

The payload cap itself stays covered by the verify tests under "over-long
and undecodable input". All 121 tests pass. Returning the token without
the verify round trip fails the new tests. Dropping verify's isValid check
does not: verify then builds new Email(email), which applies the same
check and throws into its catch, so the token is refused either way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 9222a2d)
simlarsen pushed a commit that referenced this pull request Oct 19, 2026
…gned-in user's stored address reads again

SignedInRedirect.test.tsx has failed 3/3 on master since #5025. A local run
of the Common shards the 14.0.75 gate had not reached found it. #5025 made
Email.isValid, and so Email's constructor, refuse everything that is not
text, so that a list or an object from JSON is never read as the text it
would turn into. An Email is not text either. UserUtil.getEmail reads the
address local storage holds (LocalStorage.getItem gives it back as an
Email) and copies it with new Email(...). With no email cookie (it expires
after 30 days, or this host never had one) that threw, and isLoggedIn()
with it, on every page that asks whether someone is signed in.

The email setter now reads an Email handed to it by its text, which it
checks again like any other. A copy of a malformed Email (a stored value
read leniently) is still refused, as are lists and plain objects.
Email.isValid is unchanged: every caller hands it text.

Tests:
- EmailOneMailboxAndLinearTime: a valid Email copies; a malformed one is
  refused.
- UserEmailFromLocalStorage (new): with no cookie, the stored address is
  the user's and they are signed in; the cookie is read first; with
  nothing stored, no one is.
Without the setter's Email branch, the copy test, the new suite's first
test and SignedInRedirect fail. Taking an Email's text without checking it
again fails the copy test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
simlarsen pushed a commit that referenced this pull request Oct 19, 2026
…with "Please enter one email address." again

StatusPageSubscriberMessageLimits has failed 3 tests on master since #5025.
A local run of the Common shards the 14.0.75 gate had not reached found
them. The subscribe and manage-link routes built new Email(subscriberEmail)
before #5019's one-address check (PublicMessageRateLimit.forEmail). Since
#5025, Email's constructor refuses a list itself, in its own words ("Email
a@example.com,b@example.com is not in valid format."). So these two routes
no longer answered as #5019 has every public message route answer.

StatusPageAPI.readSubscriberEmail now holds the value to forEmail's
one-mailbox check first, then builds the Email. A list, a group, a name
with an address or a trailing dot is refused with "Please enter one email
address." again, before anything is counted or looked up. An address with
an apostrophe still signs up.

Without the check, the three tests fail again: two sign-up tests and the
manage link's "refused before anything is looked up".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
simlarsen pushed a commit that referenced this pull request Oct 19, 2026
…before #5025

Email.test.ts "should create an instance of Email through the transformer"
has failed on master since #5025: getDatabaseTransformer().from(new
Email("test@test.com")) threw "Email test@test.com is not in valid
format.". #5025's lenient read (fromStoredValueLenient) refuses everything
that is not text, and fromDatabase handed it the Email as it came. No gate
cut with #5025 reached that Common shard. It is not the setter's path that
38176ea fixed.

fromDatabase now reads an Email handed to it by its text, through the same
lenient read. A valid one reads as itself; one that is not one mailbox
stays malformed, and is still refused on write. Lists and plain objects
are still refused.

Tests:
- The transformer test passes again.
- EmailOneMailboxAndLinearTime gains a test that fromDatabase and the
  transformer read an Email by its text, with a malformed one staying
  malformed.
Without the new branch, both fail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
simlarsen pushed a commit that referenced this pull request Oct 19, 2026
…in one pass

Code scanning alert #2311 (js/polynomial-redos, high) failed CodeQL on the
14.0.76 gate (#5040). It is open on master since #5028.
VercelMarketplaceMonitors.normalizeDomain removed a domain's path with
.replace(/\/.*$/, ""). The pattern starts at every slash, and . stops at a
line break, so on a run of slashes followed by one each start scanned to
the break and backtracked: 40,000 characters took 1.5 s, and 400,000 about
two and a half minutes. The value is the production domain a Vercel
webhook names.

The path is now cut at the first slash with indexOf, and a trailing dot
with endsWith. A value with a line break in it is refused first, as the
old pattern refused it: its . never crossed a break, so any such value
reached DOMAIN_PATTERN with the break still in it. Every other value reads
as before.

Tests (VercelMarketplaceMonitors.test.ts):
- the host alone, from values with paths;
- a value with a line break, wherever the break is, is no domain;
- 400,000 characters of slashes, with and without a host and a break,
  read in under 2 s through fastestRun (about 1 ms here).
With the old code back, at 80,000 characters, the two inputs with a break
take about 7.8 s each and fail. All 16 tests in the file pass.

Recheck over every new pattern since 14.0.75's base flags five others, in
#5025 and #5038's email and URL scrubbers. Timed on its own attack strings
at 400,000 characters, each takes 137-178 ms: bounded repeats, linear in
practice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant