Repository navigation
fix(validation): a value is read in time in line with its length, and an email is one mailbox - #5025
Merged
Merged
Conversation
Email.isValid and several email-shaped checks that read text an outside caller can make long are now read once from end to end, in time that stays in line with the input's length, with a length bound where the value has one. Email.isValid also now requires the whole value to be ONE mailbox: a list, a display name with an address, a group or any surrounding text is refused rather than read as the address inside it. The telemetry redactors (log/trace scrub, exception fingerprinting, AI tool-result redaction) keep finding and replacing real addresses and tokens, bounded to the sizes a real address or token has. A value stored before the one-mailbox check still reads and renders (fromDatabase / fromJSON stay lenient), but can never be written back or newly created (toDatabase stays strict), mirroring how URL keeps a legacy value it can no longer parse. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
…ail goes to one mailbox Every pattern that finds a JWT in text - server log redaction, tool-result redaction, cluster and resource output redaction, exception fingerprinting and the instance health page's scrub - now looks for a token once per run of the characters a token is made of, from the first place in the run one may begin, and puts the run's text before it back. It replaces exactly the tokens it replaced before, of any length (no bound on a segment), while each run is read a bounded number of times. - A line of cluster or resource output is read for its key once: an unquoted key ends on a character that is not a space, so the spaces before the colon belong to the gap after it (the resource agent's copy is synced). - The health page's connection-string scrub reads a scheme of at most 64 characters; the masked text reads exactly as before. - The address of "Name <address>" in an inbound email's From and To is read up to the next "<"; where several open before a ">", it is what follows the last. - A Splunk field longer than the longest written address (45 characters) is never handed to the address patterns. - The mobile recorder's email-shaped route segment check is bounded and anchored, like the web recorder's. - Email: isValid reads only text; fromJSON refuses a value that is not text, or none, as before (it stays lenient for stored text); a stored value of nothing but spaces reads as no value. - MailService.send refuses a recipient that is not one mailbox, so a value stored before the one-mailbox rule is never sent to. Tests: the JWT rules at 400,000 characters of each awkward shape, real tokens with headers and payloads of kilobytes, and drawn texts proving each rule replaces exactly what it did before; a guard holding every JWT pattern in product code to the once-per-run form; key lines; inbound headers; the Splunk bound; the health page scrub; the mobile route; one-mailbox sends; and the anchored email-shape validators on the shape that was slow for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…inear-time-review
… that names no credential The token cases ran inside an Authorization header, where a Bearer or key rule also masks the token: a token rule that missed a long token still passed. They now run in plain text and expect each redactor's own token marker, whole. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…inear-time-review
simlarsen
pushed a commit
that referenced
this pull request
Oct 19, 2026
…t text; an address stored before the one-mailbox check is read "a widget whose stored JSON cannot be read back keeps its JSON, and the others are read" failed Common Test 4/8 on the 14.0.75 gate (#5026, job 117888307421). Its unreadable argument was { _type: "Email", value: "not an email" }. Since #5025 (e8ac0b7), Email.fromJSON keeps a stored value that is not one mailbox as a malformed Email, so stored rows still read and render; only a value that is not text, or none, is refused as before. The widget was read, and its owner came back as an Email, not its JSON. The test now stores an owner whose value is 42, which Email still refuses. A new test pins the other side: a widget holding an address stored before the check (a name with an address) is read like its other arguments, its owner an Email marked malformed, kept as stored. All 262 tests in the file pass. Rethrowing in readComponent's catch fails the first test; Email.fromJSON strict again (new Email(value)) fails the new one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
simlarsen
pushed a commit
that referenced
this pull request
Oct 19, 2026
…s; a longer stored one gets no token Three generate tests failed App Test 3/3 on the 14.0.75 gate (#5026, job 117888381129) with "Email aaa...@example.com is not in valid format", thrown while the test built its address. They made Emails as long as the token's payload cap allows (about 600 characters), one past it, and 5000. Since #5025, Email holds at most MAX_EMAIL_ADDRESS_LENGTH (254) characters, and verify reads the address with Email.isValid. So no address an Email holds reaches the payload cap, and a longer one cannot be built with new Email at all. The tests now say what holds: - the longest address an Email holds mints a token, inside the cap (the cap's longest address is pinned as longer than it); - a stored address one character longer, read with Email.fromDatabase as a row stored before the check would be (kept, marked malformed), gets no token rather than one the route would refuse; - so does a 5000-character stored address, past the payload cap too. The payload cap itself stays covered by the verify tests under "over-long and undecodable input". All 121 tests pass. Returning the token without the verify round trip fails the new tests. Dropping verify's isValid check does not: verify then builds new Email(email), which applies the same check and throws into its catch, so the token is refused either way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
simlarsen
pushed a commit
that referenced
this pull request
Oct 19, 2026
…t text; an address stored before the one-mailbox check is read "a widget whose stored JSON cannot be read back keeps its JSON, and the others are read" failed Common Test 4/8 on the 14.0.75 gate (#5026, job 117888307421). Its unreadable argument was { _type: "Email", value: "not an email" }. Since #5025 (e8ac0b7), Email.fromJSON keeps a stored value that is not one mailbox as a malformed Email, so stored rows still read and render; only a value that is not text, or none, is refused as before. The widget was read, and its owner came back as an Email, not its JSON. The test now stores an owner whose value is 42, which Email still refuses. A new test pins the other side: a widget holding an address stored before the check (a name with an address) is read like its other arguments, its owner an Email marked malformed, kept as stored. All 262 tests in the file pass. Rethrowing in readComponent's catch fails the first test; Email.fromJSON strict again (new Email(value)) fails the new one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 806dad0)
simlarsen
pushed a commit
that referenced
this pull request
Oct 19, 2026
…s; a longer stored one gets no token Three generate tests failed App Test 3/3 on the 14.0.75 gate (#5026, job 117888381129) with "Email aaa...@example.com is not in valid format", thrown while the test built its address. They made Emails as long as the token's payload cap allows (about 600 characters), one past it, and 5000. Since #5025, Email holds at most MAX_EMAIL_ADDRESS_LENGTH (254) characters, and verify reads the address with Email.isValid. So no address an Email holds reaches the payload cap, and a longer one cannot be built with new Email at all. The tests now say what holds: - the longest address an Email holds mints a token, inside the cap (the cap's longest address is pinned as longer than it); - a stored address one character longer, read with Email.fromDatabase as a row stored before the check would be (kept, marked malformed), gets no token rather than one the route would refuse; - so does a 5000-character stored address, past the payload cap too. The payload cap itself stays covered by the verify tests under "over-long and undecodable input". All 121 tests pass. Returning the token without the verify round trip fails the new tests. Dropping verify's isValid check does not: verify then builds new Email(email), which applies the same check and throws into its catch, so the token is refused either way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 9222a2d)
simlarsen
pushed a commit
that referenced
this pull request
Oct 19, 2026
…gned-in user's stored address reads again SignedInRedirect.test.tsx has failed 3/3 on master since #5025. A local run of the Common shards the 14.0.75 gate had not reached found it. #5025 made Email.isValid, and so Email's constructor, refuse everything that is not text, so that a list or an object from JSON is never read as the text it would turn into. An Email is not text either. UserUtil.getEmail reads the address local storage holds (LocalStorage.getItem gives it back as an Email) and copies it with new Email(...). With no email cookie (it expires after 30 days, or this host never had one) that threw, and isLoggedIn() with it, on every page that asks whether someone is signed in. The email setter now reads an Email handed to it by its text, which it checks again like any other. A copy of a malformed Email (a stored value read leniently) is still refused, as are lists and plain objects. Email.isValid is unchanged: every caller hands it text. Tests: - EmailOneMailboxAndLinearTime: a valid Email copies; a malformed one is refused. - UserEmailFromLocalStorage (new): with no cookie, the stored address is the user's and they are signed in; the cookie is read first; with nothing stored, no one is. Without the setter's Email branch, the copy test, the new suite's first test and SignedInRedirect fail. Taking an Email's text without checking it again fails the copy test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
simlarsen
pushed a commit
that referenced
this pull request
Oct 19, 2026
…with "Please enter one email address." again StatusPageSubscriberMessageLimits has failed 3 tests on master since #5025. A local run of the Common shards the 14.0.75 gate had not reached found them. The subscribe and manage-link routes built new Email(subscriberEmail) before #5019's one-address check (PublicMessageRateLimit.forEmail). Since #5025, Email's constructor refuses a list itself, in its own words ("Email a@example.com,b@example.com is not in valid format."). So these two routes no longer answered as #5019 has every public message route answer. StatusPageAPI.readSubscriberEmail now holds the value to forEmail's one-mailbox check first, then builds the Email. A list, a group, a name with an address or a trailing dot is refused with "Please enter one email address." again, before anything is counted or looked up. An address with an apostrophe still signs up. Without the check, the three tests fail again: two sign-up tests and the manage link's "refused before anything is looked up". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
simlarsen
pushed a commit
that referenced
this pull request
Oct 19, 2026
…before #5025 Email.test.ts "should create an instance of Email through the transformer" has failed on master since #5025: getDatabaseTransformer().from(new Email("test@test.com")) threw "Email test@test.com is not in valid format.". #5025's lenient read (fromStoredValueLenient) refuses everything that is not text, and fromDatabase handed it the Email as it came. No gate cut with #5025 reached that Common shard. It is not the setter's path that 38176ea fixed. fromDatabase now reads an Email handed to it by its text, through the same lenient read. A valid one reads as itself; one that is not one mailbox stays malformed, and is still refused on write. Lists and plain objects are still refused. Tests: - The transformer test passes again. - EmailOneMailboxAndLinearTime gains a test that fromDatabase and the transformer read an Email by its text, with a malformed one staying malformed. Without the new branch, both fail. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
simlarsen
pushed a commit
that referenced
this pull request
Oct 19, 2026
…in one pass Code scanning alert #2311 (js/polynomial-redos, high) failed CodeQL on the 14.0.76 gate (#5040). It is open on master since #5028. VercelMarketplaceMonitors.normalizeDomain removed a domain's path with .replace(/\/.*$/, ""). The pattern starts at every slash, and . stops at a line break, so on a run of slashes followed by one each start scanned to the break and backtracked: 40,000 characters took 1.5 s, and 400,000 about two and a half minutes. The value is the production domain a Vercel webhook names. The path is now cut at the first slash with indexOf, and a trailing dot with endsWith. A value with a line break in it is refused first, as the old pattern refused it: its . never crossed a break, so any such value reached DOMAIN_PATTERN with the break still in it. Every other value reads as before. Tests (VercelMarketplaceMonitors.test.ts): - the host alone, from values with paths; - a value with a line break, wherever the break is, is no domain; - 400,000 characters of slashes, with and without a host and a break, read in under 2 s through fastestRun (about 1 ms here). With the old code back, at 80,000 characters, the two inputs with a break take about 7.8 s each and fail. All 16 tests in the file pass. Recheck over every new pattern since 14.0.75's base flags five others, in #5025 and #5038's email and URL scrubbers. Timed on its own attack strings at 400,000 characters, each takes 137-178 ms: bounded repeats, linear in practice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
Two rules did not hold for the way value validators and redactors read text:
A value was not read in time in line with its length.
Email.isValid,and several checks and redactors that run over text an outside caller can
make long, could be handed a value whose shape made the match take time
out of all proportion to its length.
Email.isValidreads values anycaller can send, unauthenticated ones included, and much else.
An email value was not held to one mailbox.
Email.isValidmatched avalue that merely contained an address, so a list ("a@x.com,b@y.com"), a
display name with an address, a group, or a value with surrounding text was
accepted and stored as one address - and a mail sender would then read it as
the several addresses inside it.
What changed
Email.isValid(Common/Types/Email.ts) now caps the value at the longestreal address (254) before any pattern runs, and matches one whole mailbox,
anchored at both ends. It still accepts every real single address it accepted
before - plain, plus-addressed, sub-domains, a quoted local part, an address
literal such as
user@[192.168.1.1], apostrophes and the other RFC 5322 localcharacters. It now refuses anything that is more than one mailbox.
other email-shaped checks that read outside text: the session-replay URL
scrubber (runs in the browser over visited URLs), the on-call calendar feed
display-name check, the on-call "who to page" setting reader, and the video
call connection-settings address check.
pattern, exception fingerprinting, and AI tool-result redaction - have their
email pattern bounded to the sizes a real address has, so finding one in a
long ingested body stays in line with the body's length. They still find and
replace exactly the real addresses they did before.
What this means for existing customers
accepted. New sign-ups, subscriptions, invitations and the like that pass
more than one address in a single-address field now get a clear "not in valid
format" error instead of silently storing the whole string. This is the
intended tightening - a single-address field is for one address.
(
fromDatabase/fromJSON) keep a stored value that is no longer a singlemailbox, so a single such row cannot fail the query that reads it; writes stay
strict, so that value can never be written back or newly created. This mirrors
how
URLkeeps a legacy value it can no longer parse.token or address stops being validated or redacted.
Tests added (what each proves)
Common/Tests/Types/EmailOneMailboxAndLinearTime.test.ts- one mailbox only(accepts every real shape, refuses lists / names / groups / surrounding text);
the 254 length bound; worst-case inputs of 400,000 characters each decided in
under two seconds, plus the short awkward value that was slow before; and that
a stored non-mailbox value reads but never writes.
Common/Tests/Types/ValueValidatorsLinearTime.test.ts- the URL scrubber,on-call display-name check and who-to-page reader each decide a 400,000-char
value in under two seconds and still handle a real address.
Common/Tests/Server/Utils/Telemetry/TelemetryRedactionLinearTime.test.ts-AI tool-result redaction, exception fingerprinting and exception-message
sanitization each read 400,000-char worst cases in under two seconds and still
redact a real address and a real token.
App/Tests/Telemetry/ScrubBuiltInEmailLinearTime.test.ts- the log and tracebuilt-in email scrub reads a 400,000-char body in under two seconds and still
redacts a real address, end to end through
scrubLog.VideoCallConnectionSettings.test.ts- the address check decides long valuesquickly and still accepts/refuses correctly.
A mutation check confirmed the new one-mailbox and timing assertions fail on the
old behaviour and pass on the new.
What I ran
tscon Common and App; Commoncompile-tests(all three test programs); thenew suites and the existing suites that read the changed files (Email, URL
scrubber, on-call feed, video call, public-message limits, identity reset/sign-up
limits, scrub-rule engine); all Common guard suites; the App root-level guards;
the repository ReDoS timing check; prettier and eslint on every changed file.
The repository ReDoS timing check reports a theoretical polynomial degree for
the telemetry redactors' global email patterns; that is the expected report for
a bounded repeat inside a global scan, and the committed worst-case tests at
400,000 characters show each of those patterns runs in well under two seconds.
Review
A second pass tried to get around the change and widened it to every rule of
the same kind it found. What changed in that pass:
Every pattern that finds a JWT in text now looks for a token once per run.
A token is
eyJand runs of base64url characters split by dots. The JWT rulesof the server's log redaction (
LogRedaction), AI tool-result redaction(
Serializer), cluster and resource output redaction(
KubectlOutputRedactor, its copy inResourceOutputRedactorand the resourceagent's synced copy), exception fingerprinting (
ExceptionSanitizer) and theinstance health page's scrub (
AdminHealthProbes) each begin with alook-behind that lets a match start only where a run of those characters
starts, and settle with a look-ahead on the run's first place a token may
begin; the run's text before it is put back. A later place in the same run
has a shorter first segment and can end no better, so each rule replaces
exactly the tokens it replaced before - of any length, with no bound on a
segment (the first pass's 512/16,384-character bounds are gone, so a token
whose header carries a certificate chain is redacted and fingerprinted as
before) - while each run is read a bounded number of times. Committed drawn-text
tests compare each rule with the rule as it was on 20,000 texts each.
Other rules of the same kind:
key ends on a character that is not a space or a tab, so the spaces before
the colon belong to the gap after it. The same keys are found (pinned line by
line against the earlier redactor's output).
characters; the masked text reads exactly as before.
to the next "<" (
SendGridInboundProvider,IncomingEmailMonitorAddress).isIpAddress) never hands a field longer than thelongest written address (45 characters) to its patterns.
bounded and anchored, like the web recorder's.
Email.isValidreads only text;Email.fromJSONrefuses a value that is nottext, or none, as it did before (it stays lenient for stored text, which is
what the browser needs); a stored value of nothing but spaces reads as no
value.
MailService.send- where every email of the instance leaves - refuses arecipient that is not one mailbox, so a value stored before the one-mailbox
rule is never handed to a transport.
More for existing customers:
that is not one mailbox - a list, or a name with an address - is never sent
to: the send is refused and recorded as failed (the notification API already
builds its recipient with the strict check, and
MailService.sendchecksagain). The value still shows where it is stored; setting it to one address
(a shared mailbox, for several people) makes mail reach it again. A field
meant for several addresses (billing's finance and accounting email) already
reads each address of its list on its own and is unchanged.
mailto:link with several recipients is no longer accepted where a URL isstored, since a URL's mailto address is an email value; stored ones still
read.
the address after the last of them.
as an address.
Tests added in this pass (what each proves):
Common/Tests/Server/Utils/JsonWebTokenRedactionLinearTime.test.ts- all sixJWT redactors read 400,000 characters of each awkward shape in under two
seconds; still redact an ordinary token, one with a header of kilobytes and
one with a payload of tens of kilobytes; find tokens where they found them
before (after a hyphen for the word-boundary rules, glued to the text before
for the others, side by side, counted once each); and replace exactly what
the rule as it was replaced, on drawn texts.
Common/Tests/Server/Utils/TokenPatternsGuard.test.ts- a guard over allproduct code (ee/ and the agents' copies too, when checked out): every
JWT-shaped pattern begins with the run-start look-behind unless listed with
the reason it is linear on its own, and every one reads the shapes that were
slow in under two seconds.
Common/Tests/Utils/AiRemediation/KubectlOutputKeyLinesLinearTime.test.ts-both output redactors read 400,000-character lines in under two seconds, and
mask 22 key lines exactly as the earlier redactor did.
Common/Tests/Server/Services/InboundEmail/InboundEmailAddressLinearTime.test.ts-From/To headers and recipients of 400,000 characters are read in under two
seconds; names, bare addresses and a monitor's generated address are read as
before.
Common/Tests/Utils/SecurityEvent/Connectors/SplunkIpAddressLength.test.ts-long fields are decided at once; every real IPv4/IPv6 form, up to the
45-character one, is still an address.
App/Tests/AdminDashboard/AdminHealthScrubLinearTime.test.ts- the healthpage scrub reads 400,000 characters in under two seconds and masks
connection-string passwords and tokens exactly as before.
App/Tests/Notification/MailServiceOneMailbox.test.ts- an ordinary addressis sent as before; a stored list, name, group or address with text after it
is refused and never reaches a transport.
MobileRecorder/Tests/SanitizeRouteLinearTime.test.ts- long route segmentsare scrubbed in under two seconds and real addresses still taken out.
EmailOneMailboxAndLinearTime.test.tsgained the remaining spellings anaddress can arrive in (headers, comments, encoded names,
mailto:, ports,bracketed lists, dot placement), quoted local parts as the one mailbox they
are, the text-only rule for
isValid/fromJSON/fromDatabase, and themost awkward values at the length bound;
ValueValidatorsLinearTimeandVideoCallConnectionSettingsgained the shape that was slow for the anchoredemail-shape checks (their earlier cases passed on the old code too).
Verification in this pass. The first pass's suites were run against
master's version of each source file it changed: the timing and one-mailbox
cases failed (one never finished) - but the anchored email-shape cases passed
on the old code too, which is why they gained the shape that was slow for
them. Then a mutation run put back the old form of each fix - master's form,
or the first pass's bounded token patterns; 24 mutations - one at a time, in a
copy of the packages, and ran the suites that should notice: every one was
caught (a first run showed the token cases let a long-token miss hide behind
the Authorization header they sat in; they now run in plain text).
Also run, after the last merge of master:
tscon Common, App and the mobilerecorder; Common
compile-tests; the suites that read every changed file (logredaction and logger, serializer, exception fingerprinting, kubectl and
resource redaction with both copy-parity suites, inbound email, Splunk, admin
health, every Notification suite), the new ones with billing on and off and
with ee/ hidden; ee's admin health suites with ee/ present; the resource
agent's own suite; all Common guard suites; App's root-level tests; the ReDoS
checker (no new pattern that is not linear, apart from the bounded repeats
explained beside them); prettier and eslint on every changed file.
🤖 Generated with Claude Code