Correction (October 7, revised the same evening). A reader, arhancanli, pointed out that the two holdouts below are not the same trades. I replayed the 3x/3x bracket on only the 70 holdout trades both bar sets cover: public bars −$75.54, her own bars −$75.54, identical on every one of the 70 trades, and the bracket is ahead in 26% of day-resamples on both. The whole sign flip comes from 18 trades that are in her own bars' holdout and not in the public one. My first version of this note said the public bars "never had" those trades. That was wrong, and the same reader's follow-up question made me check: 17 of the 18 are in the public bars, in the training 70%. The public feed lacks 60 older trades (50 of them before the feed starts on September 3), so its 70/30 cut lands a day later, and her own bars' holdout starts on September 23 instead of September 24. Those 18 trades sit on three days, 15 of them on September 23, where the bracket made +$338.62 against her +$40.12; the 18th is one GC trade on September 29 with no public file at all. Two trades carry $322 of the $425 gap; without them the other 16 sum to +$103.50 with a bootstrap 95% interval of −$260 to +$479 and 7 of 16 positive, which is nothing. And the sign depends on where the cut falls: a holdout starting Sept 22 gives the bracket +$430 over her exits, the 70/30 cut (Sept 23, 01:18 KST) +$174, Sept 23 at midnight −$64, Sept 24 −$125, Sept 25 +$85. (An earlier version of this note quoted a 93% trade-level resample on the 18; the same reader pointed out that 15 of them share one day, so that number treated three days as eighteen observations. Withdrawn.) So the public bars were right where they existed. What moved the answer was where the holdout cut fell, and mostly one day. The headline overstates it; the rest of the post stands, with that reading. What settles it is a forward window fixed before it starts, so one is now written down and committed: PREREGISTRATION.md, October 8 to 31, days as the unit, a sign test with the thresholds stated, reported either way.

Last week I published a result about my partner's futures trading: her hand exits beat every fixed take-profit/stop bracket I could fit. It was out of sample, it had a sanity check, and it was wrong. Not because of a bug. Because of the bars.

This is what the check looked like, what it couldn't see, and the version of the table I now trust.

The check that passed

Her NinjaTrader export has one row per trade with entry time and fill price, on the PC's clock. The 1-minute bars I had at first were public (Yahoo Finance), in UTC. To replay her entries on bars you need two things: the clock offset between the two, and some confidence that each fill actually happened inside the bar you're about to use.

The tool searches offsets in 30-minute steps across a day in each direction, and for every offset counts how many entries have a fill price between that bar's low and high. The offset with the highest share wins:

def align(trades, bars):
    best = (pd.Timedelta(0), -1.0)
    for half_hours in range(-28, 29):
        off = pd.Timedelta(minutes=30 * half_hours)
        m = _fill_bars(trades, bars, off)          # one row per entry: its bar at this offset
        inside = ((m["entry_price"] >= m["low"]) & (m["entry_price"] <= m["high"])).sum() / len(m)
        if inside > best[1]:
            best = (off, float(inside))
    return best

On her September export the answer was an offset of 9 hours (Korea), and every entry that had a bar sat inside it. The output said:

233 of 293 entries have bars that match their fills (clock offset 0 days 09:00:00).

Every bar that existed was the right bar. I read that line as "the data is good" and moved on.

What it couldn't see

The 60 missing entries were not random. The public feed started on September 3, so her first days were gone, and it only carried the front-month contract, so anything she traded on another expiry had no bars at all. The check only scores entries that have a bar. It says nothing about the ones that don't, and it can't, because there is nothing to compare them against.

Then she exported her own 1-minute bars from NinjaTrader. All 293 entries had a bar, and the same alignment code found the same offset.

The same test, both ways

The test: fit a bracket (target and stop as multiples of her usual move) on her older 70% of trades, score it on the newest 30% it never saw, and compare with what her hand exits made on those same trades.

Same code, same bracket, opposite sign. On public bars her exits looked like the strongest part of her record, and I wrote that up. On her own bars a wide symmetric bracket would have made about $170 more than her hands this month.

How much of that is noise? Her trades cluster by day, so I resampled the 88 holdout trades by day 1,000 times: the bracket came out ahead in 66% of resamples. Better than a coin flip, not by much. The honest summary is "open question, next month's export decides", and that is what the report says now.

It wasn't only the bracket

The rules table moved too. She trades with a rule about the 15-minute 200 EMA. Checked at the moment of each click, no look-ahead:

The direction of the gap survived (with the trend wins more). The size of everything else didn't. On public bars breaking the rule looked like a losing habit. On her own bars it was 161 trades that made money, with a ten-point lower win rate. Those lead to different conversations.

What I do differently now

1. Coverage is a result, not a precondition. The replay prints "N of M entries have bars" before any table, and the next change is to list which entries are missing and why (date range, contract month). A check that only scores what exists needs a second line about what doesn't.
2. The trader's own bars, or nothing. Public minute data for futures is fine for charts and bad for replaying someone's fills: wrong start date, front month only, and no way to tell from the inside.
3. Report the sign flip. I put a correction on the first post rather than quietly updating the number. If a reader made a decision on the old table, they deserve to know it moved.
4. Fill-inside-bar is necessary, not sufficient. It catches the wrong offset and the wrong contract. It cannot catch absence.

The code is open source and runs on your machine; the replay prints the coverage line and the bracket table from a plain NinjaTrader export plus your own bars:

If you'd rather I run it on your export and write it up, the details are here: https://ssap-pa.github.io/tilt-check/