Skip to content

fix(finetune): the Apple Silicon and research scripts fit temperatures outside the runtime clamp - #851

Merged
NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
Bruce-Yii:fix/finetune-fitted-temperature-clamp
Oct 4, 2026
Merged

NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
Bruce-Yii:fix/finetune-fitted-temperature-clamp

Conversation

@Bruce-Yii

@Bruce-Yii Bruce-Yii commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

What

#642 aligned the Kaggle fine-tuning notebook with the runtime temperature bounds, but two other documented training entry points still clamp fitted temperatures to 0.1..10.0:

  • notebooks/laya_finetune_typed_decisions_mps.py
  • research/scripts/finetune_single_device.py

This PR switches both to TEMP_MIN / TEMP_MAX from laya.common.

It also extends the existing calibration regression so the supported fine-tuning fitters are checked against the same runtime bounds instead of testing only the Kaggle notebook.

Why

laya.Agent clamps stored temperatures again when loading a checkpoint.

On current main, the two remaining scripts can fit values outside the runtime range:

fitter peaked dipped served
Kaggle notebook 5.0 0.5 unchanged
Apple Silicon script 10.0 0.2848 5.0 / 0.5
single-device research script 10.0 0.2848 5.0 / 0.5

That means the calibration measured during training can differ from the temperature actually used at inference time.

Using the shared constants keeps fit and serve on the same contract.

How verified

Rebased onto current main (fa9a2a7). Run from the repository root, in a virtualenv, on CPU with no downloads:

python tests/test_calibration_persistence.py      # Ran 7 tests ... OK
python tests/test_packaging.py                    # 131 passed, 0 failed
ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120   # All checks passed!
python -m compileall -q laya/ tests/              # no output

The regression exercises both ends of the allowed range. Reverting the two production changes reproduces the mismatch: the Apple Silicon and research fitters reach the wider training bounds while the already-fixed Kaggle fitter remains within the runtime range.

The rebased test file also preserves the records_from_labeled loaded-agent regression added in #832.

GitHub Actions: 23/23 visible checks passed on head 45c94ec.

Scope

No training behavior changes beyond aligning the fitted-temperature bounds with the existing runtime contract. No API or documentation changes are required.

@Bruce-Yii
Bruce-Yii force-pushed the fix/finetune-fitted-temperature-clamp branch from 4c12e81 to 3990370 Compare October 2, 2026 18:33
@Bruce-Yii
Bruce-Yii force-pushed the fix/finetune-fitted-temperature-clamp branch from 3990370 to 45c94ec Compare October 2, 2026 18:39
@NandhaKishorM
NandhaKishorM merged commit 8a6e132 into NandhaKishorM:main Oct 4, 2026
23 checks passed
NandhaKishorM pushed a commit that referenced this pull request Oct 4, 2026
…d scripts to share

The training loop lives in three drifting copies (the Kaggle notebook, the MPS script and
research/scripts/finetune_single_device.py). This adds it to the package once, as the first
step of #887. TrainConfig() trains the way the published notebook does: loss="rlcd", options
in their listed order, and an effective batch of 64 (micro_batch=8 x grad_accum=8).

- loss="soft-ce" trains on the soft cross-entropy alone (the variant #741 tested).
- shuffle_options re-encodes items of the named types with a random option_order each epoch,
  through the same path inference uses, and permutes the target to match. Off by default.
- Questions go through Agent._check_question and Agent._to_internal, so training sees the
  option text inference shows (list-form choices, custom noul labels, non-string instructions).
- Calibration uses laya.calibrate.fit_temperature_map, so the fit follows the runtime clamp and
  buckets; an inherited temperature_by_options is dropped unless refitted.
- Questions that cannot be trained on are skipped and counted by reason (invalid_question,
  invalid_target, target_mismatch, options_collapsed per #538).
- The max_len / head_max_len used in training are written into the saved config, with the
  TrainConfig under training.laya_train.
- freeze_encoder trains the head only and keeps the encoder in eval mode.

The existing scripts are left untouched in this change; #851 and #885 are open against them.
tests/test_train.py is weight-free (one-layer BERT, word-level tokenizer, no downloads) and is
registered in the shared suite list and the Windows CI loop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged.

Two PRs fixed this and I took yours for the test, not the fix. The one-line change is identical in both: TEMP_MIN / TEMP_MAX from laya.common instead of the legacy 0.1..10.0 in the MPS notebook and the single-device script. @Swatantra-66's #885 adds two hardcoded tests, one per script, each exec-ing the extracted fitter.

Yours walks notebooks/ and research/scripts/ and holds every file whose fit clamps exp() of a log-temperature, with a count assertion so a fitter that stops clamping is reported rather than passing unnoticed. On the merged tree that finds all three:

notebooks/laya_finetune_typed_decisions_2xT4_kaggle.ipynb
notebooks/laya_finetune_typed_decisions_mps.py
research/scripts/finetune_single_device.py

That matters because the bug being fixed twice is itself the symptom. #887 opens with it: "Fixes have to be made once per copy: #851 and #885 were the same temperature-clamp bug in two of them." An enumerating test would need a third entry when a fourth script appears; yours would catch it.

I mutation-tested the gate rather than trusting it. Reverting one fitter to 0.1, 10.0:

AssertionError: 10.0 not less than or equal to 5.0 :
  notebooks/laya_finetune_typed_decisions_mps.py: fit_temperature(peaked) fitted 10
FAILED (failures=1)

Red, and the message names the file and the fitted value, so whoever breaks it does not have to go looking. Restored, it is OK at 7 tests.

Reading the notebook's %%writefile cells rather than the whole JSON is the detail that makes this work on a notebook at all, since the executed source is only in those cells.

This completes the #642 parity fix across all three documented entry points. #899 landed in the same pass and deliberately left these scripts alone to avoid conflicting with you, so both are in.

Thank you.

@sang0907791651-coder sang0907791651-coder left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hhh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants