“Frozen” has to be true in the optimizer, not just in the training plan. Suppose a custom loop includes base parameters in an AdamW parameter group and forcibly sets their gradients to zero to prevent learning. A zero tensor is still a gradient value. AdamW's decoupled weight decay can shrink a parameter on an optimizer step even when the data gradient is zero. Existing optimizer moments may also move it. By contrast, in PyTorch's implementation, parameters with grad is None are omitted from the step. You can see that selection in the Adam optimizer source, and the AdamW documentation specifies decoupled weight decay.

This is a conditional failure, not a claim that ordinary PEFT training changes frozen base weights. If base parameters have requires_grad=False, are absent from optimizer groups and never acquire gradients, they should stay fixed. A careless optimizer rebuild after checkpoint restore, an adapter wrapper that leaves some base parameters trainable, or a hook replacing None gradients with zeros can break that invariant.

I would take an immutable digest of selected base tensors before the first step and after a small known number of steps. List the optimizer parameter groups by stable parameter name, requires-grad flag, gradient state and weight-decay value. Compare both grad is None and all-zero gradients under one optimizer step in a minimal reproduction. If the base changes, distinguish weight decay from a real backprop gradient, an EMA swap, mixed-precision copy, or a checkpoint/load mismatch.

The fix is to construct the optimizer from the intended trainable adapter parameters and any explicitly allowed extra modules. Assert that the intersection with the frozen base set is empty, then check the base hash at intervals. Setting weight decay to zero for accidentally included base weights may stop one route to drift but leaves the wrong ownership boundary in place.

If the interviewer says the loss still improves, that proves nothing about which weights changed. A small unintended base update can even help the training metric while invalidating the cost, reproducibility and serving contract for a LoRA-only artifact.