rattus 60ee574748 retune lowVramPatch VRAM accounting (#11173) ...

In the lowvram case, this now does its math in the model dtype in the
post de-quantization domain. Account for that. The patching was also
put back on the compute stream getting it off-peak so relax the
MATH_FACTOR to only x2 so get out of the worst-case assumption of
everything peaking at once.

2025-12-08 15:18:06 -05:00

57 KiB

Raw Blame History

View Raw