| commit | 5b201d7e6522b0b488bb146f967ca9eb6b37e65d | [log] [tgz] |
|---|---|---|
| author | Anthony Stange <stange@google.com> | Thu Jun 11 09:04:32 2020 -0400 |
| committer | Anthony Stange <stange@google.com> | Thu Jun 11 09:04:32 2020 -0400 |
| tree | 9d1deca57ec977f90403394b413a8ffc2b0d87ab | |
| parent | 73b522de919441111ab0c90d085a4bc3e5ad707b [diff] |
Requantize by resolution / 8 Requantizing by resolution / 4 causing the calibration error / min on calibrated sensors to be too high. Changing to 8 reduces that error enough to keep sensor accuracy below our thresholds. Bug: 158451802 Test: Verify that calibration error / min is low Change-Id: I350980734cc0a46f4f06878371e0d9bd7ce07de0