diff options
| author | Eric Kunze <eric.kunze@arm.com> | 2022-06-30 14:46:07 -0700 |
|---|---|---|
| committer | Eric Kunze <eric.kunze@arm.com> | 2022-06-30 14:46:07 -0700 |
| commit | b8522ba72949b3eca03410f931f2655ee8485fa5 (patch) | |
| tree | d9eba9eb23b099c3d2fc00fca680844d5652bc3a | |
| parent | 1718f3a29657bb59d694cac4b1e1ab62cff2250e (diff) | |
| download | specification-b8522ba72949b3eca03410f931f2655ee8485fa5.tar.gz | |
Relax limit on input value for RESCALE
The current value is too strict, and can lead to some cases to fail.
By making the new value limit between (-1 << shift-1) and (1 << shift-1),
we still restrict the input value a range that works for optimized implementations
that left shift their values before multiplying.
Signed-off-by: Eric Kunze <eric.kunze@arm.com>
Change-Id: Ib4f49732e361418884888d398087560da3ccd6e8
| -rw-r--r-- | chapters/introduction.adoc | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/chapters/introduction.adoc b/chapters/introduction.adoc index 93206ca..be475d4 100644 --- a/chapters/introduction.adoc +++ b/chapters/introduction.adoc @@ -352,7 +352,7 @@ The values to achieve a scaling of 1.0 are shift=30, multiplier=1<<30 for apply_ int32_t apply_scale_32(int32_t value, int32_t multipler, uint6_t shift, bool_t double_round=false) { REQUIRE(multiplier >= 0); REQUIRE(2 <= shift && shift <= 62); - REQUIRE(value >= (-1<<(shift-2)) && value < (1<<(shift-2)); + REQUIRE(value >= (-1 << (shift - 1)) && value < (1 << (shift - 1)); int64_t round = 1 << (shift - 1); if (double_round) { if (shift > 31 && value >= 0) round += 1<<30; |