Timeline for awk arithmetic differs from expr
Current License: CC BY-SA 4.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 21, 2018 at 14:53 | comment | added | user232326 |
@ilkkachu (1) Yes, Sorry about OFMT, yes, it is g not f. (2) If the result of a division require a decimal (some non-zero digit after the dot) then that result is internally converted to a float with the precision given by PREC. Any integer gets as many digits (before the dot) as it may need. Try gawk -M -vPREC=20 -vOFMT='%.30f' 'BEGIN{ a=22^22; b=101; print a,a/22,b/10; }' The first two numbers are integers and are exact, the last is as imprecise as given by PREC=20. Change it to PREC=200 to see the change on the last number only.(4)Yes, rounding could be adjusted.
|
|
| Aug 21, 2018 at 8:50 | comment | added | ilkkachu |
And there's of course ROUNDMODE for setting the rounding mode if "round to even" isn't what you want
|
|
| Aug 21, 2018 at 8:46 | comment | added | ilkkachu |
@Isaac, well, the online manual says the default for OFMT is %.6g, so I get 5.05051e+17 for gawk -M -vPREC=quad 'BEGIN{ a = 11111111111111111111; print a/22; }'. As for just using -M, gawk -M 'BEGIN{ a = 11111111111111111111; print a/22; }' gives me the wrong answer 505050505050505024. I'm not sure if that's a bug or if it automatically converts the number to a float when it's no longer an integer (a/11 is an integer, and it gives the right answer with just -M).
|
|
| Aug 18, 2018 at 15:52 | history | edited | ilkkachu | CC BY-SA 4.0 |
added 624 characters in body
|
| Aug 17, 2018 at 0:58 | comment | added | steeldriver |
Note that at least newer versions of GNU awk have features for Getting the Accuracy You Need e.g. gawk -M -v PREC="quad" -v OFMT="%.0f" 'BEGIN{ a = 11111111111111111111; print a, a/22; }'
|
|
| Aug 16, 2018 at 20:27 | history | edited | ilkkachu | CC BY-SA 4.0 |
added 12 characters in body
|
| Aug 16, 2018 at 20:18 | history | answered | ilkkachu | CC BY-SA 4.0 |