Tuesday 8 September 2009

NOTE: More on precision

Further to the earlier article on precision, the SAS papers listed below provide a lot of insight. However, unless I'm reading them wrongly, they don't tell me how many digits I can rely on when performing calculations.

This code, run on Windows, suggests the maximum is 15 digits

69 data x;
70   format num num2 best32.;
71   /* 0----+----1----+----2----+----3 */
72   char='0.123456789012345678901234567890';
73   num = input(char,best32.);
74   num2 = num + num;
75   put _all_;
76 run;
num=0.12345678901234 num2=0.24691357802469 char=0.123456789012345678901234567890 _ERROR_=0 _N_=1

SAS(R) 9.2 Language Reference: Concepts: Numeric precision in SAS software

TS-DOC: TS-654 - Numeric Precision 101