Tuesday 8 September 2009

NOTE: More on precision

Further to the earlier article on precision, the SAS papers listed below provide a lot of insight. However, unless I'm reading them wrongly, they don't tell me how many digits I can rely on when performing calculations.

This code, run on Windows, suggests the maximum is 15 digits

69 data x;
70   format num num2 best32.;
71   /* 0----+----1----+----2----+----3 */
72   char='0.123456789012345678901234567890';
73   num = input(char,best32.);
74   num2 = num + num;
75   put _all_;
76 run;
num=0.12345678901234 num2=0.24691357802469 char=0.123456789012345678901234567890 _ERROR_=0 _N_=1


SAS(R) 9.2 Language Reference: Concepts: Numeric precision in SAS software
http://support.sas.com/documentation/cdl/en/lrcon/61722/HTML/default/a000695157.htm

TS-DOC: TS-654 - Numeric Precision 101
http://support.sas.com/techsup/technote/ts654.pdf