[LON-CAPA-users] Any way around this?

[Mills, Douglas G]

HI All,

I’m finding that if I define a variable as follows:

$CB1=&format(8.000e-6,"4s");

OR

$CB1=&format(8.000e-6,"3e");

And use it as the answer to a numerical problem requiring 4 significant figures, I get the error message that “the computer’s answer is incorrect (‘8e-06’).”

I know (and have tested to confirm) that if I simply change the final zero to a 1 (so 8.001e-6) everything is fine. I had thought the &format command was the key to insisting on the correct number of sig figs in a variable, but I guess hadn’t run into this specific situation before.

Any help from the pros?  Thanks,

Doug

Douglas Mills
Director of Instructional Technologies
Department of Chemistry
University of Illinois
dmills at illinois.edu
(217) 244-5739

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.lon-capa.org/pipermail/lon-capa-users/attachments/20120821/9c4a8e7d/attachment.html>