# [LON-CAPA-users] How Best to Resolve Floating Point Issues?

Raeburn, Stuart raeburn at msu.edu
Wed Nov 27 00:51:54 EST 2019

```Doug,

Borrowing from the fp_equal() subroutine used to compare whether two floating-point numbers are equal, to a particular number of decimal places, found at:
http://perldoc.perl.org/perlop.html#Floating-point-Arithmetic

you could test whether the values of \$molt4 and \$actualM rounded to the desired precision (e.g., 3 decimal places) are the same, and if so, set the \$percenterror to 0, otherwise calculate \$percenterror, using the method you used in your example.

\$precision = 3;
if (&roundto(\$molt4,\$precision) eq &roundto(\$actualM,\$precision)) {
\$percenterror = 0;
} else {
\$percenterror=abs(\$molt4-\$actualM)*100/\$actualM;
}

For any user supplied input you should also validate that the experimental data are reasonable.  Note: you could also set an absolute tolerance for the case where the \$percenterror is 0, and use a relative tolerance otherwise.  (Note: any tolerance defined within the problem could be overridden within a course).

if (&roundto(\$molt4,\$precision) eq &roundto(\$actualM,\$precision)) {
\$temp = 0;
\$percenterror = 0;
\$tol = 0.1;
} else {
\$tol = '5%';
\$temp = \$molt4-\$actualM;
\$percenterror=abs(\$molt4-\$actualM)*100/\$actualM;
}

<numericalresponse format="3s" answer="\$percenterror" id="11">
<responseparam name="tol" type="tolerance" default="\$tol" description="Numerical Tolerance" />
<responseparam name="sig" type="int_range" default="3,4" description="Significant Digits" />
</numericalresponse>

Stuart Raeburn

________________________________________
From: LON-CAPA-users <lon-capa-users-bounces at mail.lon-capa.org> on behalf of Mills, Douglas G <dmills at illinois.edu>
Sent: Tuesday, November 26, 2019 4:55 PM
To: Discussion list for LON-CAPA users
Cc: Mills, Douglas G
Subject: [LON-CAPA-users] How Best to Resolve Floating Point Issues?

Hi All,

I've run into a situation where the binary representation of base 10 numbers is catching up to us for a very small percentage of our students. To illustrate this I've created a public problem demonstrating the issue. Unlike the problems the students get this one, for the sake of demonstration, walks you through the calculations but uses the exact same numbers that one of our students had to work with resulting in a case where what should be zero is not equal to zero. What's the best way of preventing this from happening? (The source code is open also in case you want to have a look but it's pretty well spelled out in the problem itself for demonstration purposes).

https://access3.lon-capa.illinois.edu/res/uiuc/dmills/scratchwork/FloatingPointIssueDemo.problem<https://urldefense.com/v3/__https://access3.lon-capa.illinois.edu/res/uiuc/dmills/scratchwork/FloatingPointIssueDemo.problem__;!ioFpBMP7lJU!kYg_ctrV6fkf5-Q0k15HPZw7eDzJHG_oXmJmgLbFYGCVB4UljJ9noqPl_BVJGg\$>

Thanks in advance and, on that topic, Happy Thanksgiving!

Doug

DOUGLAS G MILLS
Director of Instructional Technology

Department of Chemistry

University of Illinois at Urbana-Champaign
Chemistry Annex Box A2
601 S. Mathews | M/C 712
Urbana, IL 61801
217.244.5739 | fax: 217.244.8029 | dmills at illinois.edu<mailto:dmills at illinois.edu>
chemistry.illinois.edu<https://urldefense.com/v3/__http://chemistry.illinois.edu__;!ioFpBMP7lJU!kYg_ctrV6fkf5-Q0k15HPZw7eDzJHG_oXmJmgLbFYGCVB4UljJ9noqMWvh09LQ\$>

[/var/folders/kv/f3wpy1cs51521x5grm_qx4tw0000gn/T/com.microsoft.Outlook/WebArchiveCopyPasteTempFiles/P698ojxP4tc7j8gMqgPv+E4KQAAAABJRU5ErkJggg==]<https://urldefense.com/v3/__http://illinois.edu/__;!ioFpBMP7lJU!kYg_ctrV6fkf5-Q0k15HPZw7eDzJHG_oXmJmgLbFYGCVB4UljJ9noqPPexT8GA\$>

Under the Illinois Freedom of Information Act any written communication to or from university employees regarding university business is a public record and may be subject to public disclosure.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 2604 bytes
Desc: image001.png
URL: <http://mail.lon-capa.org/pipermail/lon-capa-users/attachments/20191127/79e168ea/attachment.png>
```