# [LON-CAPA-users] How Best to Resolve Floating Point Issues?

Mills, Douglas G dmills at illinois.edu
Wed Nov 27 13:14:56 EST 2019

```Thank you Stuart and Happy Thanksgiving!

Doug

-sent from mobile-

> On Nov 26, 2019, at 11:53 PM, Raeburn, Stuart <raeburn at msu.edu> wrote:
>
> ﻿Doug,
>
> Borrowing from the fp_equal() subroutine used to compare whether two floating-point numbers are equal, to a particular number of decimal places, found at:
> http://perldoc.perl.org/perlop.html#Floating-point-Arithmetic
>
> you could test whether the values of \$molt4 and \$actualM rounded to the desired precision (e.g., 3 decimal places) are the same, and if so, set the \$percenterror to 0, otherwise calculate \$percenterror, using the method you used in your example.
>
> \$precision = 3;
> if (&roundto(\$molt4,\$precision) eq &roundto(\$actualM,\$precision)) {
>    \$percenterror = 0;
> } else {
>    \$percenterror=abs(\$molt4-\$actualM)*100/\$actualM;
> }
>
> For any user supplied input you should also validate that the experimental data are reasonable.  Note: you could also set an absolute tolerance for the case where the \$percenterror is 0, and use a relative tolerance otherwise.  (Note: any tolerance defined within the problem could be overridden within a course).
>
> if (&roundto(\$molt4,\$precision) eq &roundto(\$actualM,\$precision)) {
>    \$temp = 0;
>    \$percenterror = 0;
>    \$tol = 0.1;
> } else {
>    \$tol = '5%';
>    \$temp = \$molt4-\$actualM;
>    \$percenterror=abs(\$molt4-\$actualM)*100/\$actualM;
> }
>
>    <responseparam name="tol" type="tolerance" default="\$tol" description="Numerical Tolerance" />
>    <responseparam name="sig" type="int_range" default="3,4" description="Significant Digits" />
> </numericalresponse>
>
> Stuart Raeburn
>
> ________________________________________
> From: LON-CAPA-users <lon-capa-users-bounces at mail.lon-capa.org> on behalf of Mills, Douglas G <dmills at illinois.edu>
> Sent: Tuesday, November 26, 2019 4:55 PM
> To: Discussion list for LON-CAPA users
> Cc: Mills, Douglas G
> Subject: [LON-CAPA-users] How Best to Resolve Floating Point Issues?
>
> Hi All,
>
> I've run into a situation where the binary representation of base 10 numbers is catching up to us for a very small percentage of our students. To illustrate this I've created a public problem demonstrating the issue. Unlike the problems the students get this one, for the sake of demonstration, walks you through the calculations but uses the exact same numbers that one of our students had to work with resulting in a case where what should be zero is not equal to zero. What's the best way of preventing this from happening? (The source code is open also in case you want to have a look but it's pretty well spelled out in the problem itself for demonstration purposes).
>
> https://access3.lon-capa.illinois.edu/res/uiuc/dmills/scratchwork/FloatingPointIssueDemo.problem<https://urldefense.com/v3/__https://access3.lon-capa.illinois.edu/res/uiuc/dmills/scratchwork/FloatingPointIssueDemo.problem__;!ioFpBMP7lJU!kYg_ctrV6fkf5-Q0k15HPZw7eDzJHG_oXmJmgLbFYGCVB4UljJ9noqPl_BVJGg\$>
>
> Thanks in advance and, on that topic, Happy Thanksgiving!
>
> Doug
>
> DOUGLAS G MILLS
> Director of Instructional Technology
>
> Department of Chemistry
>
> University of Illinois at Urbana-Champaign
> Chemistry Annex Box A2
> 601 S. Mathews | M/C 712
> Urbana, IL 61801
> 217.244.5739 | fax: 217.244.8029 | dmills at illinois.edu<mailto:dmills at illinois.edu>
> chemistry.illinois.edu<https://urldefense.com/v3/__http://chemistry.illinois.edu__;!ioFpBMP7lJU!kYg_ctrV6fkf5-Q0k15HPZw7eDzJHG_oXmJmgLbFYGCVB4UljJ9noqMWvh09LQ\$>
>
> [/var/folders/kv/f3wpy1cs51521x5grm_qx4tw0000gn/T/com.microsoft.Outlook/WebArchiveCopyPasteTempFiles/P698ojxP4tc7j8gMqgPv+E4KQAAAABJRU5ErkJggg==]<https://urldefense.com/v3/__http://illinois.edu/__;!ioFpBMP7lJU!kYg_ctrV6fkf5-Q0k15HPZw7eDzJHG_oXmJmgLbFYGCVB4UljJ9noqPPexT8GA\$>
>
> Under the Illinois Freedom of Information Act any written communication to or from university employees regarding university business is a public record and may be subject to public disclosure.
>
> _______________________________________________
> LON-CAPA-users mailing list
> LON-CAPA-users at mail.lon-capa.org
> http://mail.lon-capa.org/mailman/listinfo/lon-capa-users
```