[LON-CAPA-cvs] cvs: modules /gerd/correlpaper correlations.tex
www
lon-capa-cvs@mail.lon-capa.org
Wed, 09 Aug 2006 14:26:02 -0000
This is a MIME encoded message
--www1155133562
Content-Type: text/plain
www Wed Aug 9 10:26:02 2006 EDT
Modified files:
/modules/gerd/correlpaper correlations.tex
Log:
Per reviewer comments:
* new title
* better figure caption
* 95% intervals
--www1155133562
Content-Type: text/plain
Content-Disposition: attachment; filename="www-20060809102602.txt"
Index: modules/gerd/correlpaper/correlations.tex
diff -u modules/gerd/correlpaper/correlations.tex:1.5 modules/gerd/correlpaper/correlations.tex:1.6
--- modules/gerd/correlpaper/correlations.tex:1.5 Tue Jul 11 22:10:06 2006
+++ modules/gerd/correlpaper/correlations.tex Wed Aug 9 10:25:59 2006
@@ -36,7 +36,7 @@
%\preprint{APS/123-QED}
-\title{Correlations between Student Discussion Behavior, Student Attitudes, and Student Learning in an Introductory Physics Course}% Force line breaks with \\
+\title{Correlations between Student Discussion Behavior, Attitudes, and Learning}% Force line breaks with \\
\author{Gerd Kortemeyer}
@@ -242,7 +242,7 @@
Correlations with $|R|<0.1$ are indicated by a dash, correlations with $|R|>0.5$ are printed boldface. The values in brackets are the result of calculations limited to students with at least five discussion contributions over the course of the semester.
\begin{table*}
-\caption{\label{fullresults}Complete correlation results ($R$-values). Calculated correlations whose absolute value was lower than 0.1 are indicated by "---." Correlations with an absolute value higher than 0.5 have been printed in boldface. The values given in brackets have been calculated including only students who contributed more than five discussion entries over the course of the semester.}
+\caption{\label{fullresults}Complete correlation results ($R$-values). In the columns we list the grade performance, final exam performance, FCI scores and gains, MPEX scores and gains, as well as the prominences of different online discussion characteristics. In the rows we list the grade performance, final exam performance, FCI scores and gains, MPEX scores and gains, as well as the performance on the different MPEX clusters. Calculated correlations whose absolute value was lower than 0.1 are indicated by "---." Correlations with an absolute value higher than 0.5 have been printed in boldface. The values given in brackets have been calculated including only students who contributed more than five discussion entries over the course of the semester. As an example, the correlation between the prominence of solution-oriented discussion contributions of students who made more than five contributions over the course of the semester and the FCI gain is -0.44, indicating that students who make more solution-oriented discussion contributions have a lower gain on the FCI between the beginning and the end of the semester.}
\begin{ruledtabular}
\begin{tabular}{rllllllllllll}
&Grade&Final&FCI &FCI &MPEX&MPEX&Solution &Math &Physics &Surface &Proce-&Concep-\\
@@ -274,40 +274,40 @@
\end{ruledtabular}
\end{table*}
-Of particular interest is the lower right corner of Table~\ref{fullresults}, as it lists the correlations between student attitudes and expectations (as measured by the MPEX clusters) with the prominence of discussion behavior classes. One would have expected strong correlations between for example the score on the Concepts Cluster and the prominence of conceptual discussion contributions ($R=0.14 (0.15); n=84 (51)$), or the comfort level with the usage of mathematics as a language and the corresponding lack of purely mathematical contributions (non significant, and $R=-0.14$ ($n=51$) when including only students with more than five contributions overall). The Coherence and Effort Clusters are most strongly correlated with discussions, the Math Link Cluster -- surprisingly -- the least.
+Of particular interest is the lower right corner of Table~\ref{fullresults}, as it lists the correlations between student attitudes and expectations (as measured by the MPEX clusters) with the prominence of discussion behavior classes. One would have expected strong correlations between for example the score on the Concepts Cluster and the prominence of conceptual discussion contributions ($R=0.14 [-0.08 - 0.34] (0.15 [-0.13 - 0.41]); n=84 (51)$), or the comfort level with the usage of mathematics as a language and the corresponding lack of purely mathematical contributions ($|R|<0.1$, and $R=-0.14 [-0.4 - 0.14]$ ($n=51$) when including only students with more than five contributions overall). However, in the 95\% confidence intervals (given in square brackets) include zero. The Coherence and Effort Clusters are most strongly correlated with discussions, the Math Link Cluster -- surprisingly -- the least.
The upper right and the lower left corner list the correlations of student discussion behavior and the MPEX, respectively, with measures of student learning. Correlations are again low, but of comparable magnitude, where the MPEX appears to be slightly more correlated with grade and final exam performance, while the discussion is more correlated with the FCI. In fact, some of the strongest correlations in the study occur between the prominence of solution-oriented and physics-related discussions and the FCI. We will analyze correlations with grades in more detail in subsection~\ref{gradecorrel}, and with the FCI in subsection~\ref{fcicorrel}.
-The Coherence Cluster of the MPEX appears to be more strongly correlated to other performance indicators than the other clusters. Out of that cluster, agreement with the statement "In doing a physics problem, if my calculation gives a result that differs significantly from what I expect, I'd have to trust the calculation" (53\% unfavorable responses) has $R=-0.3$ ($n=97$) with the grade in the course, $R=-0.3$ ($n=97$) with the final FCI Score, and $R=0.3$ ($n=84$) with solution-oriented discussion postings. Out of the Concepts Cluster, agreement with the single statement "The most crucial thing in solving a physics problem is finding the right equation to use" (45\% unfavorable responses) correlates with $R=-0.3$ ($n=96$) with the final FCI score and also with $R=-0.3$ ($n=85$) with the FCI Gain, i.e., stronger than the cluster it belongs to.
+The Coherence Cluster of the MPEX appears to be more strongly correlated to other performance indicators than the other clusters. Out of that cluster, agreement with the statement "In doing a physics problem, if my calculation gives a result that differs significantly from what I expect, I'd have to trust the calculation" (53\% unfavorable responses) has $R=-0.3 [-0.47 - -0.11]$ ($n=97$) with the grade in the course, $R=-0.3 [-0.47 - -0.11]$ ($n=97$) with the final FCI Score, and $R=0.3 [-0.48 - -0.09]$ ($n=84$) with solution-oriented discussion postings. Out of the Concepts Cluster, agreement with the single statement "The most crucial thing in solving a physics problem is finding the right equation to use" (45\% unfavorable responses) correlates with $R=-0.3 [-0.47 - -0.11]$ ($n=96$) with the final FCI score and also with $R=-0.3 [-0.48 - -0.09]$ ($n=85$) with the FCI Gain, i.e., stronger than the cluster it belongs to.
-Going beyond the analysis of the large discussion superclasses, when considering the intersection of student discussion characteristics, only a few relatively strong correlations can be found. For example, the prominence of discussion contributions that were both conceptual and physics-related correlates with $R=0.2$ ($n=173$) with the grade in the course, and with $R=0.29$ ($n=95$) and $R=0.3$ ($n=84$) with the final FCI Score and Gain, respectively. The prominence of contributions that are both solution-oriented and surface-level correlates with $R=-0.29$ ($n=95$) and $R=-0.13$ ($n=84$) with the FCI Score and Gain, respectively.
+Going beyond the analysis of the large discussion superclasses, when considering the intersection of student discussion characteristics, only a few relatively strong correlations can be found. For example, the prominence of discussion contributions that were both conceptual and physics-related correlates with $R=0.2$ ($n=173$) with the grade in the course, and with $R=0.29 [-0.464 - -0.09]$ ($n=95$) and $R=0.3 [0.09 - 0.48]$ ($n=84$) with the final FCI Score and Gain, respectively. The prominence of contributions that are both solution-oriented and surface-level correlates with $R=-0.29 [-0.46 - -0.09]$ ($n=95$) and $R=-0.13 [-0.34 - 0.08]$ ($n=84$) with the FCI Score and Gain, respectively.
\subsection{\label{gradecorrel}Correlations with the Overall Course Grade and Final Exam}
-Figure~\ref{fcimpexgrade} shows the correlation between the final FCI and MPEX scores with the final course grade percentage. With an $R$ of 0.56 ($n=110$) and 0.30 ($n=97$), respectively, these -- particularly for the MPEX -- turned out lower than expected. As pointed out in section~\ref{setting}, however, the course grade is based on a number of factors, some of which are simply a matter of diligence or effort.
+Figure~\ref{fcimpexgrade} shows the correlation between the final FCI and MPEX scores with the final course grade percentage. With an $R$ of 0.56 [0.41 - 0.68] ($n=110$) and 0.30 [0.11 - 0.47] ($n=97$), respectively, these -- particularly for the MPEX -- turned out lower than expected. As pointed out in section~\ref{setting}, however, the course grade is based on a number of factors, some of which are simply a matter of diligence or effort.
\begin{figure*}
\includegraphics[width=9cm]{fcipostgrade}\includegraphics[width=9cm]{mpexpostgrade}
-\caption{\label{fcimpexgrade}Correlation between the FCI score (left; $R=0.56$; $n=110$) and the MPEX score (right; $R=0.3$; $n=97$) with the course grade percentage. 58\% was the minimum percentage to pass the course. More students participated in the FCI than in the MPEX.}
+\caption{\label{fcimpexgrade}Correlation between the FCI score (left; $R=0.56 [0.41 - 0.68] $; $n=110$) and the MPEX score (right; $R=0.3 [0.11 - 0.47] $; $n=97$) with the course grade percentage. 58\% was the minimum percentage to pass the course. More students participated in the FCI than in the MPEX.}
\end{figure*}
-Correlations with the MPEX were generally weak, with $R=0.36$ ($n=97$) between the score on the Coherence cluster and the course grade percentage being the highest value. Dancy~\cite{dancy02} found similarly low correlations with the performance on homework, tests, and final exams: direct comparison with the performance on the final exams found $R=0.37$ for the correlation with the total MPEX score ($R=0.27$ here), $R=0.39$ with the Independence Cluster ($R=0.25$ here), $R=0.24$ with the Coherence Cluster ($R=0.36$ here), $R=0.29$ with the Concept Cluster ($R=0.25$ here), $R=-.02$ with the Reality Link cluster ($R=0.1$ here), $R=0.3$ with the Math Link cluster (no significant correlation found here), and no significant correlation with the Effort Cluster ($R=0.1$ here).
+Correlations with the MPEX were generally weak, with $R=0.36 [0.17 - 0.52]$ ($n=97$) between the score on the Coherence cluster and the course grade percentage being the highest value. Dancy~\cite{dancy02} found similarly low correlations with the performance on homework, tests, and final exams: direct comparison with the performance on the final exams found $R=0.37$ for the correlation with the total MPEX score ($R=0.27$ here), $R=0.39$ with the Independence Cluster ($R=0.25$ here), $R=0.24$ with the Coherence Cluster ($R=0.36$ here), $R=0.29$ with the Concept Cluster ($R=0.25$ here), $R=-.02$ with the Reality Link cluster ($R=0.1$ here), $R=0.3$ with the Math Link cluster (no significant correlation found here), and no significant correlation with the Effort Cluster ($R=0.1$ here).
Figure~\ref{physicsgrade} shows the correlation between the prominence of physics-related discussions and the course grade percentage (for better statistics, only students who contributed at least five discussion entries over the course of the semester were considered). The correlation is stronger than with the MPEX Score, yet smaller than with the FCI.
\begin{figure}
\includegraphics[width=9cm]{physicsgrade}
-\caption{\label{physicsgrade}Correlation of percentage physics-related discussions with grade percentage ($R=0.33$; $n=111$).}
+\caption{\label{physicsgrade}Correlation of percentage physics-related discussions with grade percentage ($R=0.33 [0.15 - 0.49]$; $n=111$).}
\end{figure}
\subsection{\label{fcicorrel}Correlations with the FCI}
-Figure~\ref{mpexfci} shows how the final FCI and MPEX scores correlated with each other, i.e, $R=0.24$ ($n=97$).
-Coletta and Philips~\cite{coletta05} found a strong correlation between the FCI Gain and the MPEX Score ($R=0.52; n=37$), while the same correlation turned out much lower in this study ($R=0.17; n=84$ here). The correlations reported here are in the same range that
+Figure~\ref{mpexfci} shows how the final FCI and MPEX scores correlated with each other, i.e, $R=0.24 [0.04 - 0.42]$ ($n=97$).
+Coletta and Philips~\cite{coletta05} found a strong correlation between the FCI Gain and the MPEX Score ($R=0.52 [0.24 - 0.72]; n=37$), while the same correlation turned out much lower in this study ($R=0.17 [-0.05 - 0.37]; n=84$ here). The correlations reported here are in the same range that
Perkins et al.~\cite{perkins04} found when investigating the influence of beliefs on conceptual learning, using the CLASS~\cite{adams04} and the Force and Motion Conceptual Evaluation (FMCE)~\cite{thornton98} instruments.
-Correlations with discussion characteristics turned out somewhat stronger. Figure~\ref{fciphysics} shows how the percentage of a particular student's discussion contribution that was classified as "physics-related" correlates with their final FCI score ($R=0.34$; $n= 95$). As already in Fig.~\ref{physicsgrade}, an additional analysis was carried out that was limited to students for which better statistics were available, which let to a stronger correlation ($R=0.51$; $n=57$).
+Correlations with discussion characteristics turned out somewhat stronger. Figure~\ref{fciphysics} shows how the percentage of a particular student's discussion contribution that was classified as "physics-related" correlates with their final FCI score ($R=0.34 [0.15 - 0.51]$; $n= 95$). As already in Fig.~\ref{physicsgrade}, an additional analysis was carried out that was limited to students for which better statistics were available, which let to a stronger correlation ($R=0.51[0.29 - 0.68]$; $n=57$).
\begin{figure}
\includegraphics[width=9cm]{fcipostmpexpost}
@@ -316,12 +316,12 @@
\begin{figure*}
\includegraphics[width=9cm]{fcipostphysics}\includegraphics[width=9cm]{fcipostphysicsT}
-\caption{\label{fciphysics}Correlation between the FCI score and the percentage of that student's discussion that was classified as "physics" ($R=0.34$; $n= 95$). The figure on the right only includes students who contributed more than five discussion entries over the course of the semester ($R=0.51$; $n=57$).}
+\caption{\label{fciphysics}Correlation between the FCI score and the percentage of that student's discussion that was classified as "physics" ($R=0.34 [0.15 - 0.51]$; $n= 95$). The figure on the right only includes students who contributed more than five discussion entries over the course of the semester ($R=0.51 [0.29 - 0.68]$; $n=57$).}
\end{figure*}
-While physics-related discussions positively correlate with FCI scores and grades (Fig.~\ref{physicsgrade}), solution-oriented discussions negatively correlate (Fig.~\ref{solutionfci}; $R=-0.58$; $n=57$).
+While physics-related discussions positively correlate with FCI scores and grades (Fig.~\ref{physicsgrade}), solution-oriented discussions negatively correlate (Fig.~\ref{solutionfci}; $R=-0.58 [-0.73 - -0.38]$; $n=57$).
\begin{figure}
\includegraphics[width=9cm]{fcipostsolutionT}
-\caption{\label{solutionfci}Correlation of percentage solution-oriented discussions with final FCI score ($R=-0.58$; $n=57$).}
+\caption{\label{solutionfci}Correlation of percentage solution-oriented discussions with final FCI score ($R=-0.58 [-0.73 - -0.38]$; $n=57$).}
\end{figure}
\section{Discussion of the Results}
Correlations between Grade, Final Exam, FCI, MPEX, and student discussion behavior have turned out lower than expected. The strongest correlations exist with the final score on the FCI, namely $R=0.56$ with the grade percentage in the course, $R=0.51$ with the prominence of physics-related discussions, and $R=-0.58$ with the prominence of solution-oriented discussions.
--www1155133562--