[LON-CAPA-cvs] cvs: modules /gerd/correlpaper correlations.bib correlations.tex
www
lon-capa-cvs@mail.lon-capa.org
Tue, 11 Jul 2006 02:33:05 -0000
This is a MIME encoded message
--www1152585185
Content-Type: text/plain
www Mon Jul 10 22:33:05 2006 EDT
Modified files:
/modules/gerd/correlpaper correlations.bib correlations.tex
Log:
Almost there, still needs better examples.
--www1152585185
Content-Type: text/plain
Content-Disposition: attachment; filename="www-20060710223305.txt"
Index: modules/gerd/correlpaper/correlations.bib
diff -u modules/gerd/correlpaper/correlations.bib:1.1 modules/gerd/correlpaper/correlations.bib:1.2
--- modules/gerd/correlpaper/correlations.bib:1.1 Mon Jul 10 12:00:24 2006
+++ modules/gerd/correlpaper/correlations.bib Mon Jul 10 22:33:03 2006
@@ -7,6 +7,15 @@
title = "The problem with problems"
}
+@ARTICLE{thornton98,
+ author = "Ron Thornton and David Sokoloff",
+ year = "2005",
+ journal = "Am. J. Phys.",
+ volume = "66",
+ pages = " 338-352",
+ title = "Assessing student learning of Newton's laws: The force and motiona conceptual evaluation and the evaluation of active learning laboratory and lecture curricula"
+}
+
@ARTICLE{kortemeyer05ana,
author = "Gerd Kortemeyer",
year = "2005",
@@ -136,7 +145,21 @@
year = "2004",
booktitle = "Proc. NARST Annual Meeting",
author = "Andrea Pascarella",
- title = " The Influence of Web-Based Homework on Quantitative Problem-Solving in a University Physics Class"
+ title = "The Influence of Web-Based Homework on Quantitative Problem-Solving in a University Physics Class"
+}
+
+@CONFERENCE{perkins04,
+ year = "2004",
+ booktitle = "Proc. PERC 2004",
+ author = "Katherine Perkins and and Wendy Adams and Noah Finkelstein and Steven Pollock and Carl Wieman",
+ title = "Correlating Student Beliefs With Student Learning Using The Colorado Learning Attitudes about Science Survey,"
+}
+
+@CONFERENCE{adams04,
+ year = "2004",
+ booktitle = "Proc. PERC 2004",
+ author = "Wendy Adams",
+ title = "Design and Validation of the Colorado Learning About Science Survey,"
}
@CONFERENCE{dancy02,
@@ -180,7 +203,16 @@
pages = "212-224",
title = "Student Expectations in Introductory Physics"
}
-
+
+@ARTICLE{ebaps,
+ author="Andrew Elby",
+ year="2001",
+ journal="Am. J. Physics",
+ volume="69",
+ pages="S54-S64",
+ title="Helping physics students learn about learning"
+}
+
@MISC{fci,
author = "Ibrahim Halloun and Rchard R. Hake and E. P. Mosca and David Hestenes",
url= "http://modeling.la.asu.edu/R&E/Research.html",
Index: modules/gerd/correlpaper/correlations.tex
diff -u modules/gerd/correlpaper/correlations.tex:1.2 modules/gerd/correlpaper/correlations.tex:1.3
--- modules/gerd/correlpaper/correlations.tex:1.2 Mon Jul 10 11:59:52 2006
+++ modules/gerd/correlpaper/correlations.tex Mon Jul 10 22:33:03 2006
@@ -36,8 +36,7 @@
%\preprint{APS/123-QED}
-\title{Correlations between FCI, MPEX, Student Discussion Behavior,\\
- and Grade Performance in an Introductory Physics Course}% Force line breaks with \\
+\title{Correlations between Student Discussion Behavior, Student Attitudes, and Student Learning in an Introductory Physics Course}% Force line breaks with \\
\author{Gerd Kortemeyer}
@@ -52,86 +51,211 @@
% but any date may be explicitly specified
\begin{abstract}
-In the Physics Education Research community, the Force Concept Inventory (FCI) and the Maryland Physics Expectations Survey are frequently used instruments to assess students' conceptual understanding of and attitudes and expectations toward physics. In this paper, we investigate how these instruments correlate with each other, and how they correlate with the characteristics of students' contributions to online discussions and their overall performance in the course.\end{abstract}
+An important result of Physics Education Research is that students' learning and success in a course is correlated with their beliefs, attitudes, and expectations. In this study, we investigate the possibility of using the evaluation of online student discussion behavior as an indicator of a student's approach to physics. We investigate correlations with a traditional instrument, namely the Maryland Physics Expectations Survey (MPEX), as well as correlations with the Force Concept Inventory (FCI), the final exam grade, and the overall course performance as a measure of the student's learning. To gauge the outcomes, we also investigate correlations between these measures.\end{abstract}
\pacs{01.40.Fk}% PACS, the Physics and Astronomy
% Classification Scheme.
%\keywords{Suggested keywords}%Use showkeys class option if keyword
%display desired
\maketitle
+\section{\label{intro}Introduction}
+A traditional way of assessing student beliefs, attitudes, and expectations about physics is the deployment of surveys, for example the Maryland Physics Expectations Survey (MPEX)~\cite{mpex}, the Epistemological Beliefs Assessment for Physical Science (EBAPS)~\cite{ebaps}, or the Colorado Learning Attitudes about Science Survey (CLASS)~\cite{adams04}. While these instruments take different approaches and have different philosophies behind their designs, they do have in common that the students need to react to artificial statements outside of the normal course activity, and that they need to self-report their responses.
+
+Online student discussions associated with online physics problems are different in that they are generated within the real context of the course, and students have a vested interest in making these discussions as productive as possible, given their understanding of how physics is done and their approach to it. They could thus be a ``reality check'' of students' beliefs, attitudes, and expectations.
+
+Online discussions are a rich source of feedback to the instructor~\cite{kortemeyer05feedback}, and their quality and character was found to be correlated with the type and difficulty of the associated problems~\cite{kortemeyer05ana}, i.e., data exists regarding the influence of {\it problem} characteristics on associated discussions. Unfortunately, less data exists on the correlation between {\it student} characteristics and discussion behavior, because only very few student characteristics were known, such as the students' overall performance in the course. One of the findings was the fact that certain discussion behavior, most prominently exhibited on ``non-sanctioned'' discussion sites external to the course, is negatively correlated with performance in the course~\cite{kashy03,kortemeyer05ana}.
+
+In this study, we first aim to answer the question if and how student discussion characteristics are related to their beliefs and attitudes as measured by a traditional instrument. In a second step, we try to test the hypothesis that discussion characteristics can be used to assess students' attitudes and expectations regarding physics in place of or in conjunction with traditional instruments.
\section{\label{setting}Setting}
The project was carried out in an introductory calculus-based physics course with initially 214 students. Most of the students in this course plan on pursuing a career in a medical field. The course had three traditional lectures per week. It did not use a textbook, instead, all course materials were available online. Topics were introductory mechanics, as well as sound and thermodynamics. There was twice-weekly online homework: one small set as reading problems due for the topic was dealt with in class, and a larger set of traditional end-of-the-chapter style homework at the end of each topic. The students had weekly recitation sessions, and a traditional lab was offered in parallel. The course grade was determined from the students' performance on the biweekly quizzes, the final exam, the recitation grades, and the homework performance.
\section{\label{measures}Measures and Instruments}
+\subsection{\label{discussion}Discussion Analysis}
+We also analyzed the online student discussions that were associated with the online homework given in the course, using the scheme first suggested in Ref.~\cite{kortemeyer05ana}. There were a total of 2405 such online discussion contributions over the course of the semester.
-We deployed the revised Force Concept Inventory (FCI)\cite{fci} and the he Maryland Physics Expectations Survey (MPEX)\cite{mpex} both at the beginning and the end of the mechanics semester. Participation was voluntary. We calculated the score (for the MPEX in comparison to the expert responses given in Ref.~\cite{mpex}) on the final deployment, as well as, for students who participated in both of each instrument, the gain. The same analysis was done for the clusters of the MPEX:
-\begin{itemize}
-\item{\it Independence:} student takes responsibility for contructing their own understanding rather than takes what is given by authorities (teacher, materials) without evaluation
-\item{\it Coherence:} student believes that physics needs to be considered as a connected, consistent framework rather than a set of unrelated facts or ``pieces''
-\item{\it Concepts:} student stresses the understanding of the underlying ideas and concepts rather than the memorization and usage of formulas
-\item{\it Reality Link:} student believes that the ideas learned in physics are relevant and useful in a wide variety of real contexts rather than having little to do with experiences outside the classroom
-\item{\it Math Link:} student considers mathematics as a convenient way of representing physical phenomena rather than viewing physics and mathematics as having little or no relationship
-\item{\it Effort:} student makes the effort to use the available information and tries to make sense of it
-\end{itemize}
+Each contribution was classified according to the classification scheme of Ref.~\cite{kortemeyer05ana}, however, with the additional refinement that each contribution could be member of more than one class, and that the contributions were weighted by their length. For example, a certain contribution might include both a procedural solution-oriented question and a surface-level mathematical answer, and would thus receive 50\% membership in both classes, weighted by its total length.
-We also analyzed the online student discussions that were associated with the online homework given in the course, using the scheme first suggested in Ref.~\cite{kortemeyer05ana}. There were a total of 2405 such online discussion contributions over the course of the semester. As a refinement, the contributions were weighted by their length, and one particular contribution could belong to more than one class. For each student, the percentage of their contributions falling into the following classes~\cite{kortemeyer05ana} was calculated:
+The analysis was carried out based on discussion superclasses~\cite{kortemeyer05ana}, for example, all conceptual classes were combined, independent of their features. A given contribution can thus belong to more than one superclass.
+The following list shows the superclasses taken into consideration, as well as illustrative examples that have partial membership in each superclass, taken from this course:
\begin{itemize}
\item Discussion contributions were classified as {\it surface} if they
dealt with surface features of the problem or were surface level requests
for help.
-
+\begin{quote}
+- I know this sounds stupid, but I can't figure out how to find angular speed. I
+know that v=r*w, so I solved for w to get.... w=v/r. is r the circumference?
+But what is v? which distance do you use to find this?
+\end{quote}
\item {\it Procedural} contributions describe or inquire about mechanisms
of solving a problem without mentioning the underlying concepts or
reasoning.
+\begin{quote}
+- ok in this problem make sure your temp. is in K and your c
+is in J. So for water use 4186 and for Alcohol use 2430.
+The problem is set up the same way that Mr. Anonymous has
+posted above. Hope that helps
+\end{quote}
\item {\it Conceptual} contributions deal with the underlying concepts of
the problem.
+\begin{quote}
+- Just common sense.
+I didnt use math, I just sorta looked at how loud the
+intensities they gave were and compared it to other noises
+that were at the same intensity. For instance:
+
+a nuclear explosion is around 220 dB from 500 m away.
+
+a rock concert is around 120 dB
+
+a person shouting is 80 dB.
+
+So look at the answers and think about the question logically.
+\end{quote}
+\begin{quote}
+- I am totally lost on this problem. Could someone try to
+explain how you would approach this problem. I just don't
+understand how Newton's third law applies and why we would
+completely ignore the mass of the truck and just use the
+mass of the car.
+\end{quote}
\item {\it Solution-oriented} -- the goal of the contribution is to arrive at
the correct answer without mentioning or dealing with the mathematics or
physics of the problem.
+\begin{quote}
+- The formula in the first post is correct. Here's how i did
+it...
+
+P1/T1=P2/T2
+
+so..
+
+P2= (P1/T1) * T2
+
+remember to convert your degrees C to K by adding 273.15.
+so my T1 was 47 C which equals 320.15 K
+my T2 was -90 C which equals 183.15 K
+
+enter these values into the derived equation and you get..
+
+P2= (2260 Pa/ 320.15) * 183.15 = 1292.89
+those were my numbers, hopefully that helped clarify the
+situation, k peace!
+\end{quote}
\item {\it Mathematical} -- the contribution deals mostly with the
mathematical aspects of the problem.
+\begin{quote}
+- Change your commas to plus signs. Its asking you for the
+scalar product which means you must add the product of
+each component.
+\end{quote}
-\item {\it Physics} -- the contribution deals mostly with the physics-related aspects
+\item {\it Physics} -- the contribution deals with the physics-related aspects
of the problem.
+\begin{quote}
+- Here is how it goes:
+First, for clarity, set mass of cup as (m1), temperature
+of cup as (T1), specific heat of cup as (c1), mass of
+coffee as (m2), temperature of coffee as (T2), specific
+heat of coffee as (c2), and final temperature as (Tf).
+
+From lecture, it was stated that heat lost by the hotter
+object is equal to the opposite of heat gained by the
+colder object. So, in this case, since the cup is gaining
+heat in this case (since it has the smaller temperature),
+we can say:
+(Q1) = -(Q2)
+
+And we know that Q = c*m*$\Delta$T, so we can say:
+(c1)(m1)((Tf)-(T1)) = -(c2)(m2)((Tf)-(T2))
+
+With a lot of mathematical hoo-hah, you get:
+(Tf)*((c1)*(m1)+(c2)*(m2)) = (c1)*(m1)*(T1) + (c2)*(m2)*
+(T2)
+
+Solve for (Tf), and you have your final temperature.
+\end{quote}
+\begin{quote}
+- While the disk is spun up, what is the direction of the
+acceleration of the sphere?
+
+why isn't it the vector pointing towards the center?
+shouldn't Centripetal acceleration point towards the
+center?
+\end{quote}
+\end{itemize}
+\subsection{\label{mpex}The MPEX}
+We deployed the Maryland Physics Expectations Survey (MPEX)\cite{mpex} both at the beginning and the end of the mechanics semester. Participation was voluntary. We calculated the score in comparison to the ``favorable" expert responses given in Ref.~\cite{mpex} on the final deployment, as well as, for students who participated both times, the gain. The same analysis was done for the clusters of the MPEX (example statements are given, including the expert answer):
+\begin{itemize}
+\item{\it Independence:} student takes responsibility for constructing their own understanding rather than takes what is given by authorities (teacher, materials) without evaluation
+\begin{quote}
+Unfavorable: In this course, I do not expect to understand equations in an intuitive sense; they must just be taken as givens.
+\end{quote}
+\item{\it Coherence:} student believes that physics needs to be considered as a connected, consistent framework rather than a set of unrelated facts or ``pieces''
+\begin{quote}
+Unfavorable: Knowledge in physics consists of many pieces of information each of which applies primarily to a specific situation.
+\end{quote}
+\item{\it Concepts:} student stresses the understanding of the underlying ideas and concepts rather than the memorization and usage of formulas
+\begin{quote}
+Favorable: When I solve most exam or homework problems, I explicitly think about the concepts that underlie the problem.
+\end{quote}
+\item{\it Reality Link:} student believes that the ideas learned in physics are relevant and useful in a wide variety of real contexts rather than having little to do with experiences outside the classroom
+\begin{quote}
+Unfavorable: Physical laws have little relation to what I experience in the real world.
+\end{quote}
+\item{\it Math Link:} student considers mathematics as a convenient way of representing physical phenomena rather than viewing physics and mathematics as having little or no relationship
+\begin{quote}
+Unfavorable: All I learn from a derivation or proof of a formula is that the formula obtained is valid and that it is OK to use it in problems.
+\end{quote}
+\item{\it Effort:} student makes the effort to use the available information and tries to make sense of it
+\begin{quote}
+Favorable: I go over my class notes carefully to prepare for tests in this course.
+\end{quote}
\end{itemize}
-Online student discussions were earlier found to be a rich source of feedback to the instructor~\cite{kortemeyer05feedback}, and their quality and character was found to be correlated with the type and difficulty of the problems given~\cite{kortemeyer05ana}. This followup study was motivated in part by the hope and expectation to find strong correlations with student attitudes and expectations.
-As an additional measure of student performance, the final grade percentage for each student was taken into consideration.
+
+\subsection{\label{performance}Measures of Student Learning}
+As a measure of student conceptual understanding and learning, we deployed the revised Force Concept Inventory (FCI)\cite{fci} at the beginning and the end of the course, again with voluntary participation. As an additional measure of student performance, the performance on the final exam and the course grade for each student were taken into consideration. For the grade we used the raw percentage score, not the number grades, since it provides finer grained information about the overall student performance in the course.
\section{\label{results}Results}
+When considering the evaluation of student discussion characteristics as replacement of or complementary to surveys such as the MPEX, it is important to understand the correlations among all of the instruments.
+\subsection{Correlations with the Overall Course Grade}
Figure~\ref{fcimpexgrade} shows the correlation between the final FCI and MPEX scores with the final course grade percentage. With an $r$ of 0.56 and 0.30, respectively, these -- particularly for the MPEX -- turned out lower than expected. As pointed out in section~\ref{setting}, however, the course grade is based on a number of factors, some of which are simply a matter of diligence or effort.
-Figure~\ref{mpexfci} shows how the final FCI and MPEX scores correlated with each other.
+
\begin{figure*}
\includegraphics[width=9cm]{fcipostgrade}\includegraphics[width=9cm]{mpexpostgrade}
-\caption{\label{fcimpexgrade}Correlation between the FCI score (left) and the MPEX score (right) with the course grade percentage. 58\% was the minimum percentage to pass the course. More students participated in the FCI than in the MPEX.}
+\caption{\label{fcimpexgrade}Correlation between the FCI score (left; $R=0.56$; $n=110$) and the MPEX score (right; $R=0.3$; $n=97$) with the course grade percentage. 58\% was the minimum percentage to pass the course. More students participated in the FCI than in the MPEX.}
\end{figure*}
+
+Figure~\ref{physicsgrade} shows the correlation between the prominence of physics-related discussions and the course grade percentage (for better statistics, only students who contributed at least five discussion entries over the course of the semester were considered). The correlation is stronger than with the MPEX Score, yet smaller than with the FCI.
+\begin{figure}
+\includegraphics[width=9cm]{physicsgrade}
+\caption{\label{physicsgrade}Correlation of percentage physics-related discussions with grade percentage ($R=0.33$; $n=111$).}
+\end{figure}
+\subsection{Correlations with the FCI}
+Figure~\ref{mpexfci} shows how the final FCI and MPEX scores correlated with each other, i.e, $R=0.24$ ($n=97$). Correlations with discussion characteristics turned out somewhat stronger. Figure~\ref{fciphysics} shows how the percentage of a particular student's discussion contribution that was classified as "physics-related" correlates with their final FCI score ($R=0.34$; $n= 95$). As already in Fig.~\ref{physicsgrade}, an additional analysis was carried out that was limited to students for which better statistics were available, which let to a stronger correlation ($R=0.51$; $n=57$).
+
\begin{figure}
\includegraphics[width=9cm]{fcipostmpexpost}
-\caption{\label{mpexfci}Correlation of the final FCI score with the MPEX score.}
+\caption{\label{mpexfci}Correlation of the final FCI score with the MPEX score ($R=0.24$; $n=97$).}
\end{figure}
-Correlations with discussion characteristics turned out somewhat stronger, particularly with the FCI. Figure~\ref{fciphysics} shows how the percentage of a particular student's discussion contribution that was classified as "physics-related" correlates with their final FCI score. An additional analysis was carried out including only students who contributed more than five discussion entries over the course of the semester, and for whom thus their discussion behavior can be determined with better statistics.
+
\begin{figure*}
\includegraphics[width=9cm]{fcipostphysics}\includegraphics[width=9cm]{fcipostphysicsT}
-\caption{\label{fciphysics}Correlation between the FCI score (left) and the percentage of that student's discussion that was classified as "physics." The figure on the right only includes students who contributed more than five discussion entries over the course of the semester.}
+\caption{\label{fciphysics}Correlation between the FCI score and the percentage of that student's discussion that was classified as "physics" ($R=0.34$; $n= 95$). The figure on the right only includes students who contributed more than five discussion entries over the course of the semester ($R=0.51$; $n=57$).}
\end{figure*}
-\begin{figure}
-\includegraphics[width=9cm]{physicsgrade}
-\caption{\label{physicsgrade}Correlation of percentage physics-related discussions with grade percentage.}
-\end{figure}
-
-While physics-related discussions positively correlate with FCI scores and grades (Fig.~\ref{physicsgrade}), solution-oriented discussions negatively correlated (Fig.~\ref{solutionfci}).
+While physics-related discussions positively correlate with FCI scores and grades (Fig.~\ref{physicsgrade}), solution-oriented discussions negatively correlated (Fig.~\ref{solutionfci}; $R=-0.58$; $n=57$).
\begin{figure}
\includegraphics[width=9cm]{fcipostsolutionT}
-\caption{\label{solutionfci}Correlation of percentage solution-oriented discussions with final FCI score.}
+\caption{\label{solutionfci}Correlation of percentage solution-oriented discussions with final FCI score ($R=-0.58$; $n=57$).}
\end{figure}
-
+\subsection{Full Results}
Table~\ref{fullresults} shows the complete results of the study, including correlations with gains on FCI and MPEX. In the columns, the table shows Grade percentage, Final Exam percentage, final FCI Score, FCI Gain, final MPEX Score, MPEX Gain, and discussion characteristics, in the rows, again Grade, Final Exam, FCI, MPEX, as well as the individual scores on the final MPEX clusters.
\begin{table*}
\caption{\label{fullresults}Complete correlation results ($r$-values). Calculated correlations whose absolute value was lower than 0.1 are indicated by "---." Correlations with an absolute value higher than 0.5 have been printed in boldface. The values given in brackets have been calculated including only students who contributed more than five discussion entries over the course of the semester.}
@@ -166,24 +290,25 @@
\end{ruledtabular}
\end{table*}
-The Coherence Cluster of the MPEX appears to be more strongly correlated to other performance indicators than the other clusters. Out of that cluster, agreement with the statement "In doing a physics problem, if my calculation gives a result that differs significantly from what I expect, I'd have to trust the calculation" (53\% unfavorable responses) has $r=-0.3$ ($N=97$) with the grade in the course, $r=-0.3$ ($N=97$) with the final FCI Score, and $r=0.3$ ($N=84$) with solution-oriented discussion postings. Out of the Concepts Cluster, agreement with the single statement "The most crucial thing in solving a physics problem is finding the right equation to use" (45\% unfavorable responses) correlates with $r=-0.3$ ($N=96$) with the final FCI score and also with $r=-0.3$ ($N=85$) with the FCI Gain, i.e., stronger than the cluster it belongs to.
+The Coherence Cluster of the MPEX appears to be more strongly correlated to other performance indicators than the other clusters. Out of that cluster, agreement with the statement "In doing a physics problem, if my calculation gives a result that differs significantly from what I expect, I'd have to trust the calculation" (53\% unfavorable responses) has $R=-0.3$ ($n=97$) with the grade in the course, $R=-0.3$ ($n=97$) with the final FCI Score, and $R=0.3$ ($n=84$) with solution-oriented discussion postings. Out of the Concepts Cluster, agreement with the single statement "The most crucial thing in solving a physics problem is finding the right equation to use" (45\% unfavorable responses) correlates with $R=-0.3$ ($n=96$) with the final FCI score and also with $R=-0.3$ ($n=85$) with the FCI Gain, i.e., stronger than the cluster it belongs to.
-When considering the intersection of student discussion characteristics, only a few relatively strong correlations can be found. For example, the prominence of discussion contributions that were both conceptual and physics-related correlates with $r=0.2$ ($N=173$) with the grade in the course, and with $r=0.29$ ($N=95$) and $r=0.3$ ($N=84$) with the final FCI Score and Gain, respectively. The prominence of contributions that are both solution-oriented and surface-level correlates with $r=-0.29$ ($N=95$) and $r=-0.13$ ($N=84$) with the FCI Score and Gain, respectively.
+When considering the intersection of student discussion characteristics, only a few relatively strong correlations can be found. For example, the prominence of discussion contributions that were both conceptual and physics-related correlates with $R=0.2$ ($n=173$) with the grade in the course, and with $R=0.29$ ($n=95$) and $R=0.3$ ($n=84$) with the final FCI Score and Gain, respectively. The prominence of contributions that are both solution-oriented and surface-level correlates with $R=-0.29$ ($n=95$) and $R=-0.13$ ($n=84$) with the FCI Score and Gain, respectively.
\section{Discussion of the Results}
-Correlations between Grade, Final Exam, FCI, MPEX, and student discussion behavior have turned out lower than expected. The strongest correlations exist with the final score on the FCI, namely $r=0.56$ with the grade percentage in the course, $r=0.51$ with the prominence of physics-related discussions, and $r=-0.58$ with the prominence of solution-oriented discussions.
+Correlations between Grade, Final Exam, FCI, MPEX, and student discussion behavior have turned out lower than expected. The strongest correlations exist with the final score on the FCI, namely $R=0.56$ with the grade percentage in the course, $R=0.51$ with the prominence of physics-related discussions, and $R=-0.58$ with the prominence of solution-oriented discussions.
-Correlations with the MPEX were generally weak, with $r=0.36$ between the score on the Coherence cluster and the course grade percentage being the highest value. Dancy~\cite{dancy02} found similarly low correlations with the performance on homework, tests, and final exams: direct comparison with the performance on the final exams found $r=0.37$ for the correlation with the total MPEX score ($r=0.27$ here), $r=0.39$ with the independence cluster ($r=0.25$ here), $r=0.24$ with the coherence cluster ($r=0.36$ here), $r=0.29$ with the concepts cluster ($r=0.25$ here), $r=-.02$ with the reality link cluster ($r=0.1$ here), $r=0.3$ with the math link cluster (no significant correlation found here), and no significant correlation with the effort cluster ($r=0.1$ here).
+Correlations with the MPEX were generally weak, with $R=0.36$ between the score on the Coherence cluster and the course grade percentage being the highest value. Dancy~\cite{dancy02} found similarly low correlations with the performance on homework, tests, and final exams: direct comparison with the performance on the final exams found $R=0.37$ for the correlation with the total MPEX score ($R=0.27$ here), $R=0.39$ with the independence cluster ($R=0.25$ here), $R=0.24$ with the coherence cluster ($R=0.36$ here), $R=0.29$ with the concepts cluster ($R=0.25$ here), $R=-.02$ with the reality link cluster ($R=0.1$ here), $R=0.3$ with the math link cluster (no significant correlation found here), and no significant correlation with the effort cluster ($R=0.1$ here).
-Coletta and Philips~\cite{coletta05} found a strong correlation between the FCI Gain and the MPEX Score ($r=0.52; N=37$), while the same correlation turned out much lower in this study ($r=0.17; N=84$ here).
+Coletta and Philips~\cite{coletta05} found a strong correlation between the FCI Gain and the MPEX Score ($R=0.52; n=37$), while the same correlation turned out much lower in this study ($R=0.17; n=84$ here). The correlations reported here are in the same range that
+Perkins et al.~\cite{perkins04} found when investigating the influence of beliefs on conceptual learning, using the CLASS~\cite{adams04} and the Force and Motion Conceptual Evaluation (FMCE) instruments~\cite{thornton98}.
-An unexpected result were the low correlations between the MPEX cluster scores and the student discussion behavior. One would have expected strong correlations between for example the score on the Concepts Cluster and the prominence of conceptual discussion contributions ($r=0.14 (0.18)$), or the comfort level with the usage of mathematics as a language and the corresponding lack of purely mathematical contributions (non significant or $r=-0.14$ when including only students with more than five contributions overall).
+An unexpected result were the low correlations between the MPEX cluster scores and the student discussion behavior. One would have expected strong correlations between for example the score on the Concepts Cluster and the prominence of conceptual discussion contributions ($R=0.14 (0.18)$), or the comfort level with the usage of mathematics as a language and the corresponding lack of purely mathematical contributions (non significant or $R=-0.14$ when including only students with more than five contributions overall).
The relative weakness of many of the expected correlations with the MPEX might indicate that maybe -- in spite of the efforts of the author -- the students did not take the MPEX very seriously or did not carefully read the statements. An argument for this possible explanation is that the overall scores of the students on the MPEX were low (Independence 42\%; Coherence 46\%; Concepts 48\%; Reality Link 55\%; Math Link 40\%; Effort 47\%). Also, students relatively frequently chose the answer "3" ("Neutral") on the MPEX Likert scale, which is by definition never correct --- answering that way could indicate true indifference, confusion regarding the statement, or simply "don't care." By the same token, students appear to be taking the FCI more seriously, probably because it more closely relates to the other (grade-relevant) assessments they encounter in the course.
\section{Conclusions}
-In this introductory calculus-based course, correlations between different performance and attitude indicators were found to be lower than expected. The FCI is most strongly correlated with grade performance and the final exam grade, while in turn the prominence of solution-oriented and physics-related discussion contributions are most strongly correlated with the FCI.
+In this introductory calculus-based course, correlations between different performance and attitude indicators were found to be lower than expected. Student discussion behavior generally correlates more strongly with student performance (FCI, final exam, grade) than MPEX results. Particularly the prominence of solution-oriented and physics-related discussions correlate relatively strongly with the FCI.
-Correlations with the MPEX were generally low, where the Coherence Cluster appears to be the strongest single indicator of success in the course. The individual statements "In doing a physics problem, if my calculation gives a result that differs significantly from what I expect, I'd have to trust the calculation" (Coherence Cluster) and "The most crucial thing in solving a physics problem is finding the right equation to use" (Concept Cluster) stand out as the most indicative ones.
+Regarding the hypothesis that the analysis of student discussion characteristics might be used as a means of determining student attitudes and expectations, the results are inconclusive, since the expected correlation between MPEX clusters and the prominence of different classes of student discussion behavior is largely missing. The reason for this might be that the mechanisms -- even in related areas -- measure different things, that at least one of them in fact measures very little, or that the students did not bother responding to the MPEX with sufficient diligence.
\begin{acknowledgments}
Supported in part by the National Science Foundation under NSF-ITR 0085921 and NSF-CCLI-ASA 0243126. Any opinions, findings, and conclusions or recommendations expressed in this
publication are those of the author and do not necessarily reflect the views of the National Science Foundation. The author would like to thank the students in his course for their participation in this study.
--www1152585185--