[LON-CAPA-cvs] cvs: modules /gerd/correlpaper correlations.bib correlations.tex
www
lon-capa-cvs@mail.lon-capa.org
Tue, 11 Jul 2006 18:18:26 -0000
This is a MIME encoded message
--www1152641906
Content-Type: text/plain
www Tue Jul 11 14:18:26 2006 EDT
Modified files:
/modules/gerd/correlpaper correlations.bib correlations.tex
Log:
Re-organize subsections
--www1152641906
Content-Type: text/plain
Content-Disposition: attachment; filename="www-20060711141826.txt"
Index: modules/gerd/correlpaper/correlations.bib
diff -u modules/gerd/correlpaper/correlations.bib:1.2 modules/gerd/correlpaper/correlations.bib:1.3
--- modules/gerd/correlpaper/correlations.bib:1.2 Mon Jul 10 22:33:03 2006
+++ modules/gerd/correlpaper/correlations.bib Tue Jul 11 14:18:24 2006
@@ -113,6 +113,10 @@
title = "Performance on multiple-choice diagnostics and complementary exam problems"
}
+@MISC{loncapa,
+ title="The LearningOnline Network with CAPA",
+ url="http://www.lon-capa.org/"
+}
@ARTICLE{lin,
author = "Herbert Lin",
@@ -123,6 +127,15 @@
title = "Learning physics vs. passing courses"
}
+@ARTICLE{henderson,
+ author="Charles Henderson",
+ title="Common concerns about the Force Concept Inventory",
+ journal="Phys. Teach.",
+ volume="40",
+ year="2002",
+ pages="542-547"
+}
+
@ARTICLE{lawson01,
author = "Anton Lawson",
year = "2001",
Index: modules/gerd/correlpaper/correlations.tex
diff -u modules/gerd/correlpaper/correlations.tex:1.3 modules/gerd/correlpaper/correlations.tex:1.4
--- modules/gerd/correlpaper/correlations.tex:1.3 Mon Jul 10 22:33:03 2006
+++ modules/gerd/correlpaper/correlations.tex Tue Jul 11 14:18:24 2006
@@ -19,7 +19,7 @@
% 4) latex apssamp.tex
%
\documentclass[twocolumn,showpacs,preprintnumbers,amsmath,amssymb]{revtex4}
-%\documentclass[preprint,showpacs,preprintnumbers,amsmath,amssymb]{revtex4}
+%\documentclass[preprint,showpacs,preprintnumbers,amsmath,amssymb,floatfix]{revtex4}
% Some other (several out of many) possibilities
%\documentclass[preprint,aps]{revtex4}
@@ -65,10 +65,10 @@
Online discussions are a rich source of feedback to the instructor~\cite{kortemeyer05feedback}, and their quality and character was found to be correlated with the type and difficulty of the associated problems~\cite{kortemeyer05ana}, i.e., data exists regarding the influence of {\it problem} characteristics on associated discussions. Unfortunately, less data exists on the correlation between {\it student} characteristics and discussion behavior, because only very few student characteristics were known, such as the students' overall performance in the course. One of the findings was the fact that certain discussion behavior, most prominently exhibited on ``non-sanctioned'' discussion sites external to the course, is negatively correlated with performance in the course~\cite{kashy03,kortemeyer05ana}.
-In this study, we first aim to answer the question if and how student discussion characteristics are related to their beliefs and attitudes as measured by a traditional instrument. In a second step, we try to test the hypothesis that discussion characteristics can be used to assess students' attitudes and expectations regarding physics in place of or in conjunction with traditional instruments.
+In this study, we aim to answer the question if and how student discussion characteristics are related to their beliefs and attitudes as measured by the MPEX. We investigate correlations with the MPEX, and compare correlations with measures of student learning.
\section{\label{setting}Setting}
-The project was carried out in an introductory calculus-based physics course with initially 214 students. Most of the students in this course plan on pursuing a career in a medical field. The course had three traditional lectures per week. It did not use a textbook, instead, all course materials were available online. Topics were introductory mechanics, as well as sound and thermodynamics. There was twice-weekly online homework: one small set as reading problems due for the topic was dealt with in class, and a larger set of traditional end-of-the-chapter style homework at the end of each topic. The students had weekly recitation sessions, and a traditional lab was offered in parallel. The course grade was determined from the students' performance on the biweekly quizzes, the final exam, the recitation grades, and the homework performance.
+The project was carried out in an introductory calculus-based physics course with initially 214 students. Most of the students in this course plan on pursuing a career in a medical field. The course had three traditional lectures per week. It did not use a textbook, instead, all course materials were available online. Topics were introductory mechanics, as well as sound and thermodynamics. There was twice-weekly online homework: one small set as reading problems due before the topic was dealt with in class (implementing JiTT~\cite{jitt}), and a larger set of traditional end-of-the-chapter style homework at the end of each topic. The problems in the course were randomized using the LON-CAPA system, i.e., different students would receive different versions of the same problem (different graphs, numbers, images, options, formulas, etc)~\cite{loncapa,kashyd01}. The students had weekly recitation sessions, and a traditional lab was offered in parallel. The course grade was determined from the students' performance on the biweekly quizzes, the final exam, the recitation grades, and the homework performance.
\section{\label{measures}Measures and Instruments}
\subsection{\label{discussion}Discussion Analysis}
@@ -85,9 +85,11 @@
dealt with surface features of the problem or were surface level requests
for help.
\begin{quote}
-- I know this sounds stupid, but I can't figure out how to find angular speed. I
-know that v=r*w, so I solved for w to get.... w=v/r. is r the circumference?
-But what is v? which distance do you use to find this?
+- You use for the first part, c(ice)=.5, c(water)=1, Lfluid=80
+ right?
+\end{quote}
+\begin{quote}
+- what do d and g stand for?
\end{quote}
\item {\it Procedural} contributions describe or inquire about mechanisms
of solving a problem without mentioning the underlying concepts or
@@ -98,7 +100,14 @@
The problem is set up the same way that Mr. Anonymous has
posted above. Hope that helps
\end{quote}
-
+\begin{quote}
+- ok, my original angle is 67 degrees. When i seperate the strings into their two
+right-angle triangles, the angle becomes 33.5 with the adjacent side being
+43.7N (my force of gravity). to solve for the string, or my hypotenuse, i used
+h=43.7N/cos(33.5)= 52.41... then i tried to plug all of this into T= 52.41/2cos
+(67)=10.23 and that was wrong so then i did T=52.41/2cos(33.5) = 31.4 which
+was also wrong... why am i so wrong?! please help!!!
+\end{quote}
\item {\it Conceptual} contributions deal with the underlying concepts of
the problem.
\begin{quote}
@@ -127,24 +136,16 @@
the correct answer without mentioning or dealing with the mathematics or
physics of the problem.
\begin{quote}
-- The formula in the first post is correct. Here's how i did
-it...
-
-P1/T1=P2/T2
+- First you want to take your total mass of the car and the
+two people (In my case 800kg of car + 100kg of people) and
+divide it by 4.
-so..
+Then plug that mass into the equation: w=(k/m)\^{}.5 K is
+given to you in the problem.
-P2= (P1/T1) * T2
+Then plug w into the equation Period=2pi/w
-remember to convert your degrees C to K by adding 273.15.
-so my T1 was 47 C which equals 320.15 K
-my T2 was -90 C which equals 183.15 K
-
-enter these values into the derived equation and you get..
-
-P2= (2260 Pa/ 320.15) * 183.15 = 1292.89
-those were my numbers, hopefully that helped clarify the
-situation, k peace!
+And theres your answer. I hope that helps!
\end{quote}
\item {\it Mathematical} -- the contribution deals mostly with the
@@ -158,38 +159,22 @@
\item {\it Physics} -- the contribution deals with the physics-related aspects
of the problem.
\begin{quote}
-- Here is how it goes:
-
-First, for clarity, set mass of cup as (m1), temperature
-of cup as (T1), specific heat of cup as (c1), mass of
-coffee as (m2), temperature of coffee as (T2), specific
-heat of coffee as (c2), and final temperature as (Tf).
-
-From lecture, it was stated that heat lost by the hotter
-object is equal to the opposite of heat gained by the
-colder object. So, in this case, since the cup is gaining
-heat in this case (since it has the smaller temperature),
-we can say:
-(Q1) = -(Q2)
-
-And we know that Q = c*m*$\Delta$T, so we can say:
-(c1)(m1)((Tf)-(T1)) = -(c2)(m2)((Tf)-(T2))
-
-With a lot of mathematical hoo-hah, you get:
-(Tf)*((c1)*(m1)+(c2)*(m2)) = (c1)*(m1)*(T1) + (c2)*(m2)*
-(T2)
-
-Solve for (Tf), and you have your final temperature.
+- the SI unit of Power is a Watt (W)
\end{quote}
\begin{quote}
-- While the disk is spun up, what is the direction of the
-acceleration of the sphere?
-
-why isn't it the vector pointing towards the center?
-shouldn't Centripetal acceleration point towards the
-center?
+- velocity=flow/area
+\end{quote}
+\begin{quote}
+- Why does the value of G not depend on which planetary system
+you study? That doesn't make any sense to me. I thought
+the gravitational forces of different planets are different.
+ That's why we float on the moon...
\end{quote}
\end{itemize}
+Note that correctness of the contribution was not considered. For example, in the last physics-related example, the fact that the student confused the gravitational constant and the gravitational acceleration was not taken into account.
+
+As already found in Ref.~\cite{kortemeyer05ana}, most students are quite prolific in their online discussions, but a few students only made a small number of contributions, leading to small statistics on their actual discussion behavior. For each of the discussion correlations, we thus carried out a second calculation limited to students who contributed at least five entries over the course of the semester.
+
\subsection{\label{mpex}The MPEX}
We deployed the Maryland Physics Expectations Survey (MPEX)\cite{mpex} both at the beginning and the end of the mechanics semester. Participation was voluntary. We calculated the score in comparison to the ``favorable" expert responses given in Ref.~\cite{mpex} on the final deployment, as well as, for students who participated both times, the gain. The same analysis was done for the clusters of the MPEX (example statements are given, including the expert answer):
\begin{itemize}
@@ -224,41 +209,40 @@
\subsection{\label{performance}Measures of Student Learning}
As a measure of student conceptual understanding and learning, we deployed the revised Force Concept Inventory (FCI)\cite{fci} at the beginning and the end of the course, again with voluntary participation. As an additional measure of student performance, the performance on the final exam and the course grade for each student were taken into consideration. For the grade we used the raw percentage score, not the number grades, since it provides finer grained information about the overall student performance in the course.
\section{\label{results}Results}
-When considering the evaluation of student discussion characteristics as replacement of or complementary to surveys such as the MPEX, it is important to understand the correlations among all of the instruments.
-\subsection{Correlations with the Overall Course Grade}
-Figure~\ref{fcimpexgrade} shows the correlation between the final FCI and MPEX scores with the final course grade percentage. With an $r$ of 0.56 and 0.30, respectively, these -- particularly for the MPEX -- turned out lower than expected. As pointed out in section~\ref{setting}, however, the course grade is based on a number of factors, some of which are simply a matter of diligence or effort.
+In this section, we present the correlations among the different instruments and measures.
+\subsection{Correlation Table}
+Table~\ref{fullresults} shows the complete results of the study. In the columns of the table, we listed:
+grade performance;
+final exam performance;
+final (post) FCI score;
+FCI gain;
+final (post) MPEX score;
+MPEX gain;
+prominence of solution-oriented discussion contributions;
+prominence of math-related discussion contributions;
+prominence of physics-related discussion contributions;
+prominence of surface-level discussion contributions;
+prominence of procedural discussion contributions;
+prominence of conceptual discussion contributions.
+
+In the rows of the table, we listed:
+grade performance;
+final exam performance;
+final (post) FCI score;
+FCI gain;
+final (post) MPEX score;
+MPEX gain;
+MPEX Independence Cluster score;
+MPEX Coherence Cluster score;
+MPEX Concept Cluster score;
+MPEX Reality Link Cluster score;
+MPEX Math Link Cluster score;
+MPEX Effort Cluster score.
-\begin{figure*}
-\includegraphics[width=9cm]{fcipostgrade}\includegraphics[width=9cm]{mpexpostgrade}
-\caption{\label{fcimpexgrade}Correlation between the FCI score (left; $R=0.56$; $n=110$) and the MPEX score (right; $R=0.3$; $n=97$) with the course grade percentage. 58\% was the minimum percentage to pass the course. More students participated in the FCI than in the MPEX.}
-\end{figure*}
-
-Figure~\ref{physicsgrade} shows the correlation between the prominence of physics-related discussions and the course grade percentage (for better statistics, only students who contributed at least five discussion entries over the course of the semester were considered). The correlation is stronger than with the MPEX Score, yet smaller than with the FCI.
-\begin{figure}
-\includegraphics[width=9cm]{physicsgrade}
-\caption{\label{physicsgrade}Correlation of percentage physics-related discussions with grade percentage ($R=0.33$; $n=111$).}
-\end{figure}
-\subsection{Correlations with the FCI}
-Figure~\ref{mpexfci} shows how the final FCI and MPEX scores correlated with each other, i.e, $R=0.24$ ($n=97$). Correlations with discussion characteristics turned out somewhat stronger. Figure~\ref{fciphysics} shows how the percentage of a particular student's discussion contribution that was classified as "physics-related" correlates with their final FCI score ($R=0.34$; $n= 95$). As already in Fig.~\ref{physicsgrade}, an additional analysis was carried out that was limited to students for which better statistics were available, which let to a stronger correlation ($R=0.51$; $n=57$).
-
-\begin{figure}
-\includegraphics[width=9cm]{fcipostmpexpost}
-\caption{\label{mpexfci}Correlation of the final FCI score with the MPEX score ($R=0.24$; $n=97$).}
-\end{figure}
+Correlations with $|R|<0.1$ are indicated by a dash, correlations with $|R|>0.5$ are printed boldface. The values in brackets are the result of calculations limited to students with at least five discussion contributions over the course of the semester.
-\begin{figure*}
-\includegraphics[width=9cm]{fcipostphysics}\includegraphics[width=9cm]{fcipostphysicsT}
-\caption{\label{fciphysics}Correlation between the FCI score and the percentage of that student's discussion that was classified as "physics" ($R=0.34$; $n= 95$). The figure on the right only includes students who contributed more than five discussion entries over the course of the semester ($R=0.51$; $n=57$).}
-\end{figure*}
-While physics-related discussions positively correlate with FCI scores and grades (Fig.~\ref{physicsgrade}), solution-oriented discussions negatively correlated (Fig.~\ref{solutionfci}; $R=-0.58$; $n=57$).
-\begin{figure}
-\includegraphics[width=9cm]{fcipostsolutionT}
-\caption{\label{solutionfci}Correlation of percentage solution-oriented discussions with final FCI score ($R=-0.58$; $n=57$).}
-\end{figure}
-\subsection{Full Results}
-Table~\ref{fullresults} shows the complete results of the study, including correlations with gains on FCI and MPEX. In the columns, the table shows Grade percentage, Final Exam percentage, final FCI Score, FCI Gain, final MPEX Score, MPEX Gain, and discussion characteristics, in the rows, again Grade, Final Exam, FCI, MPEX, as well as the individual scores on the final MPEX clusters.
\begin{table*}
-\caption{\label{fullresults}Complete correlation results ($r$-values). Calculated correlations whose absolute value was lower than 0.1 are indicated by "---." Correlations with an absolute value higher than 0.5 have been printed in boldface. The values given in brackets have been calculated including only students who contributed more than five discussion entries over the course of the semester.}
+\caption{\label{fullresults}Complete correlation results ($R$-values). Calculated correlations whose absolute value was lower than 0.1 are indicated by "---." Correlations with an absolute value higher than 0.5 have been printed in boldface. The values given in brackets have been calculated including only students who contributed more than five discussion entries over the course of the semester.}
\begin{ruledtabular}
\begin{tabular}{rllllllllllll}
&Grade&Final&FCI &FCI &MPEX&MPEX&Solution &Math &Physics &Surface &Proce-&Concep-\\
@@ -290,25 +274,66 @@
\end{ruledtabular}
\end{table*}
+Of particular interest is the lower right corner of Table~\ref{fullresults}, as it lists the correlations between student attitudes and expectations (as measured by the MPEX clusters) with the prominence of discussion behavior classes. One would have expected strong correlations between for example the score on the Concepts Cluster and the prominence of conceptual discussion contributions ($R=0.14 (0.18)$), or the comfort level with the usage of mathematics as a language and the corresponding lack of purely mathematical contributions (non significant, and $R=-0.14$ when including only students with more than five contributions overall).
+
+The upper right and the lower left corner list the correlations of student discussion behavior and the MPEX, respectively, with measures of student learning. Correlations are again low, but of comparable magnitude, where the MPEX appears to be slightly more correlated with grade and final exam performance, while the discussion is more correlated with the FCI. In fact, some of the strongest correlations in the study occur between the prominence of solution-oriented and physics-related discussions and the FCI. We will analyze correlations with grades in more detail in subsection~\ref{gradecorrel}, and with the FCI in subsection~\ref{fcicorrel}.
+
The Coherence Cluster of the MPEX appears to be more strongly correlated to other performance indicators than the other clusters. Out of that cluster, agreement with the statement "In doing a physics problem, if my calculation gives a result that differs significantly from what I expect, I'd have to trust the calculation" (53\% unfavorable responses) has $R=-0.3$ ($n=97$) with the grade in the course, $R=-0.3$ ($n=97$) with the final FCI Score, and $R=0.3$ ($n=84$) with solution-oriented discussion postings. Out of the Concepts Cluster, agreement with the single statement "The most crucial thing in solving a physics problem is finding the right equation to use" (45\% unfavorable responses) correlates with $R=-0.3$ ($n=96$) with the final FCI score and also with $R=-0.3$ ($n=85$) with the FCI Gain, i.e., stronger than the cluster it belongs to.
-When considering the intersection of student discussion characteristics, only a few relatively strong correlations can be found. For example, the prominence of discussion contributions that were both conceptual and physics-related correlates with $R=0.2$ ($n=173$) with the grade in the course, and with $R=0.29$ ($n=95$) and $R=0.3$ ($n=84$) with the final FCI Score and Gain, respectively. The prominence of contributions that are both solution-oriented and surface-level correlates with $R=-0.29$ ($n=95$) and $R=-0.13$ ($n=84$) with the FCI Score and Gain, respectively.
+Going beyond the analysis of the large discussion superclasses, when considering the intersection of student discussion characteristics, only a few relatively strong correlations can be found. For example, the prominence of discussion contributions that were both conceptual and physics-related correlates with $R=0.2$ ($n=173$) with the grade in the course, and with $R=0.29$ ($n=95$) and $R=0.3$ ($n=84$) with the final FCI Score and Gain, respectively. The prominence of contributions that are both solution-oriented and surface-level correlates with $R=-0.29$ ($n=95$) and $R=-0.13$ ($n=84$) with the FCI Score and Gain, respectively.
-\section{Discussion of the Results}
-Correlations between Grade, Final Exam, FCI, MPEX, and student discussion behavior have turned out lower than expected. The strongest correlations exist with the final score on the FCI, namely $R=0.56$ with the grade percentage in the course, $R=0.51$ with the prominence of physics-related discussions, and $R=-0.58$ with the prominence of solution-oriented discussions.
-Correlations with the MPEX were generally weak, with $R=0.36$ between the score on the Coherence cluster and the course grade percentage being the highest value. Dancy~\cite{dancy02} found similarly low correlations with the performance on homework, tests, and final exams: direct comparison with the performance on the final exams found $R=0.37$ for the correlation with the total MPEX score ($R=0.27$ here), $R=0.39$ with the independence cluster ($R=0.25$ here), $R=0.24$ with the coherence cluster ($R=0.36$ here), $R=0.29$ with the concepts cluster ($R=0.25$ here), $R=-.02$ with the reality link cluster ($R=0.1$ here), $R=0.3$ with the math link cluster (no significant correlation found here), and no significant correlation with the effort cluster ($R=0.1$ here).
+
+
+\subsection{\label{gradecorrel}Correlations with the Overall Course Grade}
+Figure~\ref{fcimpexgrade} shows the correlation between the final FCI and MPEX scores with the final course grade percentage. With an $r$ of 0.56 and 0.30, respectively, these -- particularly for the MPEX -- turned out lower than expected. As pointed out in section~\ref{setting}, however, the course grade is based on a number of factors, some of which are simply a matter of diligence or effort.
+
+\begin{figure*}
+\includegraphics[width=9cm]{fcipostgrade}\includegraphics[width=9cm]{mpexpostgrade}
+\caption{\label{fcimpexgrade}Correlation between the FCI score (left; $R=0.56$; $n=110$) and the MPEX score (right; $R=0.3$; $n=97$) with the course grade percentage. 58\% was the minimum percentage to pass the course. More students participated in the FCI than in the MPEX.}
+\end{figure*}
+
+Correlations with the MPEX were generally weak, with $R=0.36$ between the score on the Coherence cluster and the course grade percentage being the highest value. Dancy~\cite{dancy02} found similarly low correlations with the performance on homework, tests, and final exams: direct comparison with the performance on the final exams found $R=0.37$ for the correlation with the total MPEX score ($R=0.27$ here), $R=0.39$ with the Independence Cluster ($R=0.25$ here), $R=0.24$ with the Coherence Cluster ($R=0.36$ here), $R=0.29$ with the Concept Cluster ($R=0.25$ here), $R=-.02$ with the Reality Link cluster ($R=0.1$ here), $R=0.3$ with the Math Link cluster (no significant correlation found here), and no significant correlation with the Effort Cluster ($R=0.1$ here).
+
+Figure~\ref{physicsgrade} shows the correlation between the prominence of physics-related discussions and the course grade percentage (for better statistics, only students who contributed at least five discussion entries over the course of the semester were considered). The correlation is stronger than with the MPEX Score, yet smaller than with the FCI.
+\begin{figure}
+\includegraphics[width=9cm]{physicsgrade}
+\caption{\label{physicsgrade}Correlation of percentage physics-related discussions with grade percentage ($R=0.33$; $n=111$).}
+\end{figure}
+
+\subsection{\label{fcicorrel}Correlations with the FCI}
+Figure~\ref{mpexfci} shows how the final FCI and MPEX scores correlated with each other, i.e, $R=0.24$ ($n=97$).
Coletta and Philips~\cite{coletta05} found a strong correlation between the FCI Gain and the MPEX Score ($R=0.52; n=37$), while the same correlation turned out much lower in this study ($R=0.17; n=84$ here). The correlations reported here are in the same range that
Perkins et al.~\cite{perkins04} found when investigating the influence of beliefs on conceptual learning, using the CLASS~\cite{adams04} and the Force and Motion Conceptual Evaluation (FMCE) instruments~\cite{thornton98}.
-An unexpected result were the low correlations between the MPEX cluster scores and the student discussion behavior. One would have expected strong correlations between for example the score on the Concepts Cluster and the prominence of conceptual discussion contributions ($R=0.14 (0.18)$), or the comfort level with the usage of mathematics as a language and the corresponding lack of purely mathematical contributions (non significant or $R=-0.14$ when including only students with more than five contributions overall).
+Correlations with discussion characteristics turned out somewhat stronger. Figure~\ref{fciphysics} shows how the percentage of a particular student's discussion contribution that was classified as "physics-related" correlates with their final FCI score ($R=0.34$; $n= 95$). As already in Fig.~\ref{physicsgrade}, an additional analysis was carried out that was limited to students for which better statistics were available, which let to a stronger correlation ($R=0.51$; $n=57$).
+
+\begin{figure}
+\includegraphics[width=9cm]{fcipostmpexpost}
+\caption{\label{mpexfci}Correlation of the final FCI score with the MPEX score ($R=0.24$; $n=97$).}
+\end{figure}
+
+\begin{figure*}
+\includegraphics[width=9cm]{fcipostphysics}\includegraphics[width=9cm]{fcipostphysicsT}
+\caption{\label{fciphysics}Correlation between the FCI score and the percentage of that student's discussion that was classified as "physics" ($R=0.34$; $n= 95$). The figure on the right only includes students who contributed more than five discussion entries over the course of the semester ($R=0.51$; $n=57$).}
+\end{figure*}
+While physics-related discussions positively correlate with FCI scores and grades (Fig.~\ref{physicsgrade}), solution-oriented discussions negatively correlated (Fig.~\ref{solutionfci}; $R=-0.58$; $n=57$).
+\begin{figure}
+\includegraphics[width=9cm]{fcipostsolutionT}
+\caption{\label{solutionfci}Correlation of percentage solution-oriented discussions with final FCI score ($R=-0.58$; $n=57$).}
+\end{figure}
+\section{Discussion of the Results}
+Correlations between Grade, Final Exam, FCI, MPEX, and student discussion behavior have turned out lower than expected. The strongest correlations exist with the final score on the FCI, namely $R=0.56$ with the grade percentage in the course, $R=0.51$ with the prominence of physics-related discussions, and $R=-0.58$ with the prominence of solution-oriented discussions.
+
+An unexpected result were the low correlations between the MPEX cluster scores and the student discussion behavior. We can thus not conclude that student discussion behavior is strongly correlated with student attitudes and expectations as measured by the MPEX. Student discussions and the MPEX also differently correlate to measures of learning, i.e., student discussion more strongly correlates to the FCI, and MPEX more strongly to course grades and the final exam.
+
+The relative weakness of many of the expected correlations with the MPEX might indicate that maybe -- in spite of the efforts of the author -- the students did not take the MPEX very seriously or did not carefully read the statements. An argument for this possible explanation is that the overall scores of the students on the MPEX were low (Independence 42\%; Coherence 46\%; Concepts 48\%; Reality Link 55\%; Math Link 40\%; Effort 47\%). Also, students relatively frequently chose the answer "3" ("Neutral") on the MPEX Likert scale, which is by definition never correct --- answering that way could indicate true indifference, confusion regarding the statement, or simply "don't care." By the same token, students appear to be taking the FCI more seriously, probably because it more closely matches the other (grade-relevant) assessments they encounter in the course, and students tend to based their relative value system regarding a subject area on the assessments used~\cite{lin}. The FCI seems to be fairly robust in ungraded settings, see for example Henderson~\cite{henderson}, who found only 0.5 points difference between graded and ungraded administration of the FCI --- the MPEX, which is never graded, may in fact be far less robust to perception of ``not counting."
-The relative weakness of many of the expected correlations with the MPEX might indicate that maybe -- in spite of the efforts of the author -- the students did not take the MPEX very seriously or did not carefully read the statements. An argument for this possible explanation is that the overall scores of the students on the MPEX were low (Independence 42\%; Coherence 46\%; Concepts 48\%; Reality Link 55\%; Math Link 40\%; Effort 47\%). Also, students relatively frequently chose the answer "3" ("Neutral") on the MPEX Likert scale, which is by definition never correct --- answering that way could indicate true indifference, confusion regarding the statement, or simply "don't care." By the same token, students appear to be taking the FCI more seriously, probably because it more closely relates to the other (grade-relevant) assessments they encounter in the course.
\section{Conclusions}
In this introductory calculus-based course, correlations between different performance and attitude indicators were found to be lower than expected. Student discussion behavior generally correlates more strongly with student performance (FCI, final exam, grade) than MPEX results. Particularly the prominence of solution-oriented and physics-related discussions correlate relatively strongly with the FCI.
-Regarding the hypothesis that the analysis of student discussion characteristics might be used as a means of determining student attitudes and expectations, the results are inconclusive, since the expected correlation between MPEX clusters and the prominence of different classes of student discussion behavior is largely missing. The reason for this might be that the mechanisms -- even in related areas -- measure different things, that at least one of them in fact measures very little, or that the students did not bother responding to the MPEX with sufficient diligence.
+The expected correlation between MPEX clusters and the prominence of different classes of student discussion behavior is largely missing. The reason for this lack of correlation could not be determined in the framework of this study: it might be that the mechanisms -- even in related areas -- measure different things, or that at least one of them in fact measures very little, or that the students did not bother responding to the MPEX with sufficient diligence.
\begin{acknowledgments}
Supported in part by the National Science Foundation under NSF-ITR 0085921 and NSF-CCLI-ASA 0243126. Any opinions, findings, and conclusions or recommendations expressed in this
publication are those of the author and do not necessarily reflect the views of the National Science Foundation. The author would like to thank the students in his course for their participation in this study.
--www1152641906--