[LON-CAPA-cvs] cvs: modules /gerd/correlpaper correlations.bib correlations.tex

[www]

This is a MIME encoded message

--www1155657994
Content-Type: text/plain

www		Tue Aug 15 12:06:34 2006 EDT

  Modified files:              
    /modules/gerd/correlpaper	correlations.bib correlations.tex 
  Log:
  Is this it?
  
  
--www1155657994
Content-Type: text/plain
Content-Disposition: attachment; filename="www-20060815120634.txt"

Index: modules/gerd/correlpaper/correlations.bib
diff -u modules/gerd/correlpaper/correlations.bib:1.3 modules/gerd/correlpaper/correlations.bib:1.4
--- modules/gerd/correlpaper/correlations.bib:1.3	Tue Jul 11 14:18:24 2006
+++ modules/gerd/correlpaper/correlations.bib	Tue Aug 15 12:06:31 2006
@@ -7,6 +7,43 @@
    title = "The problem with problems"
 }
 
+@ARTICLE{schommer93,
+   author = "Marlene Schommer",
+   year = "1993",
+   journal = "Journal of Educational Psychology",
+   volume = "85",
+   pages = "406-411",
+   title = "Epistemological Development and Academic Performance among Secondary Students"
+}
+
+@ARTICLE{may02,
+   author = "David B. May and Eugenia Etkina",
+   year = "2002",
+   journal = "Am. J. Phys.",
+   volume = "70",
+   pages = "1249-1258",
+   title = "College Physics Students' Epistemological Self-Reflection and Its Relationship to Conceptual Learning"
+}
+
+@ARTICLE{hammer94,
+   author = "David Hammer",
+   year = "1994",
+   journal = "Cognition and Instruction",
+   volume = "12",
+   pages = "151-183",
+   title = "Epistemological Beliefs in Introductory Physics"
+}
+
+@ARTICLE{hogan99,
+   author = "Kathleen Hogan",
+   year = "1999",
+   journal = "Science Education",
+   volume = "83",
+   pages = "1-32",
+   title = "Relating Students' Personal Frameworks for Science Learning to Their Cognition in Collaborative Contexts"
+}
+
+
 @ARTICLE{thornton98,
    author = "Ron Thornton and David Sokoloff",
    year = "2005",
Index: modules/gerd/correlpaper/correlations.tex
diff -u modules/gerd/correlpaper/correlations.tex:1.9 modules/gerd/correlpaper/correlations.tex:1.10
--- modules/gerd/correlpaper/correlations.tex:1.9	Mon Aug 14 13:35:37 2006
+++ modules/gerd/correlpaper/correlations.tex	Tue Aug 15 12:06:31 2006
@@ -59,11 +59,11 @@
                               %display desired
 \maketitle
 \section{\label{intro}Introduction}
-A traditional way of assessing student beliefs, attitudes, and expectations about physics is the deployment of surveys, for example the Maryland Physics Expectations Survey (MPEX)~\cite{mpex}, the Epistemological Beliefs Assessment for Physical Science (EBAPS)~\cite{ebaps}, or the Colorado Learning Attitudes about Science Survey (CLASS)~\cite{adams04}. While these instruments take different approaches and have different philosophies behind their designs, they do have in common that the students need to react to artificial statements outside of the normal course activity, and that they need to self-report their responses.
+A traditional way of assessing student beliefs, attitudes, and expectations about physics is the deployment of surveys, for example the Maryland Physics Expectations Survey (MPEX)~\cite{mpex}, the Epistemological Beliefs Assessment for Physical Science (EBAPS)~\cite{ebaps}, or the Colorado Learning Attitudes about Science Survey (CLASS)~\cite{adams04}, or through structured interviews (see for example~\cite{hammer94,hogan99}). While these instruments take different approaches and have different philosophies behind their designs, they do have in common that the students need to react to statements outside of the normal course activity, and that they need to self-report their responses.
 
 Online student discussions associated with online physics problems are different in that they are generated within the real context of the course, and students have a vested interest in making these discussions as productive as possible, given their understanding of how physics is done and their approach to it. They could thus be a ``reality check'' of students' beliefs, attitudes, and expectations. 
 
-Online discussions are a rich source of feedback to the instructor~\cite{kortemeyer05feedback}, and their quality and character was found to be correlated with the type and difficulty of the associated problems~\cite{kortemeyer05ana}, i.e., data exists regarding the influence of {\it problem} characteristics on associated discussions. Unfortunately, less data exists on the correlation between {\it student} characteristics and discussion behavior, because usually only very few student characteristics are known, with the exception of the students' overall performance in the course. Thus, one of the few findings was the fact that certain discussion behavior, most prominently exhibited on ``non-sanctioned'' discussion sites external to the course, is negatively correlated with performance in the course~\cite{kashy03,kortemeyer05ana}. 
+Online discussions are a rich source of feedback to the instructor~\cite{kortemeyer05feedback}, and their quality and character was found to be correlated with the type and difficulty of the associated problems~\cite{kortemeyer05ana}, i.e., data exists regarding the influence of {\it problem} characteristics on associated discussions. Unfortunately, less data exists on the correlation between {\it student} characteristics and discussion behavior, because usually only very few student characteristics are known, with the exception of the students' overall performance in the course. Thus, one of the few findings was the fact that certain discussion behavior, most prominently exhibited on ``non-sanctioned'' discussion sites external to the course, is negatively correlated with performance in the course~\cite{kashy03,kortemeyer05ana}. Also, Hogan~\cite{hogan99} assessed eight graders' epistemological frameworks through interviews and then analyzed their discussion behavior in a science course with a particular focus on collaboration, finding a number of correlations.
 
 In this study, we aim to answer the question if and how student discussion characteristics are related to their beliefs, attitudes, and expectations, as measured by the MPEX. We investigate correlations with the MPEX, and compare correlations with measures of student learning.
 
@@ -347,7 +347,9 @@
 \section{Discussion of Possible Causal Relationships}
 A purely correlational study does not allow any conclusions regarding causal relationships. In this section, we are discussing some possible causal relations and additional experiments that were conducted to confirm some of these.
 \subsection{Discrepancy in the Correlational Power of the MPEX and the FCI}
-A surprising result is the relative weakness of many of the expected correlations with the MPEX, particularly compared to and correlated with the FCI. A hypothesis was formed that the students do not take the MPEX very seriously or don't find it relevant, and that they do not care greatly how they are performing on it. An argument for this possible explanation is that the overall scores of the students on the MPEX were low (Independence 42\%; Coherence 46\%; Concepts 48\%; Reality Link 55\%; Math Link 40\%; Effort 47\%). 
+A surprising result is the relative weakness of many of the expected correlations with the MPEX, particularly compared to and correlated with the FCI, as well as other course-specific performance measures. Previous studies indicate that correlations between epistemological beliefs and academic performance exist, both directly and indirectly \cite{schommer93,may02}.
+
+A hypothesis was formed that the students do not take the MPEX very seriously or don't find it relevant, and that they do not care greatly how they are performing on it. An argument for this possible explanation is that the overall scores of the students on the MPEX were low (Independence 42\%; Coherence 46\%; Concepts 48\%; Reality Link 55\%; Math Link 40\%; Effort 47\%). 
 
 To give a more definitive answer, an additional survey was deployed online after the end of the course regarding both the MPEX and the FCI.
 
@@ -355,9 +357,9 @@
 
 The most surprising result was that only 32\% of the students stated that they would be frustrated or very frustrated if they did not do well on the FCI, and only 30\% of the students stated the same for the MPEX. Particularly the FCI percentage is smaller than expected, since the FCI is generally believed to be fairly robust in ungraded settings, see for example Henderson~\cite{henderson}, who found only 0.5 points difference between graded and ungraded administration of the FCI.
 
-In summary, it can be confirmed that the correlation results with and between the surveys might be weak because the students --- in spite of the best efforts of the author --- do not really care that much, particularly not how well they are doing on them. The main difference between the surveys the surveys is that the students find the FCI more relevant than the MPEX, likely because the FCI more closely matches the other grade-relevant assessments they encounter in the course, and students tend to based their relative value system regarding a subject area on the assessments used~\cite{lin}. 
+In summary, it can be confirmed that the correlation results with and between the MPEX and the FCI might be weak because the students --- in spite of the best efforts of the author --- do not really care that much about them, particularly not how well they are doing on them. The main difference between the two instruments is that the students find the FCI more relevant than the MPEX, likely because the FCI more closely matches the other grade-relevant assessments they encounter in the course, and students tend to base their relative value system regarding a subject area on the assessments used~\cite{lin}. 
 
-On the other hand, student discussions correlate more strongly with performance measures. Students are taking them seriously, likely because they are perceived as helpful and relevant. In the same post-course survey, 90\% of the students found the discussions either helpful or very helpful, and 73\% stated that they used the discussions to learn physics, as opposed to 34\% who said they often or very often just used the discussions to get the correct result as quickly as possible. They may be an authentic reflection of what the students perceive as good problem solving strategy:  While an expert would characterize most postings as ``bad strategy,''  
+On the other hand, student discussions correlate more strongly with performance measures. Students are taking them seriously, likely because they are perceived as helpful and relevant. In the same post-course survey, 90\% of the students found the discussions either helpful or very helpful, and 73\% stated that they used the discussions to learn physics, as opposed to 34\% who said they often or very often just used the discussions to get the correct result as quickly as possible. Discussions appear to be an authentic reflection of what the students perceive as good problem solving strategy:  while an expert would characterize most postings as ``bad strategy,''  
 only 16\% admitted that they often against better knowledge used bad problem solving strategies to get the correct result as soon as possible, and 48\% stated that they rarely or never did so (36\% were not sure). 
 
 
@@ -370,9 +372,10 @@
 
 As it turns out, the first correlations are significant, with $R=-0.44 [-0.65 - -0.18] (n=47)$ for FCI gain versus solution-oriented discussions, and $R=0.4 [0.13 - 0.62] (n=47)$ for FCI gain versus physics-related discussions. Such significant correlations do not occur for FCI gain versus any of the MPEX cluster scores.
 
-On the other hand, the correlations with discussion-gain are not significant: $0.24 [-0.05 -- 0.49] (n=47)$ for FCI gain versus gain in solution-oriented discussions, and $-0.12 [-0.39 -- 0.17] (n=47)$ for FCI gain versus gain in physics-related discussions. Note that these correlations have the opposite sign than expected, however, the confidence intervals include zero in both cases. When looking at the absolute values, the average gain in solution-oriented discussions between the two halves of the semester is $2.4\%$, and the gain in physics-oriented discussions $-0.3\%$ --- in other words, the students did not really change their discussion behavior over the course of the semester, and their discussion behavior does not improve co-measured with their increasing understanding of physics.
+On the other hand, the correlations with discussion-gain are not significant: $0.24 [-0.05 -- 0.49] (n=47)$ for FCI gain versus gain in solution-oriented discussions, and $-0.12 [-0.39 -- 0.17] (n=47)$ for FCI gain versus gain in physics-related discussions. Note that these correlations have the opposite sign than expected, however, the confidence intervals include zero in both cases. When looking at the absolute values, the average gain in solution-oriented discussions between the two halves of the semester is $2.4\%$, and the gain in physics-oriented discussions $-0.3\%$ --- in other words, the students did not really change their discussion behavior over the course of the semester, and their discussion behavior does not improve co-measured with their increasing understanding of physics. 
 
-Thus, the discussion behavior appears to be a property of the students that is almost constant over the course of the semester, probably reflective of their epistemologies. A more expert-like approach that is reflected in more desirable discussion behavior causes students to have higher learning gains in physics.
+Thus, the discussion behavior appears to be a property of the students that is almost constant over the course of the semester, just like Hammer~\cite{hammer94} already pointed out that it is unlikely that epistemological beliefs are changed implicitly by physics instruction.
+A more expert-like approach that is reflected in more desirable discussion behavior causes students to have higher learning gains in physics.
 
 \section{Conclusions}
 In this introductory calculus-based course, correlations between different performance and attitude indicators were found to be lower than expected. Student discussion behavior generally correlates more strongly with student performance (FCI, final exam, grade) than MPEX results. Particularly the prominence of solution-oriented and physics-related discussions correlate relatively strongly with the FCI. A more expert-like approach to physics, which is reflected in more desirable discussion behavior, causes students to have higher learning gains in physics. On the downside, a physics course appears to do little in terms of changing students' approaches to physics.

--www1155657994--