[LON-CAPA-cvs] cvs: modules /gerd/correlpaper correlations.bib correlations.tex
www
lon-capa-cvs@mail.lon-capa.org
Wed, 03 Jan 2007 02:20:58 -0000
This is a MIME encoded message
--www1167790858
Content-Type: text/plain
www Tue Jan 2 21:20:58 2007 EDT
Modified files:
/modules/gerd/correlpaper correlations.bib correlations.tex
Log:
Here we go again ... I love this paper.
--www1167790858
Content-Type: text/plain
Content-Disposition: attachment; filename="www-20070102212058.txt"
Index: modules/gerd/correlpaper/correlations.bib
diff -u modules/gerd/correlpaper/correlations.bib:1.5 modules/gerd/correlpaper/correlations.bib:1.6
--- modules/gerd/correlpaper/correlations.bib:1.5 Thu Sep 28 11:18:57 2006
+++ modules/gerd/correlpaper/correlations.bib Tue Jan 2 21:20:55 2007
@@ -273,3 +273,50 @@
url= "http://modeling.la.asu.edu/R&E/Research.html",
title = "Force Concept Inventory"
}
+
+@ARTICLE{heuvelen,
+ author="Alan Van Heuvelen",
+ year="1991",
+ journal="Am. J. Physics",
+ volume="59",
+ pages="891-897",
+ title="Learning to think like a physicist: A review of research-based instructional strategies"
+}
+
+@ARTICLE{larkin,
+ author="Jill Larkin and John McDermott and Dorothea P. Simon and Herbert A. Simon",
+ year="1980",
+ journal="Science",
+ volume="208",
+ pages="1335-1342",
+ title="Expert and novice performance in solving physics problems"
+ }
+
+ @ARTICLE{hammer96,
+ author = "David Hammer",
+ year = "1996",
+ journal = "Am. J. Phys.",
+ volume = "64",
+ pages = "1316-1325",
+ title = "More than misconceptions: Multiple perspectives on student knowledge and reasoning, and an appropriate role for education research"
+}
+
+@ARTICLE{elby99,
+ author = "Andrew Elby",
+ year = "1999",
+ journal = "Am. J. Phys.",
+ volume = "67",
+ pages = "S52-S57",
+ title = "Another reason that physics students learn by rote"
+}
+
+@ARTICLE{lising05,
+ author = "Laura Lising and Andrew Elby",
+ year = "2005",
+ journal = "Am. J. Phys.",
+ volume = "73",
+ pages = "372-382",
+ title = "The impact of epistemology on learning: A case study from introductory physics"
+}
+
+
\ No newline at end of file
Index: modules/gerd/correlpaper/correlations.tex
diff -u modules/gerd/correlpaper/correlations.tex:1.17 modules/gerd/correlpaper/correlations.tex:1.18
--- modules/gerd/correlpaper/correlations.tex:1.17 Fri Nov 24 10:56:08 2006
+++ modules/gerd/correlpaper/correlations.tex Tue Jan 2 21:20:55 2007
@@ -84,7 +84,12 @@
\end{itemize}
\section{\label{background}Background}
-Previous studies indicate that correlations between epistemological beliefs and academic performance exist, both directly and indirectly. For example, Schommer~ \cite{schommer93} found that belief in ``quick learning'' (characterized by seeking single answers, avoiding ambiguity, and relying on authority) negatively correlates with the GPA of secondary students, even after controlling for general intelligence. May~\cite{may02} found possible correlations between epistemological beliefs extracted from extensive lab reports and conceptual learning gain in introductory physics courses. For example, students who stated that they learned formulas (rather than investigated their conceptual implications), relied on authority, and made no efforts to interpret results were found to have lower gains on the Force Concept Inventory, Mechanics Baseline Test, and Conceptual Survey of Electricity and Magnetism.
+For an expert physicist, physics is much more than a vast unconnected collection of facts or the mechanics of manipulating given formulas; instead, it is a way of thinking. Students, on the other hand, tend to work from a not yet coherent set of physics factoids organized by surface features rather than physics concepts~\cite{chi} and often focus on finding the right formula for the situation rather than on the construction of new knowledge and relationships~\cite{heuvelen,larkin}.
+These and other beliefs about what constitutes knowledge in physics and how one develops knowledge are described as epistemological beliefs~\cite{hammer94,hammer96}. Different epistemological beliefs can lead to very different understandings of the same scenario. For example, expert physicists see the derivation of a formula as a way to embed the new knowledge into their existing framework, while students tend to see it as a proof that a formula is true and ``okay to be used"~\cite{mpex}. As a result, the whole process is frequently ineffective in lectures, since students would see a proof of correctness as irrelevant and would rather just trust the ``authority" (for example, a student evaluation of the course under consideration in this study stated: ``Don't derive anything, really, you have a Ph.D. I'll believe what you tell me.'')
+
+Previous studies indicate that correlations between epistemological beliefs and academic performance exist, both directly and indirectly. For example, Schommer~ \cite{schommer93} found that belief in ``quick learning'' (characterized by seeking single answers, avoiding ambiguity, and relying on authority) negatively correlates with the GPA of secondary students, even after controlling for general intelligence. May~\cite{may02} found possible correlations between epistemological beliefs extracted from extensive lab reports and conceptual learning gain in introductory physics courses. For example, students who stated that they learned formulas (rather than investigated their conceptual implications), relied on authority, and made no efforts to interpret results were found to have lower gains on the Force Concept Inventory, Mechanics Baseline Test, and Conceptual Survey of Electricity and Magnetism. It should be remarked that the performance on conceptual tests is not necessarily directly connected to good course grades, which is an issue that students are aware of~\cite{lin} and can lead them to act contrary to their beliefs.
+
+Correlations between epistemologies and learning alone do not imply causal relationships. Based on videotaped classroom scenarios, written work, and interviews, Lising and Elby~\cite{lising05} argue that a more expert-like epistemology indeed leads to better learning, and thus, curricular materials and teaching techniques should explicitly attend to students' epistemological beliefs.
The problem is how to measure these beliefs, and techniques include surveys, guided interviews, and observations. While interviews and observations are likely resulting in better data, the effort in conducting them also limits the scale at which they can be conducted. Surveys do not have this scalability problem, but
research results regarding the predictive power of these instruments is not always conclusive: for example, Coletta and Philips~\cite{coletta05} found a strong correlation between the MPEX and FCI Gain, while Dancy~\cite{dancy02} found low correlations between the MPEX and the performance on homework, tests, and final exams. It is unclear why these studies would come to such different results regarding the predictive power of the MPEX on an individual student level. Until more insights are gained, it remains a good idea to abide by the ``Product Warning Lab''~\cite{mpexwarning} that the survey is best used to gain insights into the beliefs of the class as a whole.
@@ -262,8 +267,16 @@
The most surprising result was that only 31\% of the students stated that they would be frustrated or very frustrated if they did not do well on the FCI, and only 30\% of the students stated the same for the MPEX. Particularly the FCI percentage is smaller than expected, since the FCI is generally believed to be fairly robust in ungraded settings, see for example Henderson~\cite{henderson}, who found only 0.5 points difference between graded and ungraded administration of the FCI. Also, the FCI is similar to the tests and exams used in the course, and students tend to base their relative value system regarding a subject area on the assessments used~\cite{lin}.
-On the other hand, student discussions correlate more strongly with performance measures. Students are taking them seriously, likely because they are perceived as helpful and relevant. In the same post-course survey, 89\% of the students found the discussions either helpful or very helpful, and 73\% stated that they used the discussions to learn physics, as opposed to 35\% who said they often or very often just used the discussions to get the correct result as quickly as possible. Discussions appear to be an authentic reflection of what the students perceive as effective, not necessarily good, problem solving strategy: while an expert would characterize most postings as ``bad strategy,''
-only 17\% of the students admitted that they often against better knowledge used bad problem solving strategies to get the correct result as soon as possible, and 48\% stated that they rarely or never did so (35\% were not sure).
+On the other hand, student discussions correlate more strongly with performance measures. Students are taking them seriously, likely because they are perceived as helpful and relevant. In the same post-course survey, 89\% of the students found the discussions either helpful or very helpful, and 73\% stated that they used the discussions to learn physics, as opposed to 35\% who said they often or very often just used the discussions to get the correct result as quickly as possible.
+
+While experts would characterize most postings as ``bad strategy,''
+only 17\% of the students admitted that they often against better knowledge used bad problem solving strategies to get the correct result as soon as possible, and 48\% stated that they rarely or never did so (35\% were not sure). So, about half of the students claim that their online discussion
+behavior reflects their epistemological views about physics problem solving;
+but the other half either ``aren't sure'' or explicitly admit that their
+strategies reflect expediency rather than their views about how best to
+learn physics. This finding corresponds to the study by Elby~\cite{elby99}, who found that students can perceive ``trying to understand physics deeply'' and ``pursuing good grades'' to be a different activities. Therefore, for the class as a whole, online discussion
+behavior reflects a combination of students' epistemological beliefs,
+expectations about what's rewarded in the class, and expediency.
@@ -309,7 +322,7 @@
\section{Discussion of Possible Causal Relationships}
-The study showed that there is a relatively strong correlation between solution-oriented discussion behavior (negative) and physics-oriented discussion behavior (positive) and the final FCI score. It is an interesting question whether the students learned physics better because of their more expert-like approach, or vice versa.
+The study showed that there is a relatively strong correlation between solution-oriented discussion behavior (negative) and physics-oriented discussion behavior (positive) and the final FCI score. It is an interesting question whether the students learned physics better because of their more expert-like approach (as argued by Lising and Elby~\cite{lising05}), or vice versa.
In an attempt to answer this question, we are considering the FCI gain as a rough measure of how much physics the students {\it learned} (versus, for example, knew already). We also introduced a measure of discussion behavior gain by splitting the semester in half and calculating the the difference between the prominence of discussion behaviors in the first and the second half of the semester.
@@ -337,10 +350,18 @@
\end{equation*}
with an explained variance of 47.9\% of the Post FCI score. Both coefficients are significant, the solution-oriented discussion has $p=0.019$. Thus, controlling for pre-test FCI score, for each 10 percent increase in solution-oriented discussion, the predicted post-test FCI score goes down by 0.42 points. Students who do not make any solution-oriented contributions would on the average gain 7.6 points on the 30 item FCI due to instruction, while at the other extreme, students who only make solution-oriented discussions would on the average only gain 3.4 points -- less than half.
\section{Conclusions}
-Online student discussions have very little correlation with MPEX outcomes, but appear to be a good reflection of students' individual beliefs regarding the most effective strategy of problem solving in physics. Students who exhibit more expert-like views and strategies have higher learning success, even when controlling for prior physics knowledge.
+Students' online discussion behavior is not a pure reflection of
+students' epistemological beliefs; other factors, notably expectations and
+expediency, also feed into their online behavior. This behavior, however,
+reflects how students actually approach their physics homework problems.
+Students who exhibit more expert-like approaches have higher learning
+success, even when controlling for prior physics knowledge. Indeed, the
+correlation between on-line discussion behavior and conceptual gains is
+stronger than the correlation between MPEX scores and conceptual gains,
+showing the value of online discussion behavior as a diagnostic tool.
\begin{acknowledgments}
Supported in part by the National Science Foundation under NSF-ITR 0085921 and NSF-CCLI-ASA 0243126. Any opinions, findings, and conclusions or recommendations expressed in this
-publication are those of the author and do not necessarily reflect the views of the National Science Foundation. The author would like to thank the students in his course for their participation in this study, as well as Deborah Kashy from Michigan State University for assistance with the statistical analysis of the data, and Stephen Pellathy from the University of Pittsburgh for carrying out the interrater reliability study.
+publication are those of the author and do not necessarily reflect the views of the National Science Foundation. The author would like to thank the students in his course for their participation in this study, as well as Deborah Kashy from Michigan State University for assistance with the statistical analysis of the data, Stephen Pellathy from the University of Pittsburgh for carrying out the interrater reliability study, and anonymous reviewers for their constructive criticism.
\end{acknowledgments}
\bibliography{correlations}% Produces the bibliography via BibTeX.
--www1167790858--