[LON-CAPA-cvs] cvs: modules /gerd/roleclicker description.tex
www
lon-capa-cvs@mail.lon-capa.org
Tue, 17 May 2005 20:18:19 -0000
This is a MIME encoded message
--www1116361099
Content-Type: text/plain
www Tue May 17 16:18:19 2005 EDT
Modified files:
/modules/gerd/roleclicker description.tex
Log:
Shorten
--www1116361099
Content-Type: text/plain
Content-Disposition: attachment; filename="www-20050517161819.txt"
Index: modules/gerd/roleclicker/description.tex
diff -u modules/gerd/roleclicker/description.tex:1.51 modules/gerd/roleclicker/description.tex:1.52
--- modules/gerd/roleclicker/description.tex:1.51 Tue May 17 16:14:01 2005
+++ modules/gerd/roleclicker/description.tex Tue May 17 16:18:18 2005
@@ -18,7 +18,7 @@
\headsep = 0.0in
-\pagestyle{empty}
+\pagestyle{plain}
\begin{document}
@@ -38,24 +38,14 @@
Based on the initial response distribution, the educator might decide to follow up with a second round of having the learners
discuss the problem with each other (``think-pair-share''), and then responding again.
-At the heart of peer-instruction are these learner-learner discussions - as learners are explaining concepts to each other, they gain deeper
-understanding. This outcome is established through both per-question pre-/post-discussion response analyses,
-and through course-wide pre-/post-test scores on concept inventories, where the gain is consistently higher in courses using
-peer-instruction techniques.
-
-Peer-instruction is a promising method for affecting fundamental, systemic
-improvement in
-science education \cite{mref21,mref22,mref23}.
-It demands that students think critically about the material and participate actively in the learning
-process; in addition, it uncovers student misunderstandings in real time so that they can be identified and
-corrected at once. peer-instruction is also particularly efficient because it helps those who
-get the answer right as well
-as those who get it wrong. Students answering correctly improve their own understanding by explaining
-CTs to others (consistent with research that shows high-ability students benefit from collaboration
-\cite{mref25,mref26}), and students answering incorrectly benefit from individualized explanations and the opportunity
+Peer-instruction is a promising method for affecting fundamental, systemic improvement in science education \cite{mref21,mref22,mref23}.
+It demands that students think critically about the material and participate actively in the learning process; in addition, it uncovers student misunderstandings in real time so that they can be identified and
+corrected at once. Peer-instruction is also particularly efficient because it helps those who get the answer right as well as those who get it wrong. Students answering correctly improve their own understanding by explaining
+problems
+to others (consistent with research that shows high-ability students benefit from collaboration\cite{mref25,mref26}), and students answering incorrectly benefit from individualized explanations and the opportunity
to ask follow-up questions of their classmates.
-
+At the heart of peer-instruction are these learner-learner discussions
However, to our knowledge, formal research data on the discussion process itself is missing:
while most instructors employing peer-instruction would walk around the classroom during discussion periods and eavesdrop on
learners, we are not aware of a systematic study of these discussions. Are they as effective as they could be?
@@ -138,7 +128,7 @@
Instruments have been developed to assess epistemological beliefs, for example the Epistemological Beliefs Assessment for Physical Science (EBAPS) Instrument~\cite{EBAPS}. Related to epistemological beliefs are learners' expectations and attitudes, and of particular interest is the Maryland Physics Expectations (MPEX) survey~\cite{MPEX}.
-We will use existing concept inventory surveys as both pre- and post-tests.
+To assess prior knowledge of the subject area, we will use existing concept inventory surveys as both pre- and post-tests.
The qualitative Force Concept Inventory~\cite{fci} and the quantitative companion Mechanical Baseline Test~\cite{hestenesmech} have been used in a large number of studies connected to the teaching of introductory mechanics. The Foundation Coalition has been developing a number of relevant concept inventories~\cite{foundation}, namely the Thermodynamics Concept Inventory, the Dynamics Concept Inventory, and the Electromagnetics Concept Inventory (with two subcomponents, namely Waves and Fields). Since these were designed from an engineering point of view, some adjustment might be necessary. In addition, the Conceptual Survey of Electricity and Magnetism (CSEM)~\cite{maloney} is available for the second semester course.
\subsection{\label{subsec:problemcat}Problem Classification}
@@ -147,12 +137,15 @@
Steinberg~\cite{steinberg} also analyzed student performance on multiple-choice diagnostics and open-ended exam problems, and found that while those correlate in general, for certain students
and certain questions, responses differ greatly.
For this project, we are choosing a finer-grained classification scheme of question types (adapted from Redish~\cite{redish}):
-\begin{description}
-\item[Single-Response Multiple-Choice] The most basic and most easily computer-evaluated type of question, where only one option is correct,
-see for example the original ConcepTest problems on the left of Figs.~\ref{repre} and \ref{reprecoll}.
-
-\item[Multiple-Response Multiple-Choice] This type of problem, a first step beyond conventional problems, requires a student to evaluate each statement and make a decision about it. The problem on the right side of Fig.~\ref{repre} is of this type.
+\noindent{\bf Single-Response Multiple-Choice:} The most basic and most easily computer-evaluated type of question, where only one option is correct,
+see for example the original ConcepTest problems on the left of Figs.~\ref{repre} and \ref{reprecoll}.\newline
+{\bf Multiple-Response Multiple-Choice} This type of problem, a first step beyond conventional problems, requires a student to evaluate each statement and make a decision about it. The problem on the right side of Fig.~\ref{repre} is of this type.
+\newline
+{\bf Numerical and Formula Short Answer:} Numerical answers (potentially in multiple dimensions and including physical units), such as ``\verb!17 kg/m^3!" or mathematical expressions (potentially in multiple dimensions), such as ``\verb!1/2*m*(vx^2+vy^2)!", are expected. \newline
+{\bf Ranking-Tasks:} This type of problem requires a student to rank a number of statements, scenarios, or objects with respect to a certain feature. For example, a student might be asked to rank a number of projectiles in the order that they will hit the ground, or a number of locations in order of the strength of their local electric potential.Several options may have the same rank (``tie'').The left panel of Fig.~\ref{reprecoll} is of this type.
+\newline
+{\bf Click-on-Image:} Learners need to click on different parts in an image, for example on where to cut a wire in order to brighten up a lightbulb elsewhere in a circuit diagram.
\begin{figure} [t]
\includegraphics[width=8cm]{emfOrig}
@@ -161,20 +154,22 @@
\caption{Example of two problems addressing the same concepts in two different representations. The problem on the left is an original ConcepTest problem of type single-response multiple-choice, the problem on the right is multiple-response multiple-choice. Both problems require representation translation since data is provided in graphical form.\label{repre}}
\end{figure}
-\item[Numerical and Formula Short Answer] Numerical answers (potentially in multiple dimensions and including physical units), such as ``\verb!17 kg/m^3!" or mathematical expressions (potentially in multiple dimensions), such as ``\verb!1/2*m*(vx^2+vy^2)!", are expected.
+\begin{figure} [t]
+\includegraphics[width=8cm]{collOrig}
+\includegraphics[width=8cm]{collRankNR}
+
+\caption{Example of two problems addressing the same concepts in two different representations. The problem on the left is an original ConcepTest problem of type single-response multiple-choice, the problem on the right is rank-response.\label{reprecoll}}
+\end{figure}
-\item[Ranking-tasks] This type of problem requires a student to rank a number of statements, scenarios, or objects with respect to a certain feature. For example, a student might be asked to rank a number of projectiles in the order that they will hit the ground, or a number of locations in order of the strength of their local electric potential.
-Several options may have the same rank (``tie'').
-The left panel of Fig.~\ref{reprecoll} is of this type.
-\item[Click-on-Image] Learners need to click on different parts in an image, for example on where to cut a wire in order to brighten up a lightbulb elsewhere in a circuit diagram.
-\end{description}
In addition, we consider the following features, which may or may not apply to any question type (adapted from Redish~\cite{redish}):
-\begin{description}
-\item[Representation-Translation] This type of problem requires a student to translate between different representations of the same situation, for example from a graphical to a numerical or textual representation. The answer might be required in different formats, see for example
+
+\noindent
+{\bf Representation-Translation:} This type of problem requires a student to translate between different representations of the same situation, for example from a graphical to a numerical or textual representation. The answer might be required in different formats, see for example
Fig.~\ref{repre}. Translation between representations can be surprisingly challenging for physics learners~\cite{mcdermott,beichner}.
-\item[Context-based Reasoning] The distinguishing characteristic of these problems is that they are set in the context of real-world scenarios and not in the context of the artificial ``zero-friction" laboratory scenarios of typical textbook problems.
-\end{description}
-The same classification scheme was successfully used for an analysis of online asynchronous discussions~\cite{discpaper}.
+\newline
+{\bf Context-based Reasoning:} The distinguishing characteristic of these problems is that they are set in the context of real-world scenarios and not in the context of the artificial ``zero-friction" laboratory scenarios of typical textbook problems.
+
+The same classification scheme was successfully used for an analysis of online asynchronous discussions associated with online homework problems~\cite{discpaper}.
\subsection{Process-Oriented Evaluation}
The process-oriented evaluation will focus on the actual discussion process. Since currently no baseline data exists for this study, we will assess the quality of student discussion both
before and after the introduction of extensions to the current peer-instruction technique.
@@ -193,24 +188,22 @@
\subsubsection{\label{subsec:disccat}Discussion Classification}
Student discussion entries are classified into four types and with ten possible features. The four types~\cite{discpaper} are
-\begin{description}
-\item[Emotional] - discussion contributions were classified as ``emotional" if they mostly communicated opinions,
-complaints, gratitude, feelings, etc. Two subtypes were ``positive" and ``negative."
-\item[Surface] - discussion contributions were classified as ``surface" if they dealt with surface features of the
-problem or where surface level requests for help. Two subtypes were ``question" and ``answer."
-\item[Procedural] - contributions that describe or inquire about a mechanisms to solve the problem without
-mention of the underlying concepts or reasoning. Two subtypes were ``question" and ``answer."
-\item[Conceptual] - contributions that deal with the underlying concepts of the problem. Two subtypes were
-``question" and ``answer."
-\end{description}
+
+\noindent{\bf Emotional:} discussion contributions were classified as ``emotional" if they mostly communicated opinions,
+complaints, gratitude, feelings, etc. Two subtypes were ``positive" and ``negative."\newline
+{\bf Surface:} discussion contributions were classified as ``surface" if they dealt with surface features of the
+problem or where surface level requests for help.\newline
+{\bf Procedural:} contributions that describe or inquire about a mechanisms to solve the problem without
+mention of the underlying concepts or reasoning.\newline
+{\bf Conceptual:} contributions that deal with the underlying concepts of the problem.
In addition, discussion contributions were classified by the following features~\cite{discpaper}:
-\begin{description}
-\item[Unrelated] - the contribution is not related to the problem.
-\item[Solution-oriented] - the goal of the contribution is to arrive at the correct answer without mentioning or
+
+\noindent{\bf Unrelated:} the contribution is not related to the problem.
+\newline{\bf Solution-oriented:} the goal of the contribution is to arrive at the correct answer without mentioning or
dealing with the mathematics or physics of the problem.
-\item[Mathematical] - the contribution deals mostly with the mathematical aspects of the problem.
-\item[Physics] - the contribution deals mostly with the physics aspects of the problem.
-\end{description}
+\newline{\bf Mathematical:} the contribution deals mostly with the mathematical aspects of the problem.
+\newline{\bf Physics:} the contribution deals mostly with the physics aspects of the problem.
+
In addition, the following features are considered, which were used in
an earlier study of discussions around group exercises in an introductory Computer Science course at Michigan State University (derived from Johnson et al.~\cite{johnson}):
Contributes Idea; Encourages Participation; Summarizes/Integrates; Check for Understanding; Relates New to Old Learning; Gives Direction to Work.
@@ -235,15 +228,15 @@
A&And they're gonna move after the crash?&Conceptual; Question; Physics.\\\hline
C&Maybe.&Conceptual; Answer; Physics.\\\hline
A&How would you know? Is momentum conserved here in inelastic?&Conceptual; Question; Physics.\\\hline
-B&This problem sucks. Who like comes up with this [\ldots] anyway?&Emotional; Negative.\\\hline
+B&This problem is way too hard. Who like comes up with this [\ldots] anyway?&Emotional; Negative.\\\hline
C&Let's just calculate it. So, $E=\frac12mv^2$, and it's like two cars before and one pile of junk afterwards.&Procedural; Solution-oriented; Gives direction to work.\\\hline
B&What about the wall?&Surface; Question.\\\hline
\end{tabular}
\end{table}
\subsubsection{Previous Results of Discussion Analysis}\label{prevdiscrev}
-An analysis of asynchronous online discussions has previously been carried out by the PIs, and we expect some of the results to transfer to in-class discussion settings.
-
+The MSU group previously carried out discussion analyses of learners collaborating around introductory Computer Science problems, as well as asynchronously around online homework problems in introductory Physics.
+While it is to be researched if any results transfer to in-class discussion settings, we are encouraged by statistically significant outcomes of these earlier studies.
In the study of online discussions~\cite{discpaper}, the majority of the discussion contributions were of type surface-level or procedural, followed by emotional
contributions. The vast majority of discussion contributions had the feature of being solution-oriented,
yet a considerable number dealt with the physics
@@ -436,7 +429,7 @@
\section{Research Phase}
\begin{figure} [t]
\begin{center}
-\includegraphics[width=5in]{overview.eps}
+\includegraphics[width=4.5in]{overview.eps}
\caption{Overview of the implementation process.\label{overview}
}
\end{center}
@@ -589,32 +582,23 @@
\section{Results from Prior NSF Support}\label{results}
Gerd Kortemeyer is PI on the current NSF-ITR grant Investigation of a Model for Online Resource Creation and Sharing in Educational Settings (\#0085921, \$2,055,000, September 15, 2000 through July 31, 2005), which uses LON-CAPA as its model system.
The project developed into a content sharing network of more than 15 institutions of higher education including community colleges and four-year institutions, as well as over 15 middle and high schools. In addition, LON-CAPA houses commercial textbook content from seven major publishing companies, and a commercial service company was established around the product at the end of 2004. The project maintains a gateway server to the National Science Digital Library, and the LON-CAPA shared resource pool is searchable and accessible from {\tt http://nsdl.org/}.
+LON-CAPA is used by approximately 25,000 students every semester.
Gerd Kortemeyer is Co-PIs on the current NSF-CCLI-ASA grant Diagnostic Question Clusters: Development and Testing in Introductory Geology and Biology (\#0243126, \$491,606, September 15, 2003 through August 31, 2006) to develop diagnostic questions for college students in both biology and geology. In the ASA project, a pool of peer-reviewed, diagnostic question clusters to assess students' understanding will be developed, including tools for analysis, peer review, and online publication of these question clusters.
The group of Eric Mazur has been supported by the National Science Foundation to develop, evaluate, and produce
-materials for effective undergraduate science pedagogy and assessment since 1993.
-Award No: DUE \#925407 Peer Instruction: Stimulating renewed interest in physics and other
-science and engineering courses
-Amount: \$223,500
-Dates: 2/1/93 - 1/31/96
-Award No: DUE \#9653438 Faculty Enhancement Conference: Teaching Physics, Conservation
-Laws First
-Amount: \$131,000
-Dates: 10/1/97 - 10/1/98
-Award No: DUE \#9554870 On-line server of educational resources
-Amount: \$800,000
-Dates: 3/1/96 - 3/1/00
-Award No: DUE \#9980802 Creating a community of Peer Instruction users: dissemination and
-electronic resources
-Amount: \$290,000
-Dates: 4/01/00 -3/31/02
-Award No: DUE \#123899 Distinguished Teaching Scholar Award: On-line Resources for
-Teaching With Peer Instruction
-Amount: \$305,000
-Dates: 9/15/01 - 8/31/05
-
-Under NSF sponsorship, we developed Project Galileo, a store of extensive resources forinteractive learning pedagogies, targeting both large and small classroom teaching techniques, which areavailable to the entire teaching community. Using funds from a NSF Director's Distinguished TeachingScholar Award, we created the Interactive Learning Toolkit, a learning management system that allowsinstructors to implement several proven innovative teaching techniques and to share and review materialsthey create for these techniques. The ILT is currently in use at a number of institutions nationwide,including Vanderbilt, University of Southern California, University of Massachusetts-Boston, Salem StateCollege, Massachusetts Institute of Technology, Swarthmore College, with a student user base of severalthousand students per semester.
+materials for effective undergraduate science pedagogy and assessment since 1993
+(DUE \#925407, Peer Instruction: Stimulating renewed interest in physics and other
+science and engineering courses,
+\$223,500,
+2/1/93 - 1/31/96).
+The group developed Project Galileo (DUE \#9554870, On-line server of educational resources, \$800,000, 3/1/96-3/1/00; DUE \#9980802 Creating a community of Peer Instruction users: dissemination and
+electronic resources, \$290,000, 4/01/00-3/31/02), a store of extensive resources for
+interactive learning pedagogies, targeting both large and small classroom teaching techniques, which areavailable to the entire teaching community.
+ Using funds from a NSF Director's Distinguished Teaching Scholar Award (DUE \#123899, Distinguished Teaching Scholar Award: On-line Resources for Teaching With Peer Instruction, \$305,000, 9/15/01-8/31/05),
+we created the Interactive Learning Toolkit, a learning management system that allows instructors to implement several proven innovative teaching techniques and to share and review materials they create for these techniques.
+The ILT is currently in use at a number of institutions nationwide, including Vanderbilt, University of Southern California, University of Massachusetts-Boston, Salem State College, Massachusetts Institute of Technology,
+Swarthmore College, with a student user base of several thousand students per semester.
%
% references
@@ -663,7 +647,7 @@
\bibitem{mazur96} Eric Mazur, {\it The Problem with Problems}, Optics and Photonics News {\bf 6}, 59-60 (1996)
-\bibitem{discpaper} Gerd Kortemeyer, {\it An Analysis of Asynchronous Online Homework Discussions in Introductory Physics Courses}, Am. J. Phys., submitted.
+\bibitem{discpaper} Gerd Kortemeyer, {\it An Analysis of Asynchronous Online Homework Discussions in Introductory Physics Courses}, Am. J. Phys., submitted. Available online at http://www.lon-capa.org/publications.html
\bibitem{physlets} Wolfgang Christian et al., {\it Physlets}, http://webphysics.davidson.edu/Applets/Applets.html
\bibitem{johnson} D. W. Johnson, R. T. Johnson, E. Holubec, {\it Circles of Learning: Cooperation in the Classroom}, Interaction Book Company (1990)
--www1116361099--