[LON-CAPA-cvs] cvs: modules /gerd/roleclicker description.aux description.tex
www
lon-capa-cvs@mail.lon-capa.org
Thu, 12 May 2005 15:06:06 -0000
This is a MIME encoded message
--www1115910366
Content-Type: text/plain
www Thu May 12 11:06:06 2005 EDT
Modified files:
/modules/gerd/roleclicker description.aux description.tex
Log:
Shortened
--www1115910366
Content-Type: text/plain
Content-Disposition: attachment; filename="www-20050512110606.txt"
Index: modules/gerd/roleclicker/description.aux
diff -u modules/gerd/roleclicker/description.aux:1.14 modules/gerd/roleclicker/description.aux:1.15
--- modules/gerd/roleclicker/description.aux:1.14 Wed May 11 16:27:43 2005
+++ modules/gerd/roleclicker/description.aux Thu May 12 11:06:05 2005
@@ -23,13 +23,14 @@
\newlabel{inventories}{{2.1.2}{3}}
\newlabel{subsec:problemcat}{{2.2}{3}}
\@writefile{toc}{\contentsline {subsection}{\numberline {2.2}Problem Classification}{3}}
-\citation{mcdermott}
-\citation{beichner}
\@writefile{lof}{\contentsline {figure}{\numberline {1}{\ignorespaces Example of two problems addressing the same concepts in two different representations. The problem on the left is an original ConcepTest problem of type single-response multiple-choice, the problem on the right is multiple-response multiple-choice. Both problems require representation translation since data is provided in graphical form.}}{4}}
\newlabel{repre}{{1}{4}}
\@writefile{lof}{\contentsline {figure}{\numberline {2}{\ignorespaces Example of two problems addressing the same concepts in two different representations. The problem on the left is an original ConcepTest problem of type single-response multiple-choice, the problem on the right is rank-response.}}{4}}
\newlabel{reprecoll}{{2}{4}}
\citation{mazur96}
+\citation{redish}
+\citation{mcdermott}
+\citation{beichner}
\citation{discpaper}
\citation{discpaper}
\citation{discpaper}
@@ -49,16 +50,11 @@
\citation{beichner}
\citation{kashyd01}
\citation{physlets}
+\citation{pascarella02}
\@writefile{lot}{\contentsline {table}{\numberline {2}{\ignorespaces Influence of question types and features on discussions. The values indicate the percentage prominence of the discussion superclasses, types, and features (columns) for discussions associated with questions of a certain type or with certain features (rows). The values in brackets result from an analysis with ``chat'' excluded.}}{8}}
\newlabel{table:qtype}{{2}{8}}
-\citation{pascarella02}
-\@writefile{lof}{\contentsline {figure}{\numberline {4}{\ignorespaces Pre- and post-discussion compiled from 5000 student-responses to 40 ConcepTests.}}{9}}
-\newlabel{beforeafter}{{4}{9}}
-\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.3.3}Procedure}{9}}
-\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.3.4}Interviews}{9}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {2.4}Outcome-Oriented Evaluation}{9}}
-\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.4.1}Pre-/Post-Discussion Answer Distribution}{9}}
-\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.4.2}Pre-/Post-Performance on Concept Inventories}{9}}
+\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.3.3}Procedure}{8}}
+\citation{kashyd01}
\citation{mref1}
\citation{mref2}
\citation{mref3}
@@ -70,6 +66,13 @@
\citation{mref9}
\citation{mref10}
\citation{mref13}
+\@writefile{lof}{\contentsline {figure}{\numberline {4}{\ignorespaces Pre- and post-discussion compiled from 5000 student-responses to 40 ConcepTests.}}{9}}
+\newlabel{beforeafter}{{4}{9}}
+\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.3.4}Interviews}{9}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {2.4}Outcome-Oriented Evaluation}{9}}
+\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.4.1}Pre-/Post-Discussion Answer Distribution}{9}}
+\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.4.2}Pre-/Post-Performance on Concept Inventories}{9}}
+\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.4.3}Correlation of In-Class and Exam Performance}{9}}
\citation{mref19}
\citation{mref20}
\citation{mref21}
@@ -81,50 +84,50 @@
\citation{mref28}
\citation{mref11}
\citation{mref12}
+\citation{mref11}
\@writefile{lof}{\contentsline {figure}{\numberline {5}{\ignorespaces Pre- and post-scores on the Force Concept Inventory of three courses at Harvard.}}{10}}
\newlabel{prepostfci}{{5}{10}}
-\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.4.3}Correlation of In-Class and Exam Performance}{10}}
\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.4.4}Previous Results}{10}}
-\citation{mref11}
\citation{physlets}
\@writefile{toc}{\contentsline {section}{\numberline {3}Materials Devolopment}{11}}
\newlabel{matdev}{{3}{11}}
\@writefile{toc}{\contentsline {subsection}{\numberline {3.1}Existing Material}{11}}
\@writefile{toc}{\contentsline {subsection}{\numberline {3.2}Porting of Content}{11}}
\newlabel{porting}{{3.2}{11}}
+\@writefile{toc}{\contentsline {section}{\numberline {4}Implementation}{11}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {4.1}Existing System Functionality}{11}}
+\newlabel{existing}{{4.1}{11}}
+\@writefile{toc}{\contentsline {subsubsection}{\numberline {4.1.1}Interactive Learning Toolkit (ILT)}{11}}
\citation{bq1}
\citation{bq2}
-\@writefile{toc}{\contentsline {section}{\numberline {4}Implementation}{12}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {4.1}Existing System Functionality}{12}}
-\newlabel{existing}{{4.1}{12}}
-\@writefile{toc}{\contentsline {subsubsection}{\numberline {4.1.1}Interactive Learning Toolkit (ILT)}{12}}
+\citation{features}
+\citation{edutools}
\@writefile{toc}{\contentsline {subsubsection}{\numberline {4.1.2}Beyond Question (BQ)}{12}}
\@writefile{toc}{\contentsline {subsubsection}{\numberline {4.1.3}ILT/BQ integration}{12}}
\@writefile{toc}{\contentsline {subsubsection}{\numberline {4.1.4}The Learning{\it Online} Network with CAPA (LON-CAPA)}{12}}
\newlabel{loncapa}{{4.1.4}{12}}
-\citation{features}
-\citation{edutools}
-\@writefile{toc}{\contentsline {subsection}{\numberline {4.2}Resource-Pool Organization}{13}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {4.3}Computer-Guided Group Formation}{13}}
-\newlabel{groupform}{{4.3}{13}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {4.4}Different Question Types}{13}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {4.5}Randomized Questions}{13}}
-\@writefile{lof}{\contentsline {figure}{\numberline {6}{\ignorespaces Three randomized variations of the problems in Fig.\nobreakspace {}1\hbox {} and 2\hbox {}. The graphs in the emf problem are dynamically generated on-the-fly, the cars and the brick wall in the collision problem are randomly selected images. Each randomization leads to different answers for different students. It should be noted that in this particular example, the collision problem became more difficult if the learner realized early on that in all combinations of Fig.\nobreakspace {}2\hbox {}, the combined object after the collision will be at rest -- the corresponding constraints could have been implemented in the randomizing problem. }}{14}}
-\newlabel{rando}{{6}{14}}
-\@writefile{lof}{\contentsline {figure}{\numberline {7}{\ignorespaces Computer-guided group formation. }}{15}}
-\newlabel{formation}{{7}{15}}
+\@writefile{lof}{\contentsline {figure}{\numberline {6}{\ignorespaces Three randomized variations of the problems in Fig.\nobreakspace {}1\hbox {} and 2\hbox {}. The graphs in the emf problem are dynamically generated on-the-fly, the cars and the brick wall in the collision problem are randomly selected images. Each randomization leads to different answers for different students. It should be noted that in this particular example, the collision problem became more difficult if the learner realized early on that in all combinations of Fig.\nobreakspace {}2\hbox {}, the combined object after the collision will be at rest -- the corresponding constraints could have been implemented in the randomizing problem. }}{13}}
+\newlabel{rando}{{6}{13}}
+\@writefile{lof}{\contentsline {figure}{\numberline {7}{\ignorespaces Computer-guided group formation. }}{14}}
+\newlabel{formation}{{7}{14}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {4.2}Resource-Pool Organization}{14}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {4.3}Computer-Guided Group Formation}{14}}
+\newlabel{groupform}{{4.3}{14}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {4.4}Different Question Types}{14}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {4.5}Randomized Questions}{14}}
+\newlabel{randomques}{{4.5}{14}}
\@writefile{lof}{\contentsline {figure}{\numberline {8}{\ignorespaces Rendering of a problem on PDA devices }}{15}}
\newlabel{pdaview}{{8}{15}}
+\@writefile{toc}{\contentsline {section}{\numberline {5}Dissemination}{15}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {5.1}Commodization Phase}{15}}
\@writefile{lot}{\contentsline {table}{\numberline {3}{\ignorespaces Proposed timeline by year and institution}}{16}}
\newlabel{timeline}{{3}{16}}
-\@writefile{toc}{\contentsline {section}{\numberline {5}Dissemination}{16}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {5.1}Commodization Phase}{16}}
\@writefile{toc}{\contentsline {section}{\numberline {6}Timeline}{16}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {6.1}Year 1}{17}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {6.2}Year 2}{17}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {6.3}Year 3}{17}}
-\@writefile{toc}{\contentsline {section}{\numberline {7}Expertise and Responsibilites of the PIs}{17}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {7.1}Gerd Kortemeyer}{17}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {6.1}Year 1}{16}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {6.2}Year 2}{16}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {6.3}Year 3}{16}}
+\@writefile{toc}{\contentsline {section}{\numberline {7}Expertise and Responsibilites of the PIs}{16}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {7.1}Gerd Kortemeyer}{16}}
\@writefile{toc}{\contentsline {subsection}{\numberline {7.2}Guy Albertelli}{17}}
\@writefile{toc}{\contentsline {section}{\numberline {8}Results from Prior NSF Support}{17}}
\newlabel{results}{{8}{17}}
Index: modules/gerd/roleclicker/description.tex
diff -u modules/gerd/roleclicker/description.tex:1.22 modules/gerd/roleclicker/description.tex:1.23
--- modules/gerd/roleclicker/description.tex:1.22 Wed May 11 16:27:43 2005
+++ modules/gerd/roleclicker/description.tex Thu May 12 11:06:05 2005
@@ -102,10 +102,11 @@
\subsection{Intellectual Merit}
Peer-instruction has proven successful in outcome-oriented evaluations of techniques limited by currently widely available technology. This project will add process-oriented data to the research body around peer-instruction and study the effect of extensions to this technique, which we believe can significantly change both the process and the outcome of applying these techniques.
\subsection{Broader Impact}
-Currently, every semester approximately 350,000 US students are taking introductory undergraduate physics courses similar to the ones under investigation in this project~\cite{aapt}. For many of these students, it is both their first and their last formal exposure to physics. Students will go into a large spectrum of careers, with or without an understanding of the basic concepts of the physical world.
+Currently, every semester approximately 350,000 US students are taking introductory undergraduate physics courses similar to at least one of the courses at the participating institutions~\cite{aapt}.
+For many of these students, it is both their first and their last formal exposure to physics. Students will go into a large spectrum of careers, with or without an understanding of the basic concepts of the physical world.
-This project has the potential of broader impact, since like many of the other efforts in Physics Education, it is carried out within a regular college
-venue. The three participating institutions, an ivy league school, a large state university, and a small liberal arts college, host different student
+This project has the potential of broader impact, since like many of the other efforts in Physics Education Research (PER), it is carried out within regular college
+venues. The three participating institutions, an ivy league school, a large state university, and a small liberal arts college, host different student
populations and offer different teaching environments. This allows to study a wide and diverse range of educational settings. There is also a regional
component, since the collaborating schools come from three culturally different parts of the United States.
@@ -136,12 +137,10 @@
and certain questions, responses differ greatly.
For this project, we are choosing a finer-grained classification scheme of question types (adapted from Redish~\cite{redish}):
\begin{description}
-\item[Multiple-choice and short-answer questions] The most basic and most easily computer-evaluated type of question, representing the conventional (typical back-of-chapter textbook) problem.
-
-For the purposes of this project, ``multiple choice" and ``short-answer" will be considered as separate classes, where short-answer includes numerical answers such as ``\verb!17 kg/m^3!," and formula answers, such as ``\verb!1/2*m*(vx^2+vy^2)!."
-Short-answer multiple choice questions are those where only one option is correct (``single-response multiple-choice''),
+\item[Single-Response Multiple-Choice] The most basic and most easily computer-evaluated type of question, where only one option is correct,
see for example the original ConcepTest problems on the left of Figs.~\ref{repre} and \ref{reprecoll}.
-\item[Multiple-response multiple-choice questions] This type of problem, a first step beyond conventional problems, requires a student to evaluate each statement and make a decision about it. The problem on the right side of Fig.~\ref{repre} is of this type.
+
+\item[Multiple-Response Multiple-Choice] This type of problem, a first step beyond conventional problems, requires a student to evaluate each statement and make a decision about it. The problem on the right side of Fig.~\ref{repre} is of this type.
\begin{figure}
@@ -151,6 +150,10 @@
\caption{Example of two problems addressing the same concepts in two different representations. The problem on the left is an original ConcepTest problem of type single-response multiple-choice, the problem on the right is multiple-response multiple-choice. Both problems require representation translation since data is provided in graphical form.\label{repre}}
\end{figure}
+\item[Numerical Short Answer] Numerical answers (potentially in multiple dimensions and including physical units), such as ``\verb!17 kg/m^3!" are expected.
+\item[Formula Short Answer] Mathematical expressions (potentially in multiple dimensions), such as ``\verb!1/2*m*(vx^2+vy^2)!" are expected.
+
+
\begin{figure}
\includegraphics[width=8cm]{collOrig}
\includegraphics[width=8cm]{collRankNR}
@@ -158,21 +161,23 @@
\caption{Example of two problems addressing the same concepts in two different representations. The problem on the left is an original ConcepTest problem of type single-response multiple-choice, the problem on the right is rank-response.\label{reprecoll}}
\end{figure}
-\item[Representation-translation questions] This type of problem requires a student to translate between different representations of the same situation, for example from a graphical to a numerical or textual representation. The answer might be required in different formats, see for example
-Fig.~\ref{repre}, it is a short-numerical-answer. Translation between representations can be surprisingly challenging for physics learners~\cite{mcdermott,beichner}.
-
-For the purposes of this project, ``representation-translation" will be considered a feature, which may or may not apply to any of the other problem types.
-
\item[Ranking-tasks] This type of problem requires a student to rank a number of statements, scenarios, or objects with respect to a certain feature. For example, a student might be asked to rank a number of projectiles in the order that they will hit the ground, or a number of locations in order of the strength of their local electric potential.
Several options may have the same rank (``tie'').
The left panel of Fig.~\ref{reprecoll} is of this type.
-\item[Context-based reasoning problems] The distinguishing characteristic of these problems is that they are set in the context of real-world scenarios and not in the context of the artificial ``zero-friction" laboratory scenarios of typical textbook problems.
-As in the case of ``representation-translation," ``context-based-reasoning" in this project will be considered a feature, which may apply or may not apply to any of the other problem types.
-\item[Estimation problems], also known as ``Fermi Problems," require the student to form a model for a scenario, and make reasonable assumptions. A typical example is ``How many barbers are there in Chicago?" or ``How long will I have to wait to find a parking spot?" Students do need to explain their reasoning.
+\item[Estimation problems], also known as ``Fermi Problems," require the student to form a model for a scenario, and make reasonable assumptions. A typical example is ``How many barbers are there in Chicago?" or ``How long will I have to wait to find a parking spot?"
+
+This problem type cannot be fully computer-graded, since in a full implementation, students do need to explain their reasoning. However, we do plan to poll the courses on expected results, and have
+learners explain their reasoning to each other. Displayed results of such polls can be used as a starting point for further course-wide discussions.
While students find it initially hard to believe that these questions have anything to do with physics, hardly any expert physicist would deny their significance in learning how to solve problems~\cite{mazur96}.
\end{description}
+In addition, we consider the following features, which may or may not apply to any question type (adapted from Redish~\cite{redish}):
+\begin{description}
+\item[Representation-Translation] This type of problem requires a student to translate between different representations of the same situation, for example from a graphical to a numerical or textual representation. The answer might be required in different formats, see for example
+Fig.~\ref{repre}. Translation between representations can be surprisingly challenging for physics learners~\cite{mcdermott,beichner}.
+\item[Context-based Reasoning] The distinguishing characteristic of these problems is that they are set in the context of real-world scenarios and not in the context of the artificial ``zero-friction" laboratory scenarios of typical textbook problems.
+\end{description}
The same classification scheme was successfully used for an analysis of online asynchronous discussions~\cite{discpaper}.
\subsection{Process-Oriented Evaluation}
The process-oriented evaluation will focus on the actual discussion process. Since currently no baseline data exists for this study, we will assess the quality of student discussion both
@@ -255,15 +260,15 @@
contributions. This number would be negative if the problem led to mostly negative emotional comments.
\item[Type and feature sums] - number of all related contributions belonging to a certain type, subtype, or feature.
\end{description}
+\subsubsection{Previous Results of Discussion Analysis}\label{prevdiscrev}
+An analysis of asynchronous online discussions has previously been carried out by the PIs, and we expect some of the results to transfer to in-class discussion settings.
In the study of online discussions~\cite{discpaper}, the majority of the discussion contributions were of type surface-level or procedural, followed by emotional
-contributions.
-The vast majority of discussion contributions had the feature of being solution-oriented,
+contributions. The vast majority of discussion contributions had the feature of being solution-oriented,
yet a considerable number dealt with the physics
of the problems.
-\subsubsection{Previous Results of Discussion Analysis}\label{prevdiscrev}
-An analysis of asynchronous online discussions has previously been carried out by the PIs, and we expect some of the results to transfer to in-class discussion settings.
+
\begin{figure}
\includegraphics[width=86mm]{KortemeyerFig5}% Here is how to import EPS art
\caption{\label{fig:gradecorrel}Prominance of discussion superclasses by grade.}
@@ -274,14 +279,10 @@
Figure~\ref{fig:gradecorrel}
shows the outcome of this study by discussion superclass. As an example, the figure is to be interpreted this way: within the indicated errors,
55 percent of a 3.0 student's discussion contributions were solution-oriented. The lines represent second-order polynomial fits to the data.
-\begin{itemize}
-\item The relative prominance of solution-oriented discussion contributions varies most strongly with grade, from 75 percent for a 2.0 student to 45 percent for a 4.0 student.
-\item The relative prominance of physics-related and conceptual discussion contributions on the other hand increases with grade.
-\item The relative prominance of procedural discussions does not vary significantly with grades and is consistent with 42 percent promimance across grades and gender, except for the 23 female 4.0 students, where it is $68\pm7$ percent --- the 22 male 4.0 students, by comparison, average $34\pm8$ percent procedural discussions.
-\end{itemize}
-The results are not surprising, but verify the validity of the classification approach.
-At the same time, the results confirm that conceptual and physics-related discussions are positively correlated with success in the course, while solution-oriented discussion contributions are strongly negatively correlated. While cause and effect may be arguable, in this project,
+The results are not surprising, but verify the validity of the classification approach. At the same time, the results confirm that conceptual and
+physics-related discussions are positively correlated with success in the course, while solution-oriented discussion contributions are strongly negatively correlated.
+While cause and effect may be arguable, in this project,
particular attention needs to be paid to question properties that elicit either the desirable or undesirable discussion patterns.
\item[Influence of Question Types] -
Each question was classified according to the types and features described in subsection~\ref{subsec:problemcat}, and each associated discussion entry according to~\ref{subsec:disccat}. As a measure of the prominence of a class in a given discussion,
@@ -348,8 +349,10 @@
\subsection{Outcome-Oriented Evaluation}
\subsubsection{Pre-/Post-Discussion Answer Distribution}
\begin{figure}
-\includegraphics[width=2.5in]{before}
-\includegraphics[width=2.5in]{after}
+\begin{center}
+\includegraphics[width=2in]{before}
+\includegraphics[width=2in]{after}
+\end{center}
\caption{Pre- and post-discussion compiled from 5000 student-responses to 40 ConcepTests.\label{beforeafter}}
\end{figure}
@@ -371,14 +374,15 @@
interactive course. A more complete picture would also take into account the different forms of Peer Instruction proposed.
\begin{figure}
-\includegraphics[width=2.5in]{fcipre}
-\includegraphics[width=2.5in]{fcipost}
-
+\begin{center}
+\includegraphics[width=2in]{fcipre}
+\includegraphics[width=2in]{fcipost}
+\end{center}
\caption{Pre- and post-scores on the Force Concept Inventory of three courses at Harvard.\label{prepostfci}}
\end{figure}
\subsubsection{Correlation of In-Class and Exam Performance}
-A capability of our systems is that we can use the same question without modifications in online and bubble-sheet exam mode. Since in addition, questions are randomizing, we are able to include some of the same questions used in class on exams and quizzes. A similar study was previously conducted by Kashy~\ref{kashyd01} for homework questions.
+A capability of our systems is that we can use the same question without modifications in online and bubble-sheet exam mode. Since in addition, questions are randomizing, we are able to include some of the same questions used in class on exams and quizzes. A similar study was previously conducted by Kashy~\cite{kashyd01} for homework questions.
\subsubsection{Previous Results}
We have applied
peer-instruction in both the calculus-based and the algebra-based introductory physics courses for non-majors
@@ -554,7 +558,7 @@
The system records group configurations and makes analyses of pre- and post-discussion responses within the groups possible.
\subsection{Different Question Types}
-
+Key to deploying both different question types and randomizing questions (see following subsection~\ref{randomques}) are two-way interactive devices.
\begin{figure}
\includegraphics[width=3.5in]{dell.eps}
\includegraphics[width=2.7in]{sharp2.eps}
@@ -562,8 +566,9 @@
}
\end{figure}
-\subsection{Randomized Questions}
-The LON-CAPA system is already capable of randomizing questions with a wide range of options and also provides statistical means to de-randomized analysis of responses.
+\subsection{Randomized Questions}\label{randomques}
+The LON-CAPA system is already capable of randomizing questions with a wide range of options and also provides statistical tools for the de-randomized analysis of responses. For classroom use, specialized tools similar to the existing ILT functionality need to be developed to provide the
+instructor with a quick and comprehensive overview of response patterns of the more complex randomizing question types.
\section{Dissemination}
--www1115910366--