[LON-CAPA-cvs] cvs: modules /gerd/roleclicker description.aux description.tex
www
lon-capa-cvs@mail.lon-capa.org
Thu, 12 May 2005 18:05:22 -0000
This is a MIME encoded message
--www1115921122
Content-Type: text/plain
www Thu May 12 14:05:22 2005 EDT
Modified files:
/modules/gerd/roleclicker description.aux description.tex
Log:
Shortened methodoolgy, fixed typos
--www1115921122
Content-Type: text/plain
Content-Disposition: attachment; filename="www-20050512140522.txt"
Index: modules/gerd/roleclicker/description.aux
diff -u modules/gerd/roleclicker/description.aux:1.16 modules/gerd/roleclicker/description.aux:1.17
--- modules/gerd/roleclicker/description.aux:1.16 Thu May 12 11:10:55 2005
+++ modules/gerd/roleclicker/description.aux Thu May 12 14:05:22 2005
@@ -44,14 +44,17 @@
\newlabel{prevdiscrev}{{2.3.2}{6}}
\citation{kashyd01}
\citation{discpaper}
-\@writefile{lof}{\contentsline {figure}{\numberline {3}{\ignorespaces Prominance of discussion superclasses by grade.}}{7}}
-\newlabel{fig:gradecorrel}{{3}{7}}
-\@writefile{lot}{\contentsline {table}{\numberline {2}{\ignorespaces Influence of question types and features on discussions. The values indicate the percentage prominence of the discussion superclasses, types, and features (columns) for discussions associated with questions of a certain type or with certain features (rows). The values in brackets result from an analysis with ``chat'' excluded.}}{7}}
-\newlabel{table:qtype}{{2}{7}}
+\citation{discpaper}
+\citation{kashyd01}
\citation{mcdermott}
\citation{beichner}
\citation{kashyd01}
+\citation{discpaper}
\citation{physlets}
+\@writefile{lof}{\contentsline {figure}{\numberline {3}{\ignorespaces Prominance of discussion superclasses by student grade (left panel) and question difficulty (right panel).}}{7}}
+\newlabel{fig:gradecorrel}{{3}{7}}
+\@writefile{lot}{\contentsline {table}{\numberline {2}{\ignorespaces Influence of question types and features on discussions. The values indicate the percentage prominence of the discussion superclasses, types, and features (columns) for discussions associated with questions of a certain type or with certain features (rows). The values in brackets result from an analysis with ``chat'' excluded.}}{7}}
+\newlabel{table:qtype}{{2}{7}}
\citation{pascarella02}
\@writefile{lof}{\contentsline {figure}{\numberline {4}{\ignorespaces Pre- and post-discussion compiled from 5000 student-responses to 40 ConcepTests.}}{8}}
\newlabel{beforeafter}{{4}{8}}
@@ -59,6 +62,7 @@
\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.3.4}Interviews}{8}}
\@writefile{toc}{\contentsline {subsection}{\numberline {2.4}Outcome-Oriented Evaluation}{8}}
\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.4.1}Pre-/Post-Discussion Answer Distribution}{8}}
+\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.4.2}Pre-/Post-Performance on Concept Inventories}{8}}
\citation{kashyd01}
\citation{mref1}
\citation{mref2}
@@ -73,57 +77,56 @@
\citation{mref13}
\citation{mref19}
\citation{mref20}
-\@writefile{lof}{\contentsline {figure}{\numberline {5}{\ignorespaces Pre- and post-scores on the Force Concept Inventory of three courses at Harvard.}}{9}}
-\newlabel{prepostfci}{{5}{9}}
-\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.4.2}Pre-/Post-Performance on Concept Inventories}{9}}
-\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.4.3}Correlation of In-Class and Exam Performance}{9}}
-\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.4.4}Previous Results}{9}}
\citation{mref21}
\citation{mref22}
\citation{mref23}
\citation{mref25}
\citation{mref26}
+\@writefile{lof}{\contentsline {figure}{\numberline {5}{\ignorespaces Pre- and post-scores on the Force Concept Inventory of three courses at Harvard.}}{9}}
+\newlabel{prepostfci}{{5}{9}}
+\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.4.3}Correlation of In-Class and Exam Performance}{9}}
+\@writefile{toc}{\contentsline {subsubsection}{\numberline {2.4.4}Previous Results}{9}}
\citation{mref27}
\citation{mref28}
\citation{mref11}
\citation{mref12}
\citation{mref11}
+\citation{physlets}
\@writefile{toc}{\contentsline {section}{\numberline {3}Materials Devolopment}{10}}
\newlabel{matdev}{{3}{10}}
\@writefile{toc}{\contentsline {subsection}{\numberline {3.1}Existing Material}{10}}
-\citation{physlets}
+\@writefile{toc}{\contentsline {subsection}{\numberline {3.2}Porting of Content}{10}}
+\newlabel{porting}{{3.2}{10}}
\citation{bq1}
\citation{bq2}
-\@writefile{toc}{\contentsline {subsection}{\numberline {3.2}Porting of Content}{11}}
-\newlabel{porting}{{3.2}{11}}
\@writefile{toc}{\contentsline {section}{\numberline {4}Implementation}{11}}
\@writefile{toc}{\contentsline {subsection}{\numberline {4.1}Existing System Functionality}{11}}
\newlabel{existing}{{4.1}{11}}
\@writefile{toc}{\contentsline {subsubsection}{\numberline {4.1.1}Interactive Learning Toolkit (ILT)}{11}}
\@writefile{toc}{\contentsline {subsubsection}{\numberline {4.1.2}Beyond Question (BQ)}{11}}
\@writefile{toc}{\contentsline {subsubsection}{\numberline {4.1.3}ILT/BQ integration}{11}}
+\@writefile{toc}{\contentsline {subsubsection}{\numberline {4.1.4}The Learning{\it Online} Network with CAPA (LON-CAPA)}{11}}
+\newlabel{loncapa}{{4.1.4}{11}}
\citation{features}
\citation{edutools}
-\@writefile{toc}{\contentsline {subsubsection}{\numberline {4.1.4}The Learning{\it Online} Network with CAPA (LON-CAPA)}{12}}
-\newlabel{loncapa}{{4.1.4}{12}}
\@writefile{toc}{\contentsline {subsection}{\numberline {4.2}Resource-Pool Organization}{12}}
\@writefile{toc}{\contentsline {subsection}{\numberline {4.3}Computer-Guided Group Formation}{12}}
\newlabel{groupform}{{4.3}{12}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {4.4}Different Question Types}{12}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {4.5}Randomized Questions}{12}}
+\newlabel{randomques}{{4.5}{12}}
\@writefile{lof}{\contentsline {figure}{\numberline {6}{\ignorespaces Three randomized variations of the problems in Fig.\nobreakspace {}1\hbox {} and 2\hbox {}. The graphs in the emf problem are dynamically generated on-the-fly, the cars and the brick wall in the collision problem are randomly selected images. Each randomization leads to different answers for different students. It should be noted that in this particular example, the collision problem became more difficult if the learner realized early on that in all combinations of Fig.\nobreakspace {}2\hbox {}, the combined object after the collision will be at rest -- the corresponding constraints could have been implemented in the randomizing problem. }}{13}}
\newlabel{rando}{{6}{13}}
\@writefile{lof}{\contentsline {figure}{\numberline {7}{\ignorespaces Computer-guided group formation. }}{14}}
\newlabel{formation}{{7}{14}}
\@writefile{lof}{\contentsline {figure}{\numberline {8}{\ignorespaces Rendering of a problem on PDA devices }}{14}}
\newlabel{pdaview}{{8}{14}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {4.4}Different Question Types}{14}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {4.5}Randomized Questions}{14}}
-\newlabel{randomques}{{4.5}{14}}
-\@writefile{toc}{\contentsline {section}{\numberline {5}Dissemination}{14}}
\@writefile{lot}{\contentsline {table}{\numberline {3}{\ignorespaces Proposed timeline by year and institution}}{15}}
\newlabel{timeline}{{3}{15}}
+\@writefile{toc}{\contentsline {section}{\numberline {5}Dissemination}{15}}
\@writefile{toc}{\contentsline {subsection}{\numberline {5.1}Commodization Phase}{15}}
\@writefile{toc}{\contentsline {section}{\numberline {6}Timeline}{15}}
-\@writefile{toc}{\contentsline {subsection}{\numberline {6.1}Year 1}{15}}
+\@writefile{toc}{\contentsline {subsection}{\numberline {6.1}Year 1}{16}}
\@writefile{toc}{\contentsline {subsection}{\numberline {6.2}Year 2}{16}}
\@writefile{toc}{\contentsline {subsection}{\numberline {6.3}Year 3}{16}}
\@writefile{toc}{\contentsline {section}{\numberline {7}Expertise and Responsibilites of the PIs}{16}}
Index: modules/gerd/roleclicker/description.tex
diff -u modules/gerd/roleclicker/description.tex:1.25 modules/gerd/roleclicker/description.tex:1.26
--- modules/gerd/roleclicker/description.tex:1.25 Thu May 12 12:27:22 2005
+++ modules/gerd/roleclicker/description.tex Thu May 12 14:05:22 2005
@@ -164,7 +164,7 @@
\item[Ranking-tasks] This type of problem requires a student to rank a number of statements, scenarios, or objects with respect to a certain feature. For example, a student might be asked to rank a number of projectiles in the order that they will hit the ground, or a number of locations in order of the strength of their local electric potential.
Several options may have the same rank (``tie'').
The left panel of Fig.~\ref{reprecoll} is of this type.
-
+\item[Click-on-Image] Learners need to click on different parts in an image, for example on where to cut a wire in order to brighten up a lightbulb elsewhere in a circuit diagram.
\item[Estimation problems], also known as ``Fermi Problems," require the student to form a model for a scenario, and make reasonable assumptions. A typical example is ``How many barbers are there in Chicago?" or ``How long will I have to wait to find a parking spot?"
This problem type cannot be fully computer-graded, since in a full implementation, students do need to explain their reasoning. However, we do plan to poll the courses on expected results, and have
@@ -270,8 +270,11 @@
\begin{figure}
-\includegraphics[width=86mm]{KortemeyerFig5}% Here is how to import EPS art
-\caption{\label{fig:gradecorrel}Prominance of discussion superclasses by grade.}
+\begin{center}
+\includegraphics[width=80mm]{KortemeyerFig5}% Here is how to import EPS art
+\includegraphics[width=80mm]{KortemeyerFig6}
+\end{center}
+\caption{\label{fig:gradecorrel}Prominance of discussion superclasses by student grade (left panel) and question difficulty (right panel).}
\end{figure}
\begin{description}
\item[Student Course Grade] -
@@ -294,11 +297,10 @@
\scriptsize
\begin{tabular}{p{2.3cm}cccccc}
&Emot. Clim.&Procedural&Solution&Math&Physics&Conceptual\\
-Multiple Choice&-5$\pm$3&28$\pm$7 (29$\pm$8)&66$\pm$7 (74$\pm$7)&9$\pm$6 (9$\pm$6)&16$\pm$5 (17$\pm$5)&6$\pm$3 (7$\pm$3)\\
-Short Textual&&&&&&\\
+Multi.-Choice\newline Single-Resp.&-5$\pm$3&28$\pm$7 (29$\pm$8)&66$\pm$7 (74$\pm$7)&9$\pm$6 (9$\pm$6)&16$\pm$5 (17$\pm$5)&6$\pm$3 (7$\pm$3)\\
+Multi.-Choice\newline Multi.-Resp.&1$\pm$1&15$\pm$3 (16$\pm$3)&66$\pm$4 (72$\pm$4)&1$\pm$1 (2$\pm$2)&22$\pm$3 (26$\pm$3)&14$\pm$2 (18$\pm$3)\\
Numerical&4$\pm$1&48$\pm$1 (57$\pm$1)&52$\pm$1 (63$\pm$2)&8$\pm$1 (9$\pm$1)&23$\pm$1 (27$\pm$1)&7$\pm$1 (8$\pm$1)\\
Formula&6$\pm$8&29$\pm$11 (31$\pm$10)&57$\pm$16 (64$\pm$18)&31$\pm$16 (36$\pm$18)&&\\
-Mult.-choice\newline Mult.-resp.&1$\pm$1&15$\pm$3 (16$\pm$3)&66$\pm$4 (72$\pm$4)&1$\pm$1 (2$\pm$2)&22$\pm$3 (26$\pm$3)&14$\pm$2 (18$\pm$3)\\
Ranking&2$\pm$3&24$\pm$11 (26$\pm$12)&41$\pm$18 (46$\pm$20)&&52$\pm$20 (54$\pm$20)&38$\pm$18 (39$\pm$17)\\
Click-on-Image&0$\pm$9&14$\pm$6 (18$\pm$8)&53$\pm$8 (69$\pm$11)&3$\pm$3 (5$\pm$5)&25$\pm$11 (26$\pm$11)&22$\pm$8 (25$\pm$9)\\\hline
@@ -306,34 +308,22 @@
Rep-Trans&-2$\pm$2&37$\pm$4 (45$\pm$4)&52$\pm$3 (63$\pm$4)&7$\pm$2 (9$\pm$2)&23$\pm$3 (28$\pm$3)&8$\pm$2 (10$\pm$2)\\
\end{tabular}
\end{table}
-Error boundaries on the emotional climate values are rather large and mostly include zero (neutral), indicating no significant preferences within the limited sample.
-Yet, students clearly dislike multiple-choice questions, while they clearly like numerical answer problems. The data also indicates that students prefer ``conventional'' over
-representation-translation problems.
-
-The prominence of procedural discussions is significantly higher for numerical problems than for any other problem types, and higher for ``conventional'' than for
-representation-translation problems. The latter difference vanishes when ``chat'' is excluded.
+Different question types result in different association discussion patterns. Discussions on a procedural level are more prominent for numerical problems than for any other problem type. Solution-oriented discussions are more prominent for multiple-choice style questions in an effort to short-circuit the conceptual reasoning: it was found that students in this simple question type use the discussion space to reverse-engineer the randomization process by copying-and-pasting the their correct solutions (see the surface-level solution-oriented example in Table~\ref{table:examples}).
-Solution-oriented contributions are significantly higher for multiple-choice and multiple-choice-multiple-response problems than for the other problem types with the exception
-of formula-response questions, where error-boundaries overlap. In spite of the randomization provided, in discussion entries, students frequently reverse-engineered the complete randomization space by copying their correct answer screens into the discussions
-(see the example for a surface-level solution-oriented discussion entry in Table~\ref{table:examples}). The same behaviour is to be expected in in-class discussions.
-
-The prominence of mathematical discussion contributions is the highest for formula-response questions, approximately equal for numerical and single-response multiple-choice questions, and the lowest for multiple-choice-multiple-response, ranking, and click-on-image questions.
-
-The prominence of physics-related discussion contributions was the highest for ranking and click-on-image problems, and the lowest for multiple-choice questions.
-
-Finally, when it comes to conceptual discussions, their prominence is significantly lower in single-response multiple-choice (the type currently used in peer-instruction)
+The prominance of conceptual discussions is significantly lower in single-response multiple-choice (the type currently used in peer-instruction)
and numerical problems than in the other problem types. In the
earlier study by Kashy~\cite{kashyd01}, it was also found that mastery of these same question types does not predict overall performance on the final exam as well as other question types.
-Multiple-choice problems that do not involve numbers are frequently called ``conceptual'' questions, but in~\cite{discpaper}, it was found that they do not necessarily lead to conceptual discussions.
-
-It is a surprising result that the only significant difference between ``conventional'' and representation-translation problems is that students discuss slightly less procedure in favor of
-more complaints, and that differences disappear when ``chat'' is excluded from the analysis. McDermott~\cite{mcdermott} and Beichner~\cite{beichner} on the other hand found that students have unexpected difficulties in translating for example data presented as graphs, so a stronger effect of this feature was expected. In additon, Kashy~\cite{kashyd01} found that mastery of representation-translation problems
-is the best predictor of final exam scores, even when controlling for ACT, cumulative GPA, and force-concept inventory pretests.
-Discussion behavior and final exam performance are clearly different measurements for the influence of problem types and do not necessarily need to correlate, but a connection between
+Multiple-choice problems that do not involve numbers are frequently simply called ``conceptual'' questions, but in~\cite{discpaper},
+it was found that they do not {\it necessarily} lead to conceptual discussions.
+Ranking questions showed very favorable discussion patterns, but their sample size in~\cite{discpaper} has been too small to make definitive statements.
individual discussion behavior and performance in the course clearly exists.
-It should be noted that the earlier study dealt with a relatively small set of
-representation-translation problems, some of which involved non-static time-evolving simulations as data-source, while in this study, none of the simulation-based problems were assigned. Within this project, we aim to deploy Physlets~\cite{physlets} in the classroom, and expect statistically more significant data regarding their impact.
+Differences between ``conventional'' and representation-translation problems were small, while Kashy~\cite{kashyd01} found significant differences, and McDermott~\cite{mcdermott} and Beichner~\cite{beichner} pointed out the significance of the translation step. It should be noted that the earlier study~\cite{kashyd01} dealt with a relatively small set of
+representation-translation problems, some of which involved non-static time-evolving simulations as data-source, while in~\cite{discpaper}, none of the simulation-based problems
+were assigned. Within this project, we aim to deploy Physlets~\cite{physlets} in the classroom, and expect statistically more significant data regarding their impact.
+\item[Influence of Question Difficulty] The right panel of Fig.~\ref{fig:gradecorrel} shows the prominance of different discussion contributions as a function of question
+difficulty. Very easy problems can elicit a high level conceptual discussions,
+and so can problems of mid-range difficulty. As problems become more difficult, there is no significant gain in conceptual discussions.
\end{description}
\subsubsection{Procedure}
@@ -598,7 +588,7 @@
As a derivative of the existing systems, the software will continue to be made available as open-source freeware. Generated content will be made available in the shared
LON-CAPA resource pool at the cross-institutional bottom layer of the system architecture.
-In the intitial phase, the Interactive Learning Tollkit will be integrated with BQ, forming a new software package called LT3 (Learning Togehther Through Technology). The BQ
+In the initial phase, the Interactive Learning Toolkit will be integrated with BQ, forming a new software package called LT3 (Learning Together Through Technology). The BQ
program will be transformed to a LT3 laptop-based front end. This part will host the complete Interactive Classroom features of LT3. The ILT part will continue to be a
server based software. The LT3/ILT will continue to be the environment where the instructor can prepare the content for his class by browsing the ConceptTest data base and integrate CTs into a
given lecture. An important step will be possibility to upload the CTs into the LT3/BQ part for one lecture at a time and to automatically prepare them for the Interactive Classroom. After each lecture,
--www1115921122--