[LON-CAPA-cvs] cvs: modules /gerd/discussions/paper discussions.tex
www
lon-capa-cvs@mail.lon-capa.org
Mon, 04 Apr 2005 15:56:22 -0000
This is a MIME encoded message
--www1112630182
Content-Type: text/plain
www Mon Apr 4 11:56:22 2005 EDT
Modified files:
/modules/gerd/discussions/paper discussions.tex
Log:
Additional writing
--www1112630182
Content-Type: text/plain
Content-Disposition: attachment; filename="www-20050404115622.txt"
Index: modules/gerd/discussions/paper/discussions.tex
diff -u modules/gerd/discussions/paper/discussions.tex:1.6 modules/gerd/discussions/paper/discussions.tex:1.7
--- modules/gerd/discussions/paper/discussions.tex:1.6 Sun Apr 3 08:37:34 2005
+++ modules/gerd/discussions/paper/discussions.tex Mon Apr 4 11:56:22 2005
@@ -116,8 +116,8 @@
\item[Qualitative questions] This type of questions asks students to make judgments about physical scenarios, and in that respect are somewhat similar to ranking questions. While the questions themselves are of the type ``Is this high enough?" or ``Can we safely ignore \ldots?," they often do require at least ``back-of-the-envelope" calculations to to give informed answers. As in the case of estimation problems, students do have to explain their reasoning, but the question itself is usually more structured, and at least the initial answer is more easily evaluated by a computer.
\item[Essay questions] These are ``explain why" questions. A certain scenario is presented, and students are asked to explain why it turns out the way it does. Students are not asked to recall a certain law --- it is given to them. Instead, they are asked to discuss its validity.
\end{description}
-The three courses did not include estimation, qualitative, and essay problems, even though they could have been
-mediated through the online system. Table~\ref{table:problemcat} shows the classification distribution of the online
+The three courses did not include estimation, qualitative, and essay problems, which cannot be graded automatically within the online system.
+Table~\ref{table:problemcat} shows the classification distribution of the online
problems available for this project.
\begin{table*}
@@ -127,8 +127,7 @@
context-based reasoning.\label{table:problemcat}}
\begin{ruledtabular}
\begin{tabular}{lccccccc|l}
-\hline
-&\multicolumn{4}{c}{Multiple-choice and short-answer}&Multiple-choice multiple-response&Ranking&Click-on-image&\\
+&\multicolumn{4}{c}{Multiple-choice and short-answer}&Mult.-choice mult.-resp.&Ranking&Click-on-image&\\
&Multiple-choice&Tex\-tual&Nume\-rical&For\-mula&&&\\
\hline
``Conventional''& 10& & 355& 3& 54& 4& 2&428\\
@@ -236,44 +235,79 @@
For difficulty indizes beyond 3, the prominance of conceptual discussions increases. Surprisingly, it also increases for easier
questions. This may be attributed to students feeling more confident discussing easier problems on a conceptual level, or simply
-in there being less of a need of procedural discussions in the worst.
+in there being less of a need of procedural discussions.
Overall, the prominance of conceptual discussions is disappointingly low, as it varies between 5 and 16 percent.
+Beyond a difficulty index of 5, within error boundaries, the prominance of conceptual discussions would be consistant with a constant 10 percent. If fostering them is a goal,
+and the emotional climate an indicator of ``pain,'' then beyond a difficulty index of 5 a significant increase in ``pain'' results in a non-significant gain.
+
Across all difficulties, procedural contributions dominate the discussions, with relatively little significant variance around
the 40 percent mark. The maximum occurs for questions with a difficulty index of 5.
In figure~\ref{fig:diffnochat} the same analysis was carried out, but this time excluding all ``chat" contributions
-(subsection~\ref{subsec:comp}), i.e., only related non-emotional contributions were considered.
+(subsection~\ref{subsec:comp}), i.e., only related non-emotional contributions were considered. The relative prominance of procedural and conceptual discussions systematically
+increaeses, but all observations from the full analysis remain valid. ``Chat'' mostly provides a constant background across all difficulty indices.
\begin{figure}
\includegraphics[width=92mm]{diffnochat}% Here is how to import EPS art
\caption{\label{fig:diffnochat}Discussion characteristics as a function of problem difficulty, no considering ``chat."
}
\end{figure}
-\subsection{Influence of Question Types}
-
+\subsection{\label{subsec:qtype}Influence of Question Types}
+Table~\ref{table:qtype} shows the percentage prominance of discussion contributions with a certain type or with certain features in the discussions associated with questions
+that are of a certain type or have certain features.
\begin{table*}
+\caption{Influence of question types and features on discussions.
+The values indicate the percentage prominance of the discussion supertypes, types, and features (columns) for discussions associated with questions of a certain
+type or with certain features (rows). The values in brackets result from an analysis with ``chat'' excluded.\label{table:qtype}}
\begin{ruledtabular}
-\begin{tabular}{cccccccc}
- 1&-5$\pm$3&28$\pm$7 (29$\pm$8)&66$\pm$7 (74$\pm$7)&9$\pm$6 (9$\pm$6)&16$\pm$5 (17$\pm$5)&6$\pm$3 (7$\pm$3)\\
- 2&$\pm$&$\pm$ ($\pm$)&$\pm$ ($\pm$)&$\pm$ ($\pm$)&$\pm$ ($\pm$)&$\pm$ ($\pm$)\\
- 3&4$\pm$1&48$\pm$1 (57$\pm$1)&52$\pm$1 (63$\pm$2)&8$\pm$1 (9$\pm$1)&23$\pm$1 (27$\pm$1)&7$\pm$1 (8$\pm$1)\\
- 4&6$\pm$8&29$\pm$11 (31$\pm$10)&57$\pm$16 (64$\pm$18)&31$\pm$16 (36$\pm$18)&0$\pm$0 (0$\pm$0)&0$\pm$0 (0$\pm$0)\\
- 5&1$\pm$1&15$\pm$3 (16$\pm$3)&66$\pm$4 (72$\pm$4)&1$\pm$1 (2$\pm$2)&22$\pm$3 (26$\pm$3)&14$\pm$2 (18$\pm$3)\\
- 6&2$\pm$3&24$\pm$11 (26$\pm$12)&41$\pm$18 (46$\pm$20)&0$\pm$0 (0$\pm$0)&52$\pm$20 (54$\pm$20)&38$\pm$18 (39$\pm$17)\\
- 7&0$\pm$9&14$\pm$6 (18$\pm$8)&53$\pm$8 (69$\pm$11)&3$\pm$3 (5$\pm$5)&25$\pm$11 (26$\pm$11)&22$\pm$8 (25$\pm$9)\\\hline
+\begin{tabular}{lcccccc}
+&Emot. Clim.&Procedural&Solution&Math&Physics&Conceptual\\
+Multiple Choice&-5$\pm$3&28$\pm$7 (29$\pm$8)&66$\pm$7 (74$\pm$7)&9$\pm$6 (9$\pm$6)&16$\pm$5 (17$\pm$5)&6$\pm$3 (7$\pm$3)\\
+Short Textual&&&&&&\\
+Numerical&4$\pm$1&48$\pm$1 (57$\pm$1)&52$\pm$1 (63$\pm$2)&8$\pm$1 (9$\pm$1)&23$\pm$1 (27$\pm$1)&7$\pm$1 (8$\pm$1)\\
+Formula&6$\pm$8&29$\pm$11 (31$\pm$10)&57$\pm$16 (64$\pm$18)&31$\pm$16 (36$\pm$18)&&\\
+Mult.-choice Mult.-resp.&1$\pm$1&15$\pm$3 (16$\pm$3)&66$\pm$4 (72$\pm$4)&1$\pm$1 (2$\pm$2)&22$\pm$3 (26$\pm$3)&14$\pm$2 (18$\pm$3)\\
+Ranking&2$\pm$3&24$\pm$11 (26$\pm$12)&41$\pm$18 (46$\pm$20)&0$\pm$0 (0$\pm$0)&52$\pm$20 (54$\pm$20)&38$\pm$18 (39$\pm$17)\\
+Click-on-Image&0$\pm$9&14$\pm$6 (18$\pm$8)&53$\pm$8 (69$\pm$11)&3$\pm$3 (5$\pm$5)&25$\pm$11 (26$\pm$11)&22$\pm$8 (25$\pm$9)\\\hline
- a&4$\pm$1&42$\pm$1 (50$\pm$2)&55$\pm$1 (65$\pm$2)&7$\pm$1 (8$\pm$1)&23$\pm$1 (27$\pm$1)&9$\pm$1 (10$\pm$1)\\
- b&-2$\pm$2&37$\pm$4 (45$\pm$4)&52$\pm$3 (63$\pm$4)&7$\pm$2 (9$\pm$2)&23$\pm$3 (28$\pm$3)&8$\pm$2 (10$\pm$2)\\
+``Conventional''&4$\pm$1&42$\pm$1 (50$\pm$2)&55$\pm$1 (65$\pm$2)&7$\pm$1 (8$\pm$1)&23$\pm$1 (27$\pm$1)&9$\pm$1 (10$\pm$1)\\
+Rep-Trans&-2$\pm$2&37$\pm$4 (45$\pm$4)&52$\pm$3 (63$\pm$4)&7$\pm$2 (9$\pm$2)&23$\pm$3 (28$\pm$3)&8$\pm$2 (10$\pm$2)\\
\end{tabular}
\end{ruledtabular}
\end{table*}
-
+Error boundaries on the emotional climate values are rather large and mostly include zero (neutral), indicating no significant preferences within the limited sample.
+Yet, students clearly dislike multiple-choice questions, while they clearly like numerical answer problems. The data also indicates that students prefer ``conventional'' over
+representation-translation problems.
+
+The prominance of procedural discussions is significantly higher for numerical problems than for any other problem types, and higher for ``conventional'' than for
+representation-translation problems. The latter difference vanishes when ``chat'' is excluded.
+
+Solution-oriented contributions are significantly higher for multiple-choice and multiple-choice-multiple-response problems than for the other problem types with the exception
+of formula-response questions, where error-boundaries overlap.
+
+The prominance of mathematical discussion contributions is the highest for formula-response questions, approximately equal for numerical and multiple-choice questions, and the lowest for multiple-choice-multiple-response, ranking, and click-on-image questions.
+
+The prominance of physics-related discussion contributions was the highest for ranking and click-on-image problems, and the lowest for multiple-choice questions.
+
+Finally, when it comes to conceptual discussions, their prominance is signifcantly lower in multiple-choice and numerical problems than in the other problem types.
+
+It is a surprising result that the only significant difference between ``conventional'' and representation-translation problems is discuss slightly less procedure in favor of
+more complaints, and that differences disappear when ``chat'' is excluded from the analysis.
+\subsection{Influence of course}
+Few signficant differences could be found between the algebra-based and the calculus-based course:
+\begin{itemize}
+\item discussions in the algebra-based course had a significantly higher emotional
+climate (6$\pm$1 versus 2$\pm$1)
+\item the algebra-based course had a higher prominance of ``chat'' (21$\pm$2\% versus 11$\pm$1\% (first semester) and 14$\pm$2\% (second semester))
+\item physics-related discussion were significantly higher in the calculus-based course (28$\pm$2\% (first semester) and 23$\pm$2\% (second semester) versus 17$\pm$2\%)
+\item conceptual-discussions were significantly higher in the first semester of the calculus-based course (12$\pm$2\% versus 6$\pm$2\%), but this difference vanished in the second semester (7$\pm$1\%).
+\end{itemize}
\begin{acknowledgments}
-We wish to acknowledge the support of the author community in using
-REV\TeX{}, offering suggestions and encouragement, testing new versions,
+Supported by the National Science Foundation under NSF-ITR 0085921 and NSF-CCLI-ASA 0243126. Any opinions, findings, and conclusions or recommendations expressed in this
+publication are those of the author and do not necessarily reflect the views of the National Science Foundation.
\dots.
\end{acknowledgments}
--www1112630182--