[LON-CAPA-cvs] cvs: modules /gerd/discussions/paper discussions.tex
www
lon-capa-cvs@mail.lon-capa.org
Wed, 15 Feb 2006 22:26:00 -0000
This is a MIME encoded message
--www1140042360
Content-Type: text/plain
www Wed Feb 15 17:26:00 2006 EDT
Modified files:
/modules/gerd/discussions/paper discussions.tex
Log:
Adressing editor concerns
--www1140042360
Content-Type: text/plain
Content-Disposition: attachment; filename="www-20060215172600.txt"
Index: modules/gerd/discussions/paper/discussions.tex
diff -u modules/gerd/discussions/paper/discussions.tex:1.39 modules/gerd/discussions/paper/discussions.tex:1.40
--- modules/gerd/discussions/paper/discussions.tex:1.39 Wed Feb 15 15:34:08 2006
+++ modules/gerd/discussions/paper/discussions.tex Wed Feb 15 17:25:57 2006
@@ -37,10 +37,11 @@
widespread acceptance. By using online forums the practice can be extended
outside the classroom. Over the past several years we have been using an
online system where the threaded discussion forums are directly related to
-randomizing online problems. Despite supporting research (see
-Ref.~\onlinecite{wallace} for a review), [xx why does this research
-surprise you? I don't know xx] we continue to be surprised by the richness
-of the ensuing peer-interactions. In this study we analyze student
+randomizing online problems. There is already research (see
+Ref.~\onlinecite{wallace} for a review), which suggests that online environments
+can foster peer-interactions, but
+we continue to be surprised by the richness
+of the ensuing peer-teaching we are observing. In this study we analyze student
discussion contributions, in particular with respect to of the
courses, the students, and the problems. Our goal is to first identify
online discussion behavioral patterns of successful students and to
@@ -48,8 +49,9 @@
% intro should be short and not have subsections
-LON-CAPA started in 1992 as a system to give randomized homework to students
-in introductory physics courses. [xx I suggest that you give a little more background about Lon-Capa, including its Web site and the fact that it is open source. put this info in an endnote xx] Randomized means that each student sees
+LON-CAPA\footnote{LON-CAPA is an open-source freeware system initially developed by Michigan State University. More information can be found at http://www.lon-capa.org/.}
+started in 1992 as a system to give randomized homework to students
+in introductory physics courses. Randomized means that each student sees
a different version of the same computer-generated problem: different
numbers, choices, graphs, images, simulation parameters, etc, see
Fig.~\ref{twoproblems}. Randomization was implemented as a means to both
@@ -137,7 +139,7 @@
be required in different formats; for example in the problem on the right
side of Fig.~\ref{trajectory}, it is a short numerical-answer. Conversion
between representations can be surprisingly challenging for physics
-learners.\cite{mcdermott,beichner} For t this project,
+learners.\cite{mcdermott,beichner} For this project,
representation-translation will be considered a feature that may or may
not apply to any of the other problem types.
@@ -147,6 +149,9 @@
they will hit the ground, or a number of locations in order of the strength
of their local electric potential.
+\item{\it Click-On-Image problems} require a student to click on certain areas of an image. Examples are where to cut a wire in a circuit diagram so a light bulb becomes brighter,
+or the possible locations of the center of mass for an object not to tip over.
+
\item {\it Context-based reasoning problems}
problems are set in the context of real-world scenarios
and not in the context of the artificial zero-friction laboratory
@@ -217,15 +222,14 @@
\item Discussion contributions were classified as {\it surface} if they
dealt with surface features of the problem or were surface level requests
-for help. Two subtypes were question and answer. [xx what do these subtypes
-mean? xx]
+for help. We further divided this into two subtypes, namely questions and answers.
\item {\it Procedural} contributions describe or inquire about mechanisms
of solving a problem without mentioning the underlying concepts or
-reasoning. Two subtypes were question and answer.
+reasoning. Two subtypes, again, were questions and answers.
\item {\it Conceptual} contributions deal with the underlying concepts of
-the problem. Two subtypes were question and answer.
+the problem. Two subtypes, again, were questions and answers.
\end{enumerate}
@@ -359,7 +363,7 @@
negatively correlated. Although cause and effect may be arguable, particular attention is paid in
Sec.~\ref{sec:question} to problem
properties that elicit either the desirable or undesirable discussion
-behavioral patterns. [xx note change xx]
+behavioral patterns.
Due to the smaller sample size, a correlation analysis by individual
question and answer classes yielded no statistically significant results.
@@ -373,10 +377,10 @@
in a given discussion, the number of contributions belonging to it is
divided by the total number of contributions. The discussion characteristics
of the problems were categorized by their difficulty index and the average
-percentage plotted in Fig.~\ref{fig:diff}. Only superclasses are shown, namely the emotional climate, [xx info like crosses should be in the figure caption xx] as well as all related procedural and conceptual contributions. The plot should be
-interpreted as follows: within the given error boundaries, [xx what does "within the given error boundaries" mean? xx] for a
+percentage plotted in Fig.~\ref{fig:diff}. Only superclasses are shown, namely the emotional climate, as well as all related procedural and conceptual contributions. The plot should be
+interpreted as follows: for a
problem with difficulty index of six, ten percent of the online discussion
-was conceptual. In addition, the data was fit using second order (procedural) and third order (emotional climate; conceptual) polynomials. [xx I couldn't follow this sentence xx]
+was conceptual. The lines are polynomial fits to the data.
The greatest variation is found in the emotional climate of the discussion.
As is to be expected, the climate is mostly positive for easy problems,
@@ -386,8 +390,9 @@
associated emotional comments.
For difficulty indexes beyond 3, the prominence of conceptual discussions
-increases. Surprisingly, it also increases for easier problems. This [xx
-this what? xx] may be attributed to students feeling more confident
+increases. Surprisingly, it also increases for easier problems. This increase
+of conceptual discussions for easy problems
+may be attributed to students feeling more confident
discussing easier problems on a conceptual level, or that there is
less of a need for procedural discussions. Overall, the prominence of
conceptual discussions is disappointingly low, and varies between 5 and
@@ -398,8 +403,8 @@
emotional climate an indicator of ``pain,'' then beyond a difficulty index
of 5, a significant increase in ``pain'' results in a non-significant gain.
-Across all level of difficulty procedural contributions dominate the discussions,
-with relatively little significant variance around 40\%. [xx confusing syntax. you mean that the percentage of procedural contributions was 40\%? xx] The
+Across all levels of difficulty, procedural contributions dominate the discussions, making up about
+40\% of the contributions. The
maximum occurs for problems with a difficulty index of 5.
In Fig.~\ref{fig:diffnochat} the same analysis was performed, but all chat contributions were excluded, that is, only related non-emotional
@@ -413,10 +418,8 @@
as discussed in Sec.~\ref{subsec:problemcat} and each associated discussion was classified
entry as discussed in Sec.~\ref{subsec:disccat}. As a measure of the prominence
of a class in a given discussion, the number of contributions belonging to
-it is divided by the total number of contributions. Table~\ref{table:qtype}
-shows the percentage of discussion contributions of a certain type or of
-certain features in the discussions associated with problems that are of a
-certain type or have certain features. [xx awkward sentence xx]
+it is divided by the total number of contributions. For each problem type and problem feature,
+Table~\ref{table:qtype} shows the distribution of discussion contribution types.
The error bars on the emotional climate values are rather large and mostly
include zero (neutral), indicating no significant preferences within the
@@ -459,12 +462,12 @@
found that students have unexpected difficulties in translating data presented as graphs, so a stronger effect of this feature was expected.
In addition, Kashy\cite{kashyd01} found that mastery of
representation-translation problems is the best predictor of final exam
-scores, even when controlling for ACT, [xx say what ACT is xx] cumulative GPA, and force-concept
+scores, even when controlling for scores on the ACT college entrance exam, cumulative GPA, and force-concept
inventory pretests. Discussion behavior and final exam performance are
clearly different measurements of the influence of problem types and do not
necessarily need to correlate, but a connection between individual
discussion behavior and performance in the course clearly exists (see
-Sec.~\ref{subsec:gradedep}). The earlier study [xx by whom? xx]
+Sec.~\ref{subsec:gradedep}). The earlier study by Kashy\cite{kashyd01}
considered a relatively small set of representation-translation problems,
some of which involved time-evolving simulations as data-source,
while in the present study, no simulation-based problems were assigned. A
@@ -522,10 +525,11 @@
As an example, consider the example in Fig.~\ref{fig:discussionexample}:
there is no external torque, and the problem was intended as a simple example
of angular momentum conservation. Because the disk has a several centimeter
-radius, a bug can safely be approximated as a point mass. It [xx what is it?
-xx] is
+radius, a bug can safely be approximated as a point mass. With $m_d$ being the mass of the disk,
+$m_b$ the mass of the bug, $r$ the radius of the disk, $\omega_0$ the initial angular velocity, and $\omega$ the final angular velocity which we are looking for,
+angular momentum conservaton yields
$(\frac{1}{2}m_dr^2+m_b0^2)\omega_0=(\frac{1}{2}m_dr^2+m_br^2)\omega$, and
-therefore $\omega=\omega_0m_d/(m_d+2m_b)$. [xx need to define your notation xx] As long as the disk is much
+therefore $\omega=\omega_0m_d/(m_d+2m_b)$. As long as the disk is much
larger than the bug, the result for $\omega$ is independent of the radius of the disk, and no unit
conversions are needed. Several observations are immediately obvious to
the expert reader of the discussion: (a) No student mentions the fact that
@@ -562,13 +566,14 @@
Many of these shortcomings may be correctable through early detection and
closely following the online student discussions prior to lecture,
-particularly around the assigned reading problems, which [xx what does "which" refer to? xx] may be a valid [xx "valid" is the wrong choice of word. reasonable? xx]
+particularly around the assigned reading problems. Following student discussions can thus be used as an
extension of the Just-in-Time Teaching technique.\cite{jitt}
\subsection{Comparison to other research approaches}
The present approach of understanding student problem solving
behavior is comparable to the more traditional ``thinking out loud'' or group
-discussion observations. [xx give ref xx] However, in the former, the subjects are
+discussion observations, techniques which were for example used in the study by
+Pascarella~\cite{pascarella}. However, in the former, the subjects are
keenly aware of the observer, which may influence their behavior. In almost
any course, appropriate problem solving techniques would have been
discussed, and although students might find them ``inefficient" or
@@ -588,7 +593,8 @@
in the way they believe is most efficient. The large number of discussion
contributions allows for statistically significant results. A disadvantage
is that this method depends on problem randomization, and thus can only be
-used with systems like LON-CAPA. [xx I think there are many other systems like Lon-Capa xx] If the online problems not randomized,
+used with systems like LON-CAPA.
+If the online problems are not randomized,
discussions would likely consist of one or two entries with only the final
answer, such as 17.5\,m/s or Answer B. Also, the online system must not
have a separate discussion area, but provide contextual discussion
@@ -643,7 +649,7 @@
hopefully will yield more statistically significant results.
\begin{acknowledgments}
-[xx need a subject xx] Supported by the National Science Foundation under
+This project was supported by the National Science Foundation under
NSF-ITR 0085921 and NSF-CCLI-ASA 0243126. Any opinions, findings, and
conclusions or recommendations expressed in this publication are those of
the author and do not necessarily reflect the views of the National Science
@@ -671,9 +677,9 @@
\bibitem{wallace}
R. Wallace,
-Education, Communication and Information
-\textbf{ 3}, 241--xx (2003). [xx need title and last page for all articles
-xx]
+\bibinfo{title}{Online learning in higher education: a review of research on interactions among teachers and students},
+\bibinfo{journal}{Education, Communication and Information}
+\textbf{\bibinfo{volume}{3}}, \bibinfo{pages}{241--280} (\bibinfo{year}{2003}).
\bibitem[{\citenamefont{Kashy et~al.}(2003)\citenamefont{Kashy, Albertelli,
Bauer, Kashy, and Thoennessen}}]{kashy03}
@@ -682,8 +688,9 @@
\bibinfo{author}{\bibfnamefont{W.}~\bibnamefont{Bauer}},
\bibinfo{author}{\bibfnamefont{E.}~\bibnamefont{Kashy}}, \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{M.}~\bibnamefont{Thoennessen}},
+\bibinfo{title}{Influence of non-moderated and moderated discussion sites on student success},
\bibinfo{journal}{J. Asynchronous Learning Networks}
-\textbf{\bibinfo{volume}{7}}, \bibinfo{pages}{31--xx}
+\textbf{\bibinfo{volume}{7}}, \bibinfo{pages}{31--36}
(\bibinfo{year}{2003}).
\bibitem[{\citenamefont{Kashy et~al.}(2001)\citenamefont{Kashy, Albertelli,
@@ -701,8 +708,10 @@
\bibitem[{\citenamefont{Steinberg and Sabella}(1997)}]{steinberg}
\bibinfo{author}{\bibfnamefont{R.~N.} \bibnamefont{Steinberg}}
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{M.~S.}
-\bibnamefont{Sabella}}, \bibinfo{journal}{Phys. Teach.}
-\textbf{\bibinfo{volume}{35}}, \bibinfo{pages}{150} (\bibinfo{year}{1997}).
+\bibnamefont{Sabella}},
+\bibinfo{title}{Performance on multiple-choice diagnostics and complementary exam problems},
+\bibinfo{journal}{Phys. Teach.}
+\textbf{\bibinfo{volume}{35}}, \bibinfo{pages}{150--155} (\bibinfo{year}{1997}).
\bibitem[{\citenamefont{Redish}(2003)}]{redish}
\bibinfo{author}{\bibfnamefont{E.~F.} \bibnamefont{Redish}},
@@ -714,36 +723,44 @@
\bibinfo{author}{\bibfnamefont{L.}~\bibnamefont{McDermott}},
\bibinfo{author}{\bibfnamefont{M.~L.} \bibnamefont{Rosenquist}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{E.~H.} \bibnamefont{van
-Zee}}, \bibinfo{journal}{Am. J. Phys.} \textbf{\bibinfo{volume}{55}},
-\bibinfo{pages}{503} (\bibinfo{year}{1987}).
+Zee}},
+\bibinfo{title}{Student difficulties in connecting graphs and physics: examples from kinematics},
+\bibinfo{journal}{Am. J. Phys.} \textbf{\bibinfo{volume}{55}},
+\bibinfo{pages}{503--513} (\bibinfo{year}{1987}).
\bibitem[{\citenamefont{Beichner}(1994)}]{beichner}
\bibinfo{author}{\bibfnamefont{R.~J.} \bibnamefont{Beichner}},
+\bibinfo{title}{Testing student interpretation of kinematics graphs}
\bibinfo{journal}{Am. J. Phys.} \textbf{\bibinfo{volume}{62}},
-\bibinfo{pages}{750} (\bibinfo{year}{1994}).
+\bibinfo{pages}{750--762} (\bibinfo{year}{1994}).
\bibitem[{\citenamefont{Mazur}(1996)}]{mazur96}
\bibinfo{author}{\bibfnamefont{E.}~\bibnamefont{Mazur}},
+\bibinfo{title}{The problem with problems},
\bibinfo{journal}{Optics and Photonics News} \textbf{\bibinfo{volume}{6}},
-\bibinfo{pages}{59} (\bibinfo{year}{1996}).
+\bibinfo{pages}{59--60} (\bibinfo{year}{1996}).
\bibitem[{\citenamefont{Lin}(1982)}]{lin}
-\bibinfo{author}{\bibfnamefont{H.}~\bibnamefont{Lin}}, \bibinfo{journal}{Phys.
-Teach.} \textbf{\bibinfo{volume}{20}}, \bibinfo{pages}{151}
-(\bibinfo{year}{1982}). [xx need issue \# also xx]
+\bibinfo{author}{\bibfnamefont{H.}~\bibnamefont{Lin}},
+\bibinfo{title}{Learning physics vs. passing courses},
+\bibinfo{journal}{Phys.
+Teach.} \textbf{\bibinfo{volume}{20}}, \bibinfo{issue}{3}, \bibinfo{pages}{151--157}
+(\bibinfo{year}{1982}).
\bibitem[{\citenamefont{Chi et~al.}(1981)\citenamefont{Chi, Feltovich, and
Glaser}}]{chi}
\bibinfo{author}{\bibfnamefont{M.~T.~H.} \bibnamefont{Chi}},
\bibinfo{author}{\bibfnamefont{P.~J.} \bibnamefont{Feltovich}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{R.}~\bibnamefont{Glaser}},
+\bibinfo{title}{Categorization and representation of physics problems by experts and novices},
\bibinfo{journal}{Cognitive Sci.} \textbf{\bibinfo{volume}{5}},
-\bibinfo{pages}{121} (\bibinfo{year}{1981}).
+\bibinfo{pages}{121--152} (\bibinfo{year}{1981}).
\bibitem[{\citenamefont{Pascarella}(2004)}]{pascarella}
\bibinfo{author}{\bibfnamefont{A.}~\bibnamefont{Pascarella}}, in
-Proc. NARST Annual Meeting
-(\bibinfo{year}{2004}). [xx where published? xx]
+Proc. NARST Annual Meeting,
+\url{http://www.lon-capa.org/papers/204416ProceedingsPaper.pdf}
+(\bibinfo{year}{2004}).
\bibitem[{\citenamefont{Novak et~al.}(1999)\citenamefont{Novak, Patterson,
Gavrin, and Christian}}]{jitt}
@@ -763,16 +780,18 @@
\bibinfo{author}{\bibfnamefont{S.}~\bibnamefont{Schworm}},
\bibinfo{author}{\bibfnamefont{F.}~\bibnamefont{Fischer}}, \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{R.}~\bibnamefont{Wallace}},
+\bibinfo{title}{Help Seeking and Help Design in Interactive Learning Environments},
\bibinfo{journal}{Rev. Educ. Research}
-\textbf{\bibinfo{volume}{73}}, \bibinfo{pages}{277} (\bibinfo{year}{2003}).
+\textbf{\bibinfo{volume}{73}}, \bibinfo{pages}{277--320} (\bibinfo{year}{2003}).
\bibitem[{\citenamefont{Redish et~al.}(1998)\citenamefont{Redish, Steinberg,
and Saul}}]{mpex}
\bibinfo{author}{\bibfnamefont{E.~F.} \bibnamefont{Redish}},
\bibinfo{author}{\bibfnamefont{R.~N.} \bibnamefont{Steinberg}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{J.~M.} \bibnamefont{Saul}},
+\bibinfo{title}{Student Expectations in Introductory Physics},
\bibinfo{journal}{Am. J. Phys.} \textbf{\bibinfo{volume}{66}},
-\bibinfo{pages}{212} (\bibinfo{year}{1998}).
+\bibinfo{pages}{212--224} (\bibinfo{year}{1998}).
\bibitem[{\citenamefont{Halloun et~al.}()\citenamefont{Halloun, Hake, Mosca,
and Hestenes}}]{fci}
@@ -795,7 +814,7 @@
scheme described in Sec.~\ref{subsec:problemcat} (adapted from
Ref.~\onlinecite{redish}). The columns denote the different problem types,
while the rows denote the features of required representation translation and
-context-based reasoning. [xx click on image was not discussed in text xx] [xx couldn't tell what the heading are. fix table xx]\label{table:problemcat}}
+context-based reasoning.\label{table:problemcat}}
\begin{ruledtabular}
\begin{tabular}{lccccccc|l}
&\multicolumn{4}{c}{Multiple choice and short answer} & Mult.-choice
@@ -815,7 +834,7 @@
\begin{table}[h!]
\caption{Examples of discussion contribution types and
-features.\label{table:examples} [xx I can fix your tables if you like xx]}
+features.\label{table:examples}}
\begin{ruledtabular}
\begin{tabular}{l|p{3.9cm}|p{3.9cm}|p{3.9cm}|p{3.9cm}}
&Unrelated&Solution&Math&Physics\\\hline
@@ -885,10 +904,12 @@
\begin{table}[h!]
\caption{Same as Table~\ref{table:disccat} for the first semester of the
-calculus-based class only. The table includes a small number of
+calculus-based class, the only course for which the analysis by student characteristics was performed.
+The distribution of contribution types is very similar to the full sample.
+Note that the table includes a small number of
contributions by students who eventually dropped the course, which were
included in the analysis by problem type, but not in the analysis by student
-characteristics.\label{table:disccatfirst} [xx table captions should summarize what we are supposed to learn from table xx]}
+characteristics.\label{table:disccatfirst}}
\begin{ruledtabular}
\begin{tabular}{lllllllll|l}
&\multicolumn{2}{c}{Emotional}
@@ -946,10 +967,10 @@
\begin{figure}[h!]
%\includegraphics[width=6.5in]{KortemeyerFig2}
-\caption{Example of two LON-CAPA problems that addres the same concepts. The
+\caption{Example of two LON-CAPA problems that address the same concepts. The
problem on the left is a conventional short-numerical-answer problem, and
the one on the right is of type multiple-choice
-multiple-response.\label{threemasses} [xx better to use (a) and (b) instead of left and right xx]}
+multiple-response.\label{threemasses}}
\end{figure}
\begin{figure}[h!]
@@ -982,7 +1003,7 @@
\begin{tabular}{p{9.3cm}|p{8cm}}
Student A (anonymous); female; 4.0 {\it (Emotional/Negative/Physics; Chat)}:
\begin{verbatim}
-What is that bug doing on a disk? Boo to physics. [xx better to not use verbatim xx]
+What is that bug doing on a disk? Boo to physics.
\end{verbatim}
Student B (named); male; 3.5 {\it (Procedural/Answer/Physics)}:
@@ -1127,7 +1148,12 @@
\begin{figure}[h!]
\includegraphics[width=92mm]{KortemeyerFig6}
\caption{\label{fig:diff}Discussion characteristics as a function of problem
-difficulty. }
+Only superclasses are
+shown (subsection~\ref{subsec:problemcat}), namely the emotional climate (crosses), as well as all (questions and answers) related
+procedural
+(triangles) and conceptual (diamonds) contributions.
+difficulty. In addition, the data was fit using second order (procedural, long dashes) and third order (emotional climate, short dashes; conceptual, solid) polynomials.
+}
\end{figure}
\begin{figure}[h!]
@@ -1136,4 +1162,4 @@
}
\end{figure}
-\end{document}
\ No newline at end of file
+\end{document}
--www1140042360--