[LON-CAPA-cvs] cvs: modules /gerd/concept description.tex

[www]

This is a MIME encoded message

--www1090010915
Content-Type: text/plain

www		Fri Jul 16 16:48:35 2004 EDT

  Modified files:              
    /modules/gerd/concept	description.tex 
  Log:
  Streamlining of methodology section.
  
  
--www1090010915
Content-Type: text/plain
Content-Disposition: attachment; filename="www-20040716164835.txt"

Index: modules/gerd/concept/description.tex
diff -u modules/gerd/concept/description.tex:1.17 modules/gerd/concept/description.tex:1.18
--- modules/gerd/concept/description.tex:1.17	Fri Jul 16 15:57:37 2004
+++ modules/gerd/concept/description.tex	Fri Jul 16 16:48:34 2004
@@ -298,11 +298,11 @@
 
 
 \section{Research Methodology}\label{analysis}
-\subsection{Establishment of Initial Conditions}The validity of the hypotheses may depend on both learner and assessment characteristics other than their type.\subsubsection{Learner Attitudes, Beliefs, and Expectations}Instruments have been developed to assess epistemological beliefs, for example the Epistemological Beliefs Assessment for Physical Science (EBAPS) Instrument~\cite{EBAPS}. Related to epistemological beliefs are learnerŐs expectations and attitudes, and of particular interest is the Maryland Physics Expectations (MPEX) survey~\cite{MPEX}.\subsubsection{Learner Knowledge about the Topic}\label{prepost}We will use existing concept inventory surveys as both pre- and post-tests.The qualitative Force Concept Inventory~\cite{fci} and the quantitative companion Mechanical Baseline Test~\cite{hestenesmech} have been used in a large number of studies connected to the teaching of introductory mechanics. The Foundation Coalition has been developing a number of relevant concept inventories~\cite{foundation}, namely the Thermodynamics Concept Inventory, the Dynamics Concept Inventory, and the Electromagnetics Concept Inventory (with two subcomponents, namely Waves and Fields).  Since these were designed from an engineering point of view, some adjustment might be necessary. In addition, the Conceptual Survey of Electricity and Magnetism (CSEM)~\cite{maloney} is available for the second semester course.
+\subsection{Establishment of Initial Conditions}Many educational studies result in "no-significant-difference"~\cite{russell}, and particularly the study of question type effectiveness (Sect.~\cite{effect}) may well yield the same result. Many variables may influence the impact of a particular sample of representative problems of a particular type for a particular learner, and it is imperative to understand as much of the "initial conditions" as possible, since the validity of the hypothesis may depend on them.\subsubsection{Learner Attitudes, Beliefs, and Expectations}Instruments have been developed to assess epistemological beliefs, for example the Epistemological Beliefs Assessment for Physical Science (EBAPS) Instrument~\cite{EBAPS}. Related to epistemological beliefs are learnerŐs expectations and attitudes, and of particular interest is the Maryland Physics Expectations (MPEX) survey~\cite{MPEX}.\subsubsection{Learner Knowledge about the Topic}\label{prepost}We will use existing concept inventory surveys as both pre- and post-tests.The qualitative Force Concept Inventory~\cite{fci} and the quantitative companion Mechanical Baseline Test~\cite{hestenesmech} have been used in a large number of studies connected to the teaching of introductory mechanics. The Foundation Coalition has been developing a number of relevant concept inventories~\cite{foundation}, namely the Thermodynamics Concept Inventory, the Dynamics Concept Inventory, and the Electromagnetics Concept Inventory (with two subcomponents, namely Waves and Fields).  Since these were designed from an engineering point of view, some adjustment might be necessary. In addition, the Conceptual Survey of Electricity and Magnetism (CSEM)~\cite{maloney} is available for the second semester course.
 
 \subsubsection{Problem Difficulty and Baseline Statistical Data}LON-CAPA automatically keeps tracks of the average number of attempts until a problem is solved, as well as the degree of difficulty and the degree of discrimination. This data is cumulative across semesters, and already exists for all assessment problems from their deployment in previous semesters.
-\subsection{Observables}\subsubsection{Effectiveness}Effectiveness will be measured both in terms of performance on summative assessments (quizzes and exams) and on pre-/post-test concept inventory surveys (Sect.~\ref{prepost}).  Each item on these instruments will be associated with topically corresponding formative online exercises to determine correlations and differential gain between the feedback types used with the respective online problems. A second posttest, correlated with first semester problems, will be administered at the end of the second semester to determine long-term effects.\subsubsection{Problem Solving Technique}We intend to focus on a subset of students in the LBS Collaborative Learning Laboratory, and observe them while solving problems. Schoenfeld~\cite{schoenfeld} and Foster~\cite{foster} developed instruments to categorize and document the stages and expertlike 
-characteristics~\cite{chi} of observed problem-solving activity by learners, as well as application of metacognitive skills.In addition, we will interview a group of students from all courses regarding their problem-solving strategies. Pascarella~\cite{pascarella02} developed some frameworks for these interviews, which can be built upon.Finally, for all students in all courses, LON-CAPA log data will be analyzed. Kotas~\cite{kotas} and Minaei~\cite{minaei} developed a mechanism for this log data analysis, which include submission times between attempts, and quality of the entered input. \subsubsection{Help-Seeking Behavior and Discussions}\label{discussion}It is impossible to observe all on-demand help seeking, but interactions in several settings can be analyzed:Online discussions and email communication are preserved within LON-CAPA and can be analyzed even in retrospect for past semesters with respect to relevant behavioral patterns.  Table~\ref{discussionex} shows excerpts of discussions around the two problems in Fig.~\ref{trajectory}.
+\subsection{Observables}\subsubsection{Effectiveness}\label{effect}Effectiveness will be measured both in terms of performance on summative assessments (quizzes and exams) and on pre-/post-test concept inventory surveys (Sect.~\ref{prepost}).  Each item on these instruments will be associated with topically corresponding formative online exercises to determine correlations and differential gain between the feedback types used with the respective online problems. A second posttest, correlated with first semester problems, will be administered at the end of the second semester to determine long-term effects.\subsubsection{Problem Solving Technique}We intend to focus on a subset of students in the LBS Collaborative Learning Laboratory, and observe them while solving problems. Schoenfeld~\cite{schoenfeld} and Foster~\cite{foster} developed instruments to categorize and document the stages and expertlike 
+characteristics~\cite{chi} of observed problem-solving activity by learners, as well as application of metacognitive skills.In addition, for all students, log data will be analyzed. Kotas~\cite{kotas} and Minaei~\cite{minaei} developed a mechanism for this log data analysis, which include submission times between attempts, and quality of the entered input. \subsubsection{Help-Seeking Behavior and Discussions}\label{discussion}It is impossible to observe all on-demand help seeking, but interactions in several settings can be analyzed:Online discussions and email communication are preserved within LON-CAPA and can be analyzed even in retrospect for past semesters with respect to relevant behavioral patterns.  Table~\ref{discussionex} shows excerpts of discussions around the two problems in Fig.~\ref{trajectory}.
 
 \begin{table}
 \tiny
@@ -458,7 +458,7 @@
 \caption{Excerpts from online discussion around the two problems Fig.~\ref{trajectory}\label{discussionex}}
 \end{table}
 
-Discussion contributions and states can be linked to online transactions, such as submission of homework attempts, browsing of content material, and hint usage. Wallace~\cite{wallace} reviewed existing research on such online interactions, however, some adaptation of several of the existing conceptualizations will be necessary to account for the nature of physics courses.For the subset of LBS students who come to the LBS Collaborative Learning Laboratory, group discussions can to be documented, and linked to online behavior as absolute timing and learner identify are preserved.Minaei~\cite{minaei} developed data mining strategies to categorize learner behavior, including navigational patterns between assessment and content material, the use of feedback, and communication functions.Self-reporting can be used for several other help-seeking mechanisms, such as textbook use and peer-interaction~\cite{riffell1,riffell2}.\subsubsection{Cross-Cutting Open-Ended Documentation of Learner Perceptions}We will interview focus groups of students regarding their experiences and perceived relative helpfulness of the different problem types, and ask them to also reflect on how they perceived these question types were influencing their problem-solving strategies. 
+Discussion contributions and states can be linked to online transactions, such as submission of homework attempts, browsing of content material, and hint usage. Wallace~\cite{wallace} reviewed existing research on such online interactions, however, some adaptation of several of the existing conceptualizations will be necessary to account for the nature of physics courses.For the subset of LBS students who come to the LBS Collaborative Learning Laboratory, group discussions can to be documented, and linked to online behavior as absolute timing and learner identify are preserved.Minaei~\cite{minaei} developed data mining strategies to categorize learner behavior, including navigational patterns between assessment and content material, the use of feedback, and communication functions.Self-reporting can be used for several other help-seeking mechanisms, such as textbook use and peer-interaction~\cite{riffell1,riffell2}.\subsubsection{Interviews}We will interview focus groups of students regarding their experiences and perceived relative helpfulness of the different problem types, and ask them to also reflect on how they perceived these question types were influencing their problem-solving strategies. Pascarella~\cite{pascarella02} developed some frameworks for these interviews, which can be built upon.
 
 \section{Involvement of Students in Research}
 \subsection{Undergraduate}\label{undergrad}
@@ -577,6 +577,7 @@
 \bibitem{galileo} Project Galileo, {\tt http://galileo.harvard.edu/}
 
 % Methodology
+\bibitem{russell} T. L. Russell, {\it The No Significant Difference Phenomenon, A Comparative Research Annotated Bibliography on Technology in Distance Education as Reported in 355 Research Reports, Summaries, and Papers}, ISBN 0-9668936-0-3 (2001)
 \bibitem{EBAPS} A. Elby, J. Fredriksen, C. Schwarz, and B. White, {\it Epistemological Beliefs Assessment for Physical Science (EBAPS) Instrument}, {\tt http://www2.physics.umd.edu/$\sim$elby/EBAPS/EBAPS\_items.htm} (2001)
 \bibitem{MPEX} E. F. Redish, J. M. Saul, and R. N. Steinberg, {\it Student expectations in introductory physics}, Am. J. Physics {\bf 66}, 212-224 (1998), survey available on The Physics Suite CD in Teaching Physics, ISBN 0-471-39378-9 (2003)
 \bibitem{hestenesmech} D. Hestenes and M.Wells, {\it A Mechanics Baseline Test}, Phys. Teach {\bf 30}(3), 159-166 (1992)

--www1090010915--