[LONCAPAcvs] cvs: modules /gerd/roleclicker description.tex
www
loncapacvs@mail.loncapa.org
Wed, 23 Feb 2005 01:53:46 0000
This is a MIME encoded message
www1109123626
ContentType: text/plain
www Tue Feb 22 20:53:46 2005 EDT
Modified files:
/modules/gerd/roleclicker description.tex
Log:
Little bit of cleanup.
www1109123626
ContentType: text/plain
ContentDisposition: attachment; filename="www20050222205346.txt"
Index: modules/gerd/roleclicker/description.tex
diff u modules/gerd/roleclicker/description.tex:1.3 modules/gerd/roleclicker/description.tex:1.4
 modules/gerd/roleclicker/description.tex:1.3 Tue Feb 22 14:53:20 2005
+++ modules/gerd/roleclicker/description.tex Tue Feb 22 20:53:46 2005
@@ 91,6 +91,70 @@
\subsection{The Problem with Problems  Hypotheses}\label{hypo}
+
+To quote Lin~\cite{lin}: ``The primary determinants of student performance are the specific tasks for which teachers explicitly hold students responsible (e.g. problem sets and exams), rather than the general goals of the teacher (e.g. conveying an appreciation of the power of physics in a broad context)." Mazur~\cite{mazur96} asks ``So why do we keep testing our students with conventional problems?"
+
+The answer, only too often, is scalability, and that in more than one dimension: nonconventional problems are harder to write, and even harder to grade.
+
+The scalability problem is easier to overcome in the classroom: alternative formative assessment as a classroom tool, where students are forced to verbally express their views and teach each other, rather than calculate answers~\cite{mazur}, is starting to be adopted as an effective teaching practice in more and more courses.
+
+It would clearly be advantageous to extend these effective verbalization practices outside the classroom, and offer formative assessment opportunities in which students get to work through and write about reallife problems on a conceptual level, and are explicitly graded on formulating assumptions, developing models, doing backoftheenvelope estimations, and deriving relevant formulas and solutions. Given both time and logistical constraints, except for the occasional ``project assignment," that is not a reality.
+
+This project focuses on how to move beyond conventional homework problems while operating within the realistic limitations of largeenrollment courses.
+
+Particularly in largeenrollment courses, timely feedback is often impossible without the use of computerized homework systems (e.g.~\cite{thoennessen,kashy00}). Unfortunately, an all too frequent approach to using such systems is to simply replicate conventional textbook problems in the online realm, where they are conveniently graded by the computer.
+
+The project assumes that ``the problem with problems" (a phrase borrowed from~\cite{mazur96}) is that
+\begin{quote}
+{\bf Hypothesis 1a:} Conventional calculationoriented problems affirm nonexpertlike epistemologies and encourage nonexpertlike problemsolving strategies
+\end{quote}
+
+Using computerized systems does impose limitations on which kind of problems can be made available, but does not limit educators to just these most basic types. A further assumption is that by the reverse token
+\begin{quote}
+{\bf Hypothesis 1b:} There are types of online (mostly) computerevaluated problems, which make learners confront their nonexpertlike epistemologies and encourage expertlike problemsolving strategies\end{quote}
+
+These problem types might involve both computer and humanevaluated components, where an emphasis has to be put on keeping the humanevaluated part manageable and scalable.
+
+An additional problem with conventional problems may be their mathematical nature. Hewitt in the preface to his textbook ``Conceptual Physics"~\cite{hewitt} argues that the mathematical language of physics often deters the average nonscience students, a notion which concurs with Tobias' concept of ``math anxiety"\cite{tobias}, which is a particular issue for students in the ``second tier"\cite{tobiasST} of science courses. For them, the use of mathematics in physics courses can present a hurdle, and a lack of skills or confidence to perform basic algebraic manipulations (``$V=RI\ \Rightarrow\ R=V/I"$), or even problems operating their pocket calculators, can hinder students' learning progress in physics at a very basic level.
+
+Yet, the majority of students appears to be able to correctly substitute variables and execute calculations, and is quite content with the ``plugandchug" approach. In fact, it appears to be true that their ``concept anxiety" is more prominent than any ``math anxiety."
+
+Moving beyond initial barriers, the problem with mathematics as part of a formative assessment appears to be not one of {\it operation}, but one of {\it translation}. Students see formulas in a purely operational sense~\cite{torigoe,breitenberger}, while lacking the ability to translate between the formulas and the situations~\cite{clement}, which is also illustrated in the expert and novice quotes at the beginning of Sect.~\ref{intro}.
+
+Online homework systems by the very nature of computers lend themselves to standard calcu\lationoriented problems, and are extensively used in this way. Yet, the ``plugandchug" approach is the most prominent symptom of novicelike problemsolving strategy, and calculationoriented problems may encourage just that. As a result, there is a frequent call for ``conceptual" online problems, where both instructors and students seem to define ``conceptual" simply by the absence of numbers and formulas.
+\begin{itemize}
+\item But does ``depriving" students of numbers and formulas indeed make them work on a conceptual level?
+\item Does it help both students who have problems with applying mathematical methods and those who comfortably ``plugandchug" gain conceptual understanding of physics?
+\end{itemize}
+The following hypotheses reflect these notions in the positive form:
+\begin{quote}
+{\bf Hypothesis 2:} Learners with a low level of mathematical skills or confidence will more likely develop a conceptual understanding of physics as a result of noncalculationoriented online formative assessment, by removing mathematics as a barrier to their understanding.
+\end{quote}
+\begin{quote}
+{\bf Hypothesis 3:} Learners with an average or above average level of mathematical skills or confidence will more likely develop a conceptual understanding of physics as a result of noncalculationoriented online formative assessment, by discouraging nonexpertlike problemsolving strategies.
+\end{quote}
+As evidenced in the above definition, it should be emphasized that the project does by no means attempt to establish or promote a dichotomy between ``conceptual understanding" and ``basic skills/factual knowledge." A physicist needs basic skills and factual knowledge, and the learning of these must not be underemphasized in formative assessment. However, how to best develop these through formative assessment would constitute another valid research project.
+\subsection{Intellectual Merit}
+Online homework is becoming increasingly prominent in physics education, yet research into its effect has been contradictory, sparse, and, in some cases, not very systematic~\cite{pasc04}. Differences have been reported positive~\cite{kashyda}, negative~\cite{pasc04}, and nonsignificant~\cite{bonham}. Pascarella~\cite{pascarella02} was one of the very few studies investigating problemsolving strategies, but both Pascarella~\cite{pascarella02} and Bonham~\cite{bonham} only considered the online versions of conventional textbooklike problems. Kashy~\cite{kashyd01b} found that what the authors call ``interactive problems," namely those where learners need to read relevant values from graphs or observe simulations, are better predictors of overall success in the course than other problem types, but did not investigate cause and effect relationships, or study problemsolving behavior.
+
+This study aims to provide a systematic research base regarding the effectiveness of different types of online formative assessment, especially those which do take better advantage of the medium, and inform both curriculum development efforts and practitioners.
+
+\subsection{Broader Impact/Diversity}
+Currently, every semester approximately 350,000 US students are taking introductory undergraduate physics courses similar to the ones under investigation in this project~\cite{aapt}. For many of these students, it is both their first and their last formal exposure to physics. Students will go into a large spectrum of careers, with or without an understanding of the basic concepts of the physical world.
+
+This project has the potential of broader impact, since like many of the other
+efforts in Physics Education, it is carried out within a regular college
+venue. Results from this study will be applicable especially in
+largeenrollment courses, where for logistical reasons online homework is
+frequently the only feasible formative assessment mechanism. Both the tool
+(LONCAPA, Sect.~\ref{loncapa}) and any developed, implemented, and adapted
+materials (Sect.~\ref{matdev}) will be readily available to physics
+faculty. Faculty members at the over thirty currently participating LONCAPA
+institutions will be able to profit from this project already during its
+progress.
+
+
+\subsection{PeerInstruction}
Since developing Peer Instruction (PI), a collaborative and interactive teaching technique, in 1991, we
have been 1) disseminating the technique, 2) gathering data on its effectiveness (see publication list
below), and 3) developing webbased tools to help instructors implement the method. We have applied
@@ 191,7 +255,15 @@
effective than passive learning. Our experiences with PI, as well as those of many others, who have
responded to our survey, show PI to be an effective collaborative approach to learning.
\subsection{Interactive Learning Toolkit}
+
+
+
+\section{Background and Environment}
+
+\subsection{Institutional Environment}
+Michigan State University is one of the earliest landgrant institutions in the US, and committed to providing equal educational opportunity to all qualified applicants; at the undergraduate level, the university offers comprehensive programs in the liberal arts and sciences, and provides opportunities for students of varying interests, abilities, backgrounds, and expectations. The total enrollment is approximately 44,000, 35,000 of which are undergraduates. 54\% of the student population are women, 8.1\% African American, 5.1\% Asian/Pacific Islander, 2.8\% Chicano/Other Hispanic, and 0.6\% Native American. Of the freshman class, the average high school GPA is 3.58.
+
+\subsection{Model System: Interactive Learning Toolkit}
Two years ago, we published the result of an online survey of PI users in diverse settings. \cite{mref27} This
survey alerted us to some key areas that instructors found challenging, including the time taken to
structure and organize teaching with PI efficiently, the lack of teaching materials (specifically CTs), use
@@ 249,68 +321,6 @@
\end{itemize}
To quote Lin~\cite{lin}: ``The primary determinants of student performance are the specific tasks for which teachers explicitly hold students responsible (e.g. problem sets and exams), rather than the general goals of the teacher (e.g. conveying an appreciation of the power of physics in a broad context)." Mazur~\cite{mazur96} asks ``So why do we keep testing our students with conventional problems?"

The answer, only too often, is scalability, and that in more than one dimension: nonconventional problems are harder to write, and even harder to grade.

The scalability problem is easier to overcome in the classroom: alternative formative assessment as a classroom tool, where students are forced to verbally express their views and teach each other, rather than calculate answers~\cite{mazur}, is starting to be adopted as an effective teaching practice in more and more courses.

It would clearly be advantageous to extend these effective verbalization practices outside the classroom, and offer formative assessment opportunities in which students get to work through and write about reallife problems on a conceptual level, and are explicitly graded on formulating assumptions, developing models, doing backoftheenvelope estimations, and deriving relevant formulas and solutions. Given both time and logistical constraints, except for the occasional ``project assignment," that is not a reality.

This project focuses on how to move beyond conventional homework problems while operating within the realistic limitations of largeenrollment courses.

Particularly in largeenrollment courses, timely feedback is often impossible without the use of computerized homework systems (e.g.~\cite{thoennessen,kashy00}). Unfortunately, an all too frequent approach to using such systems is to simply replicate conventional textbook problems in the online realm, where they are conveniently graded by the computer.

The project assumes that ``the problem with problems" (a phrase borrowed from~\cite{mazur96}) is that
\begin{quote}
{\bf Hypothesis 1a:} Conventional calculationoriented problems affirm nonexpertlike epistemologies and encourage nonexpertlike problemsolving strategies
\end{quote}

Using computerized systems does impose limitations on which kind of problems can be made available, but does not limit educators to just these most basic types. A further assumption is that by the reverse token
\begin{quote}
{\bf Hypothesis 1b:} There are types of online (mostly) computerevaluated problems, which make learners confront their nonexpertlike epistemologies and encourage expertlike problemsolving strategies\end{quote}

These problem types might involve both computer and humanevaluated components, where an emphasis has to be put on keeping the humanevaluated part manageable and scalable.

An additional problem with conventional problems may be their mathematical nature. Hewitt in the preface to his textbook ``Conceptual Physics"~\cite{hewitt} argues that the mathematical language of physics often deters the average nonscience students, a notion which concurs with Tobias' concept of ``math anxiety"\cite{tobias}, which is a particular issue for students in the ``second tier"\cite{tobiasST} of science courses. For them, the use of mathematics in physics courses can present a hurdle, and a lack of skills or confidence to perform basic algebraic manipulations (``$V=RI\ \Rightarrow\ R=V/I"$), or even problems operating their pocket calculators, can hinder students' learning progress in physics at a very basic level.

Yet, the majority of students appears to be able to correctly substitute variables and execute calculations, and is quite content with the ``plugandchug" approach. In fact, it appears to be true that their ``concept anxiety" is more prominent than any ``math anxiety."

Moving beyond initial barriers, the problem with mathematics as part of a formative assessment appears to be not one of {\it operation}, but one of {\it translation}. Students see formulas in a purely operational sense~\cite{torigoe,breitenberger}, while lacking the ability to translate between the formulas and the situations~\cite{clement}, which is also illustrated in the expert and novice quotes at the beginning of Sect.~\ref{intro}.

Online homework systems by the very nature of computers lend themselves to standard calcu\lationoriented problems, and are extensively used in this way. Yet, the ``plugandchug" approach is the most prominent symptom of novicelike problemsolving strategy, and calculationoriented problems may encourage just that. As a result, there is a frequent call for ``conceptual" online problems, where both instructors and students seem to define ``conceptual" simply by the absence of numbers and formulas.
\begin{itemize}
\item But does ``depriving" students of numbers and formulas indeed make them work on a conceptual level?
\item Does it help both students who have problems with applying mathematical methods and those who comfortably ``plugandchug" gain conceptual understanding of physics?
\end{itemize}
The following hypotheses reflect these notions in the positive form:
\begin{quote}
{\bf Hypothesis 2:} Learners with a low level of mathematical skills or confidence will more likely develop a conceptual understanding of physics as a result of noncalculationoriented online formative assessment, by removing mathematics as a barrier to their understanding.
\end{quote}
\begin{quote}
{\bf Hypothesis 3:} Learners with an average or above average level of mathematical skills or confidence will more likely develop a conceptual understanding of physics as a result of noncalculationoriented online formative assessment, by discouraging nonexpertlike problemsolving strategies.
\end{quote}
As evidenced in the above definition, it should be emphasized that the project does by no means attempt to establish or promote a dichotomy between ``conceptual understanding" and ``basic skills/factual knowledge." A physicist needs basic skills and factual knowledge, and the learning of these must not be underemphasized in formative assessment. However, how to best develop these through formative assessment would constitute another valid research project.
\subsection{Intellectual Merit}
Online homework is becoming increasingly prominent in physics education, yet research into its effect has been contradictory, sparse, and, in some cases, not very systematic~\cite{pasc04}. Differences have been reported positive~\cite{kashyda}, negative~\cite{pasc04}, and nonsignificant~\cite{bonham}. Pascarella~\cite{pascarella02} was one of the very few studies investigating problemsolving strategies, but both Pascarella~\cite{pascarella02} and Bonham~\cite{bonham} only considered the online versions of conventional textbooklike problems. Kashy~\cite{kashyd01b} found that what the authors call ``interactive problems," namely those where learners need to read relevant values from graphs or observe simulations, are better predictors of overall success in the course than other problem types, but did not investigate cause and effect relationships, or study problemsolving behavior.

This study aims to provide a systematic research base regarding the effectiveness of different types of online formative assessment, especially those which do take better advantage of the medium, and inform both curriculum development efforts and practitioners.

\subsection{Broader Impact/Diversity}
Currently, every semester approximately 350,000 US students are taking introductory undergraduate physics courses similar to the ones under investigation in this project~\cite{aapt}. For many of these students, it is both their first and their last formal exposure to physics. Students will go into a large spectrum of careers, with or without an understanding of the basic concepts of the physical world.

This project has the potential of broader impact, since like many of the other efforts in Physics Education, it is carried out within a regular college venue. Results from this study will be applicable especially in largeenrollment courses, where for logistical reasons online homework is frequently the only feasible formative assessment mechanism. Both the tool (LONCAPA, Sect.~\ref{loncapa}) and any developed, implemented, and adapted materials (Sect.~\ref{matdev}) will be readily available to physics faculty. Faculty members at the over thirty currently participating LONCAPA institutions will be able to profit from this project already during its progress.

\section{Background and Environment}
\subsection{PI Education and Appointments}
Dr.~Kortemeyer received his Diplom (ÒM.Sc.Ó) in physics in 1993 from the Universit\"at Hannover, Germany (Advisor Prof. P. U. Sauer), and his Ph.D. in physics from Michigan State University in 1997 (Advisor Prof. W. Bauer), both with thesis work in theoretical nuclear physics.

He has been working at Michigan State University since 1997. His first appointment has been as an Academic Specialist in the Division of Science and Mathematics Education (DSME), where he has been leading instructional technology development projects for the College of Natural Science, and is the director of the Learning{\it Online} Network with ComputerAssisted Personalized Approach (LONCAPA) project, see Sect.~\ref{loncapa}. He also taught introductory physics in a completely online mode, as well as cotaught in a more traditional oncampus setting.

Starting August 2004, Dr.~Kortemeyer has been working in a tenuretrack position as Assistant Professor of Physics Education. His appointment will be split 75/25\% between the Lyman Briggs School of Science (LBS) and DSME. He is holding an appointment as Adjunct Professor of Physics in the Department of Physics and Astronomy. His teaching responsibilities include the introductory calculusbased physics sequence (lecture and lab) in LBS, as well as seminars in special topics. His research will be focused on postsecondary science teaching and learning, with a special emphasis on the use of technology.

\subsection{Institutional Environment}
Michigan State University is one of the earliest landgrant institutions in the US, and committed to providing equal educational opportunity to all qualified applicants; at the undergraduate level, the university offers comprehensive programs in the liberal arts and sciences, and provides opportunities for students of varying interests, abilities, backgrounds, and expectations. The total enrollment is approximately 44,000, 35,000 of which are undergraduates. 54\% of the student population are women, 8.1\% African American, 5.1\% Asian/Pacific Islander, 2.8\% Chicano/Other Hispanic, and 0.6\% Native American. Of the freshman class, the average high school GPA is 3.58.
\subsection{Model System: The Learning{\it Online} Network with CAPA}\label{loncapa}
The Learning{\it Online} Network with ComputerAssisted Personalized Approach ({\tt http://www.loncapa.org/}) is a distributed learning content management, course management, and assessment system, and also the model system of the current NSFITR grant, see Sect.~\ref{results}.
www1109123626