[LON-CAPA-cvs] cvs: modules /gerd/concept description.tex

[www]

This is a MIME encoded message

--www1089835152
Content-Type: text/plain

www		Wed Jul 14 15:59:12 2004 EDT

  Modified files:              
    /modules/gerd/concept	description.tex 
  Log:
  Next round of grant proposal.
  
  
--www1089835152
Content-Type: text/plain
Content-Disposition: attachment; filename="www-20040714155912.txt"

Index: modules/gerd/concept/description.tex
diff -u modules/gerd/concept/description.tex:1.11 modules/gerd/concept/description.tex:1.12
--- modules/gerd/concept/description.tex:1.11	Sat Jul 10 16:16:28 2004
+++ modules/gerd/concept/description.tex	Wed Jul 14 15:59:11 2004
@@ -52,15 +52,31 @@
 \begin{flushright}\sc Student, MSU phy183~\cite{student}\end{flushright}
 \end{quote}
 
+\subsection{Overview}
+This five-year project focusses on online formative assessment in introductory physics education, and how formative assessment can be used to help learners re-evaluate their epistemologies, develop expertlike problem solving skills, and gain a conceptual understanding of physics. It will compare the impact of online problems which are categorized across 21 types previously identified in literature. 
+
+The study will be carried out in on-campus courses with regular classroom times, which are enhanced by online components. The NSF-supported Learning{\it Online} Network with Computer-Assisted Personalized Approach (LON-CAPA) will be used as the model system.
+
+The project has three components:
+\begin{enumerate}
+\item new development, as well as adaptation and implementation of research-based problems inside of LON-CAPA, where the currently existing library of problems does not have a sufficient number of representatives of this type 
+\item additional tool development inside of LON-CAPA to provide
+\begin{itemize}
+\item higher scalability for deploying problem types that cannot be completely evaluated by the computer
+\item better analysis tools for the purpose of this study
+\end{itemize}
+\item hypotheses testing for each problem type regarding their educational impact
+\end{enumerate}
+
 \subsection{"Thinking like a Physicist"}
 Most physicists would agree that part of "being a physicist" is to "think like a physicist"~\cite{heuvelen}, which manifests itself most prominently in the cognitive and metacognitive skills involved in problem solving~\cite{redish} --- part of the self-image of many physicists is to be expert problem solvers, even outside their own discipline~\cite{fuller}:
 \[\mbox{Think like a Physicist}\ \Rightarrow\ \mbox{Solve Problems}\]
 Yet, while basic logic tells us that the reverse statement, i.e., 
 \[\mbox{Think like a Physicist}\ \Leftarrow\ \mbox{Solve Problems}\]
 does not necessarily have to be true,
-an unfortunately only too common approach to teaching physics is to simply have students solve a lot of standard problems. 
+an unfortunately all too common approach to teaching physics is to simply have students solve a lot of standard problems. 
 
-It is broadly accepted that frequent formative assessment and feedback are a key component of the learning process~\cite{bransford}, but {\it what} the students are learning is not necessarily what educators are expecting them to learn~\cite{pellegrino}. This disconnect frequently goes undetected if the summative assessment tools are similar to the formative assessment tools~\cite{lin}. In most physics courses, these are standard problems along the lines of "A ball starts with an initial velocity of \ldots. \ldots What is the \ldots?". Deploying alternative conceptual assessment tools such as the Force Concept Inventory~\cite{fci} can reveal large discrepancies between different summative assessment types (e.g,~\cite{steinberg,mazur}).
+It is broadly accepted that frequent formative assessment and feedback are a key component of the learning process~\cite{bransford}, but {\it what} the students are learning is not necessarily what educators are expecting them to be learning~\cite{pellegrino,arons}. This disconnect frequently goes undetected if the summative assessment tools are similar to the formative assessment tools~\cite{lin}. In most physics courses, these are conventional problems along the lines of "A ball starts with an initial velocity of \ldots. \ldots What is the \ldots?". Deploying alternative conceptual assessment tools such as the Force Concept Inventory~\cite{fci} can reveal large discrepancies between different summative assessment types (e.g,~\cite{steinberg,mazur}).
 
 Expert and novice approaches to problem solving in physics have been studied extensively (e.g.~\cite{chi,larkin}). Two of the most apparent differences are that
 \begin{enumerate}
@@ -76,34 +92,44 @@
 \item Erase all information from your brain after the exam to make room for the next set of materials.
 \end{itemize}
 
-One cannot really blame learners for short-circuiting physics "learning" this way, since the cognitive and metacognitive skills, which physicists value so highly, are hardly ever made explicit, neither in instruction, nor in formative or summative assessment~\cite{lin,reif,mazur96}; in fact, they are mostly altogether "hidden"~\cite{redish} from all aspects of a course, and students are affirmed in their novice expectations~\cite{hammer} of what it is to "do physics." To quote Lin~\cite{lin}: "The primary determinants of student performance are the specific tasks for which teachers explicitly hold student responsible (e.g. problem sets and exams), rather than the general goals of the teacher (e.g. conveying an appreciation of the power of physics in a broad context)."
+One cannot really blame learners for short-circuiting physics "learning" this way, since the cognitive and metacognitive skills, which physicists value so highly, are hardly ever made explicit, neither in instruction, nor in formative or summative assessment~\cite{lin,reif,mazur96}; in fact, they are mostly altogether "hidden"~\cite{redish} from all aspects of a course, and students are affirmed in their novice expectations~\cite{hammer} of what it is to "do physics." The challenge is to move students away from treating physics as a set of unrelated factoids and formulas, as well as away from focussing on memorizing and using formulas without interpretation or sense-making~\cite{hammer}, and toward both "thinking like a physicist" and gaining conceptual understanding.
+\begin{quote}
+"Conceptual understanding" in this project is defined as insight,
+as reflected in thoughtful and effective use of knowledge and skills in varied situations,
+into abstract key ideas,
+which are generalized from particular instances.
+\end{quote}
+\subsection{The Problem with Problems - Hypotheses}\label{hypo}
+To quote Lin~\cite{lin}: "The primary determinants of student performance are the specific tasks for which teachers explicitly hold student responsible (e.g. problem sets and exams), rather than the general goals of the teacher (e.g. conveying an appreciation of the power of physics in a broad context)." Mazur~\cite{mazur96} asks "So why do we keep testing our students with conventional problems?"
 
-\subsection{The Problem with Problems}\label{hypo}
-{\bf This project focusses on formative assessment in introductory physics education, and how formative assessment can be used to help learners re-evaluate their epistemologies, develop expertlike problem solving skills, and gain a conceptual understanding of physics.} The challenge is to move students away from treating physics as a set of unrelated factoids and formulas, as well as away from focussing on memorizing and using formulas without interpretation or sense-making~\cite{hammer}, and toward "thinking like a physicist."
+The answer, only too often, is scalability, and that in more than one dimension: non-conventional problems are harder to write, and even harder to grade.
 
+The scalability problem is easier to overcome in the classroom: alternative formative assessment as a classroom tool, where students are forced to verbally  express their views and teach each other, rather than calculate answers~\cite{mazur}, is starting to be adopted as an effective teaching practice in more and more courses. 
 
-{\bf "Conceptual understanding" in this project is defined as insight,
-as reflected in thoughtful and effective use of knowledge and skills in varied situations,
-into abstract key ideas,
-which are generalized from particular instances.}
+It would clearly be advantageous to extend these effective verbalization practices outside the classroom, and offer formative assessment opportunities in which students get to work through and write about real-life problems on a conceptual level, and are explicitly grading on formulating assumptions, developing models, doing back-of-the-envelope estimations, and deriving relevant formulas and solutions. Given both time and logistical constraints, that is not a reality.
+
+This project focusses on how to move beyond conventional homework problems while operating within the realistic limitations of large-enrollment courses.
 
-Alternative formative assessment as a classroom tool, where students are forced to verbally  express their views and teach each other, rather than calculate answers~\cite{mazur}, is starting to be adopted as an effective teaching practice in more and more courses. Formative assessment outside the classroom, on the other hand, is frequently limited by logistics, particularly in large enrollment courses, where timely feedback is often impossible without the use of computerized systems (e.g.~\cite{thoennessen,kashy00}). {\bf This project will focus on online homework as formative assessment tool.} As a model system, the project will use the Learning{\it Online} Network with CAPA (LON-CAPA), described in section~\ref{loncapa}.
+Particularly in large-enrollment courses, timely feedback is often impossible without the use of computerized homework systems (e.g.~\cite{thoennessen,kashy00}). Unfortunately, an all too frequent approach to using such systems is to simply replicated conventional textbook problems in the online realm, where they are conveniently graded by the computer.
 
 The project assumes that "the problem with problems" (a phrase borrowed from~\cite{mazur96}) is that
 \begin{quote}
-{\bf Hypothesis 1a:} Standard calculation-oriented textbook problems affirm non-expertlike epistemologies and encourage non-expertlike problem-solving strategies
+{\bf Hypothesis 1a:} Conventional calculation-oriented textbook problems affirm non-expertlike epistemologies and encourage non-expertlike problem-solving strategies
 \end{quote}
-A further assumption is that by the reverse token
+
+Using computerized systems does impose limitations on which kind of problems can be made available, but does not limit one just these most basic types.  A further assumption is that by the reverse token
 \begin{quote}
 {\bf Hypothesis 1b:} There are types of online formative assessment computer-evaluated problems which make learners confront their non-expertlike epistemologies and encourage expertlike problem-solving strategies\end{quote}
 
-Hewitt in the preface to his textbook "Conceptual Physics"~\cite{hewitt} argues that the mathematical language of physics often deters the average non-science students, a notion which concurs with Tobias' concept of "math phobia"\cite{tobias}, which is a particular issue for students in the "second tier"\cite{tobiasST} of science course.  For them, the use of mathematics in physics courses can present a hurdle, and a lack of skills or confidence to perform basic algebraic manipulations ("$V=RI\ \Rightarrow\ R=V/I"$), or even problems operating their pocket calculators, can hinder students' learning progress in physics at a very basic level.
+These problem types might involve both computer- and human-evaluated components, where an emphasis has to be put on keeping the human-evaluated part managable and scalable.
+
+An additional problem with conventional problems may be their mathematical nature. Hewitt in the preface to his textbook "Conceptual Physics"~\cite{hewitt} argues that the mathematical language of physics often deters the average non-science students, a notion which concurs with Tobias' concept of "math anxiety"\cite{tobias}, which is a particular issue for students in the "second tier"\cite{tobiasST} of science courses.  For them, the use of mathematics in physics courses can present a hurdle, and a lack of skills or confidence to perform basic algebraic manipulations ("$V=RI\ \Rightarrow\ R=V/I"$), or even problems operating their pocket calculators, can hinder students' learning progress in physics at a very basic level.
 
-Yet, the majority of students appears to be able to correctly substitute variables and execute calculations, and quite content with the "plug-and-chug" approach. In fact, it appears to be true that their "concept phobia" is more prominent than any "math phobia."
+Yet, the majority of students appears to be able to correctly substitute variables and execute calculations, and quite content with the "plug-and-chug" approach. In fact, it appears to be true that their "concept anxiety" is more prominent than any "math anxiety."
 
 Moving beyond initial barriers, the problem with mathematics as part of a formative assessment  appears to be not one of {\it operation}, but one of {\it translation}. Students see formulas in a purely operational sense~\cite{torigoe,breitenberger}, while lacking the ability to translate between the formulas and the situations~\cite{clement}, which is also illustrated in the expert and novice quotes at the beginning of section~\ref{intro}.
 
-Online homework systems by the very nature of computers lend themselves to standard calcu\-lation-oriented textbook problems, and are extensively used in this way. Yet, the "plug-and-chug" approach is the most prominent symptom of novice-like problem-solving strategy, and calculation-oriented problems may encourage just that. As a result, there is a frequent call for "conceptual" online problems, where {\bf both instructors and students seem to define "conceptual" simply by the absence of numbers and formulas}. 
+Online homework systems by the very nature of computers lend themselves to standard calcu\-lation-oriented textbook problems, and are extensively used in this way. Yet, the "plug-and-chug" approach is the most prominent symptom of novice-like problem-solving strategy, and calculation-oriented problems may encourage just that. As a result, there is a frequent call for "conceptual" online problems, where both instructors and students seem to define "conceptual" simply by the absence of numbers and formulas. 
 \begin{itemize}
 \item But does "depriving" students of numbers and formulas indeed make them work on a conceptual level?
 \item Does it help both students who have problems with applying mathematical methods and those who comfortably "plug-and-chug" gain conceptual understanding of physics?
@@ -115,20 +141,23 @@
 \begin{quote}
 {\bf Hypothesis 3:} Learners with an average or above average level of mathematical skills or confidence will more likely develop a conceptual understanding of physics as a result of non-calculation-oriented online formative assessment, by discouraging non-expertlike problem-solving strategies.
 \end{quote}
-As evidenced in the above definition, it should be emphasized that the project does by no means attempt to establish or promote a dichotomomy between "conceptual understanding" and "basic skills/factual knowledge." {\it A physicist needs basic skills and factual knowledge, and their learning must not be underemphasized in formative assessment.} However, how to best develop these through formative assessment would constitute another valid research project.
-\subsection{Intellectual Merit}Much effort and many resources have been invested into developing effective curricular material and assessment, especially in the interactive or online realm, yet very little research has been done on the impact of different representations and question types on student conceptual understanding.The outcomes of this study will provide a broader research base for STEM curriculum development efforts regarding the most effective use of learner feedback. The outcomes will also inform development efforts for online course and learning content management systems, as well as provide input for educational metadata, content exchange, and interoperability standard efforts.
+As evidenced in the above definition, it should be emphasized that the project does by no means attempt to establish or promote a dichotomomy between "conceptual understanding" and "basic skills/factual knowledge." A physicist needs basic skills and factual knowledge, and the learning of these must not be underemphasized in formative assessment. However, how to best develop these through formative assessment would constitute another valid research project.
+\subsection{Intellectual Merit}The outcomes of this study will provide a broader research base for STEM curriculum development efforts regarding the most effective use of online assessment.
+
+
+The outcomes will also inform development efforts for online course and learning content management systems, as well as provide input for educational metadata, content exchange, and interoperability standard efforts.
 \subsection{Broader Impact/Diversity}
-Currently, every semester approximately 350,000 students are taking introductory undergraduate physics courses similar to the ones under investigation in this project~\cite{aapt}.
+Currently, every semester approximately 350,000 students are taking introductory undergraduate physics courses similar to the ones under investigation in this project~\cite{aapt}. For many of these students, it is both their first and their last formal exposure to physics.
 It is broadly accepted that frequent formative assessment and feedback are a key component of the learning process~\cite{bransford}. Shifting the focus from summative to formative assessment with feedback can move student motivation from an extrinsic reward to the intrinsic reward of developing understanding of the materials~\cite{stipek}. Intrinsic motivation and positive feedback promote the feelings of competence, confidence~\cite{clark,pascarella02}, and engagement that are crucial to retaining students in introductory STEM courses.  While improving student self-efficacy should have positive impacts on all student retention, Seymour and Hewitt~\cite{seymour} suggest that such changes should have a particularly strong impact on the attrition of women and underrepresented groups in science, who may often feel that science excludes them.
 
 
 
 \section{Background and Environment}
 \subsection{PI Education and Appointments}
-Dr.~Kortemeyer received his Diplom (ÒM.Sc.Ó) in physics in 1993 from the UniversitŠt Hannover, Germany (Advisor Prof. P. U. Sauer), and his Ph.D. in physics from Michigan State University in 1997 (Advisor Prof. W. Bauer).He has been working at Michigan State University since 1997. His first appointment has been as an Academic Specialist in the Division of Science and Mathematics Education (DSME), where he has been leading instructional technology development projects for the College of Natural Science, and is the director of the Learning{\it Online} Network with Computer-Assisted Personalized Approach (LON-CAPA) project, see section~\ref{loncapa}.  He also taught introductory physics in a completely online mode, as well as co-taught in a more traditional on-campus setting.
+Dr.~Kortemeyer received his Diplom (ÒM.Sc.Ó) in physics in 1993 from the Universit\"at Hannover, Germany (Advisor Prof. P. U. Sauer), and his Ph.D. in physics from Michigan State University in 1997 (Advisor Prof. W. Bauer).He has been working at Michigan State University since 1997. His first appointment has been as an Academic Specialist in the Division of Science and Mathematics Education (DSME), where he has been leading instructional technology development projects for the College of Natural Science, and is the director of the Learning{\it Online} Network with Computer-Assisted Personalized Approach (LON-CAPA) project, see section~\ref{loncapa}.  He also taught introductory physics in a completely online mode, as well as co-taught in a more traditional on-campus setting.
 
-Starting August 2004, Dr.~Kortemeyer will be working in a tenure-track position as Assistant Professor of Physics Education. His appointment will be split 75/25\% between the Lyman Briggs School of Science (LBS) and DSME. He will also be holding an appointment as Adjunct Professor of Physics in the Department of Physics and Astronomy. His teaching responsibilities will include the introductory calculus-based physics sequence (lecture and lab) in LBS, as well as seminars in special topics. His research will be focused on postsecondary science teaching and learning, with a special emphasis on the use of technology.\subsection{Michigan State University}Michigan State University is one of the earliest land-grant institutions in the United States. MSU is committed to providing equal educational opportunity to all qualified applicants; at the undergraduate level, the university offers comprehensive programs in the liberal arts and sciences, and provides opportunities for students of varying interests, abilities, backgrounds, and expectations.  The total enrollment is approximately 44,000, 35,000 of which are undergraduates. 54\% of the student population are women, 8.1\% African American, 5.1\% Asian/Pacific Islander, 2.8\% Chicano/Other Hispanic, and 0.6\% Native American. Of the freshman class, the average high school GPA is 3.58.\subsection{Lyman Briggs School of Science}The Lyman Briggs School (LBS) at Michigan State University is a residential learning community devoted to studying the natural sciences and their impact on society. All under one roof, LBS encompasses physics, chemistry, biology, and computer laboratories; classrooms; faculty, administrative, and academic support staff offices; student residences; a dining hall; and a convenience store. With approximately 1500 students, LBS offers the benefits of a small, liberal arts college with the resources of a large research university.\subsection{Division of Science and Mathematics Education}The Division of Science and Mathematics Education (DSME) was founded at Michigan State University in 1989, and is co-administered by the College of Natural Science and the College of Education. Academic specialists and faculty members with partial appointments in various departments and other colleges, graduate and undergraduate students, and professional and clerical staff work together in DSME to conduct a variety of research projects, as well as to offer courses, degree programs, and other activities in support of its mission.
-\subsection{Model System: The Learning{\it Online} Network with CAPA}\label{loncapa}For several aspects of the proposed project, the Learning{\it Online} Network with Computer-Assisted Personalized Approach (LON-CAPA; {\tt http://www.lon-capa.org/}) will be the model system. LON-CAPA is a distributed learning content management, course management, and assessment system, and also the model system of the current NSF-ITR grant, see section~\cite{results}. 
+Starting August 2004, Dr.~Kortemeyer will be working in a tenure-track position as Assistant Professor of Physics Education. His appointment will be split 75/25\% between the Lyman Briggs School of Science (LBS) and DSME. He will also be holding an appointment as Adjunct Professor of Physics in the Department of Physics and Astronomy. His teaching responsibilities will include the introductory calculus-based physics sequence (lecture and lab) in LBS, as well as seminars in special topics. His research will be focused on postsecondary science teaching and learning, with a special emphasis on the use of technology.\subsection{Michigan State University}Michigan State University is one of the earliest land-grant institutions in the United States. MSU is committed to providing equal educational opportunity to all qualified applicants; at the undergraduate level, the university offers comprehensive programs in the liberal arts and sciences, and provides opportunities for students of varying interests, abilities, backgrounds, and expectations.  The total enrollment is approximately 44,000, 35,000 of which are undergraduates. 54\% of the student population are women, 8.1\% African American, 5.1\% Asian/Pacific Islander, 2.8\% Chicano/Other Hispanic, and 0.6\% Native American. Of the freshman class, the average high school GPA is 3.58.\subsection{Lyman Briggs School of Science}The Lyman Briggs School (LBS) at Michigan State University is a residential learning community devoted to studying the natural sciences and their impact on society. All under one roof, LBS encompasses physics, chemistry, biology, and computer laboratories; classrooms; faculty, administrative, and academic support staff offices; student residences; a dining hall; and a convenience store. With approximately 1500 students, LBS offers the benefits of a small, liberal arts college with the resources of a large research university. Within LBS, 59\% of the student population are women, 15\% ethnic minority including 1\% Hispanic, 3\% African American, and 9\% Asian/Pacific.\subsection{Division of Science and Mathematics Education}The Division of Science and Mathematics Education (DSME) was founded at Michigan State University in 1989, and is co-administered by the College of Natural Science and the College of Education. Academic specialists and faculty members with partial appointments in various departments and other colleges, graduate and undergraduate students, and professional and clerical staff work together in DSME to conduct a variety of research projects, as well as to offer courses, degree programs, and other activities in support of its mission.
+\subsection{Model System: The Learning{\it Online} Network with CAPA}\label{loncapa}For several aspects of the proposed project, the Learning{\it Online} Network with Computer-Assisted Personalized Approach (LON-CAPA; {\tt http://www.lon-capa.org/}) will be the model system. LON-CAPA is a distributed learning content management, course management, and assessment system, and also the model system of the current NSF-ITR grant, see section~\ref{results}. 
 
 LON-CAPAÕs core development group is located at Michigan State University, and in addition to faculty members, has a staff of three fulltime programmers, two user support staff, one technician, one graduate student, and one project coordinator. The LON-CAPA group also offers training and support for adopters of the system.
 
@@ -141,11 +170,11 @@
 \end{figure}
 In the context of this project, this feature is important in two aspects:
 \begin{itemize}
-\item results are not tainted by students simply exchanging the answers, i.e., every student in the end has to work out their own answers
+\item results are not tainted by students simply exchanging the answers, i.e., each student in the end has to work out his or her own answers
 \item as a result, lively discussions take place, both online and in the helproom --- both of which will be analyzed in this project, see section~\ref{discussion}
 \end{itemize}
 
-Students are generally given immediate feedback on the correctness of their solutions, and in some cases additional help. They are usually granted multiple attempts to get a problem correct. This allows to follow a learner's thought process, both through statistical analysis (see~\ref{analysis}) and data-mining approaches.
+Students are generally given immediate feedback on the correctness of their solutions, and in some cases additional help. They are usually granted multiple attempts to get a problem correct. This allows the instructor to follow a learner's thought process, both through statistical analysis (see~\ref{analysis}) and data-mining approaches.
 
 The system also allows for free-form essay-type answers, which are however graded by humans with the assistance of the system (keyword-highlighting, plagiarism-checks, etc).\subsubsection{Course Management}Over the years, the system added a learning content management system and standard course management features, such as communications, gradebook, etc., which are comparable to commercial course management systems, such as BlackBoard, WebCT, or ANGEL. See 
 Refs.~\cite{features,edutools} for an overview of features, and comparisons to other systems.In addition to standard features, the LON-CAPA delivery and course management layer is designed around STEM education, for example: support for mathematical typesetting throughout (\LaTeX\ inside of XML) Ð formulas are rendered on-the-fly, and can be algorithmically modified through the use of variables inside formulas; integrated GNUplot support, such that graphs can be rendered on-the-fly, and allowing additional layered labeling of graphs and images; support for multi-dimensional symbolic math answers; and full support of physical units.
@@ -158,8 +187,6 @@
 The Lyman-Briggs School of Science Collaborative Learning Laboratory, which is expected to be completed in 2005. It is modeled in part after a setup by the North Carolina State University Physics Education R\&D Group~\cite{ncsu}, and offers a space where students can collaborate on homework while their interactions and online transactions are recorded.
 
 In addition to having whiteboards and wireless laptop computers for students to work with in flexible group settings, the facility will have integrated observation equipment to video- and audio-record student interactions. All recorded information is immediately digitized and made available for transcription and analysis using the Transana~\cite{transana} software system.
-
-
 \subsubsection{Analysis Capabilities}\label{analysis}
 LON-CAPA allows instructors to analyze student submissions both for individual students (Fig.~\ref{problemview}) and across the course (Fig.~\ref{problemanalysis}).
 
@@ -176,6 +203,8 @@
 The project will be carried out  in the two-semester LBS course sequence LBS 271/272,"Calculus-Based Introductory Physics I/II. These second-year three-credit courses have a Calculus pre-requisite, and traditionally an enrollment of over 200 students. Two separate, but associated one-credit laboratory courses (LBS 271L/272L) are required, which most but not all students choose to take simultaneously.
 
 Faculty and teaching assistants are frequently assuming shared responsibilities between the lecture and laboratory courses, with a combined staff of two faculty members and six undergraduate student assistants. The latter are responsible for particular recitation and laboratory sections, and will be involved in this research project (see Section~\ref{undergrad}). Within the duration of this project, the lecture and laboratory courses might be combined to provide greater coherence between these two venues.
+
+Students in these courses are currently solving approximately 250 online homework problems each semester, most of which currently are of the conventional type.
 \subsection{Synergy between Project and Institutional Goals}
 The mission statement of the Lyman-Briggs School of Science includes the statement
 \begin{quote}
@@ -185,8 +214,8 @@
 \begin{quote}
 to improve science and mathematics education, from kindergarten through the undergraduate years, through the professional development of preservice and inservice teachers and faculty members. 
 \end{quote}
-These ambitious goals clearly mean going beyond a surface-recapitulation of disconnected factoids, toward a deep appreciation of natural phenomena and their inter-connectedness. Research and teaching are closely intertwined in this project, with the goal of improving the learning and teaching
-of the natural sciences, even as the study itself is being carried out. The outcomes of this study will inform faculty members, and aligns with the goals of the Division of Science and Mathematics Education.
+These ambitious goals clearly infer going beyond a surface-recapitulation of disconnected factoids, toward a deep appreciation of natural phenomena and their inter-connectedness. Research and teaching are closely intertwined in this project, with the goal of improving the learning and teaching
+of the natural sciences, even as the study itself is being carried out. The outcomes of this study will inform faculty members, and aligned with the goals of the Division of Science and Mathematics Education.
 \section{Classification of Online Formative Assessment Problems}\label{class}
 Redish~\cite{redish} distinguishes eight types of exam and homework questions, an adapted version of which will form the general classification scheme for Hypothesis 1b:
 \begin{description}
@@ -198,7 +227,7 @@
 
 \begin{figure}
 \includegraphics[width=6.5in]{threemassesjpg}
-\caption{Example of two LON-CAPA problems addressing the same concepts. The problem on the left is a traditional short-numerical-answer problem, while the problem on the right is of type "multiple-choice multiple-response."\label{threemasses}}
+\caption{Example of two LON-CAPA problems addressing the same concepts. The problem on the left is a conventional short-numerical-answer problem, while the problem on the right is of type "multiple-choice multiple-response."\label{threemasses}}
 \end{figure}
 
 \item[Representation-translation questions] This type of problem requires a student to translate between different representations of the same situation, for example from a graphical to a numerical or textual representation. The answer might be given in different formats, for example in the problem on the right side of Fig.~\ref{trajectory}, it is a short-numerical-answer. Translation between representations can be surprisingly challenging for physics learners~\cite{mcdermott,beichner}.
@@ -207,7 +236,7 @@
 
 \begin{figure}
 \includegraphics[width=6.5in]{trajectoryjpg}
-\caption{Example of two LON-CAPA problems addressing the same concepts in two different representations. The problem on the left is a traditional short-numerical-answer problem, while the problem on the right requires "representation-translation."\label{trajectory}}
+\caption{Example of two LON-CAPA problems addressing the same concepts in two different representations. The problem on the left is a conventional short-numerical-answer problem, while the problem on the right requires "representation-translation."\label{trajectory}}
 \end{figure}
 
 \item[Ranking-tasks] This type of problem requires a student to rank a number of statements, scenarios, or objects with respect to a certain feature. For example, a student might be asked to rank a number of projectiles in the order that they will hit the ground, or a number of points in order of the strength of their local electric potential.
@@ -231,7 +260,7 @@
 &\multicolumn{4}{|c|}{Multiple-choice and short-answer}&Multiple-choice multiple-response&Ranking&Esti\-mation&Quali\-tative&Essay\\
 &Multiple-choice&Tex\-tual&Nume\-rical&For\-mula&&&&&\\
 \hline
-"Traditional"&S&AMS&S&AMS&S&AS&&&\\\hline
+"Conventional"&S&AMS&S&AMS&S&AS&&&\\\hline
 Repre\-sentation-translation&AS&AMS&AS&AMS&AS&AS&&&\\\hline
 Context-based&MS&AMS&MS&AMS&MS&AMS&GMS&GMS&GMS\\\hline
 \end{tabular}
@@ -444,6 +473,7 @@
 \bibitem{fuller} Robert G. Fuller, {\it Solving physics problems --- how do we do it?}, Phys. Today {\bf  35}(9), 43-47 (1982)
 \bibitem{bransford} John D. Bransford, Ann L. Brown, and Rodney R. Cocking (editors), {\it How people learn (expanded edition)}, National Research Council, ISBN 0-309-07036-8 (2000)
 \bibitem{pellegrino} James. W. Pellegrino, Naomi Chudowsky, and Robert Glaser (editors), {\it Knowing what students know}, National Academy Press, ISBN 0-309-07272-7 (2001)
+\bibitem{arons} Arnold B. Arons, {\it Teaching Introductory Physics}, Wiley, ISBN 0-471-13707-3 (1997)
 \bibitem{lin} Herbert Lin, {\it Learning physics vs. passing courses}, Phys. Teach. {\bf 20}, 151-157 (1982)
 \bibitem{fci} D. Hestenes, M. Wells, and G. Swackhamer, {\it Force Concept Inventory}, Phys. Teach. {\bf 30}, 141-158 (1992)
 \bibitem{steinberg} Richard Steinberg, {\it Performance on Multiple-Choice Diagnostics and Complementary Exam Problems}, Phys. Teach {\bf 35}(3), 150-155.

--www1089835152--