[LON-CAPA-cvs] cvs: modules /gerd/concept description.tex

[www]

This is a MIME encoded message

--www1089890305
Content-Type: text/plain

www		Thu Jul 15 07:18:25 2004 EDT

  Modified files:              
    /modules/gerd/concept	description.tex 
  Log:
  Typos
  
  
--www1089890305
Content-Type: text/plain
Content-Disposition: attachment; filename="www-20040715071825.txt"

Index: modules/gerd/concept/description.tex
diff -u modules/gerd/concept/description.tex:1.13 modules/gerd/concept/description.tex:1.14
--- modules/gerd/concept/description.tex:1.13	Wed Jul 14 17:16:30 2004
+++ modules/gerd/concept/description.tex	Thu Jul 15 07:18:24 2004
@@ -34,7 +34,6 @@
 \begin{center}
 \LARGE\sc Physics education:\\ Does "conceptual" online formative assessment lead to conceptual understanding?
 \end{center}
-
 \section{Goals and Objectives}\label{intro}
 \begin{quote}
 Mathematics is {\it not} just another language. Mathematics is a language plus reasoning; it is a language plus logic. Mathematics is a tool for reasoning. It is in fact a big collection of the results of some person's careful thought and reasoning. By mathematics it is possible to connect one statement to another.
@@ -53,19 +52,19 @@
 \end{quote}
 
 \subsection{Overview}
-This five-year project focusses on online formative assessment in introductory physics education, and how formative assessment can be used to help learners re-evaluate their epistemologies, develop expertlike problem solving skills, and gain a conceptual understanding of physics. It will compare the impact of online problems which are categorized across 21 types previously identified in literature. 
+This five-year project focusses on online formative assessment in introductory physics education, and how it can be used to help learners re-evaluate their epistemologies, develop expertlike problem solving skills, and gain a conceptual understanding of physics. It will compare the impact of online problems which are categorized across 21 types previously identified in literature. 
 
-The study will be carried out in on-campus courses with regular classroom times, which are enhanced by online components. The NSF-supported Learning{\it Online} Network with Computer-Assisted Personalized Approach (LON-CAPA; Section~\ref{loncapa}) will be used as the model system.
+The study will be carried out in on-campus courses with regular classroom times, which are enhanced by online components. The NSF-supported Learning{\it Online} Network with Computer-Assisted Personalized Approach (LON-CAPA; Sect.~\ref{loncapa}) will be used as the model system.
 
 The project has three components:
 \begin{enumerate}
-\item new development, as well as adaptation and implementation of research-based problems inside of LON-CAPA, where the currently existing library of problems does not have a sufficient number of representatives of this type (Section~\ref{matdev})
+\item new development, as well as adaptation and implementation of research-based problems inside of LON-CAPA, where the currently existing library of problems does not have a sufficient number of representatives of this type (Sect.~\ref{matdev})
 \item additional tool development inside of LON-CAPA to provide
 \begin{itemize}
-\item higher scalability for deploying problem types that cannot be completely evaluated by the computer (Section~\ref{platform})
-\item better analysis tools for the purpose of this study (Section~\ref{analysisnew})
+\item higher scalability for deploying problem types that cannot be completely evaluated by the computer (Sect.~\ref{platform})
+\item better analysis tools for the purpose of this study (Sect.~\ref{analysisnew})
 \end{itemize}
-\item hypotheses testing for each problem type regarding their educational impact (Section~\ref{analysis})
+\item hypotheses-testing for each problem type regarding their educational impact (Sect.~\ref{analysis})
 \end{enumerate}
 
 \subsection{"Thinking like a Physicist"}
@@ -106,7 +105,7 @@
 
 The scalability problem is easier to overcome in the classroom: alternative formative assessment as a classroom tool, where students are forced to verbally  express their views and teach each other, rather than calculate answers~\cite{mazur}, is starting to be adopted as an effective teaching practice in more and more courses. 
 
-It would clearly be advantageous to extend these effective verbalization practices outside the classroom, and offer formative assessment opportunities in which students get to work through and write about real-life problems on a conceptual level, and are explicitly grading on formulating assumptions, developing models, doing back-of-the-envelope estimations, and deriving relevant formulas and solutions. Given both time and logistical constraints, that is not a reality.
+It would clearly be advantageous to extend these effective verbalization practices outside the classroom, and offer formative assessment opportunities in which students get to work through and write about real-life problems on a conceptual level, and are explicitly graded on formulating assumptions, developing models, doing back-of-the-envelope estimations, and deriving relevant formulas and solutions. Given both time and logistical constraints, except for the occasional "project assignment," that is not a reality.
 
 This project focusses on how to move beyond conventional homework problems while operating within the realistic limitations of large-enrollment courses.
 
@@ -114,7 +113,7 @@
 
 The project assumes that "the problem with problems" (a phrase borrowed from~\cite{mazur96}) is that
 \begin{quote}
-{\bf Hypothesis 1a:} Conventional calculation-oriented textbook problems affirm non-expertlike epistemologies and encourage non-expertlike problem-solving strategies
+{\bf Hypothesis 1a:} Conventional calculation-oriented problems affirm non-expertlike epistemologies and encourage non-expertlike problem-solving strategies
 \end{quote}
 
 Using computerized systems does impose limitations on which kind of problems can be made available, but does not limit one just these most basic types.  A further assumption is that by the reverse token
@@ -127,9 +126,9 @@
 
 Yet, the majority of students appears to be able to correctly substitute variables and execute calculations, and quite content with the "plug-and-chug" approach. In fact, it appears to be true that their "concept anxiety" is more prominent than any "math anxiety."
 
-Moving beyond initial barriers, the problem with mathematics as part of a formative assessment  appears to be not one of {\it operation}, but one of {\it translation}. Students see formulas in a purely operational sense~\cite{torigoe,breitenberger}, while lacking the ability to translate between the formulas and the situations~\cite{clement}, which is also illustrated in the expert and novice quotes at the beginning of section~\ref{intro}.
+Moving beyond initial barriers, the problem with mathematics as part of a formative assessment  appears to be not one of {\it operation}, but one of {\it translation}. Students see formulas in a purely operational sense~\cite{torigoe,breitenberger}, while lacking the ability to translate between the formulas and the situations~\cite{clement}, which is also illustrated in the expert and novice quotes at the beginning of Sect.~\ref{intro}.
 
-Online homework systems by the very nature of computers lend themselves to standard calcu\-lation-oriented textbook problems, and are extensively used in this way. Yet, the "plug-and-chug" approach is the most prominent symptom of novice-like problem-solving strategy, and calculation-oriented problems may encourage just that. As a result, there is a frequent call for "conceptual" online problems, where both instructors and students seem to define "conceptual" simply by the absence of numbers and formulas. 
+Online homework systems by the very nature of computers lend themselves to standard calcu\-lation-oriented problems, and are extensively used in this way. Yet, the "plug-and-chug" approach is the most prominent symptom of novice-like problem-solving strategy, and calculation-oriented problems may encourage just that. As a result, there is a frequent call for "conceptual" online problems, where both instructors and students seem to define "conceptual" simply by the absence of numbers and formulas. 
 \begin{itemize}
 \item But does "depriving" students of numbers and formulas indeed make them work on a conceptual level?
 \item Does it help both students who have problems with applying mathematical methods and those who comfortably "plug-and-chug" gain conceptual understanding of physics?
@@ -154,15 +153,15 @@
 
 \section{Background and Environment}
 \subsection{PI Education and Appointments}
-Dr.~Kortemeyer received his Diplom (ÒM.Sc.Ó) in physics in 1993 from the Universit\"at Hannover, Germany (Advisor Prof. P. U. Sauer), and his Ph.D. in physics from Michigan State University in 1997 (Advisor Prof. W. Bauer).He has been working at Michigan State University since 1997. His first appointment has been as an Academic Specialist in the Division of Science and Mathematics Education (DSME), where he has been leading instructional technology development projects for the College of Natural Science, and is the director of the Learning{\it Online} Network with Computer-Assisted Personalized Approach (LON-CAPA) project, see section~\ref{loncapa}.  He also taught introductory physics in a completely online mode, as well as co-taught in a more traditional on-campus setting.
+Dr.~Kortemeyer received his Diplom (ÒM.Sc.Ó) in physics in 1993 from the Universit\"at Hannover, Germany (Advisor Prof. P. U. Sauer), and his Ph.D. in physics from Michigan State University in 1997 (Advisor Prof. W. Bauer), both with thesis work in theoretical nuclear physics.He has been working at Michigan State University since 1997. His first appointment has been as an Academic Specialist in the Division of Science and Mathematics Education (DSME), where he has been leading instructional technology development projects for the College of Natural Science, and is the director of the Learning{\it Online} Network with Computer-Assisted Personalized Approach (LON-CAPA) project, see Sect.~\ref{loncapa}.  He also taught introductory physics in a completely online mode, as well as co-taught in a more traditional on-campus setting.
 
-Starting August 2004, Dr.~Kortemeyer will be working in a tenure-track position as Assistant Professor of Physics Education. His appointment will be split 75/25\% between the Lyman Briggs School of Science (LBS) and DSME. He will also be holding an appointment as Adjunct Professor of Physics in the Department of Physics and Astronomy. His teaching responsibilities will include the introductory calculus-based physics sequence (lecture and lab) in LBS, as well as seminars in special topics. His research will be focused on postsecondary science teaching and learning, with a special emphasis on the use of technology.\subsection{Michigan State University}Michigan State University is one of the earliest land-grant institutions in the United States. MSU is committed to providing equal educational opportunity to all qualified applicants; at the undergraduate level, the university offers comprehensive programs in the liberal arts and sciences, and provides opportunities for students of varying interests, abilities, backgrounds, and expectations.  The total enrollment is approximately 44,000, 35,000 of which are undergraduates. 54\% of the student population are women, 8.1\% African American, 5.1\% Asian/Pacific Islander, 2.8\% Chicano/Other Hispanic, and 0.6\% Native American. Of the freshman class, the average high school GPA is 3.58.\subsection{Lyman Briggs School of Science}The Lyman Briggs School (LBS) at Michigan State University is a residential learning community devoted to studying the natural sciences and their impact on society. All under one roof, LBS encompasses physics, chemistry, biology, and computer laboratories; classrooms; faculty, administrative, and academic support staff offices; student residences; a dining hall; and a convenience store. With approximately 1500 students, LBS offers the benefits of a small, liberal arts college with the resources of a large research university. Within LBS, 59\% of the student population are women, 15\% ethnic minority including 1\% Hispanic, 3\% African American, and 9\% Asian/Pacific.\subsection{Division of Science and Mathematics Education}The Division of Science and Mathematics Education (DSME) was founded at Michigan State University in 1989, and is co-administered by the College of Natural Science and the College of Education. Academic specialists and faculty members with partial appointments in various departments and other colleges, graduate and undergraduate students, and professional and clerical staff work together in DSME to conduct a variety of research projects, as well as to offer courses, degree programs, and other activities in support of its mission.
-\subsection{Model System: The Learning{\it Online} Network with CAPA}\label{loncapa}For several aspects of the proposed project, the Learning{\it Online} Network with Computer-Assisted Personalized Approach (LON-CAPA; {\tt http://www.lon-capa.org/}) will be the model system. LON-CAPA is a distributed learning content management, course management, and assessment system, and also the model system of the current NSF-ITR grant, see section~\ref{results}. 
+Starting August 2004, Dr.~Kortemeyer will be working in a tenure-track position as Assistant Professor of Physics Education. His appointment will be split 75/25\% between the Lyman Briggs School of Science (LBS) and DSME. He will also be holding an appointment as Adjunct Professor of Physics in the Department of Physics and Astronomy. His teaching responsibilities will include the introductory calculus-based physics sequence (lecture and lab) in LBS, as well as seminars in special topics. His research will be focused on postsecondary science teaching and learning, with a special emphasis on the use of technology.\subsection{Michigan State University}Michigan State University is one of the earliest land-grant institutions in the United States. MSU is committed to providing equal educational opportunity to all qualified applicants; at the undergraduate level, the university offers comprehensive programs in the liberal arts and sciences, and provides opportunities for students of varying interests, abilities, backgrounds, and expectations.  The total enrollment is approximately 44,000, 35,000 of which are undergraduates. 54\% of the student population are women, 8.1\% African American, 5.1\% Asian/Pacific Islander, 2.8\% Chicano/Other Hispanic, and 0.6\% Native American. Of the freshman class, the average high school GPA is 3.58.\subsection{Lyman Briggs School of Science}The Lyman Briggs School (LBS) at MSU is a residential learning community devoted to studying the natural sciences and their impact on society. All under one roof, LBS encompasses physics, chemistry, biology, and computer laboratories; classrooms; faculty, administrative, and academic support staff offices; student residences; a dining hall; and a convenience store. With approximately 1500 students, LBS offers the benefits of a small, liberal arts college with the resources of a large research university. Within LBS, 59\% of the student population are women, 15\% ethnic minority including 1\% Hispanic, 3\% African American, and 9\% Asian/Pacific.\subsection{Division of Science and Mathematics Education}The Division of Science and Mathematics Education (DSME) was founded at MSU in 1989, and is co-administered by the College of Natural Science and the College of Education. Academic specialists and faculty members with partial appointments in various departments and other colleges, graduate and undergraduate students, and professional and clerical staff work together in DSME to conduct a variety of research projects, as well as to offer courses, degree programs, and other activities in support of its mission.
+\subsection{Model System: The Learning{\it Online} Network with CAPA}\label{loncapa}The Learning{\it Online} Network with Computer-Assisted Personalized Approach ({\tt http://www.lon-capa.org/}) is a distributed learning content management, course management, and assessment system, and also the model system of the current NSF-ITR grant, see Sect.~\ref{results}. 
 
-LON-CAPAÕs core development group is located at Michigan State University, and in addition to faculty members, has a staff of three fulltime programmers, two user support staff, one technician, one graduate student, and one project coordinator. The LON-CAPA group also offers training and support for adopters of the system.
+LON-CAPA's core development group is located at MSU, and in addition to faculty members, has a staff of three fulltime programmers, two user support staff, one technician, one graduate student, and one project coordinator. The LON-CAPA group also offers training and support for adopters of the system.
 
 LON-CAPA is open-source (GNU General Public License) freeware, there are no licensing costs associated. Both aspects are important for the success of a research project like the one proposed here. The open-source nature of the system allows researchers to modify and adapt the system in order to address the needs of their project, and the freeware character allows easier dissemination of results, in particular adaptation and implementation at other universities.\subsubsection{Shared Distributed Content Repository}LON-CAPA is designed around the concept of easy sharing and re-use of learning resources. 
-In LON-CAPA, the underlying distributed content repository spans all servers in a given cluster. Navigation through selected resources is provided by an internal sequencing tool, which allows assembling, re-using, and re-purposing content at different levels of granularity (pages, lessons, modules, chapters, etc). LON-CAPA provides highly customizable access control on resources, and has a built-in key mechanism to charge for content access. The shared content pool of LON-CAPA currently contains over 60,000 learning resources, including more than 18,000 personalized homework problems. Any content material contributed ot the pool is immediately available ready-to-use within the system at all participating sites, thus facilitating dissemination of curricular development efforts. Disciplines include astronomy, biology, business, chemistry, civil engineering, computer science, family and child ecology, geology, human food and nutrition, human medicine, mathematics, medical technology, physics, and psychology. A large fraction of these resources are also available through the gateway to the National Science Digital Library. In addition, the problem supplements to a number of commercial textbooks are available in LON-CAPA format.The network provides constant assessment of the resource quality through objective and subjective dynamic metadata. Selection of a learning resource by instructors at other institutions while constructing a learning module does both establish a de-facto peer-review mechanism and provide additional context information for each resource. In addition, access statistics are being kept, and learners can put evaluation information on each resources.\subsubsection{Formative and Summative Assessment Capabilities}LON-CAPA started in 1992 as a system to give personalized homework to students in introductory physics courses.  ÒPersonalized" means that each student sees a different version of the same computer-generated problem: different numbers, choices, graphs, images, simulation parameters, etc, Fig.~\ref{twoproblems}.
+In LON-CAPA, the underlying distributed content repository spans all servers in a given cluster. Navigation through selected resources is provided by an internal sequencing tool, which allows assembling, re-using, and re-purposing content at different levels of granularity (pages, lessons, modules, chapters, etc). LON-CAPA provides highly customizable access control on resources, and has a built-in key mechanism to charge for content access. The shared content pool of LON-CAPA currently contains over 60,000 learning resources, including more than 18,000 personalized homework problems. Any content material contributed to the pool is immediately available and ready-to-use within the system at all participating sites, thus facilitating dissemination of curricular development efforts. Disciplines include astronomy, biology, business, chemistry, civil engineering, computer science, family and child ecology, geology, human food and nutrition, human medicine, mathematics, medical technology, physics, and psychology. A large fraction of these resources are also available through the gateway to the National Science Digital Library (NSDL). In addition to faculty-provided content, the problem supplements to a number of commercial textbooks are available in LON-CAPA format.The network provides constant assessment of the resource quality through objective and subjective dynamic metadata. Selection of a learning resource by instructors at other institutions while constructing a learning module does both establish a de-facto peer-review mechanism and provide additional context information for each resource. In addition, access statistics are being kept, and learners can put evaluation information on each resources.\subsubsection{Formative and Summative Assessment Capabilities}LON-CAPA started in 1992 as a system to give personalized homework to students in introductory physics courses.  ÒPersonalized" means that each student sees a different version of the same computer-generated problem: different numbers, choices, graphs, images, simulation parameters, etc, Fig.~\ref{twoproblems}.
 \begin{figure}
 \includegraphics[width=6.5in]{atwood}
 \caption{Web-rendering of the same LON-CAPA problem for two different students.\label{twoproblems}
@@ -171,26 +170,23 @@
 In the context of this project, this feature is important in two aspects:
 \begin{itemize}
 \item results are not tainted by students simply exchanging the answers, i.e., each student in the end has to work out his or her own answers
-\item as a result, lively discussions take place, both online and in the helproom --- both of which will be analyzed in this project, see section~\ref{discussion}
+\item as a result, lively discussions take place, both online and in the helproom --- both of which will be analyzed in this project, see Sect.~\ref{discussion}
 \end{itemize}
 
-Students are generally given immediate feedback on the correctness of their solutions, and in some cases additional help. They are usually granted multiple attempts to get a problem correct. This allows the instructor to follow a learner's thought process, both through statistical analysis (see~\ref{analysis}) and data-mining approaches.
+Students are generally given immediate feedback on the correctness of their solutions, and in some cases additional help. They are usually granted multiple attempts to get a problem correct. This allows the instructor to follow a learner's thought process, both through statistical analysis (see~\ref{anatool}) and data-mining approaches.
 
 The system also allows for free-form essay-type answers, which are however graded by humans with the assistance of the system (keyword-highlighting, plagiarism-checks, etc).\subsubsection{Course Management}Over the years, the system added a learning content management system and standard course management features, such as communications, gradebook, etc., which are comparable to commercial course management systems, such as BlackBoard, WebCT, or ANGEL. See 
-Refs.~\cite{features,edutools} for an overview of features, and comparisons to other systems.In addition to standard features, the LON-CAPA delivery and course management layer is designed around STEM education, for example: support for mathematical typesetting throughout (\LaTeX\ inside of XML) Ð formulas are rendered on-the-fly, and can be algorithmically modified through the use of variables inside formulas; integrated GNUplot support, such that graphs can be rendered on-the-fly, and allowing additional layered labeling of graphs and images; support for multi-dimensional symbolic math answers; and full support of physical units.
+Refs.~\cite{features,edutools} for an overview of features, and comparisons to other systems.In addition to standard features, the LON-CAPA delivery and course management layer is designed around STEM education, for example: support for mathematical typesetting throughout (\LaTeX\ inside of XML) Ð-- formulas are rendered on-the-fly, and can be algorithmically modified through the use of variables inside formulas; integrated GNUplot support, such that graphs can be rendered on-the-fly, and allowing additional layered labeling of graphs and images; support for multi-dimensional symbolic math answers; and full support of physical units.
 
 \begin{figure}
 \includegraphics[width=6.5in]{problemview}
 \caption{Example an individual student view for problem analysis.\label{problemview}}
 \end{figure}
-\subsection{Collaborative Learning Laboratory}
-The Lyman-Briggs School of Science Collaborative Learning Laboratory, which is expected to be completed in 2005. It is modeled in part after a setup by the North Carolina State University Physics Education R\&D Group~\cite{ncsu}, and offers a space where students can collaborate on homework while their interactions and online transactions are recorded.
-
-In addition to having whiteboards and wireless laptop computers for students to work with in flexible group settings, the facility will have integrated observation equipment to video- and audio-record student interactions. All recorded information is immediately digitized and made available for transcription and analysis using the Transana~\cite{transana} software system.
-\subsubsection{Analysis Capabilities}\label{analysis}
+
+\subsubsection{Analysis Capabilities}\label{anatool}
 LON-CAPA allows instructors to analyze student submissions both for individual students (Fig.~\ref{problemview}) and across the course (Fig.~\ref{problemanalysis}).
 
-For example, Fig.~\ref{problemview} indicates that in the presence of a medium between the capacitor plates, the student was convinced that the force would increase, but also that this statement was the one he was most unsure about: His first answer was that the force would double; no additional feedback except "incorrect" was provided by the system. In his next attempt, he would change his answer on only this one statement (indicating that he was convinced of his other answers) to "four times the force" --- however, only ten seconds passed between the attempts, showing that he was merely guessing by which factor the force increased. The graphs on the right of Fig.~\ref{problemanalysis} show which statements were answered correctly on the first and on the second attempt, respectively, the graphs on the right which other options the students chose if the statement was answered incorrectly. Clearly, students have the most difficulty with the concept of how a medium acts inside a capacitor, with the absolute majority believing the capacitance would increase, and only about 20\% of the students believing the medium had no influence.
+For example, Fig.~\ref{problemview} indicates that in the presence of a medium between the capacitor plates, the student was convinced that the force would increase, but also that this statement was the one he was most unsure about: His first answer was that the force would double; no additional feedback except "incorrect" was provided by the system. In his next attempt, he would change his answer on only this one statement (indicating that he was convinced of his other answers) to "four times the force" --- however, only ten seconds passed between the attempts, showing that he was merely guessing by which factor the force increased. The graphs on the right of Fig.~\ref{problemanalysis} show which statements were answered correctly course-wide on the first and on the second attempt, respectively, the graphs on the right which other options the students chose if the statement was answered incorrectly. Clearly, students have the most difficulty with the concept of how a medium acts inside a capacitor, with the absolute majority believing the capacitance would increase, and only about 20\% of the students believing the medium had no influence.
 
 \begin{figure}
 \begin{center}
@@ -199,14 +195,19 @@
 \caption{Example of a course-wide problem analysis for the problem Fig.~\ref{problemview}.\label{problemanalysis}}
 \end{figure}
 
+\subsection{Collaborative Learning Laboratory}
+The LBS Collaborative Learning Laboratory, which is expected to be completed in 2005. It is modeled in part after a setup by the North Carolina State University Physics Education R\&D Group~\cite{ncsu}, and offers a space where students can collaborate on homework while their interactions and online transactions are recorded.
+
+In addition to having whiteboards and wireless laptop computers for students to work with in flexible group settings, the facility will have integrated observation equipment to video- and audio-record student interactions. All recorded information is immediately digitized and made available for transcription and analysis using the Transana~\cite{transana} software system.
+
 \subsection{Courses}
-The project will be carried out  in the two-semester LBS course sequence LBS 271/272,"Calculus-Based Introductory Physics I/II. These second-year three-credit courses have a Calculus pre-requisite, and traditionally an enrollment of over 200 students. 
+The project will be carried out  in the two-semester LBS course sequence LBS 271/272,"Calculus-Based Introductory Physics I/II. These second-year non-major three-credit courses have a Calculus pre-requisite, and traditionally an enrollment of over 200 students. 
 
-Starting Fall 2004, the course will be taught with less total lecturing time, where the third classroom hour will be used for peer-teaching~\cite{mazur} and more frequent summative assessment in place of the midterm exams.
+Starting Fall 2004, the course will be taught with less total lecturing time, where the third classroom hour will be used for peer-teaching~\cite{mazur} and more frequent quizzes in place of the midterm exams.
 
-Two separate, but associated one-credit laboratory courses (LBS 271L/272L) are required, which most but not all students choose to take simultaneously. Faculty and teaching assistants are frequently assuming shared responsibilities between the lecture and laboratory courses, with a combined staff of two faculty members and six undergraduate student assistants. The latter are responsible for particular recitation and laboratory sections, and will be involved in this research project (see Section~\ref{undergrad}). Within the duration of this project, the lecture and laboratory courses might be combined to provide greater coherence between these two venues.
+Two separate, but associated one-credit laboratory courses (LBS 271L/272L) are required, which most but not all students choose to take simultaneously. Faculty and teaching assistants are frequently assuming shared responsibilities between the lecture and laboratory courses, with a combined staff of two faculty members and six undergraduate student assistants. The latter are responsible for particular recitation and laboratory sections, and will be involved in this research project (see Sect.~\ref{undergrad}). Within the duration of this project, the lecture and laboratory courses might be combined to provide greater coherence between these two venues.
 
-Students in these courses are currently solving approximately 250 online homework problems each semester, most of which currently are of the conventional type.
+Students in these courses are currently solving approximately 200 online homework problems each semester, most of which currently are of the conventional type.
 \subsection{Synergy between Project and Institutional Goals}
 The mission statement of the Lyman-Briggs School of Science includes the statement
 \begin{quote}
@@ -217,14 +218,14 @@
 to improve science and mathematics education, from kindergarten through the undergraduate years, through the professional development of preservice and inservice teachers and faculty members. 
 \end{quote}
 These ambitious goals clearly infer going beyond a surface-recapitulation of disconnected factoids, toward a deep appreciation of natural phenomena and their inter-connectedness. Research and teaching are closely intertwined in this project, with the goal of improving the learning and teaching
-of the natural sciences, even as the study itself is being carried out. The outcomes of this study will inform faculty members, and aligned with the goals of the Division of Science and Mathematics Education.
+of the natural sciences, even as the study itself is being carried out. Since outcomes of this study will inform faculty members, it aligns with the goals of the Division of Science and Mathematics Education.
 \section{Classification of Online Formative Assessment Problems}\label{class}
 Redish~\cite{redish} distinguishes eight types of exam and homework questions, an adapted version of which will form the general classification scheme for Hypothesis 1b:
 \begin{description}
-\item[Multiple-choice and short-answer questions] The most basic and most easily computer-evaluated type of question, representing the typical back-of-chapter textbook problem.
+\item[Multiple-choice and short-answer questions] The most basic and most easily computer-evaluated type of question, representing the conventional (typical back-of-chapter textbook) problem.
 
 For the purposes of this project, "multiple choice" and "short-answer" will be considered as separate classes, where short-answer includes numerical answers such as "$17 kg/m^3$," and formula answers, such as "\verb!1/2*m*(vx^2+vy^2)!."  The problems on the left side of Figs.~\ref{threemasses} and \ref{trajectory} are examples of "short-(numerical)-answer" problems.
-\item[Multiple-choice multiple-response questions]  This type of problem requires a student to evaluate each statement and make a decision about it. The problems Fig.~\ref{problemview} and on the right side of Fig.~\ref{threemasses} are of this type.
+\item[Multiple-choice multiple-response questions]  This type of problem, a first step beyond conventional problems, requires a student to evaluate each statement and make a decision about it. The problems Fig.~\ref{problemview} and on the right side of Fig.~\ref{threemasses} are of this type.
 
 
 \begin{figure}
@@ -249,7 +250,7 @@
 
 While students find it initially hard to believe that these questions have anything to do with physics, hardly any expert physicist would deny their significance in learning how to solve problems~\cite{mazur96}. 
 
-A component of this project (see section~\ref{platform}) will be to find and implement mechanisms to implement these question-types within an online system in an authentic yet scalable fashion.
+A component of this project (see Sect.~\ref{platform}) will be to find and implement mechanisms to implement these question-types within an online system in an authentic yet scalable fashion.
 \item[Qualitative questions] This type of questions asks students to make judgments about physical scenarios, and in that respect are somewhat similar to ranking questions. While the questions themselves are of the type "Is this high enough?" or "Can we safely ignore \ldots?," they often do require at least "back-of-the-envelope" calculations to to give informed answers. As in the case of estimation problems, students do have to explain their reasoning, but the question itself is usually more structured, and at least the initial answer is more easily evaluated by a computer.
 \item[Essay questions] These are "explain why" questions. A certain scenario is presented, and students are asked to explain why it turns out the way it does. Students are not asked to recall a certain law --- it is given to them. Instead, they are asked to discuss its validity.
 \end{description}
@@ -266,7 +267,7 @@
 Repre\-sentation-translation&AS&AMS&AS&AMS&AS&AS&&&\\\hline
 Context-based&MS&AMS&MS&AMS&MS&AMS&GMS&GMS&GMS\\\hline
 \end{tabular}
-\caption{Classification scheme for question types, adapted from Redish~\cite{redish}, see section~\ref{class}. The symbols denote different components of the project, i.e., "A" - additional analysis tool development (section~\ref{analysisnew}); "G" - additional scalable grading tool  development (section~\ref{platform}); "M" - additional materials development (section~\ref{matdev}); "S" - this question type will be included in the study of its impact (sections~\ref{hypo} and \ref{analysis}).\label{classification}}
+\caption{Classification scheme for question types, adapted from Redish~\cite{redish}, see Sect.~\ref{class}. The symbols denote different components of the project, i.e., "A" - additional analysis tool development (Sect.~\ref{analysisnew}); "G" - additional scalable grading tool  development (Sect.~\ref{platform}); "M" - additional materials development (Sect.~\ref{matdev}); "S" - this question type will be included in the study of its impact (sections~\ref{hypo} and \ref{analysis}).\label{classification}}
 \end{table}
 \section{Preliminary Project Components}
 Several aspects of the research project can be started in the first year, while others will require additional functionality in the LON-CAPA platform, and the preparation of additional homework problems of certain types.
@@ -274,15 +275,18 @@
 In Table~\ref{classification}, problem types which are marked "S" with no additional symbols have both sufficient functionality support in the LON-CAPA system, and a sufficient library of problems of this type to conduct the study. If a problem type is marked "M," the library of problems of this type is still small, and additional problems need to be developed. If a problem type is marked "A," it means that the study would profit from the development of additional analysis tools, if it is marked "G," it means that while the platform in its current version can support it, the solution does not scale well with the number of students due to the effort required for manual grading.
 
 \subsection{Additional Materials Development}\label{matdev}
-For the question types marked "M" in Table~\ref{classification}, the currently existing library of LON-CAPA problems is not provide enough samples to carry out the study. Within this project, new homework problems of this type will be developed. Since development of completely new problems would constitute a project by itself, this component of the current project will heavily draw on existing problem collections, i.e., Redish (\cite{redish}, resource CD), McDermott~\cite{mcdermottprob}, Mazur~\cite{mazur}, and Project Galileo~\cite{galileo}. These research-based problems will be adapted and implemented in the the LON-CAPA system.
+For the question types marked "M" in Table~\ref{classification}, the currently existing library of LON-CAPA problems does not provide enough samples to carry out the study. Since development of completely new problems would constitute a project by itself, this component of the current project will heavily draw on existing problem collections, i.e., Redish (\cite{redish}, resource CD), McDermott~\cite{mcdermottprob}, Mazur~\cite{mazur}, and Project Galileo~\cite{galileo}. These research-based problems will be adapted and implemented in the the LON-CAPA system, and new problems only developed where necessary.
+
+The goal is to have 12 problems representing each type (Table~\ref{classification}) in each semester, as evenly as possible distributed over the 16 weeks of the semester.
 
 \subsection{Additional Platform Development}
 
 \subsubsection{Scalable Functionality for Manual Grading of Free-Form Answers}\label{platform}
-LON-CAPA already offers grading support for free-form student submission, such as keyword highlighting and plagiarism-checks. Additional tools will be developed for the grading of the problem types marked "G" in Table~\ref{classification}: for questions that require student submissions of the type "Explain your reasoning," better coupling between the computer- and manually-evaluated sections will be provided, for the free-form "essay" submissions better tools to compare student submissions with each other and with examplary essays.
+LON-CAPA already offers grading support for free-form student submission, such as 
+keyword-highlighting and plagiarism-checks. Additional tools will be developed for the grading of the problem types marked "G" in Table~\ref{classification}: for questions that require student submissions of the type "Explain your reasoning," better coupling between the computer- and manually-evaluated sections will be provided, for the free-form "essay" submissions, better tools to compare student submissions with each other and with examplary essays.
 
 \subsubsection{Additional Analysis Tools}\label{analysisnew}
-While the premise of this project is that feedback on formative assessment is crucial for the learner, it is almost equally important to the instructor~\cite{pellegrino}, with technology as enabler~\cite{novak,feedback}. Particularly in the context of a research project on formative assessement, timely and comprehensive feedback on student performance --- including new material (section~\ref{matdev}) --- is essential. The LON-CAPA system already has sophisticated analysis tools (see section~\ref{analysis}), but these do not support all questions types in Table~\ref{classification} equally well, and the project includes a tools development component to further enhance these mechanisms for the problem types marked "A."
+While the premise of this project is that feedback on formative assessment is crucial for the learner, it is almost equally important to the instructor~\cite{pellegrino}, with technology as enabler~\cite{novak,feedback}. Particularly in the context of a research project on formative assessement, timely and comprehensive feedback on student performance --- including new material (Sect.~\ref{matdev}) --- is essential. The LON-CAPA system already has sophisticated analysis tools (see Sect.~\ref{anatool}), but these do not support all questions types in Table~\ref{classification} equally well, and the project includes a tools development component to further enhance these mechanisms for the problem types marked "A."
 
 Data collection on a particular problem type can proceed independently  from the existence of the respective analysis tools, since LON-CAPA permanently stores all data.
 
@@ -292,7 +296,7 @@
 \subsection{Establishment of Initial Conditions}The validity of the hypotheses may depend on both learner and assessment characteristics.\subsubsection{Learner Attitudes, Beliefs, and Expectations}Instruments have been developed to assess epistemological beliefs, for example the Epistemological Beliefs Assessment for Physical Science (EBAPS) Instrument~\cite{EBAPS}. Related to epistemological beliefs are learnerÕs expectations and attitudes, and of particular interest is the Maryland Physics Expectations (MPEX) survey~\cite{MPEX}.\subsubsection{Learner Knowledge about the Topic}\label{prepost}We will use existing concept inventory surveys as both pre- and post-tests.The qualitative Force Concept Inventory~\cite{fci} and the quantitative companion Mechanical Baseline Test~\cite{hestenesmech} have been used in a large number of studies connected to the teaching of introductory mechanics. The Foundation Coalition has been developing a number of relevant concept inventories~\cite{foundation}, namely the Thermodynamics Concept Inventory, the Dynamics Concept Inventory, and the Electromagnetics Concept Inventory (with two subcomponents, namely Waves and Fields).  Since these were designed from an engineering point of view, some adjustment might be necessary. In addition, the Conceptual Survey of Electricity and Magnetism (CSEM)~\cite{maloney} is available for the second semester course.
 
 \subsubsection{Problem Difficulty and Baseline Statistical Data}LON-CAPA automatically keeps tracks of the average number of attempts until a problem is solved, as well as the degree of difficulty and the degree of discrimination. This data is cumulative across semesters, and already exists for all assessment problems from their deployment in previous semesters.
-\subsection{Observables}\subsubsection{Effectiveness}Effectiveness will be measured both in terms of performance on summative assessments (quizzes and exams) and on pre-/post-test concept inventory surveys (Section~\ref{prepost}).  Each item on these instruments will be associated with topically corresponding formative online exercises to determine correlations and differential gain between the feedback types used with the respective online problems. A second posttest, correlated with first semester problems, will be administered at the end of the second semester to determine long-term effects.\subsubsection{Problem Solving Technique}We intend to focus on a subset of students in the LBS Collaborative Learning Laboratory, and observe them while solving problems. Schoenfeld~\cite{schoenfeld} and Foster~\cite{foster} developed instruments to categorize and document the stages and expertlike 
+\subsection{Observables}\subsubsection{Effectiveness}Effectiveness will be measured both in terms of performance on summative assessments (quizzes and exams) and on pre-/post-test concept inventory surveys (Sect.~\ref{prepost}).  Each item on these instruments will be associated with topically corresponding formative online exercises to determine correlations and differential gain between the feedback types used with the respective online problems. A second posttest, correlated with first semester problems, will be administered at the end of the second semester to determine long-term effects.\subsubsection{Problem Solving Technique}We intend to focus on a subset of students in the LBS Collaborative Learning Laboratory, and observe them while solving problems. Schoenfeld~\cite{schoenfeld} and Foster~\cite{foster} developed instruments to categorize and document the stages and expertlike 
 characteristics~\cite{chi} of observed problem-solving activity by learners, as well as application of metacognitive skills.In addition, we will interview a group of students from all courses regarding their problem-solving strategies. Pascarella~\cite{pascarella02} developed some frameworks for these interviews, which can be built upon.Finally, for all students in all courses, LON-CAPA log data will be analyzed. Kotas~\cite{kotas} and Minaei~\cite{minaei} developed a mechanism for this log data analysis, which include submission times between attempts, and quality of the entered input. \subsubsection{Help-Seeking Behavior}\label{discussion}It is impossible to observe all on-demand help seeking, but interactions in several settings can be analyzed:Online discussions and email communication are preserved within LON-CAPA and can be analyzed even in retrospect for past semesters with respect to relevant behavioral patterns.  Table~\ref{discussionex} shows excerpts of discussions around the two problems in Fig.~\cite{trajectory}.
 
 \begin{table}
@@ -488,7 +492,7 @@
 % references
 \newpage
 
-\pagestyle{plain}
+\setcounter{page}{0}\pagenumbering{roman}
 \begin{thebibliography}{99}
 
 % Intro
@@ -530,9 +534,11 @@
 
 \bibitem{features} LON-CAPA feature overview, 
 {\tt http://www.lon-capa.org/features.html}
-\bibitem{edutools} EDUTOOLS Course Management System Decision Making Tools, {\tt http://www.edutools.info/course/compare/} 
-\bibitem{ncsu} North Carolina State University, Physics Education and Development Group, {\tt http://www.physics.ncsu.edu:8380/physics\_ed/Room\_Design\_files/frame.htm}
-\bibitem{transana} University of Wisconsin, Wisconsin Center for Education Research, {\tt http://www2.wcer.wisc.edu/Transana/}
+\bibitem{edutools} EDUTOOLS Course Management System Decision Making Tools,
+\newline {\tt http://www.edutools.info/course/compare/} 
+\bibitem{ncsu} North Carolina State University, Physics Education and Development Group,
+\newline {\tt http://www.physics.ncsu.edu:8380/physics\_ed/Room\_Design\_files/frame.htm}
+\bibitem{transana} University of Wisconsin, Wisconsin Center for Education Research,\newline {\tt http://www2.wcer.wisc.edu/Transana/}
 
 % cat
 \bibitem{mcdermott} Lillian McDermott, Mark L. Rosenquist, and Emily H. van Zee, {\it Student difficulties in connecting graphs and physics: Examples from kinematics}, Am. J. Phys {\bf 55}(6), 503-513 (1987)
@@ -554,8 +560,8 @@
 \bibitem{maloney} D. P. Maloney, T. L. O$'$Kuma, C. J. Hieggelke, A. van Heuvelen, {\it Surveying studentsÕ conceptual knowledge of electricity and magnetism}, Am. J. Phys. Suppl. {\bf 69}, S12 (2001)
 \bibitem{schoenfeld} A. H. Schoenfeld, {\it Mathematical Problem Solving}, Academic Press (1985)
 \bibitem{foster} T. M. Foster, {\it The Development of StudentÕs Problem-Solving Skill from Instruction Emphasizing Qualitative Problem-Solving}, dissertation, University of Minnesota (2000)
-\bibitem{minaei}  Behrouz Minaei-Bidgoli, William. F. Punch, {\it Using Genetic Algorithms for Data Mining Optimization in an Educational Web-based System}, Proceedings, Genetic and Evolutionary Computation Conference (2003)
 \bibitem{kotas} P. Kotas, Homework Behavior in an Introductory Physics Course, Masters Thesis (Physics), Central Michigan University (2000)
+\bibitem{minaei}  Behrouz Minaei-Bidgoli, William. F. Punch, {\it Using Genetic Algorithms for Data Mining Optimization in an Educational Web-based System}, Proceedings, Genetic and Evolutionary Computation Conference (2003)
 \bibitem{wallace} Raven M. Wallace, {\it Online Learning in Higher Education: a review of research on interactions among teachers and students}, Education, Communication and Information, 
 {\bf 3}(2), 241 (2003)\bibitem{riffell1} Samuel K. Riffell and Duncan F. Sibley, {\it Can hybrid course formats increase attendance in undergraduate environmental science courses?},  Journal of Natural Resources and Life Sciences Education, in press (2003)  \bibitem{riffell2} Samuel K. Riffell and Duncan F. Sibley, {\it Student perceptions of a hybrid learning format: can online exercises replace traditional lectures?}, Journal of College Science Teaching, {\bf 32}(6), 394-399 (2003)
 \end{thebibliography}

--www1089890305--