[LON-CAPA-cvs] cvs: modules /gerd/alt2007 graphing.bib graphing.tex

www lon-capa-cvs@mail.lon-capa.org
Mon, 26 Mar 2007 21:48:45 -0000


This is a MIME encoded message

--www1174945725
Content-Type: text/plain

www		Mon Mar 26 17:48:45 2007 EDT

  Modified files:              
    /modules/gerd/alt2007	graphing.bib graphing.tex 
  Log:
  With input from David
  
  
--www1174945725
Content-Type: text/plain
Content-Disposition: attachment; filename="www-20070326174845.txt"

Index: modules/gerd/alt2007/graphing.bib
diff -u modules/gerd/alt2007/graphing.bib:1.3 modules/gerd/alt2007/graphing.bib:1.4
--- modules/gerd/alt2007/graphing.bib:1.3	Tue Mar  6 16:47:16 2007
+++ modules/gerd/alt2007/graphing.bib	Mon Mar 26 17:48:43 2007
@@ -7,6 +7,31 @@
    title = "The problem with problems"
 }
 
+@BOOK{copro,
+   author = "Gerd Kortemeyer et al.",
+   year = "1993",
+   title = "Coprozessoren Programmierung mit Turbo Pascal und C++",
+   isbn="3-88322-439-1",
+   publisher="IWT Verlag (International Thompson Publishing), Vaterstetten"
+}
+
+@BOOK{serway,
+   author="Raymond A. Serway and Jerry S. Faughn and Chris Vuille and Charles A. Bennett",
+   year = "in preparation",
+   title = "College Physics",
+   publisher="Brooks Cole"
+}
+
+@ARTICLE{disessa04,
+   author = "Andrea A. diSessa",
+   year = "2004",
+   journal = "Cognition and Instruction",
+   volume = "22",
+   issue="2",
+   pages = "293-331",
+   title = "Metarepresentation: Native competence and targets for instruction"
+}
+
 @ARTICLE{schommer93,
    author = "Marlene Schommer",
    year = "1993",
@@ -170,6 +195,13 @@
    title = "Performance on multiple-choice diagnostics and complementary exam problems"
 }
 
+@BOOK{aaas93,
+  author="American Association for the Advancement of Science (AAAS)",
+  title="Benchmarks for Science Literacy",
+  year="1993",
+  publisher="Oxford University Press"
+}
+
 @MISC{loncapa,
   author="LON-CAPA Consortium",
   title="LON-CAPA Homepage",
Index: modules/gerd/alt2007/graphing.tex
diff -u modules/gerd/alt2007/graphing.tex:1.4 modules/gerd/alt2007/graphing.tex:1.5
--- modules/gerd/alt2007/graphing.tex:1.4	Wed Mar  7 16:44:10 2007
+++ modules/gerd/alt2007/graphing.tex	Mon Mar 26 17:48:43 2007
@@ -31,7 +31,7 @@
 {\Large\sc Online Assessment of Back-of-the-Envelope Graph Sketches in Introductory Physics}
 \end{center}
 \section{Introduction}
-The ability to work with graphs is a basic skill of any scientist. When scientists are discussing concepts and phenomena, they quickly resort to sketches of one variable versus another, sometimes just three lines on a paper: two axes and a graph. But how often do we intentionally teach our students about this important representation format?
+The ability to work with graphs is a basic skill of any scientist. When scientists are discussing concepts and phenomena, they quickly resort to sketches of one variable versus another, sometimes just three lines on a paper: two axes and a curve. But how often do we intentionally teach our students about this important representation format?
 
 Even with the relatively simple concept of position, velocity, and acceleration, students have unexpected difficulties translating between graphical representations and both the mathematical representation and the ``real world''~\cite{mcdermott,beichner,meltzer05,clement81}. These difficulties can sometimes have subtle reasons that lead to cognitive disconnects: for example,  using a graph showing the position of a bouncing ball versus time, Ferrara~\cite{ferrara} found that even the sign, i.e., measuring the distance from the ground versus from the launch points, can make a large difference. Students might misinterpret a graph as a pictorial representation of a situation~\cite{elby00}, mix up what is on the axes, or become confused when approximating the slope of a graph that does not start at the origin~\cite{beichner}. Students need practice both interpreting and generating these graphs, but appropriate formative assessment frequently does not happen in the typically large enrollment introductory courses due to scalability problems: there are simply not enough teaching assistants to give the students appropriate feedback on these complex tasks.
 
@@ -45,34 +45,46 @@
 \item What is the general shape of the curve (e.g., exponential growth or decay, a hysteresis, sinusoidial, asymptotic)?
 \item Are there singularities or significant points along the way?
 \end{itemize}
- (list expanded from \cite{kennedy04}). Students need to construct the curve, not reproduce it or select it from a set of prefab solutions.
+ (list expanded from \cite{kennedy04}). Students need to construct the curve, not reproduce it or select it from a set of prefabricated solutions.
 
-Sketching is an activity that should be manageable with just a few strokes to express a general relationship. Within this project, we will develop an online assessment tool for graph sketching, which can provide randomized scenarios and provide immediate feedback to graph sketches entered online with a mouse or trackpad. We will evaluate usability for both faculty and students, as well as impact on student problem solving strategies and conceptual learning.
+Sketching is an activity that should be manageable with just a few strokes to express a general relationship. Within this project, we will develop an online assessment tool for graph sketching, which will provide randomized scenarios and immediate feedback to graph sketches entered online with a mouse or trackpad. We will evaluate usability for both faculty and students, as well as impact on student problem solving strategies and conceptual learning.
 
 The tool will be developed on top of an existing course and learning content management system in order to minimize overhead. However, both the algorithms and the code will be made freely available, so they can be incorporated into other systems.
 \subsection{Learning Goals}
-Student problems in representation translation between mathematical and ``real world'' scenarios on the one hand, and graphical representations on the other, are well documented in literature. Careful plotting of functions or data does not address the conceptual visualization goals, and selecting between different options does not help students truly construct new knowledge and communicate ideas. Students will learn how to use ``back-of-the-envelope" sketches as  a tool to communicate real world and mathematical concepts.
+A sketched graph can be thought of as a pictorial description of a phenomenon.  It emphasizes the major features of the phenomenon while ignoring details that may be distracting or better represented in other ways.  Sketched graphs are regularly used by scientists and engineers in informal discourse, as means for describing and supporting claims.  In the Benchmarks for Science Literacy~\cite{aaas93}, Project 2061 makes an explicit recommendation regarding the importance of using graphs in making arguments (12D/H7).  Graphs are best used to present data, describe ideas, and support claims that involve trends, comparisons, and general behavior.  The ability to sketch a graph and use it in an argument involves not only an understanding of the holistic characteristics of graphs, such as general shape, number and general location of intersections with axes, and asymptotes, but also the general behavior of the phenomenon being graphed.  As such, opportunities to sketch graphs and receive formative feedback on them should foster students' conceptual understanding of the phenomena being graphed.
+
+We expect that the sketching tool we propose to develop will support students in developing the following skills and knowledge:\vspace*{-2mm}
+\begin{itemize}
+\item Qualitative reasoning: why sections of a graph appear as they do.\vspace*{-2mm}
+\item Translate verbal descriptions of phenomena into qualitative graphical descriptions.\vspace*{-2mm}
+\item Construct verbal descriptions of phenomena based on sketched graphs.\vspace*{-2mm}
+\item Translate from one graphical representation to another (for example, sketch a v-t graph from an x-t graph)~\cite{disessa04}.\vspace*{-2mm}
+\item Conceptual understanding of physics topics.\vspace*{-2mm}
+\end{itemize}
+We also expect that a better qualitative understanding of graphs will lead to improved quantitative generation and interpretation of graphs. When producing a graph ``dot-to-dot,'' it is easy to get lost in details. An increased qualitative understanding of graphs is expected to enable students to visualize the whole graph before embarking into the task of the ``dot-to-dot'' graphing.
+
 \subsection{Technology Goals}
 This project goes beyond offering different options for graphs or drawing graphs based on adjustable parameters to splines. It will develop algorithms that evaluate freehand input of graphs with various randomized constraints or scenarios. The constructed rule set and its tolerances will be adaptive and allow adjustment through feedback loops. Usability and accessibility testing is part of the design process, and scalability concerns will be strongly taken into account in order to develop a widely usable tool, not a laboratory-type proof-of-concept. The tools will be developed on top of an existing software infrastructure, but will be kept modular and be made available open-source, so they can be deployed in other contexts.
 \subsection{Intellectual Merit}
-The project will develop new understanding regarding the acquisition of visualization skills. Students will be observed as they perform representation translation, using both traditional tools such as multiple choice between different graphical options, and freehand sketching.  The results will inform practitioners in assessment design and lead to insights into the cognitive development of students as they are confronted with representation translation tasks.
+The proposed work will provide an empirically-supported online tool for assessing graph sketching and an assortment of open-ended and multiple choice items that can be used to support the learning and assessing of sketching in the context of an undergraduate course in introductory mechanics. There are a number of studies that have investigated the importance of constructing, reading, and critiquing numerically-based graphs of physical phenomena and the difficulties that students face when engaging in these tasks, e.g.,~\cite{mcdermott,beichner}.  However, systematic research on how to support students in distilling the key features of a phenomenon and representing them graphically without delving into numerical analyses is lacking. The proposed work will contribute to theories of learning in online environments and of the cognitive processes involved in sketching graphical representations based on textual descriptions or different graphical representations in the context of physics problems.
 \subsection{Broader Impact}
-Generating and evaluating simple data visualization in the form of ``back-of-the-envelope'' line graphs is an important skill across academic disciplines. While the project will focus on the teaching of physics, the same tools are expected to be useful not only in other natural sciences or mathematics, but also in for example economics and social sciences. The tools will be implemented within a software framework that is already used at 50 secondary and over 40 post-secondary institutions.
+Construction and validation of empirically-tested tools for online learning environments are needed to support the range of topics and skills that can be learned by diverse learners within these environments.  Using sketches to describe physical phenomena is an important skill for every scientist and engineer, but one that is not presently supported in most online learning environments.  The specific tool that we propose to develop and test can serve as a model that other developers can use to design similar tools for other online environments. More generally, the study on the use and affordances of the tool can guide other investigators in studying online tools developed to support the construction of other knowledge and skills. The tools will be implemented within a software framework that is already used at 50 secondary and over 40 post-secondary institutions.
 \section{Project Overview}
-Over the course of this project, we will:
+Over the course of this project, we will:\vspace*{-2mm}
 \begin{itemize}
-\item develop the tools to evaluate graph sketches submitted online
-\item refine the authoring aspects in a faculty usability study
-\item refine the learner interface in a student usability study
-\item develop an initial set of problems in relevant introductory physics areas
-\item analyze student problem solving strategies for these problems versus traditional numerical and graph identification/matching problems
-\item evaluate the educational effectiveness of these problems versus traditional numerical and graph identification/matching problems
-\item make algorithms and tools available open-source and freely
-\item disseminate results
+\item develop the tools to evaluate graph sketches submitted online\vspace*{-2mm}
+\item refine the authoring aspects in a faculty usability study\vspace*{-2mm}
+\item refine the learner interface in a student usability study\vspace*{-2mm}
+\item develop an initial set of problems in relevant introductory physics areas\vspace*{-2mm}
+\item analyze student problem solving strategies for these problems versus traditional numerical and graph identification/matching problems\vspace*{-2mm}
+\item evaluate the educational effectiveness of these problems versus traditional numerical and graph identification/matching problems\vspace*{-2mm}
+\item make algorithms and tools available open-source and freely\vspace*{-2mm}
+\item disseminate results\vspace*{-2mm}
 \end{itemize}
 Several of these steps will be carried out iteratively over the duration of the project while refining both tool and content
 \section{Relevant Results from Related Projects}
-\subsection{Relevant Results from Prior NSF Support to the PIs}
+\subsection{Relevant Results from Prior NSF Support to the PIs}\label{loncapa}
+David Fortus is a co-PI on the NSF grant {\it Investigating and Questioning our World through Science and Technology} (ESI-0439352, \$712,034 for MSU subcontract, 10/01/04-09/30/10). The project is developing a coordinated and comprehensive middle school science curriculum that emphasizes several scientific practices, one with is DGOA - data gathering, organization, and analysis.
 \begin{figure}
 \includegraphics[width=3.3in]{figures/coil1}
 \includegraphics[width=3.3in]{figures/coil2}
@@ -94,10 +106,10 @@
 BlackBoard, WebCT, or ANGEL. In addition to standard features, the LON-CAPA delivery and course management layer is designed around STEM education, for example: 
 \begin{itemize}
 \item support for mathematical typesetting throughout (\LaTeX\ inside of XML) (formulas are rendered on-the-fly, 
-and can be algorithmically modified through the use of variables inside formulas)
+and can be algorithmically modified through the use of variables inside formulas)\vspace*{-2mm}
 \item integrated GNUplot support (graphs can be rendered on-the-fly, 
-and allowing additional layered labeling of graphs and images)
-\item support for multi-dimensional symbolic math answers, using sampling or the integrated Maxima symbolic algebra system
+and allowing additional layered labeling of graphs and images)\vspace*{-2mm}
+\item support for multi-dimensional symbolic math answers, using sampling or the integrated Maxima symbolic algebra system\vspace*{-2mm}
 \item and full support of physical units.
 \end{itemize}
 
@@ -161,19 +173,19 @@
 \subsection{Significant versus spurious features}
 When the system evaluates a student-provided graph, it is important to distinguish significant from spurious graph features. Examples for significant features might include:
 \begin{itemize}
-\item linear versus non-linear
-\item asymptotic behavior at infinity or possibly certain points
-\item approximate position of maxima or minima
-\item approximate position of axis intercepts
+\item linear versus non-linear\vspace*{-2mm}
+\item asymptotic behavior at infinity or possibly certain points\vspace*{-2mm}
+\item approximate position of maxima or minima\vspace*{-2mm}
+\item approximate position of axis intercepts\vspace*{-2mm}
 \item curvature (convex/concave)
 \end{itemize}
 Spurious features might include:
 \begin{itemize}
-\item scale
-\item artifacts from inputting the graph using a mouse or trackpad
-\item exact position of axis intercepts, minima and maxima
+\item scale\vspace*{-2mm}
+\item artifacts from inputting the graph using a mouse or trackpad\vspace*{-2mm}
+\item exact position of axis intercepts, minima and maxima\vspace*{-2mm}
 \end{itemize}
-Which features to which degree are significant depends on the problem posed. For example, for the acceleration problem in section~\ref{accproblem} the exact shape of the graph is irrelevant, as long as the graph starts with zero and ends with zero, and is positive for a finite time in-between. In the potential versus field problem~\ref{potproblem} on the other hand, the asymptotic behavior at certain positions and the position of crossing axes is important. 
+Which features to which degree are significant depends on the problem posed. Also, features that the instructor might characterize as spurious may not be seen as such by the students - ignoring this possibility may lead authors to miss some key difficulties that students face. Thus, the problem-specific list of spurious features may need to be evaluated and revised based on the study's findings. For example, for the acceleration problem in section~\ref{accproblem} the exact shape of the graph is irrelevant, as long as the graph starts with zero and ends with zero, and is positive for a finite time in-between. In the potential versus field problem~\ref{potproblem} on the other hand, the asymptotic behavior at certain positions and the position of crossing axes is important. 
 \subsection{Rules}\label{rules}
 Rather than constructing the graph using B\'ezier curves and checking if the parameters are within acceptable tolerance~\cite{kennedy04}, we propose to evaluate the graphs using rule sets for the function itself and its derivatives. Figures~\ref{accrule} and \ref{potrule} show examples of what these rules might look like.
 \begin{figure}\begin{center}
@@ -197,19 +209,20 @@
 \end{tabular}\end{center}
 \caption{Example for a possible rule set for the potential problem~\ref{potproblem}. The variables \$pos1 and \$pos2 denote the positions of the charges and are determined by the randomization of the problem. The variable \$zerofield denotes the axis intercept and is calculated by the problem.\label{potrule}}
 \end{figure}
-Instead of applying tolerances to parameters, the system now needs to allow for degrees of ``fuzziness'' in the application of the rules: sketches are not plots, and students who correctly sketch the significant features of the graph need to receive credit. To ensure this, student input from the sketching client needs to be processed server-side and appropriate fuzzy algorithms need to be developed to apply the rules. Figure~\ref{processing} shows a possible sequence of processing steps. The server receives raw data of the student sketch, in this example, the current in an RLC-circuit (blue) and the enveloping exponential decay functions (red and green). In the next step, several of these artifacts are removed by applying a smoothing algorithm to the data. In a subsequent step, the data is fit by a function. As it turns out, in this freehand drawing, while being a correct sketch, the frequency increases slightly with time, so if in the last step, the differential equation itself is used to verify the function, sufficient fuzziness needs to be applied to accept the sketch.
+Instead of applying tolerances to parameters, the system now needs to allow for degrees of ``fuzziness'' in the application of the rules: sketches are not plots, and students who correctly sketch the significant features of the graph need to receive credit. One of the values of the tool is that it starts from the students' thinking and not from the instructors'.  The students draw what they think; they don't choose from options the instructors selected.  It will be very important to have a large enough fuzziness so that the students are not  too restricted by the instructors' categories. To ensure this, student input from the sketching client needs to be processed server-side and appropriate fuzzy algorithms need to be developed to apply the rules. Figure~\ref{processing} shows a possible sequence of processing steps. The server receives raw data of the student sketch, in this example, the current in an RLC-circuit (blue) and the enveloping exponential decay functions (red and green). In the next step, several of these artifacts are removed by applying a smoothing algorithm to the data. In a subsequent step, the data is fit by a function. As it turns out, in this freehand drawing, while being a correct sketch, the frequency increases slightly with time, so if in the last step, the differential equation itself is used to verify the function, sufficient fuzziness needs to be applied to accept the sketch.
 \begin{figure}
-\begin{tabular}{p{1.8in}l}
-\vspace*{-1.6in}
+\begin{tabular}{p{1.9in}l}
+\vspace*{-1.8in}
 
 Raw data received from client. The sketch was made freehand with a trackpad on a laptop computer in less than a minute and clearly shows the artifacts from having to move the finger multiple times across the pad.&\includegraphics[width=2.6in]{figures/damped}\\\hline
-\vspace*{-1in}
+\vspace*{-1.7in}
 
-Removal of artifacts, smoothing.&\includegraphics[width=2.6in]{figures/dampedcleaned}\\\hline
-\vspace*{-1in}
+Removal of artifacts and smoothing. This step will likely be accomplished by a convolution of the data with a Gauss function of variable width in all areas that are not identified as singularities.&\includegraphics[width=2.6in]{figures/dampedcleaned}\\\hline
+\vspace*{-1.9in}
 
-Fitting the data to a function.&\includegraphics[width=2.6in]{figures/dampedfit}\\\hline
-Applying rules. The parameters $c_1$ and $c_2$ in the differential equation need to be fit to the graph, since the problem itself does not specify their values.&{\small \begin{tabular}{|l|l|l|l|l|p{1.5in}|}\hline
+Fitting the data to functions. This step will likely be accomplished by the system piecewisely fitting a set of trial functions (including simple splines) to the smoothed data. At the end of this step, the data is represented by a piecewise set of analytic functions with known derivatives.
+&\includegraphics[width=2.6in]{figures/dampedfit}\\\hline
+Applying rules. The rule set in this example is simply the differential equation governing damped harmonic oscillation. The parameters $c_1$ and $c_2$ in the differential equation need to be fit to the graph, since the problem itself does not specify their values. In this example, the difference to the parametric spline approach of~\cite{kennedy04,kennedy98} is particularly prominent.&{\small \begin{tabular}{|l|l|l|l|l|p{1.5in}|}\hline
 {\bf Type}&{\bf From $x$}&{\bf To $x$}&{\bf From $y$}&{\bf To $y$}&{\bf Rules}\\\hline
 Interval&0&&\$i0&&$\displaystyle f+c_1\frac{df}{dx}+c_2\frac{d^2f}{dx^2}=0$;\newline $c_1>0$; $c_2>0$\\\hline\end{tabular}}
 \\
@@ -220,61 +233,83 @@
 Authoring an appropriate rule set is likely going to be a task that is perceived by the average faculty author as too complex. We are thus going to implement two sets of tools to facilitate authoring:
 \begin{itemize}
 \item Templates: we will provide templates for functions that frequently appear in physics, mathematics, and engineering, such as linear, quadratic, exponential, logarithmic, sinusoidal, etc. As in other parts of the LON-CAPA problem editor, authors can start from these templates and adapt them to their particular situation.
-\item Graph-based rule editor: in this editor mode, the author will be asked to provide a number of correct graph responses. The system will then propose a rule set which the author can adapt.
+\item Graph-based rule editor: in this editor mode, the author will be asked to provide a number of correct graph responses. The system will then propose a set of 20 to 30 rules extracted from the sketches, each with checkboxes, so the author can selectively activate or deactivate the proposed rules. In addition, the author can adapt the rules.
 \end{itemize}
 An even harder task may be the determination of the appropriate fuzziness. To this end, after the specification of the rule set, the author will be asked to provide a number of correct sketches for different randomizations of the problem. The system will then either determine the appropriate fuzziness or reject the rule set, in which case the author will be asked to modify it.
 \subsection{Refining the Rule Set}
 LON-CAPA has a built-in feedback system. When a student sends a message using this system, faculty is provided with complete contextual information, i.e., the version of the problem that the student had, and his or her previous attempts~\cite{kortemeyer05feedback}. As students are working on problems, they frequently contact instructors with questions why their solution is wrong, and at times, errors in problems get detected this way. In such cases, the instructor can manually give credit and notify the author. We will enhance this author feedback loop such that student solutions can be used to adjust the rule set or fuzziness of problems.
-\section{Tool Development}
-Most of the infrastructure for the sketching tool is already in place, including all of the content and course management features. We will need to develop a client-side tool that can take the graph input and the server-side functionality that is to be used to author and evaluate the rule sets.
-
-For the client-side functionality, different technologies such as Java applets in connection with CGI-submissions or servlet communication, or Adobe Flash (for example in connection with red5), will be tested to maximize platform compatibility and bandwidth efficiency. 
+\section{Tool Development}\label{tool}
+Most of the infrastructure for the sketching tool is already in place, including all of the content and course management features. We will need to develop
+a client-side tool that can take the graph input and the server-side functionality that is to be used to author and evaluate the rule sets.
 
+\begin{itemize}
+\item For the client-side functionality, different technologies such as Java applets in connection with CGI-submissions or servlet communication, or Adobe Flash (for example in connection with red5), will be tested to maximize platform compatibility and bandwidth efficiency. 
+\item For the server-side functionality, we will work in the LON-CAPA open-source environment (Apache modperl), using appropriate mathematical libraries.
+\end{itemize}
 The initial coding will be carried out by members of the MSU LON-CAPA group and is expected to take approximately nine months. As the tool is refined, additional coding will be necessary.
 \section{Usability Testing}
 The ease of authoring is crucial for the widespread adoption of the tool, and has been one of the limiting factors to the dissemination of the original Interactive Graphing Tool.~\cite{kennedy04}. The same is true for the student interface: a tool which students cannot use is likely not going to find wide adaption by instructors. In order to ensure that the graphing tool meets user expectations and that the interaction between the system and the user is optimized, user-centered design methodologies should be incorporated into the product development process. User-centered design means that products are developed such that they are easy, effective, accessible, and enjoyable to use from the {\it userÕs} perspective, supporting the tasks that they are trying to accomplish. We propose that conducting a usability evaluation (with representative end users) and a web accessibility evaluation will significantly enhance the toolÕs usability, thereby resulting in a more successful, usable, enjoyable product.
 \subsection{Testing Facility}
 The usability evaluation and/or usability focus group would be conducted at the MSU Usability \& Accessibility Center. The facility is equipped with Internet connectivity and video recording equipment, operated from a separate control room. The facility enables both the recording of one-on-one usability sessions and/or focus groups, as well as their live observation from a separate area~\cite{msuusabilitylab}.
-\subsection{Usability Evaluation}
+\subsection{Usability Evaluation}\label{usability}
 Usability specialists will conduct two usability evaluations: Faculty user group, consisting of 10 representative faculty members, and a Student user group comprised of 10 representative college students. The goal of the user experience testing is to assess the degree to which the product matches the way that they expect to interact with the graphing tool based on their background and experience. This study would involve conducting one-on-one user experience sessions lasting 1-1/2 hours each. Additionally, the session will consist of users performing 5-6 task scenarios that concentrate on the core functionality of the product. For the Faculty group, the tasks will include general problem editing, specifying which characteristics of the graphs are important, using the tool to test problems, and working with student results. The tasks for the Student group will concentrate on inputting their graphs, making corrections to graphs, and the quality of the feedback by the system. Key usability goals would include effectiveness, which refers to how well a system does what it is supposed to do (measures: percentage of tasks completed successfully; number and types of errors); efficiency, or the way a system supports users in carrying out their tasks (measure: time to perform a particular task successfully); and satisfaction which relates to the subjective responses users have to the system (measures: user satisfaction ratings; verbal and written feedback). This usability evaluation will save time and reduce development costs by anticipating user expectations and reactions before the product design or redesign is finalized. We will produce detailed usability report with actionable recommendations.
-\subsection{Web Accessibility Compliance Inspection}
+\subsection{Web Accessibility Compliance Inspection}\label{accessibility}
 Accessibility experts will evaluate the graphing tool and identify the improvements needed to ensure legal compliance with Section 508 standards. Coding the tool with accessibility design principles in mind will enhance the user experience of customers who use assistive technology as they interact with the product, thus increasing the ability to reach and satisfy the broadest possible audience. Additionally, including common accessibility features would dramatically improve the user experience for customers using mobile phone browsers, personal digital assistants, and even low-bandwidth connections. We will provide with a detailed report outlining the accessibility standards, whether they have been met, the code examples, and other helpful information.
-\section{Initial Content Development}
+\section{Initial Content Development}\label{content}
 Content will initially be developed in areas where there is already existing LON-CAPA content that uses representation translation, e.g.,
 \begin{itemize}
-\item Motion in one dimension
-\item Simple harmonic motion
-\item Induction
+\item Motion in one dimension\vspace*{-2mm}
+\item Simple harmonic motion\vspace*{-2mm}
+\item Induction\vspace*{-2mm}
 \item Time-Varying Currents
 \end{itemize}
 Content will be specifically designed to address the difficulties identified in~\cite{mcdermott,beichner}, and made available network-wide to all participating institutions.
 
-\section{Evaluation of Educational Effectiveness}
-\subsection{Assessment of the Tool as Instructional Aid}
-As an instrument to evaluate the effectiveness of the online homework, we will deploy the Test of Understanding Graphs in Kinematics (TUG-K)~\cite{beichner} in consecutive semesters, both before and after introducing the problems. We will also compare scores on the Force Concept Inventory (FCI)~\cite{fci}, for which already several years of pre- and post-test scores exist for the course initially under investigation.
-\subsection{Assessment of How the Tool Influences Cognitive Processes}
-Students will be observed while working on problems, where we will compare problems that are purely numerical, problems that require formula input, and problems with graph choices, with those that use the graph sketching tool. For these observations, we have two mechanisms:
+In addition, a small number of training problems will be authored to familiarize the students with the tool. In these problems, the solution will be given, for example, students will be asked to sketch a parabola or copy a given sketch. This practice has been successful with other problem types, e.g., symbolic formula input, scientific notation, and the input of physical units, since it allows students to practice mastery of the tool before embarking into more complex tasks, where they may not be able to distinguish between mastery of the tool and mastery of the physics.
+\section{Evaluation of Educational Effectiveness}\label{education}
+
+We will evaluate the impact that the tool has on studentsÕ: A) sketching of graphs of physical phenomena, B) holistic reading of graphs, and C) conceptual understanding of selected physics topics.  We will also study the cognitive processes involved in sketching graphs from textual descriptions of physical phenomena.
+We will use three instruments to gather the data needed to resolve the aforementioned issues:
+\begin{description}
+\item[\rm{1.}]	A written sketching test that will consist of two kinds of items: A) Students will be required to sketch a graph that matches a textual description of a physical phenomenon; B) Students will be required to write a verbal description of a physical phenomenon that is represented by a graph. 
+\item[{\rm 2.}]	A written physics content test that assesses studentsÕ conceptual understanding of phenomena similar to those used in the sketching test. We currently plan on using the Force Concept Inventory (FCI)~\cite{fci}, for which already several years of pre- and post-test scores exist for the course initially under investigation.
+\end{description}
+Both tests will address some physics concepts that were learned in conjunction with sketching skills and some that were learned with no connection to sketching.
+Analysis of both data sources will allow us to determine whether difficulties student faced in the sketching test were due to limited sketching abilities or to a limited conceptual understanding of the physics involved.  It will allow us to determine whether skill at sketching is content-dependent and whether sketching practice had any impact on studentsÕ conceptual understanding.
+\begin{description}
+\item[{\rm 3.}]	Think-aloud interviews in which students and physics experts will be requested to sketch graphs that match textual descriptions of physical phenomena.  The students will be interviewed three times to follow their development.
+\end{description}
+In addition, we will
 \begin{itemize}
-\item A subset of volunteer students will be observed in the Collaborative Learning Laboratory at Michigan State University, which has 20 wireless laptop computers with built-in cameras and microphones, as well as remote-access software to capture student transactions, in addition to portable video cameras. The lab has round tables, movable chairs, and white boards, and students are encouraged to work together on problem solutions. The approach allows for in-depth study and documentation of the problem-solving process. 
-\item We will also analyze the online discussions around these different problem types, which are a rich source of information~\cite{kortemeyer05ana,kortemeyer07correl}. Using the same technique, it was found that different problem types lead to different student discussion behavior.
+\item
+deploy the Test of Understanding Graphs in Kinematics (TUG-K)~\cite{beichner} in consecutive semesters, both before and after introducing the problems
+\item analyze the online discussions around these different problem types, which are a rich source of information~\cite{kortemeyer05ana,kortemeyer07correl}. Using the same technique, it was found that different problem types lead to different student discussion behavior.
 \end{itemize}
 
-\section{Dissemination}
+\section{Dissemination}\label{dissemination}
 The tool itself and its documentation will be included in the production version of LON-CAPA and thus become part of the regular distribution. The tool will be presented at the annual LON-CAPA conferences and included in the training workshops. 
 
 Any content material developed will be made available network-wide to all participating institutions.
 
 Research results will be published in the standard journals, including The Physics Teacher for application studies, and the American Journal of Physics or the Physical Review ST-PER cognitive studies. Presentations will be given at the American Association of Physics Teachers conferences and associated PER conferences.
 \section{Project Management}
-The primary project responsibility will be with the PI, Gerd Kortemeyer. Dr. Kortemeyer will supervise the postdoctoral associate. The coding of the tool will be supervised with the LON-CAPA Technical Director, Guy Albertelli.  Sarah Swierenga, Director of the Usability \& Accessibility Center at MSU, will be responsible for the direction of the usability and accessibility study.
+The primary project responsibility will be with the PI, Gerd Kortemeyer. Dr. Kortemeyer will supervise the postdoctoral associate in physics education research, who will assist in both the content development and the study of the educational effectiveness. Stuart Raeburn will be the lead programmer for the tool development. All coding efforts will be coordinated with the LON-CAPA Technical Director, Guy Albertelli.  Sarah Swierenga, Director of the Usability \& Accessibility Center at MSU, will be responsible for the direction of the usability and accessibility study.
 \section{Project Timeline}
 \subsection{Year 1}
-The rule set format as well as the fuzziness algorithms are defined. Prototypes are implemented and tested, followed by the development of the production version.
+The rule set format as well as the fuzziness algorithms are defined. Prototypes are implemented and tested, followed by the development of the production version (section~\ref{tool}).
 \subsection{Year 2}
-The usability and accessibility testing will be carried out, as well as an initial formative educational evaluation of the tool in focus group settings. In parallel, content for the tool is developed.
+The usability (section \ref{usability}) and accessibility (section~\ref{accessibility}) testing will be carried out, as well as an initial formative educational evaluation of the tool in focus group settings 
+(section~\ref{education}). In parallel, content for the tool is developed (section~\ref{content}).
 \subsection{Year 3}
 The tool becomes part of the LON-CAPA production releases.
-The assessment of its educational effectiveness in carried out in a production setting. Results are analyzed and published, as well as presented at conferences.
+The assessment of its educational effectiveness (section \ref{education}) in carried out in a production setting. Results are analyzed and published, as well as presented at conferences (section~\ref{dissemination}).
+\section{Qualifications of the PIs}
+\begin{description}
+\item[Gerd Kortemeyer] is an assistant professor of physics education at Michigan State University. He has taught introductory calculus-based physics for a number of years. He is the Principal Investigator of the LON-CAPA Project (section~\ref{loncapa}), has contributed to its code base, and has authored more than 1200 online resources (problems, text pages, and images) within the system. In addition, he is currently authoring problem for the Serway physics textbook~\cite{serway}, and authored a book on numerical coprocessors with a significant portion devoted to discrete signal processing~\cite{copro}.
+\item[David Fortus] is a senior scientist at the Weizmann Institute of Science in Israel and an assistant professor of science education at Michigan State University. He is a co-PI on the IQWST project 
+(section~\ref{loncapa}), has won research awards from the National Association for Research on Science Teaching (NARST) and from the American Psychological Association (APA), and has worked as a project manager in the hi-tech industry. 
+\item[Stuart Raeburn] is currently a postdoctoral staff member in the Division of Science and Mathematics Education (DSME) at Michigan State University, where he is a member of the LON-CAPA development team.  In the past decade at Michigan State University (prior to joining DSME) he was a visiting faculty in the Department of Geological Sciences, where he taught a variety of geology courses, from introductory to graduate level, and he was also a developer and administrator for a number of course management systems in the centrally-supported Faculty Facility for Creative Computing. 
+\end{description}
 \pagebreak
 \bibliography{graphing}
 \end{document} 
\ No newline at end of file

--www1174945725--