An implicit assumption in psychometrics and educational statistics is that the generative model for student scores on test questions is governed by the topics of those questions and each student's aptitude in those topics. That is, a function to generate the matrix of scores for m students on n questions should rely on each student's ability in a set of t topics, and the relevance of each question to those topics. In this paper, we use educational data mining techniques to analyze score matrices from university-level computer science courses, and demonstrate that no such structure can be extracted from this data.
Good teachers know their students, and exploit this knowledge to adapt or optimise their instruction. Traditional teachers know their students because they interact with them face-to-face in classroom or one-to-one tutoring sessions. In these settings, they can build student models, i.e., by exploiting the multi-faceted nature of human-human communication. In distance-learning contexts, teacher and student have to cope with the lack of such direct interaction, and this must have detrimental effects for both teacher and student. In a past study we have analysed teacher requirements for tracking student actions in computer-mediated settings. Given the results of this study, we have devised and implemented a tool that allows teachers to keep track of their learners' interaction in e-learning systems. We present the tool's functionality and user interfaces, and an evaluation of its usability.
The ability to detect fluctuation in students' comprehension of text would be very useful for many intelligent tutoring systems. The obvious solution of inserting comprehension questions is limited in its application because it interrupts the flow of reading. To investigate whether we can detect comprehension fluctuations simply by observing the reading process itself, we developed a statistical model of 7805 responses by 289 children in grades 1-4 to multiple-choice comprehension questions in Project LISTEN's Reading Tutor, which listens to children read aloud and helps them learn to read. Machine-observable features of students' reading behavior turned out to be statistically significant predictors of their performance on individual questions.
This paper describes an approach to support practitioners in the analysis of computer-supported learning processes by utilizing logfiles of learners' actions captured by the system. We enable researchers and teachers to identify and search for insightful patterns of the learning process in two ways: on the one hand patterns can be specified explicitly to be searched in logfiles; on the other hand automatic discovery of patterns that match configurable parameters is used to find most typical sequences in the logfiles. Both features have been realized in a stand-alone tool that accepts generic logfiles usable with a potentially wide variety of different learning systems. This is shown in an example of practical use of this tool in a project that supports researchers and moderators of graphical electronic discussions.
This study examined the effectiveness of an educational data mining method – Learning Factors Analysis (LFA) – on improving the learning efficiency in the Cognitive Tutor curriculum. LFA uses a statistical model to predict how students perform in each practice of a knowledge component (KC), and identifies over-practiced or under-practiced KCs. By using the LFA findings on the Cognitive Tutor geometry curriculum, we optimized the curriculum with the goal of improving student learning efficiency. With a control group design, we analyzed the learning performance and the learning time of high school students participating in the Optimized Cognitive Tutor geometry curriculum. Results were compared to students participating in the traditional Cognitive Tutor geometry curriculum. Analyses indicated that students in the optimized condition saved a significant amount of time in the optimized curriculum units, compared with the time spent by the control group. There was no significant difference in the learning performance of the two groups in either an immediate post test or a two-week-later retention test. Findings support the use of this data mining technique to improve learning efficiency with other computer-tutor-based curricula.
Building on past results establishing a benefit for using handwriting when entering mathematics on the computer, we hypothesize that handwriting as an input modality may be able to provide significant advantages over typing in the mathematics learning domain. We report the results of a study in which middle and high school students used a software tutor for algebra equation solving with either typing or handwriting as the input modality. We found that handwriting resulted in similar learning gains in much less time than typing. We also found students seem to experience a higher degree of transfer in handwriting than in typing based on performance during training. This implies that students could achieve farther goals in an intelligent tutoring system curriculum when they use handwriting interfaces vs. typing. Both of these results encourage future exploration of the use of handwriting interfaces for mathematic instruction online.
Odette Auzende, Hélène Giroire, Françoise Le Calvez
524 - 526
We propose an extension of the IMS-QTI v2 specification to define templates of mathematics exercises with template variables linked by constraints. We describe the various types of constraints we add, then we present functionalities for defining template mathematics exercises and creating associated dynamic web pages
Roger Azevedo, Amy Witherspoon, Shanna Baker, Jeffrey Greene, Daniel Moos, Jeremiah Sullins, Andrew Trousdale, Jennifer Scott
527 - 529
Eighty-two (N = 82) college students with little knowledge of the circulatory system were randomly assigned either to the control condition (SRL; self-regulated learning) or human tutoring (ERL; externally-regulated learning) condition. Learners in the SRL condition regulated their own learning, while learners in the ERL condition had access to a human tutor who facilitated their self-regulated learning. All learners were given 40 minutes to learn about the circulatory system. We collected several pretest and posttest learning measures and collected think-aloud protocols during learning. Generally, the learners in the ERL condition significantly outperformed the learners in SRL condition. In comparison to the SRL condition, results indicate that learners in the ERL condition deployed significantly more SRL processes related to planning, monitoring, and handling task difficulties. Each of the classes of SRL processes was predictive of learners' performance on different posttest measures.
In many intelligent tutoring systems, a detailed model of the task domain is constructed and used to provide students with assistance and direction. Reciprocal tutoring systems, however, can be constructed without needing to codify a full-blown model for each new domain. This provides various advantages: these systems can be developed rapidly and can be applied to complex domains for which detailed models are not yet known. In systems built on the reciprocal tutoring model, detailed validation is needed to ensure that learning indeed occurs. Here, we provide such validation for SpellBEE, a reciprocal tutoring system for the complex task domain of American-English spelling. Using a granular definition of response accuracy, we present a statistical study designed to assess and characterize student learning from collected data. We find that students using this reciprocal tutoring system exhibit learning at the word, syllable, and grapheme levels of task granularity.
COLLECT-UML is a collaborative constraint-based tutor for teaching object-oriented analysis and design using Unified Modelling Language. It is the first system in the family of constraint-based tutors to represent a higher-level skill such as collaboration using constraints. We present the full evaluation study carried out at the University of Canterbury to assess the effectiveness of the system in teaching UML class diagrams and good collaboration. The results show that COLLECT-UML is an effective educational tool. In addition to improved problem-solving skills, the participants both acquired declarative knowledge about good collaboration and did collaborate more effectively. The participants have enjoyed working with the system and found it a valuable asset to their learning.
The efficacy of a tutoring system for pre-algebra instruction plus human tutoring was compared to instruction provided to small groups of middle school students by experienced human math tutors, with instructional time held constant. Students completed pre- and post-tests of computation, fractions, algebra and rational numbers skills. Results indicated that students showed significant improvement from pre- to post-test, but there was no difference as a function of type of tutoring. The findings help to establish the efficacy of ITS instruction relative to skilled human tutoring of students in small groups.
Students do not always use efficient strategies to solve scientific problems. Students' motivation beliefs were assessed through an on-line survey instrument, along with performance on easy and challenging multimedia science problem sets. Results indicated that female students reported lower self-efficacy and higher concerns about their performance than male students. Students' initial strategies predicted overall performance on the harder problem sets. In addition, although there was no difference in initial strategy, students' motivational beliefs predicted overall performance. Results suggest that both cognitive and motivational factors influence students' strategic efficacy.
Jill Burstein, Jane Shore, John Sabatini, Yong-Won Lee, Matthew Ventura
542 - 544
K-12 teachers often need to develop text adaptations of authentic classroom texts as a way to provide reading-level appropriate materials to English language learners in their classrooms. Adaptations are done by hand, and include practices such as, writing text summaries, substituting appropriate vocabulary, and offering native language support. In light of the labor-intensiveness involved in the manual creation of text adaptations, we have implemented the Automated Text Adaptation Tool (ATA v.1.0). This tool uses natural language processing (NLP) capabilities to automatically generate text adaptations that are similar to teacher-created adaptations. We have also conducted a qualitative teacher pilot with 12 middle school teachers who teach English language learners. We present a discussion of the tool and pilot.
Automatic essay scoring is a very important tool for many educational researches. However, the linguistic differences between Chinese and English indicate a need to reconsider various issues when designing Chinese automatic essay scoring systems. This study proposes system architecture for developing a Chinese automatic essay scoring system and analyzes various issues related to feature extraction of Chinese writing.
In this study, students studied two different domains in the same Intelligent Tutoring System, Andes. Analysis of 435 log files from 22 subjects indicated that there are two types of interactive behaviours in Andes: domain-independent and domain-specific. We believe the existence of the domain-specific behaviours is one possible reason that similar meta-cognitive behaviors has not been found across domains in a single ITS curriculum . This paper describes the study and the potential applications of these findings.
One possible approach to reducing the cost of developing an intelligent tutoring system (ITS) is to reuse the components of an existing ITS. We used this approach to develop an Andes probability tutoring system by modifying the declarative knowledge of the Andes physics tutoring system. We claim that if we cluster various educational domains into groups based on their problem-solving methods , then it will be more efficient to port an existing ITS to a new domain in the same cluster than to build a new ITS from scratch.
Scotty D. Craig, Kurt Vanlehn, Soniya Gadgil, Micki Chi
554 - 556
Learning by observation has long been a traditional method of learning. Recent work has pointed toward collaboratively observing tutoring as a promising new method for observational learning. Our current study tested this new method in the PSLC physics LearnLab where students were introduced two topics of rotational kinematics by observing videos while problem solving in Andes. The students were randomly assigned to a pair condition that collaboratively observed a video of an expert tutoring or providing an example, or to a solo condition that observed a video of an expert worked example. Several robust and normal learning measures were collected, however, to date only multiple choice measures have been analyzed. Students' performance on the multiple choice questionnaires revealed significant pretest to posttest gains for all conditions. However, no differences have been found among conditions for normal learning measures.
This paper describes a system that automatically links study materials to encyclopedic knowledge, and shows how the availability of such knowledge within easy reach of the learner can improve both the quality of the knowledge acquired and the time needed to obtain such knowledge.
Benedict Du Boulay, Genaro Rebolledo Méndez, Rosemary Luckin, Erika Martínez-Mirón
563 - 565
: Motivationally intelligent systems deploy resources and tactics dynamically to maintain or increase the student's desire to learn and her willingness to expend effort in so doing. Three categories of diagnostic inputs and feedback reactions are outlined each with its associated meta-level. The meta-level includes the account which learners tell themselves, the system and others about what they know, how they feel, and the conditions under which they learn best.
Intrinsic motivation has been shown in previous research to lead to better learning. In order to increase intrinsic motivation, REAP, a tutoring system for ESL vocabulary was enhanced to prefer practice readings that match personal interests. In a randomized experiment, students receiving personalized readings indicated higher levels of interest in post-reading questionnaires. Additionally, overall post-test scores were higher (but not significantly) for students with interest-matched practice readings than for students using a previous version of REAP that did not match topics to student interests.
Arthur Graesser, Patrick Chipman, Brandon King, Bethany McDaniel, Sidney D'Mello
569 - 571
The relationship between emotions and learning was investigated by tracking the emotions that college students experienced while learning about computer literacy with AutoTutor. AutoTutor is an animated pedagogical agent that holds a conversation in natural language, with spoken contributions by the learner. Thirty students completed a multiple-choice pre-test, a 35-minute training session, and a multiple-choice post-test. The students reviewed the tutorial interaction and were stopped at strategically sampled points for emotion judgments. They judged what emotions they experienced on the basis of the dialogue history and their facial expressions. The emotions they judged were boredom, flow (engagement), frustration, confusion, delight, surprise, and neutral. A multiple regression analysis revealed that post-test scores were significantly predicted by pre-test scores and confusion, but not by any of the other emotions.
Nowadays standard technologies play important roles in enhancing sharability, reusability and interoperability of learning contents. However, there is a lack of pedagogical justification of the contents implemented with the standards. This paper discusses the standard-compliance of our ontology-based modeling framework and how the framework gives theoretical justification to standard-compliant learning/instructional scenarios in a theory-aware authoring tool.
In this paper we shall introduce a new conceptual model for learner modelling based on multinets, which are Bayesian networks mixture. This conceptuel model makes it possible to take into account in a single student model different Bayesian networks, with the same nodes but different structures. We also present experiemental results obtained with real students data.