|
Next revision
|
Previous revision
|
papers:abml:start [2017/07/06 10:22]
martin created |
papers:abml:start [2018/06/18 17:30] (current)
martin |
| ====== Learning intermediate goals for human problem solving ====== | ====== Identifying typical approaches and errors in Prolog programming with argument-based machine learning ====== |
| |
| In programming courses with high student-teacher ratios, it is nearly | In programming courses with high student-teacher ratios, it is nearly |
| impossible to provide feedback to all homeworks submitted by students. | impossible to provide feedback to all homeworks submitted by students. |
| The alternative would be to give a general feedback to all students or to | The alternative would be to generate automatic feedback, however |
| implement a programming tutor that can automatically generate personalized | this would require teachers to specify all possible errors made by students. |
| feedback. Both approaches are, however, difficult to implement, since teachers | Instead we propose a data-driven algorithm for identification of typical approaches and errors. We present the problem as supervised rule learning, where each learned rule can be used as a hint to a student. To learn rules, we use programs submitted by students as learning instances that are classified either as correct or incorrect solutions. The attributes for machine learning correspond to patterns from an abstract syntax tree. As the space of patterns is immense and we wish to use only meaningful patterns, we used the argument-based machine learning (ABML) process, where an expert and ABML interactively exchange arguments until the model is good enough. We provide a step-by-step demonstration of the ABML process, describe some of the induced rules and explain how are they used to automatically generate hints to students. Finally, we evaluate classification accuracy of learned rules on 42 Prolog exercises from the Programming Languages Course at University of Ljubljana. |
| need to consider all possible errors made by students. In this paper, | |
| we propose to semi-automatically | |
| learn typical patterns of correct and incorrect programs, which can be used to identify different approaches and errors in Prolog programming. The attributes describing programs are extracted from abstract syntax trees (AST). However, as the space of all patterns is immense and not all patterns are meaningful, we propose a new rule learning algorithm based on argument-based machine learning (ABML), where experts can articulate | |
| their relevant knowledge through arguments. We describe the application of our algorithm to Prolog programs submitted by students during Programming Languages Course at University of Ljubljana. | |
| We tested our approach on 42 Prolog exercises and obtained promising results. The average default classification accuracy was 0.65, while the accuracy of our method was above 90\% in 34 cases, | |
| in 7 cases the accuracy was between 85\% and 90\%, however only in one case | |
| our method failed to learn good rules and scored 72\%. We also provide a step-by-step description of gathering new knolwedge accomplished with the ABML interaction loop, and describe some examples of learned rules to demonstrate how can they be used to explain typical errors and approaches in Prolog programs. | |
| |
| Paper submitted to journal. | * TTT |
| | * {{:papers:ABCN2e.pdf|ABCN2e: an extension of ABCN2 (white paper)}} |
| | * [[https://github.com/martinmozina/orange3-abml |Implementation of ABCN2e as an Orange 3 add-on]] |
| * [[https://github.com/martinmozina/orange3-abml |Source code of the Orange 3 add-on]] | |
| |
