Differences

This shows you the differences between two versions of the page.

--- ast-patterns:start [2018/06/19 23:02]
timotej
+++ ast-patterns:start [2018/07/04 18:42]
timotej
@@ Line 1: / Line 1: @@
 ====== AST patterns ======
-We propose using abstract-syntax-tree (AST) patterns as features for learning rules to distinguish between correct and incorrect programs. These rules can be used for debugging student programs: rules for incorrect programs contain patterns indicating mistakes, whereas rules for correct programs cover subsets of submissions sharing the same solution strategy.
-===== Papers =====
-==== Syntax-based analysis of programming concepts in Python (2018) ====
 Writing programs is essential to learning programming. Most programming courses encourage students to practice with lab and homework assignments. By analyzing solutions to these exercises teachers can discover mistakes and concepts students are struggling with, and use that knowledge to improve the course. Students however tend to submit many different programs even for simple exercises, making such analysis difficult.
-We propose using tree regular expressions to encode common patterns in programs. Based on these patterns we induce rules describing common approaches and mistakes for a given assignment. In this paper we present a case study of rule-based analysis for an introductory Python exercise. We show that our rules are easy to interpret, and can be learned from a relatively small set of programs.
+We propose using abstract-syntax-tree (AST) patterns as features for learning rules to distinguish between correct and incorrect programs. These rules can be used for analyzing student programs: rules for incorrect programs contain patterns indicating mistakes, whereas rules for correct programs cover subsets of submissions sharing the same solution strategy.
-  * {{:papers:aied2018.pdf|Paper}}
+===== Code =====
-==== Identifying typical approaches and errors in Prolog programming with argument-based machine learning (2018) ====
+Check out the WIP source code with ''git clone https://ailab.si/code/patterns.git'', or browse it online by following the same link.
-In programming courses with high student-teacher ratios, it is nearly
+Automatic hints for Prolog are implemented in the [[https://codeq.si|CodeQ programming tutor]] (use account ''demo-auto'' with password ''demo''). Note that the application is not fully translated into English. Source code for CodeQ is also [[http://codeq.si/code/|available]].
-impossible to provide feedback to all homeworks submitted by students.
-The alternative would be to generate automatic feedback, however
-this would require teachers to specify all possible errors made by students.
-Instead we propose a data-driven algorithm for identification of typical approaches and errors. We present the problem as supervised rule learning, where each learned rule can be used as a hint to a student. To learn rules, we use programs submitted by students as learning instances that are classified either as correct or incorrect solutions. The attributes for machine learning correspond to patterns from an abstract syntax tree.
+===== Papers =====
-As the space of patterns is immense and we wish to use only meaningful patterns, we used the argument-based machine learning (ABML) process, where an expert and ABML interactively exchange arguments until the model is good enough. We provide a step-by-step demonstration of the ABML process, describe some of the induced rules and explain how are they used to automatically generate hints to students. Finally, we evaluate classification accuracy of learned rules on 42 Prolog exercises from the Programming Languages Course at University of Ljubljana.
+{{:papers:aied2018.pdf|Syntax-based analysis of programming concepts in Python}} (2018). In this paper we present a case study of rule-based analysis for an introductory Python exercise. We show that our rules are easy to interpret, and can be learned from a relatively small set of programs.
-  * {{:papers:abcn2_knowledge_acquisition.pdf|Paper}}
+{{:papers:abcn2_knowledge_acquisition.pdf|Identifying typical approaches and errors in Prolog programming with argument-based machine learning}} (2018). As the space of AST patterns is immense and we wish to use only meaningful patterns, we used the argument-based machine learning (ABML) process, where an expert and ABML interactively exchange arguments until the model is good enough. We provide a step-by-step demonstration of the ABML process, describe some of the induced rules and explain how are they used to automatically generate hints to students. Finally, we evaluate classification accuracy of learned rules on 42 Prolog exercises from the Programming Languages Course at University of Ljubljana.
   * {{:papers:ABCN2e.pdf|ABCN2e: an extension of ABCN2 (white paper)}}
-  * [[https://github.com/martinmozina/orange3-abml |Implementation of ABCN2e as an Orange 3 add-on]]
+  * [[https://github.com/martinmozina/orange3-abml|Implementation of ABCN2e as an Orange 3 add-on]]
-==== Automatic extraction of AST patterns for debugging student programs (2017) ====
-When implementing a programming tutor, it is often difficult to manually consider all possible errors encountered by students. An alternative is to automatically learn a bug library of erroneous patterns from students’ programs.
-We propose using abstract-syntax-tree (AST) patterns as features for learning rules to distinguish between correct and incorrect programs. These rules can be used for debugging student programs: rules for incorrect programs (buggy rules) contain patterns indicating mistakes, whereas rules for correct programs cover subsets of submissions sharing the same solution strategy.
-To generate hints, we first check all buggy rules and point out incorrect patterns. If no buggy rule matches, rules for correct programs are used to recognize the student’s intent and suggest patterns that still need to be implemented.
-We evaluated our approach on past student programming data for a number of Prolog problems. For 31 out of 44 problems, the induced rules correctly classified over 85% of programs based only on their structural features. For approximately 73% of incorrect submissions, we were able to generate hints that were implemented by the student in some subsequent submission.
+{{:papers:aied2017:aied2017.pdf|Automatic extraction of AST patterns for debugging student programs}} (2017). We propose using abstract-syntax-tree (AST) patterns as features for learning rules to distinguish between correct and incorrect programs. We evaluated our approach on past student programming data for a number of Prolog problems. For 31 out of 44 problems, the induced rules correctly classified over 85% of programs based only on their structural features. For approximately 73% of incorrect submissions, we were able to generate hints that were implemented by the student in some subsequent submission.
+    * Results ({{:papers:aied2017:results.tsv|.tsv}}, {{:papers:aied2017:results.ods|.ods}})
-Paper presented at the [[http://iaied.org/conf/event/14/|18th conference on AI in Education]].
+{{:papers:tlt2017:tlt2017.pdf|Rewrite rules for debugging student programs in programming tutors}} (2017). Before AST patterns we formalized the notion of a »code edit« to support automatic program rewriting, described in this paper.
-  * {{:papers:aied2017:aied2017.pdf|Paper}}
-  * [[https://ailab.si/code/aied2017.git/|Source code]]
-  * Results ({{:papers:aied2017:results.tsv|.tsv}}, {{:papers:aied2017:results.ods|.ods}})

A.I.lab

User Tools

Site Tools

Differences

Page Tools