Add outline and files for the AIED2017 paper

8 files changed, 110 insertions, 0 deletions

diff --git a/aied2017/aied2017.tex b/aied2017/aied2017.tex
new file mode 100644
index 0000000..d342a9c
--- /dev/null
+++ b/aied2017/aied2017.tex
@@ -0,0 +1,23 @@
+\documentclass{llncs}
+
+\usepackage[utf8]{inputenc}
+
+\begin{document}
+
+\title{TODO}
+\author{TODO}
+\institute{University of Ljubljana, Faculty of Computer and Information Science, Slovenia}
+\maketitle
+
+\begin{abstract}
+TODO
+\end{abstract}
+
+\input{introduction}
+\input{background}
+\input{dataset}
+\input{method}
+\input{evaluation}
+\input{conclusion}
+
+\end{document}
diff --git a/aied2017/background.tex b/aied2017/background.tex
new file mode 100644
index 0000000..563e5ea
--- /dev/null
+++ b/aied2017/background.tex
@@ -0,0 +1 @@
+\section{Background}
\ No newline at end of file
diff --git a/aied2017/conclusion.tex b/aied2017/conclusion.tex
new file mode 100644
index 0000000..47c04f1
--- /dev/null
+++ b/aied2017/conclusion.tex
@@ -0,0 +1 @@
+\section{Conclusion}
\ No newline at end of file
diff --git a/aied2017/dataset.tex b/aied2017/dataset.tex
new file mode 100644
index 0000000..e8d168d
--- /dev/null
+++ b/aied2017/dataset.tex
@@ -0,0 +1 @@
+\section{Dataset}
\ No newline at end of file
diff --git a/aied2017/evaluation.tex b/aied2017/evaluation.tex
new file mode 100644
index 0000000..22e4253
--- /dev/null
+++ b/aied2017/evaluation.tex
@@ -0,0 +1 @@
+\section{Evaluation}
\ No newline at end of file
diff --git a/aied2017/introduction.tex b/aied2017/introduction.tex
new file mode 100644
index 0000000..ac26833
--- /dev/null
+++ b/aied2017/introduction.tex
@@ -0,0 +1 @@
+\section{Introduction}
\ No newline at end of file
diff --git a/aied2017/method.tex b/aied2017/method.tex
new file mode 100644
index 0000000..3a14d53
--- /dev/null
+++ b/aied2017/method.tex
@@ -0,0 +1 @@
+\section{Method}
\ No newline at end of file
diff --git a/aied2017/outline.org b/aied2017/outline.org
new file mode 100644
index 0000000..a70c1a3
--- /dev/null
+++ b/aied2017/outline.org
@@ -0,0 +1,81 @@
+* Introduction (1)						    :Timotej:
+** Generics
+ITSs are useful, but expensive to create. Internet increases the reach of ITSs (think MOOC) and allows us to gather a lot of data. The challenge is to extract useful information from this data to improve feedback to students.
+  - koedinger2013new
+
+** Programming ITS
+A particularly challenging domain:
+  - no meaningful “solutions steps”
+  - no easily discernible and independent “program features”
+
+** Problem
+Find good features to describe student programs that:
+  - are automatically learnable
+  - can serve as a basis for generating hints
+
+** Contributions
+A pattern language for describing program features.
+Rules for classifying correct/incorrect Prolog programs.
+
+* Background (2)						    :Timotej:
+Model-tracing is not well-suited for programming tutors.
+  - Hint Factory (Barnes & Stamper) for programming (Jin, Rivers)
+  - student traces are basically insane anyway, so learning from them is not a good idea (Piech 2015)
+Instead we focus on analyzing individual submissions - CBM (maybe just /like/ CBM?).
+  - why: human tutors are usually able to comment without referring to previous program versions
+
+Analyzing program based on formal descriptions (usually defined manually):
+  - plans (Johnson)
+  - techniques (Brna/Huang/Le)
+  - strategies (Gerdes)
+Comparing programs directly / after canonicalization (Rivers 2015)
+  - not easy to determine which differences are crucial
+
+Defining “programming attributes/features”:
+  - AST subtrees (Nguyen 2014)
+  - linkage graphs (Jin 2012)
+  - n-grams or other purely textual attributes
+    - mostly used for plagiarism (MOSS) / code clone detection 
+
+Tgrep / trx for structural patterns in NLP - use the same for describing code!
+
+* Dataset (3)							    :Timotej:
+Solution traces
+** Motivating example: ancestor/2
+  - show and explain correct / buggy program
+  - examples of patterns & rules
+  - buggy rules ≈ error classes
+  - correct rules ≈ program strategies or “plans” → determine intent
+  - how correct / buggy rules can be used to automatically determine intent and generate feedback
+  - using rules to assist authoring tutors
+
+** Patterns
+Programming is all about manipulating data (variables/literals).
+Patterns encode (only) relations between data nodes in an AST, and are more granular than subtrees.
+Patterns abstract away some information 
+Algorithm for extracting patterns.
+  - which patterns are considered
+
+* Method (2)							     :Martin:
+Rule learning.
+  - first learn correct, then buggy rules
+Automatic hint generation.
+
+* Evaluation (3)					     :Martin:Timotej:
+Example: describe patterns/rules for ancestor/2 (and/or another domain).
+  - example trace
+CA/AUC for rules/RF/….
+  Using e.g. RF with patterns for attributes achieves a high classification accuracy for correct/incorrect programs.
+  Rules are less accurate, but can be used for hints (how can we improve this)?
+Simulated hints: for how many submissions would we suggest a hint that was ultimately implemented in the final solution?
+  Hints from buggy rules are almost always implemented; correct rules are less certain.
+
+* Conclusions / future work (1)
+  - syntactic variations (e.g. using = instead of implicit matching)
+    - combine with canonicalization
+  - ABML for rule learning
+  - integrate into a tutor (CodeQ / CloudCoder) to test in situ
+
+* Misc
+Problems:
+  - predicates with similar clauses (e.g. union/3)