diff options
author | Timotej Lazar <timotej.lazar@fri.uni-lj.si> | 2017-04-01 20:32:46 +0200 |
---|---|---|
committer | Timotej Lazar <timotej.lazar@fri.uni-lj.si> | 2017-04-01 20:32:46 +0200 |
commit | c868100877b4dafcb1251b600bfb85c82532d57c (patch) | |
tree | c0643cb743790be83e98591286eeddc6d9240200 /aied2017/outline.org | |
parent | ad732e88887a72485ca03fae288bb9dc39911d15 (diff) |
Rename “aied2017” directory to “paper”
Diffstat (limited to 'aied2017/outline.org')
-rw-r--r-- | aied2017/outline.org | 81 |
1 files changed, 0 insertions, 81 deletions
diff --git a/aied2017/outline.org b/aied2017/outline.org deleted file mode 100644 index a70c1a3..0000000 --- a/aied2017/outline.org +++ /dev/null @@ -1,81 +0,0 @@ -* Introduction (1) :Timotej: -** Generics -ITSs are useful, but expensive to create. Internet increases the reach of ITSs (think MOOC) and allows us to gather a lot of data. The challenge is to extract useful information from this data to improve feedback to students. - - koedinger2013new - -** Programming ITS -A particularly challenging domain: - - no meaningful “solutions steps” - - no easily discernible and independent “program features” - -** Problem -Find good features to describe student programs that: - - are automatically learnable - - can serve as a basis for generating hints - -** Contributions -A pattern language for describing program features. -Rules for classifying correct/incorrect Prolog programs. - -* Background (2) :Timotej: -Model-tracing is not well-suited for programming tutors. - - Hint Factory (Barnes & Stamper) for programming (Jin, Rivers) - - student traces are basically insane anyway, so learning from them is not a good idea (Piech 2015) -Instead we focus on analyzing individual submissions - CBM (maybe just /like/ CBM?). - - why: human tutors are usually able to comment without referring to previous program versions - -Analyzing program based on formal descriptions (usually defined manually): - - plans (Johnson) - - techniques (Brna/Huang/Le) - - strategies (Gerdes) -Comparing programs directly / after canonicalization (Rivers 2015) - - not easy to determine which differences are crucial - -Defining “programming attributes/features”: - - AST subtrees (Nguyen 2014) - - linkage graphs (Jin 2012) - - n-grams or other purely textual attributes - - mostly used for plagiarism (MOSS) / code clone detection - -Tgrep / trx for structural patterns in NLP - use the same for describing code! - -* Dataset (3) :Timotej: -Solution traces -** Motivating example: ancestor/2 - - show and explain correct / buggy program - - examples of patterns & rules - - buggy rules ≈ error classes - - correct rules ≈ program strategies or “plans” → determine intent - - how correct / buggy rules can be used to automatically determine intent and generate feedback - - using rules to assist authoring tutors - -** Patterns -Programming is all about manipulating data (variables/literals). -Patterns encode (only) relations between data nodes in an AST, and are more granular than subtrees. -Patterns abstract away some information -Algorithm for extracting patterns. - - which patterns are considered - -* Method (2) :Martin: -Rule learning. - - first learn correct, then buggy rules -Automatic hint generation. - -* Evaluation (3) :Martin:Timotej: -Example: describe patterns/rules for ancestor/2 (and/or another domain). - - example trace -CA/AUC for rules/RF/…. - Using e.g. RF with patterns for attributes achieves a high classification accuracy for correct/incorrect programs. - Rules are less accurate, but can be used for hints (how can we improve this)? -Simulated hints: for how many submissions would we suggest a hint that was ultimately implemented in the final solution? - Hints from buggy rules are almost always implemented; correct rules are less certain. - -* Conclusions / future work (1) - - syntactic variations (e.g. using = instead of implicit matching) - - combine with canonicalization - - ABML for rule learning - - integrate into a tutor (CodeQ / CloudCoder) to test in situ - -* Misc -Problems: - - predicates with similar clauses (e.g. union/3) |