| Age | Commit message (Collapse) | Author |
|---|
|
For each query:
- generate a single solution
- incorrect solution / timeout / other error → FAIL
- generate (up to) ten solutions
- more than one distinct solution found → FAIL
- timeout is OK (a correct solution was found above)
- if reached this point → PASS
|
|
|
|
This prevents incorrect student solutions from overwriting or adding to
predicates in solution<n> modules.
|
|
Place functions for internal use at the end.
|
|
Using erase/1 keeps the dynamic property for all defined predicates in
the testing module. This overrides the clauses in solution modules. For
example, if the user defines conc/3 in the solution for the dup/2
problem, the correct conc/3 would not get used even after unloading all
user's clauses.
This commit also plugs a couple of memory leaks by opening (and later
discarding) a new Prolog frame in functions test, load_solution and
mark_solved.
|
|
|
|
Turns out unix:dup/2 exists and is autoloaded if the student solution
for dup/2 does not contain the predicate with this name; this can mess
up subsequent tests.
|
|
This is always required for the safe_goal/1 predicate.
|
|
|
|
|
|
Testing logic now lives in PrologEngine. The engine now has some notion
of problems and users, which is necessary to avoid repeatedly loading
code into Prolog.
TODO:
- support library loading
- fix PrologEngine.test for unusual cases (more than one solution, …)
- memoization of correct answers
|
|
This will allow us to match line edits to original source locations.
|
|
Use exceptions to report errors. Used in the server branch.
|
|
Use findnsols/4 to allow limiting the number of solutions found by
prolog.engine.query. Add a basic test case to prolog.engine.
|
|
|
|
Also disable error messages printed by swipl library.
|
|
- only run queries with correct code once
- use msort/2 to implement quicksort/3, used for some tests
- correctly split programs in the presence of =.. operator
|
|
|
