Hint Generation
and Evaluation
Design, generate, and mathematically evaluate step-by-step hints for open questions.
Stop guessing and start measuring hint quality.
Design
Start by inserting a factual question.
Generate
Use LLMs to create multiple hint sequences that guide students step-by-step.
Evaluate
Analyze metrics like Leakage and Convergence to pick the perfect hints.
Evaluation Metrics
Measures how strongly the hint steers the learner towards the specific target solution.
Detects if the hint inadvertently gives away the answer too early in the sequence.
Estimates if the concepts used are appropriate for the target student's knowledge level.
Ensures the hint is contextually related to the solution path without drifting.
Classifies linguistic complexity: Easy (0), Intermediate (1), or Difficult (2).
Example: "Capital of Australia"
Question
Generated Hints
It is not the largest city.
It was chosen as a compromise and is inland.
The name starts with 'C' and houses Parliament.
| Hint | Conv. | Leak | Read. |
|---|---|---|---|
| Hint 1 | 0.45 | 0.96 | 0.94 |
| Hint 2 | 0.70 | 0.88 | 0.90 |
| Hint 3 | 0.92 | 0.62 | 0.86 |