[Cognitive Task Analysis] Geometry Task

05–418/818 CMU Design Educational Games Knowledge Elicitation Assignment Theoretical CTA Diagram

Task Description:

The 3 reference theorems provided in the geometry task
The 2 question sheets that the participants will be provided

Theoretical CTA Synthesis

  1. Identify known quantities
  2. Identify quantities that need to be solved
  3. Identify the theorem that can be used to solve the unknown quantities
  4. Apply the theorem with known quantities
  • At step 1, they may use the diagram to annotate the quantities to help visualize the known information, so they need to decide whether it’s an arc or an angle and deal with them differently.
  • In step 3, if the identified theorem couldn’t be applied since learners don’t have all the necessary quantities available, they can go back to step 2 and try to solve those quantities first.
  • At step 4, learners may use some heuristics like whether the solved quantity matches the diagram to verify their answer, and if it seems very wrong (e.g., on the graph it looks like an obtuse angle but the solution is 60˚), they may mark it as an unknown quantity and go back to step 3 to try to re-solve it.
  • Finally, learners may examine the final answer before submission (e.g., check if the arithmetics are correct), and return to any of the previous steps depending on the mistake identified.
The worked problems of my roommate (Q4–7). She didn’t use write anything on the table so only the graph is presented.
  • Mistakenly traces the minor arc instead of the major arc
  • Identify the wrong theorem to apply,
  • Identify the wrong unknown or known quantities,
  • Can’t identify a theorem because the application direction is backward,
  • Forget about the 1/2 part of theorem 1 & 2,
  • Struggle with the cognitive load of arithmetics (the numbers aren’t nice like 10, 30, 60) and all these different variables (much more than 7 letters on a busy graph),
  • Depend too much on intuition, use it as a way to solve problems rather than a sanity check.

Empirical CTA Participant Requirement:

Empirical CTA Participant Strategy & Individual Diagrams

Participant 1: Q1 (left) & Q2 (right)
Empirical CTA diagram: participant 1
  • What was their answer? 15˚ (wrong)
  • What was the strategy they used?
(1) Read the problem, label the provided angles & arcs:
∠ASE=85˚, ACE, and ∠ATC
(2) Write the exterior angle theorem equation with known and unknown quantities:
ATC = 35˚ = (BD — AC)/2
(3) Apply existing geometry knowledge that “opposite angles are equal” and wrongly derive:
BD = ∠BSD = ∠ASE = 85˚
(4) Solve the equation with one known quantity and relabel unknown quantity as x:
35˚ = (85˚ — x) / 2, so x = AC = 85˚ — 70˚ = 15˚
  • What difficulties did they have? (1) Struggled with major and minor arc ACE, so they explicitly trace the major arc to emphasize that. (2) Confused arc and angles, e.g., equates arc BD as ∠BSD. (3) Almost forgot the /2 part of the theorem, so they originally derive AC as 50˚.
  • Additional Notes: Geometry knowledge outside of the provided theorems was correctly activated but wrongly applied. The participant aimed for speed and forgot that a provided quantity ACE = 280˚ wasn’t even used, and was pretty confident and did not examine the solution.
  • What was their answer? 16˚ (correct answer, but a wrong process)
  • What was the strategy they used?
(1) Read the problem, label the provided angles & arcs: 
AHC = 316˚, GH = 54˚, DB = 114˚
(2) Apply existing geometry knowledge of opposite angles (mistakenly think of AG as ∠ASE) to derive:
∠ASE = (AHC — GH) / 2 = (316˚ — 54˚) / 2 = 262˚/2 = 131˚
(3) Apply another existing geometry knowledge to derive:
ESF = 180˚ — 131˚ = 49˚
(4) Write the exterior angle theorem equation, substituting known quantities with numbers and unknown quantity EF with a variable x:
49˚ = (114˚ - x)/2
(5) Solve the equation correctly and derive:
EF = 114˚ — (49˚x 2) = 16˚
  • What difficulties did they have? (1) Initially confused by the plenty of variables and asked where’s D, where’s B. (2) Confused arc and angles again, e.g., equates arc AG as ∠ASE. (3) Complained about their slow speed in solving arithmetics.
  • Additional Notes: Another existing geometry knowledge “angles on one side of a straight line add to 180˚” was activated but wrongly applied. The participant used the wrong approach with the same confusion of arc and angle but managed to get the correct answer.
Participant 2: Q1 (left) & Q2 (right)
Empirical CTA diagram: participant 2
  • What was their answer? 20˚ (correct)
  • What was the strategy they used?
(1) Read the problem, label the provided angles & arcs and the asked (goal) quantity:
∠ASE=85˚, ∠ATC = 35˚, AC (want to solve)
(2) Trace the major arc ACE and apply the major-minor arc theorem to solve for AE:
AE = 360˚-280˚ = 80˚
(3) Write the equation including known, unknown and goal quantities and rewrite the goal in terms of known and unknown quantities:
ACE = AC + CE = 280˚, AC = 280˚ - CE
(4) Check the provided theorem sheet, wrongly identify ∠ATC as the interior angle and write the equation:
35˚ = (AC˚ + BD˚) / 2
(5) Recheck the theorem sheet and self-correct ∠ATC as the exterior angle, modify the equation and simplify:
35˚ = (BD˚ - AC˚) / 2, BD˚ - AC˚ = 70˚
(6) Identify the correct interior angle, write the theorem equation and simplify:
85˚ = (AE + BD) / 2, AE + BD = 170˚
(7) Recall AE solved earlier and use the 3 equations involving BD, AC, and AE to solve for AC
AE + BD = 170˚, BD - AC = 70˚, AE = 360˚ - 280˚ = 80˚
BD = 90˚, AC = 20˚
  • What difficulties did they have? (1) Struggled with the major and minor arc ACE. “Is ACE 280˚? How it can be 280˚? It seems to be much less… oh it’s on this side.” (2) Struggled with differentiating the interior and exterior angle theorems and their specific forms.
  • Additional Notes: The participant was more like logically manipulating the provided theorems rather than doing the math: “I’m not doing geometry, I just used reasoning.” The participant was also more careful with each step, used heuristics like how large an arc looks like in the graph to verify the answer, and double-checked the reference theorem sheet more extensively.
  • What was their answer? 16˚ (correct)
  • What was the strategy they used?
(1) Read the problem, underline and trace the major arc AHC, write it as a sum of two minor arcs, and apply the major-minor arc theorem to solve for AC:
AH + CH = AHC = 316˚, AC = 360˚ - 316˚ = 44˚
(2) Continue reading the problem, label the provided angles & arcs and the asked (goal) quantity:
GH = 54˚, DB = 114˚, EF (want to solve)
(3) Mistakenly think of EF as ∠ESF, and apply existing geometry knowledge on opposite angles:
EF = ∠ESF = ∠ASC
(4) Refer to the theorem sheet and apply interior angle theorem to solve for ∠ASC:
∠ASC = (AC + 54˚) / 2
(5) Wrongly apply the exterior angle theorem on DB and GH to solve for ∠ESF:
∠ESF = (114˚ - 54˚) / 2 = 30˚, so EF = 30˚
(6) Identify that some provided quantities weren’t used, re-examine if EF is solved correctly and self-corrected the application of exterior angle theorem, explicitly noted that the exterior angle ∠ESF for circle N is the interior angle for circle M:
∠ESF = (114˚ - EF) / 2
(7) Rearrange the equations in terms of ∠ESF = ∠ASC and solve for EF
AC + 54˚ = 114˚ - EF, AC = 44˚
44˚ + 54˚ = 98˚ = 114˚ - EF, EF = 16˚
  • What difficulties did they have? (1) Struggled with major arc AHC: “Is AHC = AC + AH? Along the straight line? Oh, it’s all the way around.” (2) Confused arc EF of circle N as angle ∠ESF in circle M. (3) Confused exterior angle theorem for arcs in different circles use BD (circle N) and GH (circle M) instead of BD and EF (both on circle N).
  • Additional Notes: This participant likes to do whatever they can with theorems while reading the problem, which is why they always apply the major-minor arc theorem first (previously known by the participant) to solve for unknown quantity immediately. Heuristics like “all information provided in the problem should be helpful” was used to verify the answer: “It’s offering extra information? Oh, it’s not yet the solution.”

Empirical CTA Unified Diagram

Empirical CTA Synthesis

  1. Label known quantities
  2. Identify quantities that is related to a known theorem
  3. Apply the theorem with unknown and known quantities with substitution to list all equations they can find
  4. Check if the target quantity can be solved given all listed equations
  • At step 1, novices annotate the quantities to help visualize the known information, some also highlight the target quantity (e.g., uniquely using wavy line as in participant 2).
  • At step 2, novices need to decide if a theorem is provided, newly learned, or it’s an unprovided theorem pre-existing in their prior knowledge, and if it’s the former, novices tend to double-check the reference sheet to make sure they have written the formula in the correct way.
  • At step 4, novices may use heuristics like whether the solved quantity match the diagram or whether all provided information had been used to verify their answer, and if it doesn’t seem right, they may mark the target as an unknown quantity again and go back to step 2. But novices may also forget or too rush to double-check their solution before output the answer.
  • Confused major and minor arc
  • Confused arc and interior angle
  • Confused the new theorems (interior and exterior angle theorems)
  • Confused by the plenty of variables and non-straightforward arithmetics
  • Can’t remember the new theorems and sometimes wrongly apply (e.g., forgot 1/2, identify the wrong quantities)

Reflection and Discussion

  • Immediate feedback system should be built to check for learners’ misunderstandings (e.g., if learners mistakenly equates an arc and an interior angle, highlight and correct it).
  • Animations can be used to illustrate confusing concepts (e.g., major arc can be highlighted as if someone is tracing it according to the letters).
  • The problem presentation should reduce extraneous cognitive load (e.g., labeling the circle as M & N isn’t necessary for solving the problem and just makes the diagram look more busy).
  • To encourage more goal-driven search for theorems like experts, the game should scaffold the explicit process of checking what can be used to solve for the target and unknown quantities (e.g., although the provided table wasn’t used by either participant, it can be helpful to guide novices’ thinking process, so the game can build the stepped solution into its gameplay, and the steps could be reversed to emphasize the logic of starting with the target).
  • To develop true understandings on the theorems and avoid superficial manipulation, make sure the game help learners digest the theorems (either provide them a good intuition or guide them through the proof).

--

--

I go by Christina too, and I’m a Ph.D. student at CMU HCII. Find my Chinese blogs at WeChat 一千只海鸥

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Qianou Ma

I go by Christina too, and I’m a Ph.D. student at CMU HCII. Find my Chinese blogs at WeChat 一千只海鸥