Mar 4, 2021
[Cognitive Task Analysis] Geometry Task
Task Description:
The final task for participants is a variation of the provided geometry task package, with the easy problems 1–3 (which are a literal application of the theorems provided) and the hard problems 6–7 eliminated. Therefore, for this task, participants will only do problems 4 and 5 (relabel them as Q1 & Q2 from now on), which as a set will take about 10–20 minutes to solve for a novice in the domain. According to preliminary testing results on my roommate, who’s a Math major so a supposed expert in this area, a task with problems 4–7 took her more than 30 minutes, which should be too long for novices.
In this task, participants will be provided with the reference theorems sheet first, and they will need to understand the theorems before starting the 2 questions. The table along with the problem description will also be provided to guide their thinking process. The researcher (me) will give instructions about the think-aloud protocol and take notes during the process.
Theoretical CTA Synthesis
The Theoretical CTA Diagram is presented at the beginning of this article, and it is constructed by performing CTA on my roommate.
The overall problem-solving strategy for the geometry task would mainly be a loop of
- Identify known quantities
- Identify quantities that need to be solved
- Identify the theorem that can be used to solve the unknown quantities
- Apply the theorem with known quantities
During the process, learners need to make several decisions.
- At step 1, they may use the diagram to annotate the quantities to help visualize the known information, so they need to decide whether it’s an arc or an angle and deal with them differently.
- In step 3, if the identified theorem couldn’t be applied since learners don’t have all the necessary quantities available, they can go back to step 2 and try to solve those quantities first.
- At step 4, learners may use some heuristics like whether the solved quantity matches the diagram to verify their answer, and if it seems very wrong (e.g., on the graph it looks like an obtuse angle but the solution is 60˚), they may mark it as an unknown quantity and go back to step 3 to try to re-solve it.
- Finally, learners may examine the final answer before submission (e.g., check if the arithmetics are correct), and return to any of the previous steps depending on the mistake identified.
Some difficulties novices may have with the task:
- Mistakenly traces the minor arc instead of the major arc
- Identify the wrong theorem to apply,
- Identify the wrong unknown or known quantities,
- Can’t identify a theorem because the application direction is backward,
- Forget about the 1/2 part of theorem 1 & 2,
- Struggle with the cognitive load of arithmetics (the numbers aren’t nice like 10, 30, 60) and all these different variables (much more than 7 letters on a busy graph),
- Depend too much on intuition, use it as a way to solve problems rather than a sanity check.
Empirical CTA Participant Requirement:
The participants recruited are my parents, since I don’t have access to English-speaking geometry novices (my college friends are all better at math than me). Both participants had a college background, but they work in the business field that’s not related to geometry problem-solving, so they haven’t done geometry for decades. The translated version of the theorems was provided (as follows, note that they already know theorem 3, so only theorem 1 & 2 are new to them), and both participants were asked to think aloud in their native language.
圆内角定理：圆内角的度数=此角及其对顶角所对的弧度数之和的一半
圆外角定理：圆外角的度数=所夹的两段弧度数的差（大减小）的一半
对应优弧和劣弧的度数和为360˚
Empirical CTA Participant Strategy & Individual Diagrams
Q1
- What was their answer? 15˚ (wrong)
- What was the strategy they used?
(1) Read the problem, label the provided angles & arcs:
∠ASE=85˚, ACE, and ∠ATC(2) Write the exterior angle theorem equation with known and unknown quantities:
ATC = 35˚ = (BD — AC)/2(3) Apply existing geometry knowledge that “opposite angles are equal” and wrongly derive:
BD = ∠BSD = ∠ASE = 85˚(4) Solve the equation with one known quantity and relabel unknown quantity as x:
35˚ = (85˚ — x) / 2, so x = AC = 85˚ — 70˚ = 15˚
- What difficulties did they have? (1) Struggled with major and minor arc ACE, so they explicitly trace the major arc to emphasize that. (2) Confused arc and angles, e.g., equates arc BD as ∠BSD. (3) Almost forgot the /2 part of the theorem, so they originally derive AC as 50˚.
- Additional Notes: Geometry knowledge outside of the provided theorems was correctly activated but wrongly applied. The participant aimed for speed and forgot that a provided quantity ACE = 280˚ wasn’t even used, and was pretty confident and did not examine the solution.
Q2
- What was their answer? 16˚ (correct answer, but a wrong process)
- What was the strategy they used?
(1) Read the problem, label the provided angles & arcs:
AHC = 316˚, GH = 54˚, DB = 114˚(2) Apply existing geometry knowledge of opposite angles (mistakenly think of AG as ∠ASE) to derive:
∠ASE = (AHC — GH) / 2 = (316˚ — 54˚) / 2 = 262˚/2 = 131˚(3) Apply another existing geometry knowledge to derive:
ESF = 180˚ — 131˚ = 49˚(4) Write the exterior angle theorem equation, substituting known quantities with numbers and unknown quantity EF with a variable x:
49˚ = (114˚ - x)/2(5) Solve the equation correctly and derive:
EF = 114˚ — (49˚x 2) = 16˚
- What difficulties did they have? (1) Initially confused by the plenty of variables and asked where’s D, where’s B. (2) Confused arc and angles again, e.g., equates arc AG as ∠ASE. (3) Complained about their slow speed in solving arithmetics.
- Additional Notes: Another existing geometry knowledge “angles on one side of a straight line add to 180˚” was activated but wrongly applied. The participant used the wrong approach with the same confusion of arc and angle but managed to get the correct answer.
Q1
- What was their answer? 20˚ (correct)
- What was the strategy they used?
(1) Read the problem, label the provided angles & arcs and the asked (goal) quantity:
∠ASE=85˚, ∠ATC = 35˚, AC (want to solve)(2) Trace the major arc ACE and apply the major-minor arc theorem to solve for AE:
AE = 360˚-280˚ = 80˚(3) Write the equation including known, unknown and goal quantities and rewrite the goal in terms of known and unknown quantities:
ACE = AC + CE = 280˚, AC = 280˚ - CE(4) Check the provided theorem sheet, wrongly identify ∠ATC as the interior angle and write the equation:
35˚ = (AC˚ + BD˚) / 2(5) Recheck the theorem sheet and self-correct ∠ATC as the exterior angle, modify the equation and simplify:
35˚ = (BD˚ - AC˚) / 2, BD˚ - AC˚ = 70˚(6) Identify the correct interior angle, write the theorem equation and simplify:
85˚ = (AE + BD) / 2, AE + BD = 170˚(7) Recall AE solved earlier and use the 3 equations involving BD, AC, and AE to solve for AC
AE + BD = 170˚, BD - AC = 70˚, AE = 360˚ - 280˚ = 80˚
BD = 90˚, AC = 20˚
- What difficulties did they have? (1) Struggled with the major and minor arc ACE. “Is ACE 280˚? How it can be 280˚? It seems to be much less… oh it’s on this side.” (2) Struggled with differentiating the interior and exterior angle theorems and their specific forms.
- Additional Notes: The participant was more like logically manipulating the provided theorems rather than doing the math: “I’m not doing geometry, I just used reasoning.” The participant was also more careful with each step, used heuristics like how large an arc looks like in the graph to verify the answer, and double-checked the reference theorem sheet more extensively.
Q2
- What was their answer? 16˚ (correct)
- What was the strategy they used?
(1) Read the problem, underline and trace the major arc AHC, write it as a sum of two minor arcs, and apply the major-minor arc theorem to solve for AC:
AH + CH = AHC = 316˚, AC = 360˚ - 316˚ = 44˚(2) Continue reading the problem, label the provided angles & arcs and the asked (goal) quantity:
GH = 54˚, DB = 114˚, EF (want to solve)(3) Mistakenly think of EF as ∠ESF, and apply existing geometry knowledge on opposite angles:
EF = ∠ESF = ∠ASC(4) Refer to the theorem sheet and apply interior angle theorem to solve for ∠ASC:
∠ASC = (AC + 54˚) / 2(5) Wrongly apply the exterior angle theorem on DB and GH to solve for ∠ESF:
∠ESF = (114˚ - 54˚) / 2 = 30˚, so EF = 30˚(6) Identify that some provided quantities weren’t used, re-examine if EF is solved correctly and self-corrected the application of exterior angle theorem, explicitly noted that the exterior angle ∠ESF for circle N is the interior angle for circle M:
∠ESF = (114˚ - EF) / 2(7) Rearrange the equations in terms of ∠ESF = ∠ASC and solve for EF
AC + 54˚ = 114˚ - EF, AC = 44˚
44˚ + 54˚ = 98˚ = 114˚ - EF, EF = 16˚
- What difficulties did they have? (1) Struggled with major arc AHC: “Is AHC = AC + AH? Along the straight line? Oh, it’s all the way around.” (2) Confused arc EF of circle N as angle ∠ESF in circle M. (3) Confused exterior angle theorem for arcs in different circles use BD (circle N) and GH (circle M) instead of BD and EF (both on circle N).
- Additional Notes: This participant likes to do whatever they can with theorems while reading the problem, which is why they always apply the major-minor arc theorem first (previously known by the participant) to solve for unknown quantity immediately. Heuristics like “all information provided in the problem should be helpful” was used to verify the answer: “It’s offering extra information? Oh, it’s not yet the solution.”
Empirical CTA Unified Diagram
Empirical CTA Synthesis
The overall problem-solving strategy novices use for the geometry task mainly include a loop of
- Label known quantities
- Identify quantities that is related to a known theorem
- Apply the theorem with unknown and known quantities with substitution to list all equations they can find
- Check if the target quantity can be solved given all listed equations
During the process, learners need to make several decisions.
- At step 1, novices annotate the quantities to help visualize the known information, some also highlight the target quantity (e.g., uniquely using wavy line as in participant 2).
- At step 2, novices need to decide if a theorem is provided, newly learned, or it’s an unprovided theorem pre-existing in their prior knowledge, and if it’s the former, novices tend to double-check the reference sheet to make sure they have written the formula in the correct way.
- At step 4, novices may use heuristics like whether the solved quantity match the diagram or whether all provided information had been used to verify their answer, and if it doesn’t seem right, they may mark the target as an unknown quantity again and go back to step 2. But novices may also forget or too rush to double-check their solution before output the answer.
Some common difficulties novices have encountered:
- Confused major and minor arc
- Confused arc and interior angle
- Confused the new theorems (interior and exterior angle theorems)
- Confused by the plenty of variables and non-straightforward arithmetics
- Can’t remember the new theorems and sometimes wrongly apply (e.g., forgot 1/2, identify the wrong quantities)
Reflection and Discussion
The majority of predicted difficulties occurred to novices in this study, but some of them didn’t and that’s probably because the participants were adults who have finished formal education and learned transferable skills. For example, I predicted that novices might “depend too much on intuition” and “can’t identify theorems’ backward application direction,” but participants of this study didn’t exhibit these mistakes. A possible explanation could be that participants as experienced adults have learned that intuition and heuristics cannot replace mathematical reasoning, and have had enough practices in logical inferences to apply given theorems both forward and backward.
The biggest difference between experts and novices in their problem-solving strategies is that experts solve for unknown quantities if they identify it to be relevant in solving for the target, and they apply theorems only as needed; in comparison, novices tend to unselectively apply whatever they know about the theories with known or unknown quantities first to see if anything can be solved. So experts are more goal-oriented, while novices are more like “listing all I have and see if anything works out.”
Another interesting difference is that novices tend to be satisfied with shallowly manipulating variables instead of digesting geometry, while experts tried to integrate new theorems into their existing knowledge. For example, novices spent more time looking at and understanding the theorems, but are not curious about how are they derived. This may lead to the fact that novices commonly mix up the interior and exterior angle theorems or apply them in a wrong way, since they don’t truly understand why are the theorems true.
In terms of designing educational games in this domain, a lot of mechanisms can be incorporated, which are consistent with both CTA results and learning science principles. For example,
- Immediate feedback system should be built to check for learners’ misunderstandings (e.g., if learners mistakenly equates an arc and an interior angle, highlight and correct it).
- Animations can be used to illustrate confusing concepts (e.g., major arc can be highlighted as if someone is tracing it according to the letters).
- The problem presentation should reduce extraneous cognitive load (e.g., labeling the circle as M & N isn’t necessary for solving the problem and just makes the diagram look more busy).
- To encourage more goal-driven search for theorems like experts, the game should scaffold the explicit process of checking what can be used to solve for the target and unknown quantities (e.g., although the provided table wasn’t used by either participant, it can be helpful to guide novices’ thinking process, so the game can build the stepped solution into its gameplay, and the steps could be reversed to emphasize the logic of starting with the target).
- To develop true understandings on the theorems and avoid superficial manipulation, make sure the game help learners digest the theorems (either provide them a good intuition or guide them through the proof).
