Institutional Repository

Tracing search paths and overcoming finite-space limits in large language models for mathematical problem solving

Show simple item record

dc.contributor.author Kabwe, Chansa
dc.date.accessioned 2026-10-20T08:25:00Z
dc.date.available 2026-10-20T08:25:00Z
dc.date.issued 2026-07
dc.identifier.uri https://ir.unisa.ac.za/handle/10500/33182
dc.description.abstract Large Language Models (LLMs) can produce fluent explanations while returning unreliable exact answers when mathematical prompts impose large representations, repeated operations, or restrictive output budgets. This dissertation examines that observable weakness using TokenPath, a trace-capture instrument, and the Token-Stress-Test (TST), a procedurally generated collection of mathematical stress tasks. Finite-Space Constraints (FSC) are treated as a theoretical motivation; an FSC-consistent stress outcome is defined operationally as an evaluable incorrect, missing, or truncated answer to a registered TST item under a recorded finite output budget. This rule identifies outcomes under stress but does not, by itself, establish an internal causal mechanism. Reconciliation of the retained records reconstructed the historical evidence from 46 selected source files. The canonical ledger contains 9 000 eligible baseline observations across six models, six non-leaking task families, five output budgets, and 50 observations per model– family–budget cell. Exact task-specific scoring produced 499 correct answers, an accuracy of 5.54% (Wilson 95% confidence interval 5.09–6.04%). Performance varied substantially by task family: sequence prediction reached 21.53%, symbolic expansion 9.07%, large-number arithmetic 2.53%, repetitive sequences 0.13%, and base conversion and combinatorics 0%. The last three families therefore have severe floor effects and require calibrated tiered validation before they can support comparative claims. Token length was available for all six models, whereas entropy and effective-vocabulary measures were comparable only for the three open-weight models whose logits were retained. Because correct outcomes were rare and several predictors were separated, the primary multivariable analysis used Firth-penalised logistic regression. It estimated associations between correctness and output budget, realised response length, retrospective within-family difficulty, model, and family; these estimates are descriptive and not causal, and realised length is postgeneration. The historical tool-ablation archive contained 140 declared groups, but only 70 matched the same item and ground truth. After excluding 10 leaking chain-reasoning pairs, 60 valid pairs remained; neither condition was correct, giving a sample recovery efficacy of zero without establishing a population-level absence of tool benefit. A separate tool-event funnel records requests, executions, successes, and verifications without treating them as paired effects. The study implements TokenPath and TST and reports comparative mathematical outcomes. The intended mechanism taxonomy and low-overhead mitigation claims are not fully supported by the retained evidence. These limits qualify the achievement of the original four research objectives. en
dc.language.iso en en
dc.subject Finite-Space Constraints en
dc.subject TokenPath en
dc.subject Token-Stress-Test en
dc.subject Mathematical reasoning en
dc.subject Rare-event logistic regression en
dc.subject Tool augmentation en
dc.subject Reproducibility en
dc.title Tracing search paths and overcoming finite-space limits in large language models for mathematical problem solving en
dc.type Dissertation en


Files in this item

This item appears in the following Collection(s)

  • Unisa ETD [13659]
    Electronic versions of theses and dissertations submitted to Unisa since 2003

Show simple item record

Search UnisaIR


Browse

My Account

Statistics