Evaluating AI applications: a verification framework for writing, programming and workflows

1 From generative capability to delivery evidence

Generative systems produce fluent text, executable code and tool actions, but none automatically establishes successful delivery. We argue for shifting evaluation from what a model produces to which claims and states can be externally checked. Evidence support, test coverage and final-state verification are distinct levels of judgement. This is a conceptual synthesis and research agenda, not an experimentally validated new algorithm.

2 Literature scope and lines of evidence

This paper purposively selects 150 research documents for a question-led narrative synthesis, not a systematic review or meta-analysis. The selected works were first submitted in 2016–2025 and do not cover all developments through 2026. Titles, authors, submission dates, public abstracts, source pages and PDF access were checked on 8 September 2026. Original research, technical reports and surveys are distinguished rather than treated as equally strong experimental evidence. Selected methodological passages in FActScore, ALCE, EvalPlus and τ-bench were previously examined; we did not conduct full-text appraisal or experimental replication of every work.

The Transformer uses attention for sequence transduction, providing an architectural basis for subsequent language-generation research.[1] GPT-3 demonstrates few-shot task performance through textual examples, while reporting variation across tasks and methodological issues associated with training data.[2]

T5 frames language tasks as text-to-text problems and compares objectives, architectures and datasets; a common task format does not imply a common evaluation criterion.[3] BART studies generation and comprehension through denoising pre-training, illustrating the importance of the training objective.[4]

Retrieval-augmented generation combines parametric models with external retrieval memory for knowledge-intensive tasks.[5] REALM incorporates a knowledge retriever into language-model pre-training, exploring access to documents beyond facts stored in model parameters.[6]

InstructGPT uses demonstrations and preference feedback to improve instruction following, but its authors also report that the resulting models still make simple mistakes.[7] TruthfulQA evaluates questions that elicit common misconceptions, distinguishing imitation of language from factual truthfulness.[8]

FActScore decomposes long-form output into atomic facts and estimates the proportion supported by a reliable knowledge source.[9] ALCE evaluates fluency, correctness and citation quality separately: the presence of citations does not establish that every claim is supported.[10]

Chain-of-Verification separates drafting, verification questions, independent answers and revision, reducing hallucination in the tasks studied.[11] Lost in the Middle finds position-dependent use of relevant information in the models tested; a larger context window does not guarantee effective use of all supplied material.[12]

The Codex study evaluates functional correctness using tasks including HumanEval and examines repeated sampling and model limitations.[13] The study introducing MBPP evaluates synthesis of short Python programs from natural-language descriptions.[14]

AlphaCode combines large-scale candidate sampling with behaviour-based filtering for competitive programming, a different setting from maintaining an existing software system.[15] Pearce and colleagues find vulnerable code generated by an early version of Copilot in selected security-relevant scenarios; these historical observations do not estimate current product vulnerability rates.[16]

CodeGen studies multi-turn programme synthesis by decomposing requirements into successive subproblems.[17] StarCoder describes an openly released code-model effort, including licensing, personal-information handling and source-attribution arrangements.[18]

Code Llama offers variants for generation, completion and instruction following, highlighting differences in training and use conditions.[19] EvalPlus expands test inputs and identifies errors missed by earlier tests, demonstrating that test adequacy changes measured performance.[20]

SWE-bench evaluates issue resolution in real repositories, requiring coordinated understanding and changes across functions, classes or files.[21] SWE-agent examines how an agent–computer interface for navigation, editing and testing affects software-repair performance.[22]

Agentless uses a simplified localization, repair and patch-validation pipeline, motivating comparisons against simpler workflows.[23] LiveCodeBench collects new competition problems over time and includes self-repair, execution and test-output prediction to address contamination and evaluation coverage.[24]

Chain-of-Thought prompting improves performance on selected reasoning tasks through intermediate examples; generated reasoning is not itself external evidence.[25] Self-Consistency samples multiple reasoning paths and aggregates answers, improving performance in the reasoning tasks studied.[26]

ReAct interleaves reasoning and actions, allowing external knowledge or environmental feedback to inform subsequent decisions.[27] Toolformer studies how a model can learn when to invoke APIs, what arguments to use and how to incorporate their results.[28]

Reflexion stores textual reflections on feedback for subsequent attempts without updating model weights.[29] Self-Refine uses the same model to provide feedback and iteratively revise outputs across several tasks.[30]

Tree of Thoughts organizes intermediate reasoning as a process of exploration, evaluation and backtracking for planning or search tasks.[31] AutoGen provides configurable multi-agent conversations connecting models, tools and human input.[32]

AgentBench evaluates reasoning and decision-making in multiple interactive environments and analyses failures in long-horizon reasoning and instruction following.[33] WebArena provides reproducible web environments and long-horizon tasks evaluated by functional task completion.[34]

ToolLLM develops data, training and evaluation for API use, including tool retrieval and search over invocation paths.[35] τ-bench combines simulated users, domain policies and API tools, checking resulting database states and reliability across repeated trials.[36]

3 Extended foundations and counterevidence

The extension covers training and data conditions, retrieval and factual evaluation, software and tool environments, and privacy, bias and uncertainty. Each entry summarizes a research question or method supported by the public abstract, without importing unverified performance figures or treating a larger bibliography as validation of our framework. The cited arXiv version is not necessarily the final journal version; source existence does not guarantee correct conclusions.

3.1 Model and representation foundations

BERT learns bidirectional representations from left and right context and studies transfer to language-understanding tasks.[37] RoBERTa revisits BERT training to examine data and hyperparameter effects, motivating control of training conditions in model comparisons.[38]

XLNet learns bidirectional context through an autoregressive objective over factorization permutations, offering a different route from masked reconstruction.[39] The GPT-4 report describes image and text inputs, text outputs and benchmark evaluation, while acknowledging weaker-than-human performance in many real-world settings.[40]

LLaMA studies foundation models trained on publicly available data, providing a case for examining data accessibility alongside model capability.[41] Llama 2 distinguishes pretrained models from dialogue-tuned variants, making variant identification necessary when citing results.[42]

3.2 Scale, data and training conditions

Scaling Laws studies empirical relationships between cross-entropy loss, parameters, data and training compute, rather than dependable production delivery.[43] The Chinchilla study jointly allocates model size and training tokens under a compute budget, challenging explanations based on parameter count alone.[44]

PaLM examines scaling and few-shot performance alongside bias, toxicity and memorization.[45] OPT releases pretrained models at multiple scales together with training records to support research and inspection.[46]

BLOOM describes an open-access model trained on natural and programming languages, making training-language composition relevant to evaluation.[47] Pythia provides controlled data ordering and multiple checkpoints for studying capability, memorization and bias over training.[48]

3.3 Task adaptation and preference learning

LoRA freezes pretrained weights and trains low-rank update matrices to reduce trainable parameters during adaptation.[49] QLoRA combines a quantized frozen model with low-rank adapters, principally addressing fine-tuning memory requirements.[50]

Prefix-Tuning learns continuous task prefixes on a frozen model for lightweight adaptation to generation tasks.[51] Prompt Tuning learns soft prompts through backpropagation and studies how their effectiveness changes with model scale.[52]

DPO reformulates preference learning through a classification loss; preference alignment should still be evaluated separately from factual support and task completion.[53] Constitutional AI trains assistants using principles, self-critique and AI preference feedback, targeting behavioral constraints rather than universal factual guarantees.[54]

3.4 Data documentation, deduplication and provenance

Datasheets for Datasets proposes documenting motivation, composition, collection and intended uses, informing documentation of source materials.[55] Model Cards proposes reporting intended uses, evaluation procedures and performance across conditions rather than a single score.[56]

Research on training-data deduplication examines repeated text, memorized output and train–test overlap.[57] Kandpal and colleagues study duplicated training sequences and privacy attacks, finding that deduplication reduces attack effectiveness in their evaluated settings.[58]

Data Portraits uses compact records to support training-data membership queries and investigation of test leakage and provenance.[59] Dolma releases an English pretraining corpus and construction documentation, supporting investigation of data-curation effects.[60]

3.5 Benchmark coverage and evaluation methods

MMLU evaluates knowledge and problem solving across academic and professional tasks; its mean score is not accuracy across all applications.[61] BIG-bench assembles diverse tasks and compares performance across model scales, supporting evaluation beyond a single task.[62]

HELM evaluates accuracy, calibration, robustness, fairness, bias, toxicity and efficiency across multiple scenarios.[63] CheckList organizes behavioral tests by capabilities and test types, supplementing weaknesses obscured by average held-out accuracy.[64]

Dynabench collects challenge examples through human-and-model-in-the-loop data creation for dynamic evaluation.[65] Research on neural-network calibration tests correspondence between confidence and correctness and examines temperature scaling; its classification experiments do not guarantee calibrated text generation.[66]

3.6 Reasoning strategies and evidential limits

Zero-shot-CoT studies a short stepwise-reasoning prompt without task exemplars on selected reasoning benchmarks.[67] Least-to-Most decomposes complex problems into sequential subproblems that reuse earlier answers.[68]

PAL generates intermediate programs and delegates execution and computation to an interpreter.[69] Program of Thoughts likewise separates reasoning representation from external computation and evaluates numerical reasoning tasks.[70]

Research on unfaithful explanations finds that input biases can influence predictions without being disclosed in the generated chain of thought.[71] A study of intrinsic self-correction finds that revision without external feedback can fail or degrade performance in the reasoning settings tested.[72]

3.7 Retrievers and external knowledge

DPR uses a dual encoder to learn dense question and passage representations for evidence retrieval in open-domain question answering.[73] ColBERT uses late interaction between query and document representations to study retrieval effectiveness and computational efficiency.[74]

Research combining passage retrieval and generation examines aggregation of information from multiple passages for open-domain answers.[75] Atlas studies few-shot retrieval augmentation and examines the contents and updating of the document index.[76]

RETRO conditions token prediction on retrieved document chunks, studying explicit external memory alongside model parameters.[77] BEIR compares retrieval systems across heterogeneous tasks, cautioning against assuming cross-domain generalization from one dataset.[78]

3.8 Control and evaluation of retrieval augmentation

Self-RAG learns on-demand retrieval and reflection tokens to coordinate retrieval, generation and self-evaluation.[79] CRAG evaluates retrieved-document quality to trigger different retrieval actions, addressing failures of retrieval.[80]

IRCoT interleaves retrieval with multi-step reasoning so that later retrieval uses information already derived.[81] FLARE retrieves using an anticipated next sentence and regenerates it when low-confidence tokens indicate a need for evidence.[82]

RAPTOR recursively clusters and summarizes text chunks to build a retrieval tree at multiple abstraction levels.[83] HyDE generates a hypothetical document as a retrieval intermediary before searching a real corpus; that hypothetical document is not factual evidence to cite.[84]

3.9 Fact verification and summary faithfulness

FEVER labels claims as supported, refuted or lacking sufficient information and records evidence sentences where applicable.[85] The FactCC study jointly assesses summary consistency and locates supporting and conflicting spans.[86]

Maynez and colleagues examine hallucinations through human evaluation of summaries, distinguishing source faithfulness from general text-similarity metrics.[87] SummEval compares automatic metrics, summarization models, and expert and crowd annotations under a common evaluation setup.[88]

QAFactEval examines question generation and answerability in QA-based factual evaluation and their complementarity with entailment-based assessment.[89] TRUE standardizes data from multiple tasks for example-level factual-consistency evaluation, beyond correlations between system averages.[90]

3.10 Hallucination detection and semantic support

SelfCheckGPT uses agreement across sampled answers to identify suspect content; a shared repeated error does not thereby become an externally established fact.[91] HaluEval uses generated and human-annotated samples to test hallucination recognition; its observed rates cannot be extrapolated to all answers.[92]

Ji and colleagues survey definitions, evaluation and mitigation of hallucination in language generation; here it provides conceptual background, not an additional independent experiment.[93] Research on semantic uncertainty measures entropy over meanings rather than string differences alone, accounting for equivalent answer formulations.[94]

AlignScore learns information alignment between two text passages for factual-consistency assessment across settings.[95] Ragas separates retrieved context, faithfulness and output quality to support RAG evaluation without human reference answers.[96]

3.11 Instruction boundaries and safety evaluation

Research on indirect prompt injection shows that malicious instructions in retrieved content can interfere with applications; source content must not automatically acquire operational authority.[97] Research on transferable adversarial attacks studies input suffixes that bypass alignment constraints in tested models, motivating explicit threat models for safety evaluation.[98]

Instruction Hierarchy studies training models to distinguish instruction priorities when lower-trust content conflicts with higher-priority requirements.[99] HarmBench provides a standardized framework for comparing automated red teaming and evaluating attacks and defenses.[100]

JailbreakBench specifies threat models, prompt configurations, scoring and attack artifacts to improve comparability of jailbreak research.[101] SORRY-Bench evaluates safety refusal across fine-grained unsafe topics and linguistic variants rather than a narrow selection of topics.[102]

3.12 Code representation and program generation

CodeBERT learns representations of natural and programming languages and evaluates tasks including code search and documentation generation.[103] GraphCodeBERT incorporates data flow between variables during pretraining rather than treating code solely as a token sequence.[104]

CodeT5 uses an encoder–decoder architecture and identifier-aware objectives to study both code understanding and generation.[105] CodeT selects candidate programs using generated tests and execution agreement; the correctness of generated tests still requires separate scrutiny.[106]

InCoder studies code infilling with bidirectional context alongside left-to-right synthesis, distinguishing editing from continuation.[107] The study introducing PolyCoder compares code models across programming languages, highlighting code corpora and language distribution in evaluation.[108]

3.13 Cross-file context and execution benchmarks

RepoCoder uses iterative retrieval and generation to incorporate cross-file repository information into code completion.[109] RepoBench separates retrieval, code completion and combined-pipeline tasks, distinguishing context acquisition from context use.[110]

CrossCodeEval uses static analysis to construct multilingual completion tasks requiring cross-file context.[111] DS-1000 builds programming tasks around data-science libraries and evaluates execution results alongside constraints such as API usage.[112]

APPS evaluates program generation through natural-language coding problems of varying difficulty and associated test cases.[113] CRUXEval evaluates program input and output prediction separately, showing that generation scores and execution reasoning are distinct capabilities.[114]

3.14 Execution feedback, training and model versions

CodeRL predicts functional correctness with a critic and studies code generation using test feedback and reinforcement learning.[115] Self-Debugging studies code revision using program explanations and execution results; its feedback conditions differ from intrinsic correction without external information.[116]

LEVER learns a verifier from the natural-language request, candidate program and execution result to rerank candidates.[117] DeepSeek-Coder studies project-level code data and infilling objectives, identifying repository context and completion objectives as distinct training factors.[118]

StarCoder2 and The Stack v2 document code provenance and training-data construction alongside evaluation across languages and tasks.[119] WizardCoder applies Evol-Instruct to instruction tuning for code and evaluates code-generation benchmarks.[120]

3.15 Tool selection and interactive memory

Gorilla combines API-document retrieval with call generation and examines changes to documentation at test time.[121] API-Bank provides runnable tools and dialogue tasks to distinguish errors in API planning, retrieval and invocation.[122]

HuggingGPT uses a language model to coordinate task planning, model selection, execution and result summarization.[123] MRKL proposes a system architecture connecting language models, external knowledge and discrete reasoning modules.[124]

Voyager combines a curriculum, executable skill library and environmental feedback in Minecraft; game exploration does not directly establish reliability in real workflows.[125] Generative Agents uses memory, reflection and planning to simulate believable human behavior; behavioral believability differs from business-task correctness.[126]

3.16 Observable environments and task outcomes

Mind2Web collects tasks and action sequences from real websites to study generalization across sites and domains.[127] WorkArena defines knowledge-work tasks in enterprise software and provides an environment for interactive agent evaluation.[128]

OSWorld evaluates cross-application tasks with real computer environments, initial-state configurations and execution-based checking scripts.[129] VisualWebArena studies web tasks requiring both images and text, extending evaluation beyond text-only observations.[130]

AndroidWorld evaluates mobile agents with parameterized tasks and explicit initialization, success checks and teardown logic.[131] AgentDojo tests prompt injection in untrusted tool data, evaluating ordinary tasks alongside attacked conditions.[132]

3.17 Privacy, bias and harmful outputs

Training-data extraction research demonstrates recovery of training text from GPT-2 outputs, showing that publicly sourced inputs can still involve privacy risks.[133] Research on differentially private deep learning uses training mechanisms and privacy-cost analysis to bound leakage risk under the corresponding assumptions.[134]

StereoSet evaluates gender, profession, race and religion stereotypes in English; these categories are not treated here as an exhaustive Chinese-language evaluation.[135] CrowS-Pairs uses sentence pairs to examine masked language models' preferences for social stereotypes in a US context.[136]

RealToxicityPrompts investigates whether apparently innocuous prompts elicit toxic output and compares generation-control methods.[137] BBQ contrasts under-informative and informative question-answering contexts to examine how social bias affects answer selection.[138]

3.18 Uncertainty and selective answering

Research on model self-knowledge distinguishes estimating answer correctness from estimating whether a question is answerable and reports cross-task calibration difficulties.[139] Just Ask for Calibration compares calibration of verbalized confidence and conditional probabilities on selected question-answering benchmarks.[140]

Linguistic Calibration studies confidence in long-form text by asking whether users can form calibrated probabilistic judgments from the output.[141] Conformal Language Modeling calibrates sampling-stop and rejection rules for answer sets with statistical guarantees, not correctness of every single response.[142]

Selective-classification research trades rejection of some instances for risk control, informing joint reporting of coverage and error.[143] Research on risk-controlling prediction sets uses held-out data to calibrate set size and expected loss; its guarantees cannot be detached from the specified sampling and calibration conditions.[144]

3.19 Subsequent models and reasoning training

DeepSeek-R1 studies reinforcement learning for reasoning and performance on verifiable tasks; it does not establish that open-ended factual statements are necessarily correct.[145] Qwen3 reports thinking and non-thinking modes with inference-budget control, reinforcing the need to record both mode and compute budget.[146]

DeepSeek-V3 describes mixture-of-experts design, load balancing and multi-token prediction; active and total parameter counts should be recorded separately.[147] Llama 3 reports multilingual, coding, reasoning and tool-use evaluation while distinguishing pretraining, post-training and multimodal experiments.[148]

OLMo 2 provides weights, training data, logs and intermediate checkpoints, distinguishing open weights from availability of the broader training artifacts.[149] Qwen2.5 describes pretraining and post-training changes and multiple released variants, making model identity necessary when citing its results.[150]

4 An evidence framework across tasks

Rather than combine heterogeneous benchmark scores into a ranking, we distinguish three acceptance objects: claims in writing, patches in programming and state transitions in workflows. All can be recorded as task–constraint–artefact–check–final state, but this shared structure does not justify an unvalidated aggregate score. Good citations cannot replace code tests, and successful receipts cannot replace final-state checks.

5 Falsifiable research propositions

H1: Does a claim–source ledger improve supported-claim precision without worsening required-fact coverage or abstention relative to baseline? Gains achieved only by omitting difficult claims would not support the hypothesis.

Use human writing, direct generation and retrieval-augmented generation as baselines. Add an evidence ledger and independent review in the full workflow; ablate each separately while fixing materials, model version, retrieval corpus and total time budget.

H2: At fixed patch scope and cost, does independent testing improve the proportion of valid final repairs, rather than merely reveal more errors? A higher detection rate without more usable final patches must be reported separately, not as better repair effectiveness.

Compare human repair, one-shot generation and a simple localization–repair–validation pipeline. Add independent tests and regression checks in the full workflow. Freeze benchmark revisions, isolate held-out tests from patch generation, and record dependencies and environment hashes.

H3: Can state verification and idempotent retries improve repeated reliability without broader permissions or increased human takeover? Benefits depending on extra intervention or authority cannot be attributed to workflow structure alone.

Compare a single workflow, multi-agent coordination and a state-verified workflow at matched models, tool permissions, budgets and tasks. Ablate state checks and retry protection separately; inject timeouts, duplicate responses and partial failures only in recoverable environments.

6 Operational definitions and decision boundaries

Let Sᵢ be supported checkable claims in task i and Aᵢ all checkable claims. Let Kᵢ be facts required in advance and Dᵢ required facts correctly retained in the output. Report supported-claim precision Pᵢ and coverage Cᵢ separately so that omission or silence cannot inflate the apparent result. A zero denominator is not applicable; abstentions and omissions are reported separately.

Count a code task as a valid repair only if the predefined issue is resolved, independent acceptance checks pass and required regression checks do not fail. Report security review and target-environment coverage separately. This operational definition is bounded by test adequacy, not proof of correctness for every input. Errors detected, repairs completed and review time are distinct outcomes.

Let yᵢⱼ∈{0,1} indicate that task i in run j satisfies both final-state and policy requirements, with N tasks and a fixed k runs per task. R_all,k is the fraction succeeding in every run; R_any,k is the fraction succeeding at least once. Report both with k, budget and takeover rate. Neither should be replaced by taking the overall mean success rate to the kth power.

7 A testable evaluation protocol

Treat tasks, not repeated outputs from one task, as sampling units. Use a separate pilot to assess annotation feasibility, cost and plausible effects, then determine the main sample size from precision or power targets. Exclude pilot tasks from held-out evaluation. Stratify Chinese and English tasks and treat translated counterparts as paired rather than independent samples. Publish inclusion and exclusion criteria and freeze inputs, versions, permissions and budgets.

Randomize condition order and compare conditions within tasks; blind adjudicators to condition where feasible. Report raw counts, task-level effect differences and uncertainty intervals without treating repeat runs as extra independent tasks. Report task families separately and test sensitivity to missing outcomes and timeouts. Retain and distinguish timeouts, abstentions and human takeovers. Document when and why a planned procedure changes.

A reproducibility package should identify tasks, data permissions, model and tool versions, prompt templates, configurable random seeds, tests, adjudication rules, failure logs and analysis scripts. Reproducibility does not justify releasing personal or restricted information; redacted structures and controlled-access terms may be appropriate. These are proposed research outputs: this site does not currently provide a completed experimental dataset or analysis code.

8 Counterarguments, limitations and scope

More verification is not necessarily better: a reliable ledger can increase review costs, defective tests can reject correct patches, and final-state checks can miss implicit requirements. Lightweight or human-only workflows may be more economical for low-risk, reversible tasks. If additional checks increase cost without improving predefined outcomes, the proposed workflow should be reduced or rejected rather than defended through post-hoc changes to success criteria.

FActScore primarily evaluates biographies against a corresponding knowledge source; it does not directly establish the truth of complex causal arguments. ALCE’s automated entailment judgements can also fail. The proportions proposed below are operational definitions for this protocol, not full reproductions of either benchmark.[9][10]

EvalPlus shows that expanded tests can reveal errors missed by earlier evaluation and also discusses defective reference implementations. More tests do not prove correctness for every input; test validity and final-patch correctness require separate scrutiny.[20]

τ-bench checks final database states together with required responses. We adopt its distinction between at-least-once and all-trial success, but the equations below are direct empirical statistics over a fixed number of trials, not the paper’s combinatorial estimator. Descriptive computation does not require independent and identically distributed trials.[36]

Purposive selection, predominantly abstract-level checking and the absence of original data limit this paper. The present reference set cannot establish a research gap through 2026 or support priority claims. Before submission, searches need updating, key methods require full-text appraisal, version and publication information need checking, and domain researchers must assess the argument. Claims of effectiveness additionally require the proposed studies.

9 Conclusion

Dependability in AI applications should not be defined by generation performance alone. This paper proposes separate acceptance criteria for evidence support, independent tests and task completion, alongside falsifiable hypotheses and bounded evaluation measures. Its contribution is to specify what must be tested, not to declare the proposed approach effective.

10 Companion papers

• Evidence-constrained AI writing: from fluent text to verifiable claims
• AI-assisted programming: from generated code to validated repository changes
• AI workflows: tool coordination and repeatable task completion

11 Declarations and availability

Data and code: No original experimental data were generated. Sources are individually linked. Experimental and analysis code has not been implemented; website code is not an experimental replication package.

Authorship, contributions, funding and competing interests: Alsay is the website publisher, which does not establish scholarly authorship. Author identities, contributions, funding and competing interests require confirmation by accountable contributors. This draft does not assert absence of competing interests or invent affiliations, degrees or ethical approvals.

AI assistance and versions: AI assisted retrieval, organization, editing and Chinese–English translation. The two versions represent one working paper, not two independent studies; scholarly approval by accountable authors remains pending. No peer review, journal acceptance or completed preregistration is claimed.