Thinking Is Not Telling: Information Disclosure in User-Service LLM Agents

Jiatong Li1, Changdae Oh1, Hyeong Kyu Choi1, Jindong Wang2, Sharon Li1
1University of Wisconsin–Madison 2William & Mary
Transactions on Machine Learning Research (TMLR), 2026
Paper arXiv Code coming soon
A paper example comparing an agent that discloses a cancellation constraint with an enforced-thinking agent that omits it.

Private reasoning can be counter-productive in disclosing the right information when confronting user decision points.

Abstract

User-engaged LLM agents increasingly operate in service scenarios where task success depends on coordination between the agent, the user, and a stateful environment. The agent often has access to task policies, tool results, and environment states that the user cannot observe, making communication central to successful task completion.

We study a communication-level failure mode: internal reasoning can be counterproductive for interactive information disclosure. We define information disclosure as the proactive communication of decision-relevant state corrections, options, constraints, consequences, and retrieved environment information. Across five models and three user-service scenarios, enforced thinking has unstable utility, and more thinking does not reliably produce richer user-facing disclosure.

Evidence-grounded probing and turn-level counterfactual intervention show that restoring missing decision-relevant information can correct downstream environment-edit actions. A lightweight disclosure-aware prompt also mitigates degradation from enforced thinking. These findings suggest that agents should be optimized not merely to think more, but to communicate the right information at the right decision point.

Information Disclosure

The framework measures five kinds of information an agent may need to disclose before a user decision.

Category
Decision-relevant information
Example
State Correction SC
Corrects a factual misunderstanding or false assumption expressed by the user.
“This order was delivered, not cancelled.”
Option OP
Presents the choices, variants, or paths available for the next decision.
Lists all available replacement options.
Constraint CN
States policies, eligibility conditions, or limits that affect valid actions.
“Modifications are only possible before shipment.”
Consequence CQ
Explains financial impact, timing, status changes, or irreversible outcomes.
Refund amount and expected timeline.
Proactive Retrieval PR
Surfaces useful user-specific information retrieved from tools or databases.
Shows current and new addresses for comparison.

Counterfactual Intervention

For a failed target trajectory and a matched successful trajectory, the method identifies the last user decision point before the first environment edit. It then finds information that was present in the successful response but absent from the failed response, injects that evidence, and rolls the dialogue forward again.

Overview of the counterfactual intervention pipeline.
The intervention restores missing disclosure before the agent's first environment-edit action.

Results

Thinking does not reliably scale communication

Increasing internal think effort does not consistently improve task success or increase user-visible response length across models and service scenarios.

Plots comparing task reliability and response length under different thinking efforts.

Failures miss state corrections and constraints

Across both models and service scenarios, State Correction (SC) and Constraint (CN) are the most frequently injected categories. This shows that failed trajectories most often omit information that corrects the user's understanding or explains limits on valid actions, while Option (OP) is consistently the least frequent missing category.

Frequency of intervention across the five information disclosure dimensions.

Counterfactual intervention results

Table 4 reports the intervention results for each model and domain. Injecting missing golden information improves the first environment edit more consistently than simply increasing verbosity or exposing think content from the failed trajectory. Golden think content can be stronger, but it must first be obtained from a separately successful run.

11.74%Golden Info FECR
16.35%Golden Think FECR
25.55%Golden Think recovery rate
Table 4 showing first edit correction and recovery rates for four counterfactual interventions across models and domains.
Table 4. Golden information and golden think content produce the strongest corrections across most model–domain settings.
Tables 5 and 6 showing micro-averaged intervention effects and trials needed to obtain a successful reasoning trajectory.
Tables 5–6. Targeted disclosure improves micro-averaged FECR; obtaining golden think content requires more than one trial in 16.67–37.50% of settings.
Figure 4 comparing first edit correction rates across intervention types, models, and domains.
Figure 4. The advantage of targeted information is visible across both model and domain aggregations, with the largest gains from successful golden reasoning.

Disclosure-aware prompting

A lightweight prompt that asks agents to disclose decision-relevant information improves micro-averaged pass@1 by 5.26 points for GPT-4o and 3.44 points for DeepSeek-V3.1 when the think tool is enforced.

Table comparing baseline, proactive chain-of-thought, and information-disclosure prompting.

Conclusion

Raw internal thinking is not equivalent to effective user-facing communication. Information disclosure provides an interpretable way to diagnose a meaningful subset of agent failures and a practical target for improving user-service agents: convert environment information into timely, public, decision-relevant disclosure.

BibTeX

@article{li2026thinkingmakesllmagents,
    title={Thinking Is Not Telling: Information Disclosure in User-Service LLM Agents}, 
    author={Jiatong Li and Changdae Oh and Hyeong Kyu Choi and Jindong Wang and Sharon Li},
    year={2026},
    journal={arXiv preprint arXiv:2602.07796},
}