ProVer is a framework that improves credit assignment in agentic reinforcement learning by using a model judge to identify pivotal decisions in trajectories, then verifying these segments through empirical outcome comparison rather than exhaustively evaluating every step. Tested on ALFWorld, WebShop, and SearchQA, ProVer achieves 9.91% and 7.12% improvements over GRPO for Qwen models at different scales.