Data infrastructure provider Handshake AI has released sobering performance metrics regarding the reliability of frontier artificial intelligence models across the financial sector. According to Chief Executive Garrett Lord and finance lead Collin, a former bulge-bracket investment banker, leading models achieve a success rate of under 20 percent on the specialized ATLAS-Finance benchmark. These findings highlight a persistent divide between theoretical language comprehension and error-free execution in high-stakes financial operations. Despite massive capital injections from top research labs into specialized datasets and reinforcement learning environments, automated banking tasks continue to expose fundamental model limitations.
The evaluation framework was designed around standard assignments regularly handled by junior analysts at investment banks and private equity firms. Specifically, Handshake tested multi-step workflows such as parsing transaction details from incoming banking emails and integrating that data directly into financial spreadsheets. The tests exposed a consistent failure mode: while frontier models could accurately recite complex tax statutes and accounting principles when prompted directly, they failed to implement them properly in generated Python scripts and Excel equations. For example, systems repeatedly committed elementary balance sheet errors, including the improper subtraction of existing goodwill positions.
These technical bottlenecks have not dampened commercial demand for training data, but rather intensified it across the artificial intelligence sector. Handshake AI reported that its gross annualized revenue surged to nearly 1 billion dollars within roughly one year, accounting for around 950 million dollars of its 1.1 billion dollar total revenue run rate. Leading frontier AI laboratories are aggressively acquiring specialized domain data and reinforcement learning environments tailored to private equity and banking. The startup's meteoric revenue trajectory demonstrates that frontier labs are willing to commit unprecedented sums to move beyond generic web crawls toward structured domain knowledge.
The broader market for expert-level training data has undergone rapid expansion as foundation model builders compete for specialized capabilities. According to figures shared by Handshake AI, the commercial market for specialized training and evaluation data across finance, biology, and model alignment grew from roughly 1 billion dollars three years ago to approximately 10 billion dollars today. The company projects that this market could expand to 100 billion dollars by 2029 as enterprise automation demands stricter reliability standards. For institutional financial firms, this shift indicates that commercial advantage will increasingly depend on proprietary validation benchmarks rather than raw compute power alone.
The persistent underperformance on real-world workflows raises acute risk management considerations for financial institutions considering agentic deployments. Because frontier models exhibit expert-level performance in narrow conversational tasks while failing unexpectedly at procedural arithmetic, manual verification by human analysts remains essential. Industry observers argue that until autonomous systems can reliably exceed the 20 percent success threshold on complex workflows, full end-to-end automation in regulated banking environments remains unrealistic. Financial executives must therefore approach autonomous agent rollouts with rigorous oversight protocols and maintain strict operational firewalls.