📊 Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.

Recent empirical data indicates that AI systems are now capable of automating the majority of core engineering tasks involved in AI research, reaching near saturation on multiple benchmarks. Meanwhile, research activities that require creativity and hypothesis generation remain partly reliant on human input, leaving a residual gap. This development underscores a potential transformation in AI R&D workflows over the next 32 months.

According to Thorsten Meyer’s analysis of Jack Clark’s recent work, six key benchmarks measuring AI’s capabilities in core science and engineering tasks show rapid progress. Notably, the CORE-Bench, which tests research reproduction, has reached 95.5% success, with one author declaring it ‘solved.’ This indicates that AI can now reliably reproduce research papers, handling dependencies, code execution, and output analysis at a level comparable to competent post-docs.

Similarly, the MLE-Bench, which assesses performance on Kaggle competitions, has seen AI reach 64.4%, roughly matching mid-tier human Kaggle practitioners. The leaderboard for this benchmark has been paused to develop a more fair measurement process, signaling that AI capabilities have outstripped the original evaluation methods. Additionally, progress in kernel design—such as automated GPU kernel generation—further exemplifies the transition of AI from experimental to production-ready engineering tools.

Clark’s analysis suggests that these trajectories are converging, with multiple independent benchmarks nearing saturation within a similar timeframe. The implication is that engineering tasks in AI development are increasingly automated, reducing the marginal cost and time traditionally spent on engineering work. However, the research process—such as hypothesis formulation, creative problem-solving, and novel theory development—remains less automated, with some aspects still dependent on human input. The ongoing question is how much of research can be automated before it becomes indistinguishable from engineering.

Implications for AI R&D and Industry Innovation

The rapid automation of core engineering tasks in AI research suggests a fundamental shift in how AI development is conducted. As engineering bottlenecks diminish, the pace of AI innovation could accelerate significantly, enabling faster deployment of new models and applications. This may also reshape the role of human researchers, shifting their focus toward creative and strategic aspects rather than routine engineering. However, the residual human-driven research components could still act as a bottleneck or source of innovation, depending on how automation evolves.

For industry, this trend could lead to cost reductions, faster iteration cycles, and more accessible AI development pipelines. Conversely, it raises questions about the future of research roles and the need for new skills adapted to an increasingly automated R&D environment. The broader impact will depend on whether AI can eventually automate the remaining creative and hypothesis-driven aspects of research.

Progress in AI Engineering Capabilities and Benchmarks

Over the past year, multiple benchmarks have tracked AI progress in core research and engineering tasks. The CORE-Bench, measuring research reproduction, improved from 21.5% in September 2024 to 95.5% in December 2025, with the author of the benchmark declaring it ‘solved.’ The MLE-Bench, assessing Kaggle competition performance, advanced from 16.9% to 64.4% over the same period, prompting the organizers to pause submissions to refine measurement methods.

These improvements are supported by advances in kernel design, including automated GPU code generation and optimized deep learning kernels, demonstrating that AI systems are transitioning from experimental tools to production-grade engineering solutions. The convergence of these trajectories indicates that AI’s capability to handle engineering tasks is approaching saturation, while research remains less fully automated.

“Clark’s conclusion is correct and possibly understated for engineering. The residual research question is real but may be less binding than the framing suggests.”

— Thorsten Meyer

Unresolved Aspects of AI Research Automation

It is still unclear to what extent AI can automate the creative and hypothesis-driven aspects of research, such as novel theory generation and experimental design. While engineering tasks are nearing full automation, the residual research component may still require human insight, and the timeline for full automation remains uncertain. Additionally, the impact of these developments on research roles and industry workflows is still being evaluated.

Next Milestones in AI R&D Automation

Over the next 32 months, focus will likely be on refining measurement benchmarks to better capture AI’s capabilities, especially in research creativity and hypothesis generation. Expect continued advances in automated kernel design, model training, and deployment tools. Industry adoption of these automated engineering solutions may accelerate, while research automation remains an open challenge. Monitoring how these capabilities influence research productivity and innovation cycles will be critical.

Key Questions

How close is AI to fully automating research activities?

Currently, AI has automated core engineering tasks such as reproducing research, optimizing kernels, and performing well in competitions. However, the creative, hypothesis-driven aspects of research are still partly human-dependent. Full automation of research remains an open question.

What are the implications for human researchers?

As engineering tasks become automated, researchers may shift focus toward strategic, creative, and theoretical work. This could lead to a change in skill requirements and research workflows, emphasizing innovation over routine engineering.

Will this automation reduce research costs?

Yes, automating engineering tasks can lower costs and speed up development cycles. However, maintaining oversight and guiding AI-driven research will still require human expertise, especially in creative domains.

Could AI eventually automate all aspects of research?

This remains uncertain. While engineering automation is advancing rapidly, the automation of creative research processes is less certain and may require breakthroughs in AI understanding and reasoning.

Source: ThorstenMeyerAI.com