Research3 min read

Google and Artificial Analysis launch the MKA agent benchmark

The Multimodal Knowledge Agent benchmark tests how AI agents handle long professional tasks that involve visually complex files. Its 200 tasks span 83 occupations, and Google says Gemini 4 Carbon ranks second on quality.

AE
Aidose Editorial
Nov 11, 2026

Google DeepMind on November 10, 2026 introduced the Multimodal Knowledge Agent (MKA) benchmark, built with the independent evaluation firm Artificial Analysis. It measures how well AI agents reason over visually complex files, such as charts, drawings, maps and dense documents, while completing realistic knowledge-work tasks. The announcement was written by Rohan Doshi and Ben Caine of Google DeepMind.

The benchmark covers 10 professional areas and 83 occupations: business, construction, education, engineering, finance, healthcare, law and government, logistics and mapping, science, and software and data. Each task would take a professional three to eight hours. Each is graded against a rubric of about 70 yes-or-no checks, weighted 75% toward factual accuracy and 25% toward visual design.

The evaluation set has 200 tasks, which Artificial Analysis keeps private to prevent contamination. Ten public example tasks are on Hugging Face. Grading uses an agentic autorater built on a non-Google model, which Google says agreed with domain experts 96% of the time. Models used between 30 and 130 turns per task on average, run in the Stirrup agent harness.

Google says Gemini 4 Carbon ranked second in quality among the models tested and completed tasks at about six times lower average cost than the top-scoring model. The post shows the full scores only in a chart and does not name the top model in its text. A technical report has been published alongside the benchmark.

Developers can explore the benchmark and evaluate their own models on Artificial Analysis starting now. MKA adds a long, multimodal, rubric-graded test to the set of agent benchmarks, which so far have focused mostly on coding and browsing.

AE
Aidose Editorial
Nov 11, 2026 · 3 min read
Back to News