MMToM-QA Leaderboard

MMToM-QA systematically evaluates the cognitive ability to understand people's minds both on multimodal data and different unimodal data. Questions span seven categories, evaluating belief inference and goal inference in rich and diverse situations.

For details, see our project page and paper. Submission instructions live in the GitHub repo.

#	Method	Belief	Goal	All
—	Human	97.5	88.5	93.0
1	AutoToM + Model Spec. (w/ GPT-4o)SOTA Zhang et al., '25	94.0	65.7	79.8
2	BIP-ALM (w/ LLaMA 2) Jin et al., '24	80.3	73.3	76.7
3	AutoToM (w/ GPT-4o) Zhang et al., '25	88.7	62.3	75.5
4	o3-mini	88.7	40.7	64.7
5	Gemini 2.0 Flash Thinking	73.3	34.7	54.0
6	SimToM (w/ GPT-4o)Wilf et al., '23	75.7	26.3	51.0
7	Gemini 2.0 Pro	57.0	44.7	50.8
8	Gemini 2.0 Flash	62.7	33.3	48.0
9	InstructBLIP	48.7	44.7	46.7
10	GPT-4o	55.7	32.3	44.0
11	Llama 3.1 70B	51.3	36.3	43.8
12	LLaVA	43.0	44.0	43.5
13	Video-LLaMA 2	42.0	38.3	40.2
14	GPT-4V	55.3	34.7	40.0

#	Method	Belief	Goal	All
—	Human	91.0	74.0	82.5
1	o1*SOTA	95.1	59.2	76.5
2	o3-mini*	97.1	44.9	71.5
3	BIP-ALM (w/ LLaMA 2)Jin et al., '24	82.3	58.7	70.5
4	Thought-tracing + CoT (w/ GPT-4o)*Kim et al., '25	93.1	42.9	69.0
5	SymbolicToM (w/ GPT-4)Sclar et al., '23	78.3	47.7	63.0
6	Thought-tracing (w/ GPT-4o)*Kim et al., '25	78.4	37.8	60.0
7	GPT-4o*	74.5	39.8	56.5
8	SimToM (w/ GPT-4o)Wilf et al., '23	64.3	40.7	52.5
9	GPT-J	49.0	52.3	50.7
10	LLaMA 2	56.3	44.3	50.3
11	Gemini 1.5 Pro*	70.6	28.6	50.0
12	DeepSeek R1*	73.5	24.5	49.0
13	GPT-4	62.0	34.0	48.0
14	Llama 3.3 70B*	50.0	42.9	47.0
15	Qwen 2.5 72B*	51.0	32.7	41.5
16	GPT-3.5	43.7	33.0	38.3

#	Method	Belief	Goal	All
—	Human	73.3	64.6	68.9
1	BIP-ALM (w/ LLaMA 2)SOTA Jin et al., '24	64.0	58.3	61.2
2	InstructBLIP	49.3	52.3	50.8
3	Video-LLaMA 2	41.0	51.0	46.0
4	GPT-4V	45.7	46.3	46.0
5	LLaVA	39.0	45.3	42.2

* indicates that the method was tested on a 200-sample subset of MMToM-QA, as reported by Kim et al., '25.