This JSON file contains multiple-choice questions related to visual perception. Each entry includes a video clip (File_Name ), a question (Question), four original options (Option1–Option4), a shuffled version of those options (Shuffled_Option1–Shuffled_Option4), and the shuffle index map (Shuffle_Index_Map).
| Field Name | Description |
|---|---|
File_Name |
Name of the video file (e.g., 276.mp4) |
URL |
URL of the video (e.g., YouTube link) |
URL2 |
URL of the second image used in generating the latter half of the video from two images |
Start |
Start time of the clip (in seconds) |
End |
End time of the clip (in seconds) |
Pos_Left |
Left position of the video display window |
Pos_Top |
Top position of the video display window |
Width |
Width of the video display window |
Height |
Height of the video display window |
Process |
How the data is processed |
Gemini_Filter_Flag |
Indicates filtering status by Gemini (1 = filtered, 0 = not filtered) |
Category |
Indicates the category (1 = Discrepancy, 2 = Anomaly) |
Question |
The question text (multiple-choice format) |
Option1–Option4 |
The original answer choices |
Shuffled_Option1–Shuffled_Option4 |
The randomized order of answer choices |
Shuffle_Index_Map |
An array indicating how the original options were shuffled (e.g., [3, 2, 0, 1]) |
Prompt-guided-Recognition_1–Prompt-guided-Recognition_3 |
Human Annotation for (P3) No instruction. |
Prompt-guided-Robustness_1–Prompt-guided-Robustness_3 |
Human Annotation for (P2) Answer without being deceived by the illusion. |
Uninformed_1-Uninformed_3 |
Human Annotation for (P1) Answer based on the illusion. |
Anomaly_1–Anomaly_3 |
Human Annotation for Anomaly Category QA. |
Q: A question about illusion.- Meaning of options:
Option1: Illusion-basedOption2: Reality-basedOption3: Distractor 1Option4: Distractor 2
Q: A question about identifying an anomaly.- Meaning of options:
Option1: Correct answerOption2: Distractor 1Option3: Distractor 2Option4: Distractor 3
-
Shuffled_Option1–Shuffled_Option4are randomized versions of the originalOption1–Option4. -
Shuffle_Index_Mapindicates the original position of eachShuffled_Option.Example:
"Shuffle_Index_Map": [3, 2, 0, 1]
Run the following command to prepare the videos:
python process.pyIf you use this dataset in your research, please cite:
@inproceedings{VILQA,
author = {Souto Ohira and Tosho Hirasawa and Mamoru Komachi},
title = {Can Video LLMs See Through Illusions? Video-Illusion QA Benchmark Dataset},
year = {2026},
booktitle = {Proceedings of The 15th edition of the Language Resources and Evaluation Conference},
}