multi-image inference

[Mitsuha-M]

@HaozheZhao
Hi, thanks for your excellent work. I'm attempting to train the MIC dataset, but I've encountered an issue. When I input two different images and ask questions like 'What is the connection/difference between these images,' the model only seems to consider the latter image. I would like to know how to organize the prompt during training or inference. Referring to the paper:

Q: describe this image<image 0>, A cute dog. what is the content of the image?<image 1>
A: A cat lying on the floor.

Will organizing data in this similar way enable the training to achieve results similar to those in research papers, and what conditions need to be met?