Enhancing AI systems with efficient communication skills that align with human understanding is crucial for their effective assistance to human users. Proactive initiatives from the system side are needed to discern specific circumstances and interact aptly with users to solve these scenarios. In this research, we opt for a collective building assignment taken from the Minecraft dataset. Our proposed method employs language modeling to enhance task understanding through state-of-the-art (SOTA) methods using language models. These models focus on grounding multi-modal understandinging and task-oriented dialogue comprehension tasks. This focus aids in gaining insights into how well these models interpret and respond to a variety of inputs and tasks. Our experimental results provide compelling evidence of the superiority of our proposed method. This showcases a substantial improvement and points towards a promising direction for future research in this domain.
View on arXiv