A Multi-Sensor Interface to Improve the Teaching and Learning Experience in Arc Welding Training Tasks

This paper presents the development of mixed reality multi-sensor platform to improve the teaching and learning experience of arc welding tasks. Traditional methods to acquire hand-eye coordination skills are typically conducted through one-to-one instruction where trainees/trainers must wear protective helmets and conduct several hands-on tests with metal workpieces. This approach is inefficient as the harmful light emitted from the electric arc impedes the close monitoring of the welding process (practitioners can only observe a small bright spot and most geometric information cannot be perceived). Some new teaching methods have introduced the use of virtual reality as a way to safely simulate the process and visualize the geometry of the workpieces. However, the synthetic nature of these simulators reduces the effectiveness of the platforms. As a feasible solution to these problems, this work presents a new multi-sensor teaching-learning interface composed of a HDR camera (to monitor the welding spot in real-time), a depth sensor (to capture the scene's 3D geometry), and a VR headset (to visualize the process safely). In contrast with traditional systems, our new platform provides trainees with virtual cues of the seam geometry, automatic spot tracking, and a performance score (useful for instructors to assess the task). To validate the method's feasibility, we conducted a detailed experimental study with several teaching and learning welding tasks, and compared its advantages with the current practice and virtual welding solutions.