The 21st International Conference “Man and Working Environment”
SAFETY ENGINEERING & MANAGEMENT SCIENCE, INDUSTRY, EDUCATION (SEMSIE 2025)   
PROCEEDINGS OF PAPERS 
25-26 September 2025, SOKOBANJA, SERBIA  

Marija Savković , Marko Đapan , Carlo Caiazzo ,
Aleksandar Bodić , Arso Vukićević , Ivan Mačužić 

ORIGINAL SCIENTIFIC PAPER

MULTIMODAL ANALYSIS OF WORKERS' COGNITIVE AND PHYSICAL STRESS

DOI: 10.46793/SEMSIE25.073S
DOWNLOAD FULL PDF
Abstract:

In the modern contemporary environment, simultaneous monitoring of physical and mental strain is very important in order to improve ergonomic conditions and the health of workers. This research paper presents a multimodal approach to assessing the physical and mental strain of workers during assembly activities. The proposed model combines the RULA (Rapid Upper Limb Assessment) method for assessing ergonomic risk and the EEG (electroencephalography) method for evaluating mental strain. Through an experimental study conducted on a sample of 7 subjects, the implications of assuming certain body postures on the physical and cognitive state of the subjects were analyzed. The results indicated a significant association between the adoption of non-physiological and non-ergonomic body postures and increased mental effort, which indicates the need for an integrated assessment when designing ergonomic workplaces. The results obtained showed that the application of ergonomic principles contributes to the reduction of physical and mental strain. The results also confirm that high levels of physical strain lead to changes in the brain activity of the subjects that correspond to increased mental effort. The proposed approach can serve as a basis for the development of advanced systems for monitoring and optimizing worker performance and well-being.

Keywords:

Mental strain, multimodal approach, ergonomic, physical strain, RULA

ACKNOWLEDGEMENTS:

This paper is supported by the European Commission, HORIZON 2020 Marie Curie Training Network Collaborative Intelligence for Safety Critical systems project (Grant agreement ID: 955901). This paper is supported by the Ministry of Science, Technological Development and Innovation within the framework of scientific cooperation with Italy, project: AI-powered visio-tactile soft robotic hand for Agrifood farming - AI-AGRI-HAND, 2025-2027.

REFERENCES:
  • Aromaa, S., Liinasuo, M., Kaasinen, E., Bojko, M., Schmalfuß, F., Apostolakis, K. C., Zarpalas, D., Daras, P., Öztürk, C., & Boubekeur, M. (2018). User evaluation of Industry 4.0 concepts for worker engagement. In W. Karwowski & T. Ahram (Eds.), Advances in intelligent systems and computing: Vol. 876. Proceedings of the 1st International Conference on Human Systems Engineering and Design (IHSED 2018) (pp. 39–45). Springer. https://doi.org/10.1007/978-3-030-02053-8_6 
  • Ayaz, H., & Dehais, F. (2019). Neuroergonomics: The brain at work and in everyday life. Elsevier. https://doi.org/10.1016/C2016-0-02196-4 
  • Barr, A. E., Barbe, M. F., & Clark, B. D. (2004). Work-related musculoskeletal disorders of the hand and wrist: Epidemiology, pathophysiology, and sensorimotor changes. Journal of Orthopaedic & Sports Physical Therapy, 34(10), 610–627. https://doi.org/10.2519/jospt.2004.34.10.610  
  • Gevins, A., Leong, H., Du, R., Smith, M. E., Le, J., Du Rousseau, D., & Libove, J. (1995). Towards measurement of brain function in operational environments. Biological Psychology, 40(1), 169–186. https://doi.org/10.1016/0301-0511(95)05105-8  
  • Hogervorst, M. A., Brouwer, A. M., & Van Erp, J. B. (2014). Combining and comparing EEG, peripheral physiology and eye-related measures for the assessment of mental workload. Frontiers in Neuroscience, 8, 322. https://doi.org/10.3389/fnins.2014.00322  
  • Infantolino, Z. P., & Miller, G. A. (2014). Psychophysiological methods in neuroscience. In R. Biswas-Diener & E. Diener (Eds.), Noba textbook series: Psychology. Champaign, IL: DEF Publishers.  
  • McAtamney, L., & Corlett, E. N. (1993). RULA: a survey method for the investigation of work-related upper limb disorders. Applied ergonomics, 24(2), 91-99. 
  • Neumann, W. P., Kihlberg, S., Medbo, P., Mathiassen, S. E., & Winkel, J. (2002). A case study evaluating the ergonomic and productivity impacts of partial automation strategies in the electronics industry. International Journal of Production Research, 40(16), 4059–4075.  
  • Petreanu, V., & Seracin, A. M. (2017). Risk factors for musculoskeletal disorders development: Hand-arm tasks, repetitive work. National Research-Development Institute for Health and Safety, Bucarest, Romania  
  • Santos, J., Silva, F. J. G., Pinto, G., & Baptista, A. (2019). Lean and ergonomics: How to increase the productivity improving the wellbeing of the workers. In Lean manufacturing: Implementation, opportunities and challenges (pp. 189–224). Nova Science Publishers, ISBN: 978-1-53615-725-3  
  • Savković, M., Caiazzo, C., Djapan, M., Vukićević, A., Pušica, M., & Mačužić, I. (2022). Development of modular and adaptive laboratory set-up for neuroergonomic and human-robot interaction research. Frontiers in Neurorobotics, 16, Article 863637.https://doi.org/10.3389/fnbot.2022.863637  
  • Savković, M., Petrović, M., Caiazzo, C., Djapan, M., & Vukićević, A. (2023). Determine muscle strain of assembly workers by applying advanced EMG measurement. In Proceedings of the Quality Festival Conference 2023, Kragujevac, Serbia.  
  • Suzaki, K. (1987). The new manufacturing challenge: Techniques for continuous improvement. The Free Press, New York, NY 
  • Trimmel, M., Fairclough, S., & Henning, R. (2009). Psychophysiology in ergonomics. Applied Ergonomics, 40(6), 963–964. https://doi.org/10.1016/j.apergo.2009.02.003  
  • Young, M. S., Brookhuis, K. A., Wickens, C. D., & Hancock, P. A. (2015). State of science: Mental workload in ergonomics. Ergonomics, 58(1), 1–17. https://doi.org/10.1080/00140139.2014.956151

© 2023 Faculty of Occupational Safety - Multimedia