Human–Machine Interaction Using Probabilistic Neural Network for Light Communication Systems

Julian Webber, Abolfazl Mehbodniya, Rui Teng, Ahmed Arafa

Hand gestures are a natural and efficient means to control systems and are one of the promising but challenging areas of human–machine interaction (HMI). We propose a system to recognize gestures by processing interrupted patterns of light in a visible light communications (VLC) system. Our solution is aimed at the emerging light communication systems and can facilitate the human–computer interaction for services in health-care, robot systems, commerce and the home. The system exploits existing light communications infrastructure using low-cost and readily available components. Different finger sequences are detected using a probabilistic neural network (PNN) trained on light transitions between fingers. A novel pre-processing of the sampled light on a photodiode is described to facilitate the use of the PNN with limited complexity. The contributions of this work include the development of a sensing technique for light communication systems, a novel PNN pre-processing methodology to convert the light sequences into manageable size matrices along with hardware implementation showing the proof of concept under natural lighting conditions. Despite the modest complexity our system could correctly recognize gestures with an accuracy of 73%, demonstrating the potential of this technology. We show that the accuracy depends on the PNN pre-processing matrix size and the Gaussian spread function. The emerging IEEE 802.11bb ‘Li-Fi’ standard is expected to bring the light communications infrastructure into virtually every room across the world and a methodology to exploit a system for gesture sensing is expected to be of considerable interest and value to society.