INVESTIGATION OF TACTILE GAIT PARAMETERS BASED ON DEEP LEARNING OF ENERGY CONSUMPTION ESTIMATION ALGORITHM IN SPORT

INVESTIGAÇÃO SOBRE PARÂMETROS TÁCTEIS DE MARCHA BASEADOS NO APRENDIZADO PROFUNDO DO ALGORITMO DE ESTIMATIVA DE CONSUMO ENERGÉTICO NO ESPORTE

INVESTIGACIÓN SOBRE LOS PARÁMETROS TÁCTILES DE MARCHA BASADOS EN EL APRENDIZAJE PROFUNDO DEL ALGORITMO DE ESTIMACIÓN DEL CONSUMO DE ENERGÍA EN EL DEPORTE

ABSTRACT

Introduction

In medicine, Deep Learning is a type of machine learning that aims to train computers to perform human tasks by simulating the human brain. Gait recognition and gait motion simulation is one of the most interesting research areas in the field of biometrics and can benefit from this technological feature.

Objective

To use Deep Learning to format and validate according to the dynamic characteristics of gait.

Methods

Gait was used for identity recognition, and gait recognition based on kinematics and dynamic gait parameters was performed through pattern recognition, including the position and the intensity value of maximum pressure points, pressure center point, and pressure ratio.

Results

The investigation shows that the energy consumption of gait as modeled analyzed, and the model of gait energy consumption can be obtained, which is comprehensively affected by motion parameters and individual feature parameters.

Conclusion

Real-time energy measurement is obtained when most people walk. The research shows that the gait frequency and body parameters obtained from the tactile parameters of gait biomechanics can more accurately estimate the energy metabolism of exercise and obtain the metabolic formula of exercise. There is a good application prospect for assessing energy metabolism through the tactile parameters of gait. Level of evidence II; Therapeutic studies - investigating treatment outcomes.

Deep Learning; Gait Analysis; Biomechanical Phenomena; Energy Consumption

RESUMO

Introdução

Na medicina, o aprendizado profundo é um tipo de aprendizado de máquina que visa treinar computadores para a realização de tarefas humanas simulando o cérebro humano. O reconhecimento da marcha e a simulação do movimento de marcha são um dos pontos de maior interesse da investigação no campo da biometria e pode ser beneficiado com esse recurso tecnológico.

Objetivo

Utilizar o aprendizado profundo para formatar e validar, de acordo com as características dinâmicas da marcha.

Métodos

A marcha foi utilizada para o reconhecimento da identidade, e o reconhecimento da marcha baseado na cinemática e parâmetros dinâmicos de marcha foi realizado através do reconhecimento de padrões, incluindo a posição e o valor de intensidade dos pontos de pressão máxima, ponto central de pressão e relação de pressão.

Resultados

A investigação mostra que o consumo de energia da marcha como modelado analisado, e o modelo de consumo de energia da marcha pode ser obtido, o qual é afetado de forma abrangente pelos parâmetros de movimento e pelos parâmetros de características individuais.

Conclusão

A medição de energia em tempo real é obtida quando a maioria das pessoas caminha. A investigação mostra que a frequência da marcha e os parâmetros corporais obtidos a partir dos parâmetros tácteis da biomecânica da marcha podem estimar com maior precisão o metabolismo energético do exercício e obter a fórmula metabólica do exercício. Há uma boa perspectiva de aplicação para avaliar o metabolismo energético através dos parâmetros tácteis da marcha. Nível de evidência II; Estudos terapêuticos - investigação dos resultados do tratamento.

Aprendizado Profundo; Análise da Marcha; Fenômenos Biomecânicos; Consumo de Energia

RESUMEN

Introducción

En medicina, el aprendizaje profundo es un tipo de aprendizaje que pretende entrenar a los ordenadores para que realicen tareas humanas simulando el cerebro humano. El reconocimiento de la marcha y la simulación de su movimiento es uno de los puntos más interesantes de la investigación en el campo de la biometría y puede beneficiarse de este recurso tecnológico.

Objetivo

Utilizar el aprendizaje profundo para formatear y validar según las características dinámicas de la marcha.

Métodos

Se utilizó la marcha para el reconocimiento de la identidad, y el reconocimiento de la marcha basado en la cinemática y los parámetros dinámicos de la marcha se realizó mediante el reconocimiento de patrones, incluyendo la posición y el valor de la intensidad de los puntos de presión máxima, el punto de presión central y la relación de presión.

Resultados

La investigación muestra que el consumo de energía de la marcha, tal y como se analizó, y el modelo de consumo de energía de la marcha se puede obtener, que es ampliamente afectado por los parámetros de movimiento y los parámetros de las características individuales.

Conclusión

La medición de la energía en tiempo real se obtiene cuando la mayoría de la gente camina. La investigación muestra que la frecuencia de la marcha y los parámetros corporales obtenidos a partir de los parámetros táctiles de la biomecánica de la marcha pueden estimar con mayor precisión el metabolismo energético del ejercicio y obtener la fórmula metabólica del mismo. Existe una buena perspectiva de aplicación para evaluar el metabolismo energético a través de los parámetros táctiles de la marcha. Nivel de evidencia II; Estudios terapéuticos - investigación de los resultados del tratamiento.

Aprendizaje Profundo; Análisis de la Marcha; Fenómenos Biomecánicos; Consumo de Energía

INTRODUCTION

Gait is the way and style of human walking. Normal people’s gait has some commonalities and follows the same rules. ¹ However, different people have their own physiological characteristics and walking habits. We can recognize people we are familiar with from their walking posture, which reflects the personality of gait. It is precisely because gait has such personality. ² Tactile recognition based on gait also has the advantages of concealment, non-invasiveness, no need for close pedestrian cooperation, remote monitoring and so on. At the same time, it can simulate the floor by burying the pressure test board on the ground. ³ Therefore, based on gait haptic recognition, there is no complex background segmentation and occlusion problem, which is not affected by external light, environment and noise. It is a more effective and convenient biometric recognition technology. Although it has achieved a high recognition accuracy rate, it is susceptible to noise, clothing and other interference. Therefore, many scholars have begun to study gait tactile recognition technology. ⁴ It collects the tactile information of the sole of the foot for identification by burying the pressure test board on the ground. The energy consumption associated with physical activity is limited by the amount of direct receptor activity, and the adjustability is large. ⁵ Maintain energy balance between total body intake and total consumption. ⁶

Current research on gait tactile recognition mainly focuses on related manual feature modeling and traditional machine learning (non-in-depth learning) recognition algorithms. ⁷ Bilevel convolution neural network has simple structure and strong adaptability, which is suitable for feature learning. Therefore, it can be used for gait tactile recognition based on plantar pressure image. ⁸ In-depth learning model needs more research and innovation. In the simulation of gait movement, most of the algorithms aim at analyzing the three-dimensional structure of human motion and combining bionics mechanics to achieve the simulation effect of human motion. Most of these studies are in the field of automated bionics and biology. ⁹ The research in this paper focuses on the gait motion simulation algorithm in the field of computer vision. The condition-limited Boltzmann machine based on Gaussian process can well learn and predict the post-sequence of gait sequence. ¹⁰ This requires the developer to understand the problem to be solved in order to extract appropriate features for the sample for the shallow model to process. It is this complex regulation process that leads to the variability of gait movements, even if each person only has subtle differences in body structure, which may lead to larger gait tactile differences. This provides the possibility of gait recognition using the gait information of the foot gait.

In gait tactile recognition, the acquisition of gait tactile data is the basis, because it mainly uses information such as the force between sole and contact surface to recognize. Therefore, it is necessary to design a gait tactile data acquisition device which can quickly and accurately collect these force information. With the development of computer technology and people’s in-depth gait research. Spatial and temporal parameters of gait, such as step frequency and step length, are identified by using the periodicity of human walking and the linear relationship between step frequency and step length. However, the human step frequency and step length are closely related to physiological characteristics such as height and leg length. Gait biomechanical parameters acquisition circuit and sports shoes are integrated. The measured parameters are sent to the wristwatch which stores the motion energy consumption estimation model through radio frequency link. The information of motion energy consumption can be displayed, stored and managed in real time on the wristwatch. Gait tactile feature information is analyzed from the perspectives of completeness, repeatability and uniqueness. The time information, spatial information and dynamic information contained in it also have great prospects in characterizing the energy consumption characteristics of sports. The existing research results show that when the registered sample is wearing a jacket and the verification set is a normal walking gait sequence, the recognition accuracy will be significantly reduced. When identifying and simulating gait, the target data is dimension-reduced by a trained DBN model. At the same time, the low-dimensional data is reconstructed from the original high-dimensional space, and the class is identified by the minimum reconstruction error of the target data. The post-timing low-dimensional features of the target data are then predicted by the identified GCRBM model. Finally, the DBN model corresponding to the class is used to reconstruct the original spatial image of the post-timing low-dimensional features of the gait.

Related work

Rosenbaum et al. proposed in 1997 that the common method of plantar region division is to divide the plantar region into regions. They are medial heel, middle heel, lateral heel, arch, medial forefoot, middle forefoot, lateral forefoot and big toe area. When analyzing the plantar pressure of diabetic patients, the foot is divided into regions to analyze the foot pressure data of different people walking at different times.

Decision-oriented acyclic graph is a new learning architecture proposed by Platt et al. The main idea is to combine two types of training samples into one two classifiers. For example, in 2014, R. Martín-Félez et al. proposed gait recognition by learning a two-way sorting model, using different gait categories to transfer certain covariant invariant features.

MATERIALS AND METHODS

By learning projection, gait templates from different perspectives are projected into a common subspace independent of perspective for recognition.The typical correlation analysis is used to learn the low-dimensional geometry of the projection matrix mining data for a specific perspective, and the perspective-independent discriminant projection matrix is learned. The visual gait is identified as shown in Table 1 . In order to solve the problem that the principal component analysis is inconsistent with the projection direction of the typical correlation analysis, a complete canonical correlation analysis method is proposed. Draw the curve of the three-dimensional acceleration of the plantar motion continuously with time and the force curve of the forefoot, and calculate and calculate. Give data such as the time of the exercise and the frequency, distance and other data during the exercise period, and provide the raw data files and database for analysis.

Table 1
Visual Gait Contact Recognition.

	Calculation	Distinguish
Gait image extraction	0.62	1.39
Gait Motion Analysis	2.80	2.51

The visual gait recognition prediction model in deep learning is shown in Figure 1 . The number of pixels per row with maximum continuity of 1 is counted and compared, and the maximum number of pixels is selected as the width of the minimum gait template. Then scan the pixels longitudinally, count the number of pixels with the maximum continuity of 2 in each column and compare them. The maximum number of pixels is selected as the minimum template height to establish the minimum rectangular template of gait contour. The gait characteristic parameters are shown in Table 2 . The extraction and processing of gait contour is shown in Figure 2 . Each person takes a total of 70 gaits from the other three gait cycles independent of the above three gaits as a test set to calculate the recognition rate of the system. The characteristic dynamics are shown in Table 3 and Figure 3 . It shows that walking speed can be predicted by step frequency or step size, but its wider range of applicability needs further verification.

Figure 1
Visual Gait Recognition Prediction Model in Deep Learning.

Table 2
Gait characteristic parameters.

	Analysis	Distinguish
Spatio-temporal parameters	32.3	17.6
Kinematics-based parameters	52.6	19.8
Parameters Based on Dynamics	41.9	18.4

Figure 2
Extraction and Processing of Gait Periodic Contour.

Table 3
Mechanics Characteristics of Sports.

	Force measurement	Parameter
Maximum pressure point	32.6	31%
Pressure center	19.8	33%
Pressure ratio	21.4	36%

Figure 3
Mechanics Characteristics of Sports.

The regression equation between the effective parameters and energy consumption was established, and the estimation formula of energy metabolism during running was established. Fifteen groups of pre-extracted data were substituted into the formula for testing. The setting of each parameter in the network and the image preprocessing process are specially processed based on empirical knowledge. For example, the setting of the special learning rate, the selection of parameters in the hyperbolic tangent function, and the like. If the model is applied to the recognition of other objects. The gait recognition step parameters are shown in Table 4 and Figure 4 .

Table 4
Step parameters of gait recognition.

	Train	Sample
Gait data acquisition and storage	19%	41%
Data preprocessing	16%	42%
Feature extraction	14%	17%

Figure 4
Step parameters of gait recognition.

RESULT ANALYSIS AND DISCUSSION

The net energy consumption in sports refers to the energy consumption actually used in sports. The swing of thighs and legs during walking is regarded as pendulum model, and the individual characteristics of this swing are studied in frequency domain. Ground branch reaction (GRF) in gait parameters is used as feature for recognition. In order to better illustrate the validity of the weights constructed in this paper, a comparative experiment is carried out with the methods that use Euclidean distance and Gauss distance as weights. The correct rate of gait recognition is shown in Table 5 and Figure 5 . Therefore, it is necessary to synthesize the recognition results of the left and right feet. This article stipulates that only when the recognition results of the left and right feet are the same, it is the final recognition result, otherwise it is classified into the wrong recognition. Therefore, in order for this speed estimation formula to have wider applicability, certain corrections must be made to these two people with large errors. The 3D convolution and feature integration technology is introduced to extract the time information in the gait sequence and obtain higher recognition accuracy on the single model.

Table 5
Gait Recognition Correctness Rate.

	Sample	Combination
Left foot recognition	15.6	20.1
Right foot recognition	14.5	19.8
Comprehensive recognition	14.8	18.5

Figure 5
Gait Recognition Correctness Rate.

In the case of inconvenient measurement of step speed, step speed can be estimated by step frequency or step length:

$w(t)=w_2+\left(w_1-w_2\right)\frac{T-t}{T}$ (1)

Modified velocity estimation by using modified formula:

$c_1(t)=c_{11}+\left(c_{1T}-c_{11}\right)\frac{t}{T}$

$c_2(t)=c_{21}+\left(c_{2T}-c_{21}\right)\frac{t}{T}$ (2)

According to in-depth learning, the weight and threshold modification of the updated network can be obtained:

$p=\left\{\begin{array}{l}k{\sigma }^2<{\sigma }_d^2,f\left(P_g^t\right)>f_d\\ 0\end{array}\right.$ (3)

The network signal propagates forward, calculating the input of each node layer by layer from the input layer nodes:

$P_g^t={\left(P_{g_1}^t,P_{g2}^t,P_{g3}^t,\dots ,P_{gd}^t\right)}^T$

$P_{gi}^t=P_{gi}^t(1+0.5\eta ),i=1,2,\dots d$ (4)

After calculating these two partial derivatives, we can calculate the partial derivatives of the cost function of the whole sample according to them:

$e_j=-k\sum _{i=1}^n{f}_{ij}\ln f_{ij}$ (5)

$W_j=1+k\sum _{i=1}^n{f}_{ij}\ln f_{ij}/\sum _{j=1}^m\left(1+k\sum _{i=1}^n{f}_{ij}\ln f_{ij}\right)$ (6)

Calculated using the forward conduction formula based on the calculation of the feedforward conduction process:

$W_j=d_j/\sum _{j=1}^m{d}_j$ (7)

In calculating the net consumption of a sport, the amount of energy used to maintain normal life activities, that is, quiet time, must be subtracted:

$y_i=\frac{max(y)-y_i}{max(y)-min(y)}$ (8)

There is no high requirement for the external environment, the noise is small, the application value is high, and the difference between the feature vectors of different human sample data is large. Most of the feature points of the same person’s test sample fall on the feature distribution curve of the training sample, so the stability between different frame data of the same person is high. The correction method is estimated by the three speed modes of slow walking, normal walking and fast walking, and there is no significant difference between the actual distance and the actual distance. That is to say, the correction method 2 can meet the application requirements, and any one of the three speed modes of slow walking, normal walking and fast walking can be used as the correction to meet the user’s use requirements. The regulation process of walking exercise is shown in Figure 6 .

Figure 6
The Regulatory Process of Walking.

The modeling research on digital running shoes in this experiment is still in the walking stage of the research room, and will be directed to the mode of youth free activities in the natural environment. Add more pressure and time series discriminating factors to perfect. The generative method first encodes the gait features at various different perspectives through an encoder. Then, the encoded feature is transformed into a typical perspective or a certain verification set perspective through a feature transformation network, and finally the transformed feature is reconstructed by the decoder. The reconstructed static gait haptic data is collected as shown in Figure 7 .

Figure 7
Reconstructed static gait tactile data acquisition.

CONCLUSION

This paper discusses the feasibility of gait recognition based on joint angle and vertical branch response feature extraction in gait contact. Collect the plantar pressure data of each foot during static standing, and save the frame after stable standing. Dynamically collect the plantar pressure data of each foot passing through the pressure test board at constant speed, fast speed and slow speed. The foot pressure distribution is detected by the flexible array pressure sensor to obtain the real-time information of the user’s gait and gait, which is designed as a insole and embedded in the sole. Different from traditional gait recognition methods, gait recognition based on deep learning can usually achieve higher performance. Each combination trains each type of gait timing independently. In gait recognition and simulation, the target data traverses the deep network, uses the minimum criterion of target data reconstruction error to identify categories, and uses the corresponding gcrbm and DBN to predict and simulate gait timing.

REFERENCES

• ¹
  Auvinet E, Multon F, Meunier J. New Lower-Limb Gait Asymmetry Indices Based on a Depth Camera. Sensors. 2015;15(3):4605-23.
• ²
  Ben X, Zhang P, Yan R, Ge G. Gait recognition and micro-expression recognition based on maximum margin projection with tensor representation. Neural Comput Appl. 2015;27(8):1-18.
• ³
  Procházka A, Vyšata O, Vališ M, Ťupa O, Schätz M, Mařík V. Use of the image and depth sensors of the Microsoft Kinect for the detection of gait disorders. Neural Comput Appl. 2015;26(7):1621-9.
• ⁴
  Nakamura A, Funaya H, Uezono N, Nakashima K, Ishida Y, Suzuki T, et al. Low-cost Three-Dimensional Gait Analysis System for Mice with an Infrared Depth Sensor. Neurosci Res. 2015;100:55-62.
• ⁵
  Eum H, Yoon C, Lee H, Park M. Continuous Human Action Recognition Using Depth-MHI-HOG and a Spotter Model. Sensors. 2015;15(3):5197-227.
• ⁶
  Pan J, Zhang X, Manocha D. Efficient Penetration Depth Approximation using Active Learning. Pattern Recognit Lett. 2013;34(13):1476-84.
• ⁷
  Su S, Liu Z, Xu S, Li S, Ji R. Sparse auto-encoder based feature learning for human body detection in depth image. Signal Process. 2015;112:43-52.
• ⁸
  Paulo J, Peixoto P, Nunes UJ. ISR-AIWALKER: Robotic Walker for Intuitive and Safe Mobility Assistance and Gait Analysis. IEEE Trans Hum Mach Syst. 2017;47(6):1-13.
• ⁹
  Jiu M, Wolf C, Taylor G, Baskurt A. Human body part estimation from depth images via spatially-constrained deep learning. Pattern Recognit Lett. 2014;50:122-9.
• ¹⁰
  Song H, Liu Z, Du H, Sun G, Le Meur O, Ren T. Depth-aware Salient Object Detection and Segmentation via Multiscale Discriminative Saliency Fusion and Bootstrap Learning. IEEE Trans Image Process. 2017;26(9):4204-16.

Publication Dates

• Publication in this collection
  29 Aug 2022
• Date of issue
  2023

History

• Received
  04 Apr 2022
• Accepted
  28 Apr 2022