The Design of Waveguide Structures Using Improved Neural Networks

In this paper, an improved neural networks (INN) strategy is proposed to design two waveguide filters (Pseudo-elliptic waveguide filter and Broad-band e-plane filters with improved stopband). INN is trained by an efficient optimization algorithm called teaching–learning-based optimization (TLBO). To validate the effective of this proposed strategy, we compared the results of convergence and modeling obtained with a population based algorithm that is widely used in training NN namely Particle Swarm Optimization (PSO-NN). The results show that the proposed INN has given better results.


I. INTRODUCTION
The full wave EM solvers [1] have been used to design the microwave filter for a long time.With the increasing complexity of wireless and satellite communication hardware, the use of EM solver takes a considerable amount of time.Recently, artificial neural network (ANN) has been proven to be a fast and effective means of modeling complex electromagnetic devices.It has been recognized as a powerful tool for predicting device behavior for which no mathematical model is available or the device has not been analyzed properly yet.ANN can be trained to capture arbitrary input-output relationship to any degree of accuracy.Once a model is developed it can be used over and over again.
The trained model delivers the output parameters very quickly.For these attractive qualities, ANN has been applied to different areas of engineering's [2] - [4].
Training of neural networks (NN) is an important step; it is based on optimization of weights of NN to minimize the mean square error (MSE) between the NN output and the desired output.Specialized learning algorithms are used to adapt these weight values.Among those algorithms, the most popular algorithm is the back-propagation method (BP) [5] based on a gradient descending.Lately, many populations based algorithms have been proposed for training a neural network such as Particle Swarm Optimization (PSO) [6], Genetic Algorithms [7] and other optimization algorithms [8].
In this paper, we tried to improve the NN by training them by a recent and effective optimization algorithm called Teaching-Learning Based Optimization (TLBO) [9].The

A. Teacher phase
In this phase, learners of each group take their knowledge directly through the teacher, where a teacher tries to increase the mean result value of the classroom to another value, which is better than, depending on his or her capability.This follows a random process depending on many factors.In this work, the value of solution is represented as (X j,k ) S , where j means the j th design variable (i.e.subject taken by the learners), k represents the k th population member (i.e.learner), and S represents the S th group.The existing solution is updated according to the following expression (2) The above equations are for a minimization problem, the reverse is true for a maximization problem.Dj is the difference between the current mean and the corresponding result of the teacher of that group for each subject calculated by: Where h ≠ k, M j is the mean result of each group of learners in each subject and TF is the adaptive teaching factor in each iteration given by equation: Learner phase In this part, each group update the learners' knowledge with the help of the teacher's knowledge, along with the knowledge acquired by the learners during the tutorial hours, according to: Where EF = exploration factor = round (1 + rand).The above equations are for a minimization problem, the reverse is true for a maximization problem.

III. IMPROVED NEURAL NETWORKS
The training of neural networks is to find an algorithm for optimized weights of networks to minimize the mean square error (MSE) described as follows (7) Where PT the total number of training samples, Y S is the output of the network and Y is the desired output.
With f 2 and f 1 are the activation functions (typically: sigmoid, tanh ...), X is the input vector of NN.
Regarding the NN training, the most used training algorithm is the back-propagation (BP) algorithm, which is a gradient-based method.In this paper, we propose a new strategy of training a neural network based on a recently proposed algorithm called Teaching-Learning Based Optimization (TLBO) [9]; the details of this strategy are presented in the next section.

A.
Implementation of TLBO on the neural networks The step-wise procedure for the implementation of (TLBO-NN) is given in this section.
Step.1.Define the neural network architecture (number of neurons in input layer Ne, number of neurons in hidden layer Nc and number of neurons in output layer Ns) and define the optimization problem: -Design variables of the optimization problem (i.e.number of subjects offered to the learner): WE and WS the matrices of input connection weights and output connection weights respectively.WE matrix of Nc rows and Ne columns and WS matrix of Ns rows and Nc columns.
-The optimization problem (fitness function): find the optimal WE and WS which minimizes the mean square error (MSE) equation (7). Step.
=  * (,  * ) (10) Step.3.Select the best solution who acts as chief teacher for that cycle, and select randomly the other teachers.
Step.4.Arranged the teachers according to their fitness value, then assigns the learners to the teachers [9].
Step.5.Calculate the mean result of each group of learners in each design variables: MWE, MWS.
Step.6.For each group, evaluate the difference between the current mean and the corresponding result of the teacher of that group for each design variables by utilizing the adaptive teaching factor [12].
Step.8.Combine all the groups.
Step.9.Repeat the procedure from step 3 to 8 until the termination criterion is met.

B. Application examples and results
In this part, the performance of the improved neural networks (INN) is investigated for modeling two microwave filters: Pseudo-elliptic waveguide filter Fig. 2 [13] and Broad-band e-plane filters with improved stop-band Fig. 3 [14].The dimensions of the first and second filters are listed in Table I     For modeling the two structures above-mentioned.We propose multilayer feed-forward neural network architecture with a single hidden layer.We begin by selecting the input parameters and creating database, the latter starts by creating a list of points from the matrix of bounds of the input parameters.The list of the database points is of size (Ne, PNe), Ne is the number of input parameters, and P is the number of points for each parameter.In this work, we choose an equal number of points for each variable P=4, Ne=5 for the first filter corresponding to (W1, W2, W3, V1, V2) with a database equal 1024 and Ne=3 for the second filter corresponding to (W1, W2, W3) with a database equal 64.The bounds of each parameter are presented in Table III.The number of output Ns=1 corresponding to the frequency responses of the Sij parameters.For the frequency range, we chose to be (8, 12 GHz) with a number of points K=41 and (26, 36 GHz) with K=34 for the first and the second filter respectively.The choice of the number of hidden neurons is strongly related to the nature of nonlinearity to model.In our application examples, the number of hidden neurons gives a good convergence of the algorithm and a good accuracy of the neural model formed are Nc=8 for the first filter and Nc=6 for the second filter.The activation functions are hyperbolic tangent function (Tansig), and linear function (Purelin) respectively.When the architecture of NN is selected, the next step is to train NN using TLBO algorithm section (III.A).We begin by initializing the connection weights equations ( 9) and (10).Once the learning is complete, we obtained the update WE and WS and we can approximate the S parameters response to any input parameter in the boundary range using equation (8).Fig. 4 and Table IV shows the convergence of PSO and TLBO algorithms for minimize the MSE of NN for the filters above-mentioned with the effect of the number of teachers in TLBO algorithm.The common parameters for algorithms (population size NP=50, number of iterations is 300 for the first filter, and 100 for the second filter), the other specific parameters of the algorithms are given below.
PSO Settings: c 1 and c 2 are constant coefficients c 1 = c 2 = 2, w is the inertia weight decreased linearly from 0.9 to 0.2.
TLBO Settings: for TLBO there is no such constant to set.It is observed from Fig. 4 and Table IV that, the TLBO (NT=7) algorithm perform better in terms of convergence than the PSO algorithm, in which this algorithm requires less number of iterations to converge to optimum solution as compared to PSO algorithm.Fig. 5 and Fig. 6 gives the approximate parameters S11, S21 (magnitude and phase) for the first and second filters respectively, an excellent approximate can be observed.
trained networks are applied to modeling two waveguide Filters (Pseudo-elliptic waveguide filter and Broad-band e-plane filters with improved stop-band).We use a multilayer perceptron neural network (MLP-NN) to three layers.Each sub-net in the NN architecture shown in Fig.1, possesses Ne input neurons corresponding to the number of the geometry parameters of the structures, Nc neurons in the hidden layer and one output associated with the value of S ij (f k ).The entire network consists of k distinct NNs corresponding to a particular frequency with k determined by the number of approximate points in the frequency interval.Frequency responses of S parameters obtained in simulations compose the network database.The connection weight from the neurons of the input layer to the neurons of the hidden layer is WE and the connection weight from the neurons of the hidden layer to the neurons of the output layer is WS.

2 .
Initialize the optimization parameters and initialize the population -Population size (number of learners in a class) NP -Number of generations (maximum number of allowable iterations) maxit -Number of teachers NT -The initial population according to the population size and the number of neurons and evaluate the corresponding objective function value.For simplification, the population is decomposed into two groups one represents the inputs weights population WEp and the second one represents the output weights population WSp. =  * ( * , )

Fig. 3 .
Fig. 3. Broad-band E-plane filter, a) The designs of broad-band E-plane filter, b) Fin-line post structure.

Fig. 4 .
Fig. 4. Convergence of algorithms for minimizing the MSE, a) first filter, b) second filter.

TABLE I .
DIMENSIONS FOR THE DESIGNED FILTER (UNITS: MILLIMETERS).

TABLE II .
DIMENSIONS FOR THE DESIGNED FILTER (UNITS: MILLIMETERS).

TABLE III .
INPUT PARAMETERS AND THEIR LIMITS

TABLE IV .
COMPARATIVE RESULTS OF CONVERGENCE OF MSE