Recognition of Odia Vowels using Clonal

Selection Algorithm Based FLANN Model

Pushpalata Pujari1 and Babita Majhi2

Department of CSIT

Guru

Ghasidas Vishwavidyalaya

Bilaspur, India

e-mail: [email protected]om, [email protected]

Abstract— Recognition of handwritten character is still a challenging problem

due to a number of variations found in writing style. This

paper aims to develop a robust model for the recognition of handwritten

character using functional link artificial neural network (FLANN) as

classifier. The weights of the FLANN classifier are further optimized by using clonal

selection algorithm (CSA) which is inspired by the clonal selection theory of

acquired immunity. Discrete wavelet transform (DWT) is used to extract features

from the handwritten characters and principal Component Analysis (PCA) is used

to further reduce the number of features. The proposed model is applied on dataset

containing 1200 samples of Odia handwritten vowels. recognition accuracy of

85.75% is achieved with the proposed model on test dataset.

Keywords— Handwritten character recognition; Discrete wavelet transform (DWT); Functional

link artificial neural network (FLANN); Clonal Selection Algorithm (CSA)

I.

Introduction

Two or more techniques can be

integrated with an objective to overcome the weakness of one technique with the

strength of other to produce optimum solution. In literature several research

works have been reported using integration of two or more techniques. In this

paper two techniques FLANN and CSA are integrated to build a robust model for

the recognition of handwritten Odia vowels. DWT transform is used for the extraction

of features from the images of the vowels.

Most of the reported works have

used multilayer network and SVM as the classifier. But the complexity of the

multilayer neural network is more when the number of input features grows on.

Hence in this paper, functional link artificial neural network (FLANN), having

single layer and single neuron is proposed for recognition of hand written Odia

vowels. FLANN has been successfully applied in many applications like channel

equalization 2526,stock market prediction 2728, detection of impulse

noise in images 29 and classification of micro array data 30. Generally

FLANN is having its own updation rule, known as delta learning. In order to

avoid the chances of falling to local minima problem, the weights of FLANN are

optimized using real coded genetics algorithm (RCGA). Wavelet transform has

been applied in applications like tumour tissue identification 19, character

recognition 20,24 license plate localization21,22 and analysis of

protein sequence23. Being motivated by the results of WT, the features of

Odia vowels are extracted using WT in this paper.

The paper is organized as follows: Section I

presents introduction and related work on hybrid models for character

recognition. Section II discusses the dataset, preprocessing, feature

extraction and feature reduction step. Section III describes the recognition

phase. Section IV shows the simulation study and experimental results.

Conclusion and future scope of the research are discussed in section V and VI.

II. Dataset,

Preprocessing And Feature Extraction

A.

Dataset

The dataset used in this paper is taken

from computer vision and pattern recognition centre of NIT Rourkela. The

database contains 1200 samples of Odia handwritten vowels. All the samples of

the database belong to twelve classes (1-12). Each vowel (1-12) appears 120

times in the database. 80% of the dataset are used for training and the rest

are used for testing. Few samples of Odia vowels are shown in Fig.1.

Fig.1. Samples

of Odia handwritten vowels

B.

Preprocessing

Pre-processing is a series of

operations which include background noise reduction,

filtering, original image restoration etc which are performed on the input

image. This step is carried out for improving the quality of the image before

the application of other character recognition steps. In this paper first the

data is normalized to a standard size of 64X64 pixels. Then the gray scale

image of the data is generated by using Mean filtering method. The normalized images of the vowels are shown

in Fig.2.

Fig.2. Image of Odia Vowels after normalization

C.

Feature extraction by using

discrete wavelet transform

Feature extraction

is carried out to find important features to be used in the recognition phase.

In this paper DWT based approach is used for feature extraction. In DWT, a

time-scale representation of the digital signal is obtained using digital

filtering techniques 313233 and 34. Due to the better energy compaction

property of wavelet transform it provides substantial improvement in picture quality

at high compression ratio. In DWT 1-D wavelet transform is applied along the

rows of the image for obtaining 2-D DWT decomposition. The results obtained are

then decomposed along the columns which splits the given input image into four

decomposed sub band images. The sub band images are represented as LL, LH, HL

and HH frequency bands. 2-D DWT

decomposes the image into two parts: approximation and detailed part.

Approximation part contains a low frequency sub- band LL and detailed part

contains three high frequency sub bands LH, HL and HH. The process is repeated to obtain multiple

scale wavelet decomposition.

Any given signal

can be decomposed by DWT into a set of basic functions called wavelets. It is

realized from a single prototype wavelet ?(t) by mother wavelet using dilations and shifting. DWT requires a two-dimensional scaling function, ?(u, v) and three two-dimensional

wavelet functions, ?H(u,v), ?V(u,v), and ?D(u, v). They are represented as

the products of two one-dimensional functions,

(1)

where ?(?) is a one-dimensional

scaling function and ?(?) is a one dimensional

wavelet function. These wavelets measure intensity functional variations along

different directions. ?H measures variations along

columns, ?V measures variations along

rows and ?D measures variations along

diagonals.

The

two-dimensional discrete scale and translated basis functions are defined as

(2)

(3)

Where j is a scale and m, n are the translation quantities. The

transform of image f(u,v) of size M X N is expressed as

(4)

(5)

Where represents

approximation part of image f(u,v) and represents

horizontal, vertical and diagonal parts.

III. Classification

A)

Functional Link Artificial

Neural Network

Functional link artificial

neural network is a higher order neural network (HON) 25262728 used for

several applications like classification, planning, system identification,

intelligent pressure sensor, electric load forecasting, intelligent sensor,

insecurity estimation etc. In this paper FLANN is used for the classification

task. FLANN has only one neural element and link which makes it simpler than multilayer

artificial neural network (MLANN). FLANN needs less number of iterations and

computation in training phase. It can handle non-linear problems with faster

rate of convergence with less complexity. In FLANN the dimension of input space

is increased by extending the input vectors with a suitable enhanced

representation of input vectors. A single layer model based on trigonometric

expansion is presented below

For the input pattern

(6)

The enhanced pattern using trigonometric function for is

represented as

(7)

where

Fig. 3 shows the architecture of FLANN classifier. The

extended inputs are multiplied by a set of weights calculated by using eq. (8). The outputs obtained are summed to

produce the estimated output. The estimated outputs are compared with the

desired class value. The errors generated are minimized by adjusting the

weights of the FLANN by using RCGA.

Fig. 3 . Architecture of FLANN model as

a classifier for Odia vowels

A) Weight optimization by RCGA_FLANN model

III. Clonal Selection Algorithm (CSA)

Clonal selection

algorithm is an optimization technique which behaves same as an organism

behaves against a pathogen. It is based on natural theory of evolution inspired

by Darwinian principles. It undergoes through the phase of selection, reproduction

and mutation. In

reproduction phase the child is exact copy of the parent. Mutation is carried

out to maintain diversity in population. an 17 Clonal selection algorithm is

a population based algorithm which comes

under the field of artificial immune systems. The algorithm learns adaptively

in the same way the antibodies learns the features of antigen and behaves

accordingly. The algorithm starts by creating a set

of possible candidate solutions. The affinity is calculated using the antibodies.

The affinity determines the antibodies to be cloned for the next stage. The algorithm clones most simulated

antibodies. After cloning of anti bodies mutation is carried out on the cloned

values with a mutation ratio. After mutation the affinities of the antibodies are

evaluated again. After certain generation the affinity with the lowest value

provides the desired solution to the problem. The generic steps of CSA are listed as follows:

Step 1: Initialize a set of antibodies randomly where each

antibody represents a candidate solution for the specified objective function.

Step

2: Using each antibody calculate the affinity value of each candidate

solutions.

Step 3:

Starting from the lowest affinity to highest affinity sort the antibodies.

Step 4: Clone the antibody having lowest

affinity with other antibodies with some predefined ratio.

Step 5: Mutate the cloned antibodies with mutation

ratio.

Step 6: Reevaluate the affinity of each

antibody.

Step

7: Repeat the steps from 3 to 6 until the desired criteria is reached.

IV.

Weight Optimization

of FLANN Model using Clonal Selection

algorithm

The steps involved in optimization 17 of weights of

FLANN model using CSA are as follows

Step1:

Initialize the weights of the FLANN model from -1 to +1. Each weight of the FLANN

model represents an antibody. Generate m number of weight vectors where each

weight vector represents a candidate solution to the problem.

Step2:

Take ‘x’ number of input samples. Apply the input samples to the FLANN

classifier. Evaluate the output of the FLANN classifier using the weight vector

and input samples.

Step

3: Generate error terms using output obtained from the

model and target output for sample and vector using (2)

Step

4: Calculate mean square error (MSE) for each set of weight vector using (3). Best

fitness is based on the minimum MSE obtained.

The

goal of CSA is to minimize MSE to get a suitable set of weights for FLANN classifier.

Step

5: Select the weight vector having minimum MSE.

Step

6: Clone the selected weight vector with a pre defined probability.

Step

7: Randomly choose the weight vectors and mutate the weight vectors with

mutation probability

Step

8: Repeat the step 2 through 6 until stopping criteria is reached.

V. Simulation

Study for FLANN- CSA Model

The experimental work is

implemented using MATLAB software. 1200 number of handwritten Odia vowels is

used for the experimental work. 90% of the dataset is used for training and rest

is used for testing the proposed model. The vowels are categorized into one of twelve

categories. In the training phase fivefold cross validation is used to build

the model. In the preprocessing phase

the images of the vowels are normalized into 64 X 64 pixels and median

filtering is applied on the images of the vowels. For generating feature vector

DWT is used in the feature extraction phase. The dimension of the generated feature

vector is further reduced using PCA from 2519 to 33. In recognition phase FLANN is used as a

classifier. The reduced features are applied as inputs to the FLANN model. Initially

weights are randomly generated within the range from -0.5 to 0.5 and sigmoid

activation function is used for obtaining the outputs. Errors are generated

using actual and predicted outputs. The weights of the FLANN classifier are

adjusted using CSA algorithm. The error

terms obtained from the model are used for the computation of Mean square error

(MSE). In the proposed model 1000 number of generation taken. The confusion matrix obtained

for FLANN_CSA model with equal number of classes is shown in figure 5. Table 1

shows the recognition accuracy of FLANN_CSA model for each class. From

experimental result the recognition accuracy is found to be 85.25%.

TABLE 1. Confusion Matrix Of FLANN_CSA

Classifier During Testing

Class1

Class2

Class3

Class4

Class5

Class6

Class7

Class8

Class9

Class10

Class11

Class12

Class1

90

3

1

0

0

0

1

0

0

0

0

1

Class2

2

89

1

0

1

0

1

0

0

1

0

1

Class3

1

2

82

3

2

2

0

0

2

1

1

0

Class4

1

1

8

73

2

3

0

2

0

2

2

2

Class5

0

1

2

2

76

3

2

2

2

2

2

2

Class6

2

2

3

2

3

70

2

1

3

2

2

3

Class7

0

1

0

1

1

2

81

3

1

2

2

2

Class8

2

0

1

1

2

1

2

85

2

0

0

0

Class9

0

0

1

1

0

0

1

1

89

2

1

0

Class10

1

0

2

2

2

1

0

2

3

82

1

0

Class11

1

0

0

1

1

0

1

0

1

0

89

2

Class12

2

1

2

3

2

0

2

0

1

1

2

80

TABLE 2. Classification accuracy Of The FLANN_CSA model

Class

No. of Samples

No. of Correct

Prediction

Accuracy %

Training

Testing

Training

Testing

Training

Testing

Class1

96

24

90

22

93.75

91.67

Class2

96

24

89

21

92.71

87.5

Class3

96

24

82

20

85.42

83.33

Class4

96

24

73

17

76.04

70.83

Class5

96

24

76

18

79.17

75

Class6

96

24

70

16

72.92

66.67

Class7

96

24

81

19

84.38

79.17

Class8

96

24

85

21

88.54

87.5

Class9

96

24

89

22

92.71

91.67

Class10

96

24

82

20

85.42

83.33

Class11

96

24

89

22

92.71

91.67

Class12

96

24

80

19

83.33

79.17

Average Accuracy

85.59%

82.29%

IV.

Conclusion

In

this paper two techniques: FLANN and CSA are combined to produce optimum

solution for the recognition of Odia handwritten vowels. Before recognition

preprocessing, feature extraction and feature reduction steps are carried out.

DWT is used for extraction of features form the vowels and PCA is used to

further reduce the number of features from 2519 to 33. The reduced features are

applied to FLANN classifier in the recognition phase. The weights of FLANN classifier

are adjusted using CSA to obtain a suitable set of weights which provides

optimum solution for recognition of Odia vowels. From the experimental result it

is observed that the proposed system which uses a combination of DWT transform,

FLANN classifier and CSA based optimization technique has achieved 82.29% accuracy

on test dataset. Still the recognition accuracy of the system can be improved

applying more robust techniques in feature extraction and recognition phase.