On The Performance Improvement Of Neural Networks: Using Lempel-Ziv (LZ77) Algorithm to Compress Input Data in an Activation-Normalised Network

In this paper two complementary performance improval techniques are demonstrated in order to obtain better results in a particular neural network. We perform lossless data compression on the input data to reduce the data size as well as implementing layer activation normalisation to avoid internal covariate shift. Data compression is performed using the dictionary-based lossless data compression algorithm known as LZ77 which reduces the size of the input data efficiently by approximately the half of its original size. This allows us to reduce the computations involved with desired weights associated with the input layer, which furthermore results in less computational time in the backpropagation algorithm. Later on, in the feedforward process, the activations are normalised for better performance of the network. By normalising the activations, we obtain more reliable results. With having the two techniques combined, a fast and much more reliable approach intraining networks is proposed.

کلیدواژه ها:

Artificial Neural Networks ، Lempel-ziv (LZ77) ، Lossless Data Compression ، Covariate Shift ، Layer Normalisation