[01] Kernan Mzelikahle, Computer Science Department, National University of Science and Technology, Bulawayo, Zimbabwe. [02] John Trimble, Industrial Engineering Department, Tshwane University of Technology, Tshwane, South Africa. [03] Dumisani John Hlatywayo, Applied Physics Department, National University of Science and Technology, Bulawayo, Zimbabwe.

In the field of machine learning, many applications require techniques that are able to respond in real time. However, such abilities are usually achieved by trading off the accuracy rates in favour of good time complex performance. The Long Short Term Memory (LSTM) is a typical example where the technique provides good time complex performance, yet it has fairly low accuracy rates. The Bat Optimised Self Organising Map (BOSOM), on the other hand, is a relatively slow processing technique for unsupervised classification problems; however, it has fairly high accuracy rates. In this paper, a hybrid technique between the standard LSTM network and BOSOM, for use in time and space unconstrained applications, is proposed. The objective of the paper is to demonstrate that higher accuracy rates and higher recall rates are better achieved by striking a balance between turn-over time and exhaustive learning. To achieve generalisation of performance of the BOSOM-LSTM model, datasets of varying sizes from multiple domains are used. The results in this paper show that the BOSOM-LSTM hybrid model has considerably better accuracy and recall performance compared to other considered techniques. This performance establishes good ground that the BOSOM-LSTM hybrid technique is a competitive technique that can be used in application areas that require high accuracy and high reliability.

Long Short Term Memory, Bat Optimised Self Organising Map, Artificial Neural Networks, Unsupervised Learning

[01] Chaudhary, R. and Patel, H. (2015). A Survey on the Back-Propagation Algorithm for Neural Networks. International Journal for Technological Research in Engineering, 2, pp. 729-733. [02] Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H and Bengio, Y. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. [Online] arXiv preprint arXiv:1406.1078. [03] Demuth, H. B., Beale, M. H, De Jess, O. and Hagan, M. T. (2014). Neural Network Design. 2nd ed., Martin Hagan, Oklahoma, USA: Oklahoma State University. [04] Fausett, L. V. (1994). Fundamentals of Neural Networks: Architectures, Algorithms and Applications. Prentice-Hall International Editions, New Jersey, USA: Prentice-Hall. [05] Gers, F. A., Schmidhuber, J. and Cummins, F. (2000). Learning to Forget: Continual Prediction with LSTM. Neural Computation, 12 (10), pp. 2451-2471. [06] Gers, F. A. and Schmidhuber, J. (2001). LSTM Recurrent Networks Learn Simple Context Free and Context Sensitive Languages. IEEE Transactions on Neural Networks, 12 (6), pp. 1333-1340. [07] Gers, F. A., Schraudolph, N. N. and Schmidhuber, J. (2002). Learning Precise Timing with LSTM Recurrent Networks. Journal of Machine Learning Research, 3, pp. 115-143. [08] Gurney, K. (2004). An Introduction to Neural Networks. London: Taylor and Francis e-Library, University College London (UCL). [09] Hochreiter, S. and Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9 (8), pp. 1735-1780. [10] Kröse, B. J. and van der Smagt P. (1996). An Introduction to Neural Networks. 8th ed. Department of Computer Systems, University of Amsterdam, Netherlands. [11] Li, S., Li, W., Crook, C., Zhu, C. and Yanbo, G. (2018). Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN. [Online] arXiv preprint arXiv:1803.04831v3. [12] Mzelikahle, K., Mapuma, D. J., Hlatywayo, D. J. and Trimble, J. (2017). Optimisation of Self Organising Maps using the Bat Algorithm. American Journal of Information Science and Computer Engineering, 3 (6), pp. 77-83. [13] Pang, B. and Lee, L. (2005). Seeing Stars: Exploiting Class Relationships for Sentiment Categorisation with Respect to Rating Scales. In: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, pp. 115-124. [14] Pérezgonzález, J. D. and Gilbey, A. (2011). Predicting Skytrax Airport Rankings from Customer Reviews. Journal of Airport Management, 5 (4), pp. 335-339. [15] Sak, H., Senior, A. and Beaufays, F. (2014). Long Short-Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling. In: Fifteenth Annual Conference of the International Speech Communication Association. [16] Schleif, F. M. (2014). Discriminative Fast Soft Competitive Learning. In: Proceedings of the International Conference on Artificial Neural Networks, September 2014. pp. 81-88, Springer, Cham. [17] Schmidhuber, J. (2015). Deep Learning in Neural Networks: An Overview. Neural Networks, Elsevier, 61, pp. 85-117. [18] Shalev-Shwartz, S. and Ben-David, S. (2014). Understanding Machine Learning: From Theory to Algorithms. United Kingdom: Cambridge University Press. [19] Spanakis, G. and Weiss, G. (2016). AMSOM: Adaptive Moving Self-Organizing Map for Clustering and Visualization. [Online] arXiv preprint arXiv:1605.06047. [20] Taheri J. and Zomaya A. Y. (2006). Artificial Neural Networks. In: Zomaya A. Y. (eds.) Handbook of Nature-Inspired and Innovative Computing: Integrating Classical Models with Emerging Technologies. New York, USA: Springer, pp. 147–185. [21] Yang, X. S. (2010). Nature-Inspired Metaheuristic Algorithms. University of Cambridge: Luniver Press.