Title: Learning Overcomplete Representations using Leaky Linear Decoders

Issue Number: Vol. 8, No. 3
Year of Publication: Sep - 2018
Page Numbers: 174-179
Authors: Sarthak Yadav , Ankur Singh Bist
Journal Name: International Journal of Digital Information and Wireless Communications (IJDIWC)
- Hong Kong
DOI:  http://dx.doi.org/10.17781/P002444


Feature engineering is a complex and arduos undertaking that takes considerable amount of effort and domain expertise. Feature Learning, also called Representation learning, obviates manual Feature Engineering and automates the process. One such methodology is the Autoencoder. In this paper, we propose a modified Autoencoder based technique, named Leaky Linear Decoders (LLD), which help us extract quality Overcomplete Representations of the data in question. In this work, we propose the given technique, explicitly stating the LLD architecture, the training objective and the loss function used for training LLDs. Finally, use weights learned by LLDs as initial weights for Neural Network Models and apply these models to standard benchmark data sets widely used in literature.