Exchange of secret keys over public channels based on neural synchronization using a variety of learning rules offer an appealing alternative to number theory based cryptography algorithms. Though several forms of attacks are possible on this neural protocol e.g. geometric, genetic and majority attacks, our survey finds that deterministic algorithms that synchronize with the end-point networks have high time complexity, while probabilistic and population-based algorithms have demonstrated ability to decode the key during its exchange over the public channels. Our survey also discusses queries, heuristics, erroneous information, group key exchange, synaptic depths, etc, that have been proposed to increase the time complexity of algorithmic interception or decoding of the key during exchange. The Tree Parity Machine and its variants, neural networks with tree topologies incorporating parity checking of state bits, appear to be one of the most secure and stable models of the end-point networks. Our survey also mentions some noteworthy studies on neural networks applied to other necessary aspects of cryptography. We conclude that discovery of neural architectures with very high synchronization speed, and designing the encoding and entropy of the information exchanged during mutual learning, and design of extremely sensitive chaotic maps for transformation of synchronized states of the networks to chaotic encryption keys, are the primary issues in this field.