There is an interesting legend behind it. Once when Brahma was relaxing in his Vayjayanti assembly, the sage Narada entered singing the praises of Sree Narayana. Brahma was so enraptured that he fell at the feet of Narada seeing him not as a son but as Hari Himself. The other devas who saw this burst into laughter and Brahma was greatly embarrassed. Infuriated, he cursed them banishing them to earth where they could purify their minds and acquire knowledge and wisdom.
Narada consoled the devas and advised them to do penance at the place where his valkalam (garment made of bark of a tree) fall. And the place where it landed is now called varkala a colloquial form of valkalam. The devas built a temple there dedicated to Lord Krishna. This was however destroyed by sea erosion. Lord Brahma, the legend goes, came to earth one day to conduct a yoga. Lord Mahavishnu as Sree Janardana Swamy appeared before him to bless the devas and redeem them. The presiding diety of Varkala temple is thus Sree Janardana Swamy. The image has four hands one holding divine water, another, the conch, the third, the discus and the fourth, the mace.