This temple was originally the queen's palace of the Bundela King Madhukar Shah. There is a legend associated with it. The queen being an devotee of Lord Rama had got an idol of Rama from Ayodha to be kept in a newly built temple. She was told that, once the idol is kept down for the first time in Orchha, it can not be later moved out.
It so happened that, when queen arrived in Orchha, the new temple the Chaturbhuj Temple built for the Lord Rama was not still ready. Due to this the idol was placed in Queen's palace for the day. But, later it could not be relocated from this place. Because of this the palace was converted into a temple.