There are three different places by the name of Udaygiri in India as far as my understanding. All the three are famous heritage sites known for the caves in these locations. If one of them is in Madhya Pradesh state, and the other two are in Orissa state.
In Orissa Udaygiri, Jain Heritage site is very close to the state capital Bhunaneshwar. If one travels mere 6 kms one can witness this magnificent caves inhabited by Jain monks in the 2nd century BCE. There are several caves here of different sizes and some of them have unique carvings in them.
Second Udaygiri, Buddist Heritage site in Orissa is approximately 90 kms away from state capital Bhunaneshwar. Here in the forest region one can see excavated Stupas and layout of various centers that existed in between 7th and 12th century.
The Udaygiri, Hindu Heritage site in Madhya Pradesh is very close to World Heritage site Saanchi. Here in 4th to 5th century caves, we can see several forms of the Lord, the most famous being Varaha Avatar.