Oh yes, you can go in... but first you have the climb a rickety, rusty metal ladder up a 130-feet tall column of rock. The building atop the column was constructed sometime between the sixth and eighth centuries and is thought to have at first been a pagan temple (the rock being ummmm... symbolic of "fertility", parse that as you will).
The Orthodox churches credit their founding to the apostle Andrew, who journeyed to the region following the ascension of Christ. With the conversion of the pagans, the temple became a Christian chapel. Today it is being restored and is even the home of an Orthodox priest.
My brain feels staggered just looking at the photo of this thing. Just... wow.
Click here to learn more about the chapel at Chiatura, including a photo of the interior containing some beautiful icons.