Why did Hollywood drop the actor who played Jesus in Passion of the Christ?

Jim Caviezel seemed to have successful Hollywood career. But all that changed after taking the role of Jesus Christ in Mel Gibson’s controversial biblical movie  The Passion of The Christ. So what happened?