This expression means that you're really scared or anxious about something. Thousands of years ago in ancient Greece, the poet Homer used the phrase in his famous poem The Iliad, and people have been saying it ever since.
When your heart starts pounding so much that you can feel a thumping in your throat, it may feel like you "have your heart in your mouth." But the good news is that your heart can't move into your throat, mouth, or anywhere else. It stays put!