Mount Goken once had five distinct peaks, but one collapsed during an earthquake in the Edo period. The mountain's name comes from a legend that five swords fell from heaven when Kūkai trained here, which he buried for protection.

Yakuri-ji è il tempio numero 85 degli 88 templi del Pellegrinaggio di Shikoku, rendendolo una tappa ufficiale per i pellegrini henro.