Koyama-ji est associé à Yakushi Nyorai, le Bouddha de la Médecine, qui y est vénéré en tant que divinité principale du temple.

Koyama-ji se trouve à Zentsuji, dans la préfecture de Kagawa, ce qui le relie à l'une des villes de pèlerinage les plus connues de Shikoku.

Young Kobo Daishi played in this area as a child, building mud Buddha statues and plant temples. The field where he played was called Sen-yugahara (Field Where Hermits Play).

A saint appeared from a cave on Mt. Koyama and told Kobo Daishi to build a temple there. That cave, located to the left of the Daishido Hall, became the birthplace of Koyamaji Temple.