Koyama-ji está asociado con Yakushi Nyorai, el Buda de la Medicina, quien es venerado aquí como la deidad principal del templo.

Koyama-ji está en Zentsuji, Prefectura de Kagawa, lo que lo vincula con una de las ciudades de peregrinación más conocidas de Shikoku.

Young Kobo Daishi played in this area as a child, building mud Buddha statues and plant temples. The field where he played was called Sen-yugahara (Field Where Hermits Play).

A saint appeared from a cave on Mt. Koyama and told Kobo Daishi to build a temple there. That cave, located to the left of the Daishido Hall, became the birthplace of Koyamaji Temple.