Kobo Daishi prayed at the Bishamonten cave for the safe completion of Manno-ike Reservoir renovations. The project finished in just three months, and this temple was built to commemorate that achievement.

Koyama-ji está asociado con Yakushi Nyorai, el Buda de la Medicina, quien es venerado aquí como la deidad principal del templo.