Electrochemical properties of all-solid-state lithium batteries using lithium borohydride (LiBH4) as a solid electrolyte are presented for the first time. Despite its high conductivity and good compatibility with a lithium electrode, LiBH4 has not been considered suitable for practical applications partly due to its high reducing capability when used with metal oxides. In our investigation, it was confirmed that contact between this hydride solid lithium ion conductor and LiCoO2 results in a large interfacial resistance, and therefore significant capacity loss. In an attempt to minimize this effect, an intermediate Li3PO4 layer was employed. Our results indicate a significant improvement in both the rate and the cycle performance of LiBH4 solid electrolyte batteries, suggesting that LiBH4 can be used in practical applications as an electrolyte in all-solid-state lithium batteries.