All-solid-state lithium batteries (ASSLB) are promising candidates for next-generation energy storage devices. Nevertheless, the large-scale commercial application of high energy density ASSLB with the polymer electrolyte still faces challenges. In this study, a thin solid polymer composite electrolyte (SPCE) is prepared through a facile and cost-effective strategy with an infiltration of thermoplastic polyurethane (TPU), lithium salt (LiTFSI or LiFSI), and halloysite nanotubes (HNTs) in a porous framework of polyethylene separator (PE) (TPU–HNTs–LiTFSI–PE or TPU–HNTs–LiFSI–PE). The composition, electrochemical performance, and especially the effect of anions (TFSI− and FSI−) on cycling performance are investigated. The results reveal that the flexible TPU–HNTs–LiTFSI–PE and TPU–HNTs–LiFSI–PE with a thickness of 34 μm exhibit wide electrochemical windows of 4.9 and 5.1 V (vs. Li+/Li) at 60 ℃, respectively. Reduction in FSI− tends to form more LiF and sulfur compounds at the interface between TPU–HNTs–LiFSI–PE and Li metal anode, thus enhancing the interfacial stability. As a result, cell composed of TPU–HNTs–LiFSI–PE exhibits a smaller increase in interfacial resistance of solid electrolyte interphase (SEI) with a distinct decrease in charge-transfer resistance during cycling. Li|Li symmetric cell with TPU–HNTs–LiFSI–PE could keep its stable overpotential profile for nearly 1300 h with a low hysteresis of approximately 39 mV at a current density of 0.1 mA cm−2, while a sudden voltage rise with internal cell impedance-surge signals was observed within 600 h for cell composed of TPU–HNTs–LiTFSI–PE. The initial capacities of NCM|TPU–HNTs–LiTFSI–PE|Li and NCM|TPU–HNTs–LiFSI–PE|Li cells were 149 and 114 mAh g−1, with capacity retention rates of 83.52% and 89.99% after 300 cycles at 0.5 C, respectively. This study provides a valuable guideline for designing flexible SPCE, which shows great application prospect in the practice of ASSLB.