Imidazole rings were grafted on alkoxysilane with a simple nucleophilic substitute reaction to form hybrid inorganic–organic polymers with imidazole rings. Proton exchange membranes (PEM) based on these hybrid inorganic–organic polymers and H3PO4 exhibit high proton conductivity and high thermal stability in an atmosphere of low relative humidity. The grafted imidazole rings improved the proton conductivity of the membranes in the high temperature range. It is found that the proton conductivities increase with H3PO4 content and temperature, reaching 3.2×10−3 S/cm at 110°C in a dry atmosphere for a membrane with 1mole of imidazole ring and 7moles of H3PO4. The proton conductivity increases with relative humidity (RH) as well, reaching 4.3×10−2 S/cm at 110°C when the RH is increased to about 20%. Thermogravimetric analysis (TGA) indicates that these membranes are thermally stable up to 250°C in dry air, implying that they have a good potential to be used as the membranes for high-temperature PEM fuel cells.