Qdrant, Google Cloud, Azure, Openai Api, Retrieval Augmented Gen.
of the text is that the model described leverages grouped-query attention (GQA) and sliding window attention (SWA) to enable faster inference and effectively handle sequences of arbitrary length with reduced inference cost. Question: What architectural improvements did PaLM 2 introduce compared to