TeleMath: A Benchmark for Large Language Models in Telecom Mathematical
  Problem Solving
The increasing adoption of artificial intelligence in telecommunications has
raised interest in the capability of Large Language Models (LLMs) to address
domain-specific, mathematically intensive tasks. Although recent advancements
have improved the performance of LLMs in general mathematical reasoning, their
effectiveness within specialized domains, such as signal processing, network
optimization, and performance analysis, remains largely unexplored. To address
this gap, we introduce TeleMath, the first benchmark dataset specifically
designed to evaluate LLM performance in solving mathematical problems with
numerical solutions in the telecommunications domain. Comprising 500
question-answer (QnA) pairs, TeleMath covers a wide spectrum of topics in the
telecommunications field. This paper outlines the proposed QnAs generation
pipeline, starting from a selected seed of problems crafted by Subject Matter
Experts. The evaluation of a wide range of open-source LLMs reveals that best
performance on TeleMath is achieved by recent models explicitly designed for
mathematical or logical reasoning. In contrast, general-purpose models, even
those with a large number of parameters, often struggle with these challenges.
We have released the dataset and the evaluation code to ease result
reproducibility and support future research.
684bdf0f3b733ba3336871c3
Vincenzo Colle
claimed_verified
2025-06-13T14:59:22.494Z
66e81005f42f13eb43120bb3
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/B_PUi4PjBNnfpQ09Lvtn0.png
Vincenzo Colle
user
vincolle
684bdf0f3b733ba3336871c4
Mohamed Sana
684bdf0f3b733ba3336871c5
Nicola Piovesan
claimed_verified
2025-06-14T06:24:31.963Z
650ab653cc02352e1e43e43e
https://cdn-avatars.huggingface.co/v1/production/uploads/650ab653cc02352e1e43e43e/Zl_nv3yFWhJYeu3EYKwIU.jpeg
Nicola Piovesan
user
nicopi
684bdf0f3b733ba3336871c6
Antonio De Domenico
684bdf0f3b733ba3336871c7
Fadhel Ayed
684bdf0f3b733ba3336871c8
Merouane Debbah
TeleMath: A Benchmark for Large Language Models in Telecom Mathematical
  Problem Solving
The increasing adoption of artificial intelligence in telecommunications has
raised interest in the capability of Large Language Models (LLMs) to address
domain-specific, mathematically intensive tasks. Although recent advancements
have improved the performance of LLMs in general mathematical reasoning, their
effectiveness within specialized domains, such as signal processing, network
optimization, and performance analysis, remains largely unexplored. To address
this gap, we introduce TeleMath, the first benchmark dataset specifically
designed to evaluate LLM performance in solving mathematical problems with
numerical solutions in the telecommunications domain. Comprising 500
question-answer (QnA) pairs, TeleMath covers a wide spectrum of topics in the
telecommunications field. This paper outlines the proposed QnAs generation
pipeline, starting from a selected seed of problems crafted by Subject Matter
Experts. The evaluation of a wide range of open-source LLMs reveals that best
performance on TeleMath is achieved by recent models explicitly designed for
mathematical or logical reasoning. In contrast, general-purpose models, even
those with a large number of parameters, often struggle with these challenges.
We have released the dataset and the evaluation code to ease result
reproducibility and support future research.
A benchmark dataset called TeleMath evaluates Large Language Models in domain-specific mathematical problems within telecommunications, showing that models designed for mathematical reasoning perform better than general-purpose models.
Large Language Models (LLMs) mathematical reasoning signal processing network optimization performance analysis benchmark dataset QnA pairs mathematical problems numerical solutions evaluation pipeline open-source models parameter-efficient fine-tuning result reproducibility