## The Energy Cost of Artificial Intelligence of Things Lifecycle

Shih-Kai Chou 1 , Jernej Hribar , Vid Hanˇ zel 1 1 , Mihael Mohorˇ ciˇ c 1 , Carolina Fortuna 1

Abstract -Artificial Intelligence (AI) coupled with the existing Internet of Things (IoT) enables more autonomous operations across various economic sectors. While this paradigm shift results in increased energy consumption it is difficult to quantify the end-to-end energy consumption of such systems with the conventional metrics as they either focus on the communication, the computation infrastructure or model development. To address this, we propose a new metric, the Energy Cost of AI lifecycle ( eCAL ). eCAL captures the energy consumption throughout the architectural components and lifecycle of an AI-powered wireless system by analyzing the complexity of data collection and manipulation in individual components and deriving overall and per-bit energy consumption. We show that the better a model and the more it is used, the more energy efficient an inference is. For an example Artificial Intelligence of Things (AIoT) configuration, eCAL for making 100 inferences is 2 73 . times higher than for 1000 inferences. Additionally, we developed a modular open source simulation tool to enable researchers, practitioners, and engineers to calculate the end-to-end energy cost with various configurations and across various systems, ensuring adaptability to diverse use cases.

AI, the Carbon Footprint (CF) of LLMs [13] and find ways to scale models efficiently [14]. Furthermore, the networking and IoT communities that are increasingly relying on AI techniques are also investigating pathways towards net-zero carbon emissions [15], analyzing the CF of various learning techniques [16] and data modalities while also developing various approaches to optimizing energy efficiency through hardware-software optimization [17], scheduling [18], more efficient model design [19] and energy and carbon consumption testing [20].

Index Terms -AIoT lifecycle, Energy Consumption, Carbon Footprint, Metric, Methodology

## I. INTRODUCTION

Next-generation networks aim to deliver unprecedented levels of connectivity and distributed intelligence, facilitating seamless interactions between a massive number of devices, services, and applications. The trend is especially noticeable in the Internet of Things (IoT) where devices are becoming increasingly more intelligent [1], from smart home appliances [2] to industrial machinery [3], smart cities [4], intelligent car [5], and beyond [6], [7]. With the help of Artificial Intelligence (AI), we are moving away from conventional IoT applications towards Artificial Intelligence of Things (AIoT) systems [8] designed to achieve more streamlined and autonomous operations. For example, in future smart factories, the integration of AI and IoT will enable real-time monitoring and optimization of production processes, predictive maintenance, and enhanced decision-making capabilities.

While AIoT systems enable unprecedented automation and agility, their reliance on AI implies additional energy consumption and increased carbon emissions [9]. Especially in the cases when models with many parameters are employed, such as large language models (LLMs), these energy and carbon costs may be significant [10]. In the light of climate challenges, the general AI research community's efforts are intensifying to better estimate the costs of training [11] and using [12]

1 Joˇ zef Stefan Institute, Ljubljana, 1000, Slovenia. Corresponding author: Shih-Kai Chou (e-mail: shih-kai.chou@ijs.si)

Depending on the scope of the study, various metrics are employed to understand energy efficiency and CF. As well noted in [15], the traditional ITU standardized Energy-per-Bit [ J/b ] metric 'will no longer be able to reflect the environmental impact of the modern mobile services, especially network AIenabled smart services' . For instance, [17] considers normalized energy as well as energy for buffering in [ J ] , electricity [ Wh ] and [ CO eq 2 ] for the various phases involved in federated learning (FL) edge systems [15], [16], [ J ] and GFLOPS of the neural architectures [19] and energy [ Wh ] for the scheduled loads in [18]. However, these metrics are not specifically designed to capture the energy efficiency of a system or its parts.

Recently, a number of metrics appeared addressing environmental friendliness, such as accuracy per consumption (APC), which captures the performance of an AI model together with its environmental impact [21]. Nevertheless, while the metrics proposed in [21] are powerful in terms of assessing the energy/performance trade-offs of models, they are less suitable in assessing the energy efficiency of a system such as AIoT. To this end, a metric somewhat similar to the Energyper-Bit [ J/b ] would be more suitable for measuring energy efficiency of an AI-based communication system, similarly as some other efficiency metrics that have been proposed in wireless in particular and engineering and economy in general [22]. For instance, spectral efficiency, measured in [ b/s/Hz ] [23] provides means of calculating the amount of data bandwidth available in a given amount of spectrum, and lifecycle emissions for vehicles, measured in [ g/km ] [24], enables computing the CF in grams per kilometer.

Following these observations, we aimed to find a suitable energy efficiency metric that would be simple and general and would holistically quantify the environmental cost of inference carried out in AIoT systems. The contributions of this paper are as follows:

- · We propose a new metric, the Energy Cost of AIoT Lifecycle (eCAL), measured in [ J/b ] , that captures the

overall energy cost of generating an inference in an AIoT system. Unlike capturing the energy required for transmitting bits that is done by the Energy-per-Bit [ J/b ] metric, or [ J ] for AI model complexity, the proposed eCAL captures the energy consumed by all the data collection and manipulation components in an AIoT system during the lifecycle of a trained model enabling inference.

- · Following the Open Systems Interconnection (OSI) and Machine Learning Operations (MLOps) [25] architectures, we devise a methodology for determining the eCAL of an AIoT system and develop an open source modular and extensible simulation tool , namely, eCAL calculator. 1 The methodology breaks down the AIoT system into different data manipulation components, and, for each component, it analyzes the complexity of data manipulation and derives the overall energy consumption.
- · Using the proposed metric, we demonstrate that the better a model is and the more it is used, the more energyefficient an inference is. For an example AIoT configuration, the energy consumption per bit for making 100 inferences is 2 73 . times higher than for 1000 inferences. This can be explained by the fact that the energy cost of data collection, preprocessing, training, and evaluation, all necessary to develop an AI model are spread across more inferences.

The paper is structured as follows. We first review the relevant literature in Section II. We then describe the lifecycle of an AIoT system, define eCAL and the methodology for deriving it in Section III. Subsequently, the derivation of the energy consumption formulas for different data manipulation components involved in the AIoT lifecycle are provided in Sections IV to VII. In Section VIII, the proposed metric, eCAL, is derived and analyzed, demonstrating its utility in capturing the overall energy cost of an AIoT system over its lifecycle. Finally, Section IX concludes the paper and outlines future research directions.

## II. RELATED WORK

The IoT systems increasingly leverage AI techniques to optimize automation and enhance operational efficiency, essentially turning into so called AIoT systems. However, their growing dependence on AI introduces significant energy and carbon costs [9], intensifying environmental concerns [13]. In response, the AI research community is placing greater emphasis on understanding and then formalizing methods to assess the environmental impact of the increasing AI utilization. Due to the complexity of state of the art neural architectures, in many cases the energy cost of training is computed after the training by measuring the performed computation through interfaces such as the performance application programming interface [29]. Furthermore, [11] shows that the energy footprint of inference is more studied than the one for training and highlights below 70% accuracy between predicted and measured energy consumption.

1 https://github.com/sensorlab/eCAL

However, a number of approaches and tools capable of proactively estimating the energy and environmental cost of training have also emerged. LLMCarbon [13] is a very recent end-to-end CF projection model designed for both dense (all parameters are used for every input) and mixture of experts (only a subset of parameters for each input) LLMs. It incorporates critical LLM, hardware, and data center parameters, such as LLM parameter count, hardware type, system power, chip area, and data center efficiency, to model both operational and embodied CFs of a LLM. Furthermore, [14] aims at providing guidelines for scaling AI in a sustainable way. They analyze the energy efficiency of new processing units, the CF of the most prominent LLM models since GPT-3 and analyze their lifecycle carbon impact showing that inference and training are comparable. They conclude that, to enable sustainability as a computer system design principle, better tools for carbon telemetry are required, large scale carbon datasets, carbon impact disclosure and more suitable carbon metrics.

The networking and IoT communities, increasingly relying on AI techniques, are also investigating pathways towards netzero carbon emissions [15]. In [15] the authors notice that in spite of improvements in hardware and software energy efficiency, the overall energy consumption of mobile networks continues to rise, exacerbated by the growing use of resourceintensive AI algorithms. They introduce a novel evaluation framework to analyze the lifecycle of network AI implementations, identifying major emission sources. They propose the Dynamic Energy Trading and Task Allocation framework, designed to optimize carbon emissions by reallocating renewable energy sources and distributing tasks more efficiently across the network. Similar as in [14], they highlight the need for the development of new metrics to quantify the environmental impact of new network services enabled by AI.

The authors of [16] introduce a novel framework to quantify energy consumption and carbon emissions for vanilla FL methods and consensus-based decentralized approaches, identifying optimal operational points for sustainable FL designs. Two case studies are analyzed within 5G industry verticals: continual learning and reinforcement learning scenarios. Similar to the authors of [15], they consider energy [ Wh ] and [ CO eq 2 ] in their evaluations. The solution proposed in [17] is a hardware/software co-design that introduces modality gating (throttling) to adaptively manage sensing and computing tasks. This so-called adaptive modality gating (AMG) solution features a novel decoupled modality sensor architecture that supports partial throttling of sensors, significantly reducing energy consumption while maintaining data flow across multimodal data: text, speech, images, and video. More energy efficient task scheduling has been considered in [18], while more efficient neural network architecture design and subsequent model development in [19]. The energy efficiency of various programming languages is studied in [30], while energy and carbon consumption testing for AI-driven IoT services is developed in [20].

Table I summarizes the relevant existing telecommunications, data center and deep learning metrics proposed to quan-

TABLE I: State-of-the-Art Energy Efficiency Metrics in Networking, Data Centers, and AI.

| Category          | Metric            | Units of Measurement   | Description                                                                                                                                                                                                                                       |
|-------------------|-------------------|------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Telecommunication | Energy Efficiency | b/J                    | Measures the number of information bits transmitted or received per unit of energy consumed by the radio access network or communication module. Rec. ITU-R M.2083-0 [26], [27].                                                                  |
| Telecommunication | Energy per Bit    | J/b                    | Tracks the energy consumed to transmit a single bit of information [15].                                                                                                                                                                          |
| Data Center       | PUE               | %                      | Power Usage Effectiveness (PUE) measures the ratio of the total energy used by a data center to the energy delivered to its IT equipment [28].                                                                                                    |
| Data Center       | CUE               | kg/kWh                 | Carbon Usage Effectiveness (CUE) quantifies the carbon emissions (kg) pro- duced per unit of energy consumption (kWh) in a data center [28].                                                                                                      |
| Data Center       | WUE               | L/kWh                  | Water Usage Effectiveness (WUE) measures the amount of water (L) required for cooling and operations in a data center per unit of energy (kWh) supplied to IT equipment [28].                                                                     |
| Deep Learning     | APC               | %                      | Accuracy per Consumption (APC) measures the accuracy of a machine learning model relative to its weight energy consumption (WC). A lower WC indicates a system with reduced energy cost for the same accuracy. GreenAI-guided metric [21].        |
| Deep Learning     | APEC              | %                      | Accuracy per Energy Cost (APEC) is similar to APC but accounts for different parameters within the WC metric. GreenAI-guided metric [21].                                                                                                         |
| Deep Learning     | TTCAPC            | %                      | Time to Closest APC (TTCAPC) integrates training time with the APC infer- ence metric, rewarding higher accuracies while penalizing higher net energy consumption (Wh) and longer training times. GreenAI-guided metric [21].                     |
| Deep Learning     | TTCAPEC           | %                      | Time to Closest APEC (TTCAPEC) integrates training time with the APEC inference metric, rewarding higher accuracies while penalizing higher net energy consumption (measured in Euros) and longer training durations. GreenAI-guided metric [21]. |
| Proposed Metric   | eCAL              | J/b                    | Quantifies the energy consumption of AI inference in AIoT, considering both communication and computation energy costs relative to application data.                                                                                              |

tify the energy cost of data collection and manipulation. According to [27], standardization organizations, such as 3GPP have recently placed a greater emphasis on energy efficiency and energy savings as a core topic, and 3GPP, for instance, has listed energy efficiency as Key Performance Index (KPI) in its Release 19 2 set of specifications. While the existing energy efficiency and energy per bit metrics are seen as sufficient for previous generations of mobile communication systems, they are not able to fully capture the AI aspects estimated to also be used for implementing 6G network functionality as well as for optimizations [15]. A number of metrics such as Power Usage Effectiveness (PUE), Carbon Usage Effectiveness (CUE), and Water Usage Effectiveness (WUE) have also been proposed for data centers [28]. However, they only capture energy aspects of data center computation without providing insights into the communications and AI aspects.

Furthermore, novel metrics to evaluate Deep Learning (DL) models, considering not only their accuracy and speed but also their energy consumption and cost have been proposed in [21]. The four metrics used are: Accuracy Per Consumption (APC) and Accuracy Per Energy Cost (APEC) for inference, and Time To Closest APC (TTCAPC) and Time To Closest APEC (TTCAPEC) for training. These metrics aim to promote the use of energy-efficient and green energy-powered DL systems by integrating energy considerations into the performance evaluation. Nevertheless, while powerful in terms of assessing the energy/performance trade-offs of models, these metrics are less suitable in assessing the energy efficiency of a system such as AIoT.

Inspired by the simplicity and effectiveness of metrics such as transmission efficiency through energy-per-bit and the

2 https://www.3gpp.org/specifications-technologies/releases/release-19

lifecycle emissions, and responding to calls for new metrics [14], [15] that capture the energy cost in the era of AI enabled systems, this work proposes the novel Energy Cost of AIoT Lifecycle (eCAL) metric and methodology to derive it. eCAL is the only metric that, based on OSI and MLOps architectures, provides insights into the cost of inference considering the data transmission and AI model overheads.

## III. ECAL DEFINITION AND METHODOLOGY

In our work, we consider an AIoT communication and computing system whose lifecycle is depicted in Fig. 1, consisting of the following data manipulating components as per the OSI stack and MLOps architectures:

- a) Data Collection: This data-manipulating component, depicted in the lower part of Fig. 1 includes receiving telemetry data from sensors N I S S to the terminal computing infrastructure via wired or wireless technologies. The collected information can be turned into indicators such as signal strength and be interpreted by the AI models, for example, to predict the location of a user. To successfully collect all the application level data, the total energy of data collection E DC [ J ] is needed.
- b) Data Preprocessing: To ensure accuracy and reliability during the training process, the data must go through several preprocessing steps such as cleaning, feature engineering and transformation. The energy consumption for preprocessing E pre , depends on the integrity of the ingested dataset.
- c) Training and Evaluation: The training component comprises the AI/Machine Learning (ML) model development using selected AI/ML techniques, such as neural network architectures, and data. In this step of the model development process, the processed data, N S T , is fetched and utilized to

Fig. 1: Data manipulation components involved in the lifecycle of an AIoT system.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00540v2-figure-e0d88cf3-e20e-4a5e-b64e-8f7b410fd21d.jpg)

learn weights and biases in order to approximate the underlying distribution. For most neural architectures, the learning processes rely on Graphics Processing Unit (GPU) and Tensor Processing Unit (TPU) performing complex tensor processing operations and consuming training energy, E train . Once the neural network architecture weights are learned using the data in view of minimizing a loss function, the model is considered ready for evaluation and deployment. Subsequently, the quality of the learned model is evaluated on the evaluation datasets, N S E , , a process that consumes E eval energy. The data storage, preprocessing, training, and evaluation form the end-to-end training of the AI/ML model and cumulatively require E D energy to complete.

d) Inference: Once the model is trained, it can be used by applications in inference mode. Various applications can send samples of data N I P , and receive model outputs in the form of forecasts for regression problems or discrete (categorical) labels for classification problems. The energy consumption E inf of a single inference is relatively small, however for high volumes of requests it can become significant.

Based on this lifecycle, we proposed eCAL as the ratio between all the energy consumed by all data manipulation components and all the manipulated application-level bits:

$$e C A L & = & \frac { \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta } & \Delta } { \Delta }$$

In the upcoming sections, we employ the following methodology in order to finally derive the proposed metric. We follow

TABLE II: Summary of Notations

| Symbol                 | Description                                                                                                                            | Description                                                                                                                            |
|------------------------|----------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------|
| N                      | Counts No. of collected data samples                                                                                                   | Counts No. of collected data samples                                                                                                   |
| S N IoT , cycle        | No. of processing cycles per bit (IoT device)                                                                                          | No. of processing cycles per bit (IoT device)                                                                                          |
| N Gateway , cycle N    | No. of processing cycles per bit (Gateway) No. of samples used for training                                                            | No. of processing cycles per bit (Gateway) No. of samples used for training                                                            |
| S , T N                |                                                                                                                                        |                                                                                                                                        |
| S , E                  | No. of samples used for evaluation inference                                                                                           | No. of samples used for evaluation inference                                                                                           |
| N S , inf N            | No. of samples used for No. of training epochs                                                                                         | No. of samples used for No. of training epochs                                                                                         |
| epochs N               | No. of batches during training                                                                                                         | No. of batches during training                                                                                                         |
| batch I                | Sample size                                                                                                                            | Sample size                                                                                                                            |
| S γ                    | No. of inferences over model lifecycle                                                                                                 | No. of inferences over model lifecycle                                                                                                 |
| B T                    | No. of transmitting bits No. of requested bits by the application from the IoT device                                                  | No. of transmitting bits No. of requested bits by the application from the IoT device                                                  |
| B T , L OSI            | No. of bits need to be processed at the l -th layer                                                                                    | No. of bits need to be processed at the l -th layer                                                                                    |
| B T l M l              | No. of nodes in the l -th                                                                                                              | layer                                                                                                                                  |
| L OSI                  | No. of OSI layers                                                                                                                      | No. of OSI layers                                                                                                                      |
| L                      | No. of dense layers                                                                                                                    | No. of dense layers                                                                                                                    |
| C in                   | No. of input channels                                                                                                                  | No. of input channels                                                                                                                  |
| G N                    | No. of intervals in the grid                                                                                                           | No. of intervals in the grid                                                                                                           |
| f N head               | No. of filters                                                                                                                         | No. of filters                                                                                                                         |
| N                      | No. of heads                                                                                                                           | No. of heads                                                                                                                           |
| decoder blocks N I , P | No. of decoder blocks No. of samples used for inference                                                                                | No. of decoder blocks No. of samples used for inference                                                                                |
| M                      | Computational Complexity (CC)                                                                                                          | Computational Complexity (CC)                                                                                                          |
| C M minmax             | CC of decryption CC of min-max scaling                                                                                                 | CC of decryption CC of min-max scaling                                                                                                 |
| DS M norm              | CC of normalization                                                                                                                    | CC of normalization                                                                                                                    |
| DS M GADF DS           | CC of GADF                                                                                                                             | CC of GADF                                                                                                                             |
| M pre                  | CC of preprocessing                                                                                                                    | CC of preprocessing                                                                                                                    |
| M MLP M MLP , FP       | CC of a an MLP CC of a forward propagation of                                                                                          | CC of a an MLP CC of a forward propagation of                                                                                          |
| M CONV                 | CC of a convolutional layer                                                                                                            | CC of a convolutional layer                                                                                                            |
| M POOL                 |                                                                                                                                        |                                                                                                                                        |
| M CNN , FP             | CC of a pooling layer CC of a forward propagation of CNN                                                                               | CC of a pooling layer CC of a forward propagation of CNN                                                                               |
| M KAN                  | CC of a KAN layer CC of B-spline activation function across all input elements                                                         | CC of a KAN layer CC of B-spline activation function across all input elements                                                         |
| M NLF M                | CC of a forward propagation of KAN                                                                                                     | CC of a forward propagation of KAN                                                                                                     |
| KAN , FP M ATT         | CC of an attention block                                                                                                               | CC of an attention block                                                                                                               |
| M TR M TR , FP         | CC of the decoder in a transformer CC of a forward propagation of a transformer                                                        | CC of the decoder in a transformer CC of a forward propagation of a transformer                                                        |
| M model , tot          | CC of training a model Power                                                                                                           | CC of training a model Power                                                                                                           |
| P T                    | Transmitting power                                                                                                                     | Transmitting power                                                                                                                     |
| P R P IoT , cycle      | Receiving power Power consumption per processing                                                                                       | Receiving power Power consumption per processing                                                                                       |
| P Gateway , cycle      | cycle (IoT device) Power consumption per processing cycle (Gateway)                                                                    | cycle (IoT device) Power consumption per processing cycle (Gateway)                                                                    |
| P pre                  | Power consumption of the processing unit                                                                                               | Power consumption of the processing unit                                                                                               |
| E T E R                | Energy Consumption (EC) EC of transmitting data from IoT device to the reception side EC of receiving the signal at the reception side | Energy Consumption (EC) EC of transmitting data from IoT device to the reception side EC of receiving the signal at the reception side |
|                        | EC of decryption                                                                                                                       | EC of decryption                                                                                                                       |
| E C E R , tot E DC     | EC of receiving the signal and decryption                                                                                              | EC of receiving the signal and decryption                                                                                              |
| E                      | EC of the data collection component EC per bit of the data collection component                                                        | EC of the data collection component EC per bit of the data collection component                                                        |
| DC , b E               |                                                                                                                                        |                                                                                                                                        |
| pre E train            | EC of preprocessing EC of model training                                                                                               | EC of preprocessing EC of model training                                                                                               |
| E eval                 | EC of model evaluation                                                                                                                 | EC of model evaluation                                                                                                                 |
| E inf                  | EC of inference                                                                                                                        | EC of inference                                                                                                                        |
| E D                    |                                                                                                                                        |                                                                                                                                        |
| E D , b E inf , p      | EC of developing the model EC per bit of developing the EC of the inference process                                                    | EC of developing the model EC per bit of developing the EC of the inference process                                                    |
| E inf , p , b          | model                                                                                                                                  | model                                                                                                                                  |
| eCAL abs               | EC per bit of the inference process EC over the lifetime of a model in the AIoT system                                                 | EC per bit of the inference process EC over the lifetime of a model in the AIoT system                                                 |
| eCAL                   | EC per bit over the lifetime of a model in the AIoT system                                                                             | EC per bit over the lifetime of a model in the AIoT system                                                                             |
| T T T pre              | Time Transmitting Time Executing time for preprocessing                                                                                | Time Transmitting Time Executing time for preprocessing                                                                                |
|                        | Parameters                                                                                                                             | Parameters                                                                                                                             |
| f γ l                  | Bit precision Scaling factor between DP and CP                                                                                         | Bit precision Scaling factor between DP and CP                                                                                         |
| γ v                    | overheads Scaling factor for virtualization                                                                                            | overheads Scaling factor for virtualization                                                                                            |
| β OH dp ,l             | Split ratio between training and evaluating data DP overhead of the l -th OSI layer                                                    | Split ratio between training and evaluating data DP overhead of the l -th OSI layer                                                    |
| OH cp ,l               | CP overhead of the l -th OSI layer                                                                                                     | CP overhead of the l -th OSI layer                                                                                                     |
| RR dp ,l RR            | DP retransmission rate of the l -th OSI CP retransmission rate of the l -th OSI                                                        | DP retransmission rate of the l -th OSI CP retransmission rate of the l -th OSI                                                        |
| cp ,l R T              | layer Transmitting rate                                                                                                                | layer Transmitting rate                                                                                                                |
|                        | layer                                                                                                                                  | layer                                                                                                                                  |
| R R B                  | Receiving rate Batch size                                                                                                              | Receiving rate Batch size                                                                                                              |
|                        | Theoretical peak performance of a processing unit Computational power of a processor                                                   | Theoretical peak performance of a processing unit Computational power of a processor                                                   |
| PU performance M PU I  | Input tensor size (height)                                                                                                             | Input tensor size (height)                                                                                                             |
| r I c                  | Input tensor size (width)                                                                                                              | Input tensor size (width)                                                                                                              |
| K r                    | Kernel (filter) height                                                                                                                 | Kernel (filter) height                                                                                                                 |
| K c                    | Kernel (filter) width                                                                                                                  | Kernel (filter) width                                                                                                                  |
| P r                    | Padding size (height)                                                                                                                  | Padding size (height)                                                                                                                  |
| P c                    | Padding size (width) Stride value (height)                                                                                             | Padding size (width) Stride value (height)                                                                                             |
| S r                    | Stride value (width)                                                                                                                   | Stride value (width)                                                                                                                   |
| S c                    | Spline order                                                                                                                           | Spline order                                                                                                                           |
| K                      | Context length                                                                                                                         | Context length                                                                                                                         |
| C                      |                                                                                                                                        |                                                                                                                                        |
| N embed FFS            | Size of the embedding Feed forward size                                                                                                | Size of the embedding Feed forward size                                                                                                |

the data flow, highlighted with blue in Fig. 1, across the abovementioned components and derive the computational complexity and subsequently the energy consumption, highlighted with purple, in each component. For instance, in Section IV, we derive E T for the data collection component. With this formalism in place, Section VIII is concerned with deriving eCAL.

For the sake of clarity and convenience, we summarize the notations used throughout this paper in Table II.

## IV. ENERGY COST OF DATA COLLECTION

In this section, we derive the energy cost of data collection in an AI-powered wireless system. To calculate such cost, we first determine the number of bits that need to be transmitted from an IoT device to the server via wireless technology (uplink). Then, we calculate the energy consumption of both transmitting and receiving the data based on the given transmitting/ receiving power and corresponding transmission/ receiving rate. Finally, we derive the total energy consumption for collecting data from the IoT device. Note that the energy consumption of wired communication is considered negligible in comparison to the wireless part [31]. Therefore, this work focuses solely on the communication from the IoT device to the access point (AP) via wireless technologies, however the open source calculator can be extended to also include other communication segments and technologies.

## A. OSI Overhead

In OSI architecture, each layer introduces its own overhead, which includes various fields in the headers in the Data Plane (DP) and additional control and set-up messaging from the Control Plane (CP). Additionally, in some of the layers, such as link (e.g., MAC) and transport (e.g., TCP) layers, errors or packet loss are internally treated, potentially leading to retransmissions. While each protocol and every layer enables a wide selection of settings, we devise the following generalized theoretical formulation for computing the size of the transmitted data at the physical layer ( B T ):

$$B _ { \text{T} } & \left [ b \right ] = \left \lceil B _ { \text{T,LOSI} } \prod _ { l = 1 } ^ { L _ { \text{OSI} } } \underbrace { \left ( R R _ { \text{DPP,l} } \cdot \left ( 1 + O H _ { \text{DPP,l} } \right ) } _ { \text{contribution from DP} } } _ { \text{contribution from CP} } \quad & \text{refras} \\ + & \underbrace { R R _ { \text{CPP,l} } \cdot \gamma _ { l } \cdot O H _ { \text{CPP,l} } \right ) } _ { \text{contribution from CP} } \right ], \text{where } L _ { \text{S} } = 7, \quad ( 2 ) \text{ equal } \dot { \text{show} } \\ \text{where } R _ { \text{--} } \, + \quad \text{rencont} \, the \, \text{annification layer data } \, \text{i.e. } \, \text{i.e. } \, \text{i.e. } \, \text{i.e. }$$

where B T L , OSI represents the application layer data (i.e. IoT device measurements) sent from the IoT device, measured in bits. It is calculated as the product of the bit precision ( f ), the number of samples ( N S ) and the sample size ( I S ) in floating-point format, i.e., B T L , OSI = fN I S S . Typically, f equals 32 bits for single precision (float) or 64 bits for double precision format. To be able to send/receive data, in each of the seven OSI layers, represented by the product in Eq. 2, a number of data and control plane operations take place as expressed in the respective terms. The application data, B T L , OSI , is processed and is added some overhead OH DP ,l ,

Fig. 2: Total transmitting bits ( B T ) versus retransmission rate with γ l = 0 5 . , OH DP ,l = 10% and OH CP ,l = 5% .

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00540v2-figure-74779634-b885-478d-9117-4fc9c02b5d39.jpg)

in some layers it is encrypted (i.e. application, presentation and/or transport layers) and in some layers it may also require retransmissions RR DP ,l . Moreover, some layers, such as the network layer, incur additional overhead required from signaling to maintain routing tables. This is reflected in the second term of Eq. 2 through the respective CP overheads OH CP ,l , retransmissions RR CP ,l and the γ l scaling factor that adjusts for the relative contribution of the CP overhead compared to the DP counterpart at the l -th layer.

The eCAL calculator implements the computation from Eq. 2 and enables the configuration of the application data size, retransmission rates, overheads, and scaling factor. Furthermore, it enables plugging custom protocol implementations for each of the layers, based also on the existing literature [32][34], so that the community can also instantiate eCAL for a specific well-defined set of protocols in addition to the generalized theoretical approach adopted in this paper.

To illustrate the behavior of Eq. 2, we first consider a scenario where a single IoT device transmits N S of doubleprecision samples to the application accompanied by DP and CP overheads while altering the retransmission rate. In particular, we consider that each layer in the OSI model introduces a fixed overhead percentage for the DP and CP, with 10 % and 5 %, respectively. Furthermore, the relative contribution of CP overhead is scaled by a factor of 0 5 . , i.e., γ l = 0 5 . . The retransmission rates for both DP and CP are assumed to be equal and vary from 1 to 2 in increments of 0 04 . . This is shown in Fig. 2. The results show the retransmission rate has a significant impact on total transmitting bits. For example, when N S = 256 samples, there is a growth of 2 3 . , 17 , and 128 times with retransmission rates equal to 1 125 . , 1 5 . , and 2 , respectively, compared to scenarios with no retransmission.

## B. Energy Estimation

To estimate the energy consumption of the data collection component ( E DC ), which is the sum of the energy of transmitting ( E T ) and receiving the data ( E R ) introduced in the physical layer (PHY), as well as the energy consumption of computation in the DP and CP across the various layers of OSI ( E C ), we propose the following analytical expression:

Fig. 3: Energy consumption of data collection ( E DC ) (log scale) with its components and its components, including transmission energy ( E T ), receiving energy ( E R ), and computational energy ( E C ) with N S = 256 , RR DP ,l = RR CP ,l = 1 , OH DP ,l = 10% , OH CP ,l = 5% , and γ l = 0 5 . .

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00540v2-figure-41db6801-af62-4b25-9b75-ccae5b32c747.jpg)

$$E _ { \text{DC} } = \quad \underbrace { \frac { P _ { \text{T} } [ W ] } { R _ { \text{T} } [ b / s ] } \cdot B _ { \text{T} } [ b ] } _ { \text{R} } \quad + \quad \underbrace { \frac { P _ { \text{R} } [ W ] } { R _ { \text{R} } [ b / s ] } \cdot B _ { \text{T} } [ b ] } _ { \text{T} } \quad \begin{array} { c } \stackrel { \dots } { \text{it } \colon } \\ \text{inc} \end{array}.$$

︸ ︷︷ ︸ PHY layer transmitting energy ( E T ) ︸ ︷︷ ︸ PHY layer receiving energy ( E R )

$$+ \sum _ { l = 2 } ^ { L _ { \text{OSI} } } ( \underbrace { B _ { \text{T} _ { l } } [ b ] \cdot N _ { \text{IoT,cycle,l} } [ c / b ] \cdot P _ { \text{IoT,cycle} } [ J / c ] } _ { \text{from APP to MAC layer computational energy at the IoT device} } [ J / c ] } _ { \text{$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$\quad$$$

$$+ \, B _ { \text{T} _ { l } } \, \left [ b \right ] \cdot N _ { \text{gateway,cycle,l} } \, \left [ c / b \right ] \cdot P _ { \text{gateway,cycle} } \, \left [ J / c \right ] \right ) \quad d a t a$$

︸

︷︷ from MAC to APP layer computational energy at the gateway

︸

$$& = E _ { \mathrm T } + E _ { \mathrm R } + E _ { \mathrm C }, & \quad \text{and re} \\. & \quad \text{-} \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \colon$$

where P T and R T represent the transmitting power and data rate, respectively, while P R and R R are the receiving power and data rate. Furthermore, B T ,l denotes the total number of the bits that need to be processed at the l -th layer in the OSI model. Additionally, we define the computational parameters at the IoT device as N IoT cycle , ,l representing the number of processing cycles per bit at the l -th layer, and P IoT cycle , , denoting the power consumption per cycle. Similarly, at the gateway, these parameters are defined as N gateway cycle , ,l and P gateway cycle , .

Computing exact values for E DC for a broad range of B T l , N , cycle ,l and P , cycle is challenging due to the overall complexity of computing systems. For instance, for processing the same amount of data, N , cycle ,l varies across microprocessors while also not having a linear dependency with the data size [35]. Therefore, the values for B T l and N , cycle cannot be chosen arbitrarily. Next, P , cycle also depends on the microprocessor. However, in Fig. 3, we illustrate the energy consumption of E DC with a selected set of values to provide an intuition on the contribution from each term. More specifically, we analyze a scenario where N S = 256 , and we consider two sets of parameters for computing transmitting and receiving energy, i.e. E T and E R , alongside two different settings for calculating the computational energy consumption ( E C ). The results show that while transmission energy ( E T ) remains the dominant contributor in most cases, the computational energy consumption ( E C ) is not negligible and, in some configurations, constitutes a significant portion of the total energy consumption of data collection. This is particularly the case when the hardware is less optimized for the tasks [35], leading to higher cycles per bit ( N , cycle ,l ) and power per cycle ( P , cycle ). For example, when N , cycle = 100 and P , cycle = 2 × 10 -10 , E C is 7 55 . × 10 -3 [ J ] , which represents approximately 10% and 95% of the total energy cost in two different PHY layer configurations. On the other hand, when N , cycle ,l = 50 and P , cycle = 10 -10 , E C is 6 61 . × 10 -3 [ J ] , contributing to approximately 5% and 90% of the total energy consumption in data collection component.

## V. ENERGY COST OF DATA PREPROCESSING

At the server side, data is first stored on a hard drive, subsequently preprocessed, and then used for the ML model development or inference, but we are only focusing on the latter in Section VII. While storage is generally energyefficient and in terms of energy costs relatively negligible, it can become significant when handling large volumes of incoming data or when utilizing virtualized hosted storage with encryption in an edge-cloud environment [36]. In the following, we consider the dataset with N S samples collected via the wireless network, as depicted in Fig. 1 and discussed in Section IV.

In this study, we consider data preprocessing consisting of data cleaning , and data transformation steps, while leaving other more complex approaches, such as feature engineering and feature for future work. In the data cleaning step, the task of preprocessing is to identify and remove invalid samples from a list or array which incurs zero floating-point operations (FLOPs) since no floating points additions and multiplications are performed, only comparisons and memory operations. In the data transformation step, the preprocessing needs to determine the complexity of data transformation ( M DS ). We provide three examples, i.e., min-max scaling, normalization, and Gramian Angular Difference Field (GADF) process which represent typical approaches for data transformation.

a) Min-max scaling: Min-max scaling is a data preprocessing technique that transforms numerical data to fit within the [0,1] interval by scaling each value relative to the minimum and maximum values in the dataset. For each feature, the transformation subtracts the minimum value and divides by the range, ensuring that the lowest value becomes 0 and the highest value becomes 1, with all other values scaled proportionally between them. While finding the minimum and maximum values requires computational resources for comparisons, these operations do not contribute to the FLOPs count. The scaling itself requires calculating the range (one FLOP), followed by subtracting the minimum and dividing by the range for each sample ( 2 · N S · I S FLOPs). Therefore, the total number of FLOPs ( M minmax DS ) for min-max scaling can be expressed as:

$$M _ { \text{DS} } ^ { \min \max } = 2 \cdot N _ { \text{S} } \cdot I _ { \text{S} } + 1, \text{ \quad \ \ } ( 4 )$$

b) Normalization: Standard score, also known as z-score normalization, represents a more computationally intensive preprocessing technique compared to min-max scaling. The process involves three distinct computational steps. First, calculating the population mean requires N S · I S FLOPs, as we sum all values and divide by the total number of samples. Second, computing the population standard deviation demands 3 · N S · I S +1 FLOPs, which encompasses calculating squared differences from the mean, their sum, and the final square root operation. Finally, the standardization transformation itself necessitates 2 · N S · I S FLOPs, as each data point must be centered by subtracting the mean and scaled by dividing with the standard deviation. The total computational cost for normalization, expressed in FLOPs ( M norm DS ), is therefore:

$$M _ { \text{DS} } ^ { \text{norm} } = 6 \cdot N _ { \text{S} } \cdot I _ { \text{S} } + 1 \text{ \quad \ \ } ( 5 )$$

c) Gramian Angular Difference Field: To showcase a more computationally expensive preprocessing operation, we implemented a calculator to compute the energy cost of the GADF transformation [37]. This transformation converts time series data into an image representation that captures temporal correlations between each pair of values in the time series. The GADF computation consists of several sequential steps, each contributing to the overall computational complexity and, consequently, the energy consumption. The first step requires scaling the time series data using min-max scaling, which ensures values fall within the [-1, 1] interval. Following the scaling, the data undergoes conversion to polar coordinates, where the radius is calculated using the time stamp values, and the angular coordinates are computed through an inverse cosine operation. The final step involves constructing the GADF matrix through pairwise angle differences, resulting in an I S × I S matrix where I S represents the length of the input time series.

The computational complexity of GADF can be broken down into its constituent operations. The initial min-max scaling requires 2 · N S · I S +1 FLOPs as previously discussed, however in these case we have to multiply the number of samples with the length of the input time series ( I S ) as we are dealing with a series of values in a single sample increasing the FLOPs. The polar coordinate transformation and the construction of the GADF matrix itself, requires (5 · I S + I 2 S ) · N S FLOPs . Therefore, the total number of FLOPs ( M GADF DS ) for the GADF transformation can be expressed as:

$$M _ { \text{DS} } ^ { \text{GADF} } = ( 2 \cdot N _ { \text{S} } \cdot I _ { \text{S} } + 1 ) + ( 5 \cdot I _ { \text{S} } + I _ { \text{S} } ^ { 2 } ) \cdot N _ { \text{S} } \quad ( 6 ) \quad \text{To } i$$

This quadratic computational complexity makes GADF significantly more energy-intensive compared to simpler preprocessing operations like standardization or min-max scaling, particularly for longer time series.

As a result, the total number of FLOPs for preprocessing can be expressed as:

$$M _ { \text{pre} } = \begin{cases} 2 \cdot N _ { S } \cdot I _ { S } + 1, \min \text{-max scaling}, & \text{consist} \quad & \text{with $G_{i}$} \\ 6 \cdot N _ { S } \cdot I _ { S } + 1, \text{normalization}, & \text{(7)} \quad & \text{due to} \\ 2 \cdot N _ { S } \cdot I _ { S } + 1 + ( 5 \cdot I _ { S } + I _ { S } ^ { 2 } ) \cdot N _ { S }, G \text{ADF}. & \text{cost of} \end{cases}$$

Fig. 4: Energy consumption ( E pre ) of different sample size ( I S ) across different preprocessing techniques.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00540v2-figure-51cd89c9-5cf5-4739-af94-c8568ad2b2d2.jpg)

Fig. 5: Energy consumption ( E pre ) of fixed data point ( N S · I S ) across different preprocessing techniques and sample size ( I S ).

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00540v2-figure-0d8ecb1b-1ea0-4752-8ecb-4f166f22c90c.jpg)

With the computing complexity in FLOPs formalized, we can obtain total energy consumption ( E pre ) of the preprocessing ( E pre ) by utilizing the power consumption of the processing unit ( P pre ) and the executing time ( T pre ). Thus, it is expressed as follows:

$$E _ { \text{pre} } [ J ] = P _ { \text{pre} } [ W ] \cdot T _ { \text{pre} } [ s ],$$

where T pre can be calculated as the ratio between FLOPs needed for preprocessing and the computational power of a processor ( M PU ):

$$T _ { \text{pre} } [ s ] = \frac { M _ { \text{pre} } [ F L O P s ] } { M _ { \text{PU} } [ F L O P s / s ] }. \quad \quad ( 9 )$$

To illustrate the energy cost in various N S and I S configurations, consider the preprocessing energy cost E pre on a CPU with M PU = 10 GFLOPs/s and power consumption of P pre = 140 W, shown in Fig. 4 and Fig. 5. More specifically, Fig. 4 demonstrates how the energy consumption of preprocessing methods for a fixed number of samples ( N S = 256 ) increases with the sample size ( I S ). This trend is consistent across min-max scaling, normalization, and GADF, with GADF exhibiting the most pronounced energy growth due to its quadratic dependency on I S , whereas, the energy cost of the other two techniques grows linearly. In addition,

Fig. 5 focuses on the energy consumption of the preprocessing techniques while maintaining a constant total amount of data ( N S · I S = 100 000 , ). This constraint ensures that as the sample size ( I S ) increases, the number of data samples ( N S ) decreases proportionally. Consequently, the energy consumption for minmax scaling and normalization remains constant across different configurations, as their computational complexity scales linearly with N S · I S . In contrast, GADF exhibits a quadratic (with a linear component) growth in energy consumption due to the I 2 S term in its computational complexity, shown in Eq. 7, which is also confirmed by the O N ( S · I 2 S ) shown in the gray dotted line. These results highlight the tradeoffs in selecting a preprocessing method based on both the energy efficiency requirements and the specific input data characteristics required by the AI/ML technique.

## VI. ENERGY COST OF TRAINING

In order to train a model, training data and a neural network architecture are needed. The training data typically goes through a preprocessing phase as discussed in Section V and is then split into certain ratios for training and evaluation with a β split ratio as:

$$N _ { S } = \underbrace { \beta \cdot N _ { S } } _ { N _ { S, T } } + \underbrace { ( 1 - \beta ) \cdot N _ { S } } _ { N _ { S, E } }, \quad \quad ( 1 0 ) \quad \text{when} \quad \text{and} \ c$$

where N S T , and N S E , represent the number of samples for training and evaluation, respectively. While there are many neural network architectures available for developing AIoT models, in this work we focus on four representative types, namely, Multilayer Perceptrons (MLPs) [11], Convolutional Neural Networks (CNNs) [38], Kolmogorov-Arnold Networks (KANs) [39] and transformers [40]. Training a neural network involves two fundamental phases: forward propagation and backward propagation depicted with blue and red in Fig. 6 on an example MLP. During the forward propagation, input data passes through the network layers. Each layer performs specific tasks, such as computing a linear combination of weights and biases, followed by applying activation functions in dense (fully connected) layers and applying convolutional operations and pooling mechanisms in convolutional layers and pooling layers, respectively. At the end of this phase, the loss is calculated by comparing the predicted outputs to the actual target values. Subsequently, the model performs backward propagation, where it calculates gradients of the loss with respect to each parameter, updates these gradients in reverse through the layers, and adjusts the weights and biases accordingly. This process completes one epoch. Consequently, we can calculate the energy costs associated with forward and backward propagation, leading to an understanding of the overall energy consumption for training, model evaluation, and subsequent inference. In the following sections, we showcase the computational complexity of training and evaluating the aforementioned Neural Networks (NNs).

Fig. 6: A fully connected MLP architecture.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00540v2-figure-5b02a2a6-43b4-41b3-b1eb-af23d7891291.jpg)

A. Computational Complexity of MLP, CNN, KAN and transformer a) MLPs: Assuming an MLP architecture with L dense layers as depicted in Fig. 6, where L = K +2 , the total number of FLOPs for the forward propagation of a single input sample can be expressed as:

$$\text{ation} \\ \text{--} \sim.$$

where the first term in the sum represents one multiplication and one addition ( = 2 FLOPs) related to the weight and bias corresponding to the edges times the number of nodes in that layer ( = M l -1 · M l ), while the second term represents the summation of all the values of the incoming edges and the application of the activation function ( = 2 FLOPs) times the number of nodes in that layer ( M l ).

In general, however, training consists of multiple epochs and batches. Therefore, the total complexity of forward propagation is:

$$M _ { \text{MLP}, \text{FP} } & = N _ { \text{epochs} } \cdot N _ { \text{batch} } \cdot B \cdot M _ { \text{MLP} } \\ & = N _ { \text{epochs} } \cdot N _ { \text{S}, \text{T} } \cdot M _ { \text{MLP} }.$$

b) CNNs: Typical CNNs consist of three types of layers: dense layers, convolutional layers, and pooling layers. For a convolutional layer, with input tensor of size I r × I c × C in , the corresponding computational complexity is delivered as:

$$\text{ayers} \text{$(abs,} \\ \text{ts to} \\ \text{forms} \\ \text{f the} \\ \text{nts in} \\ \text{jases} \\ \text{entity}, \quad \underbrace { M _ { \text{CONV} } = \underbrace { \left ( \frac { I _ { r } - K _ { r } + 2 P _ { r } } { S _ { r } } + 1 \right ) } \cdot \underbrace { \left ( \frac { I _ { c } - K _ { c } + 2 P _ { c } } { S _ { c } } + 1 \right ) } \, \underbrace { \text{Output width} } \, \underbrace { \text{Output width} } \, \underbrace { \text{$(13)} \\ \text{$(\text{$\text{$out}$)} \, \underbrace { \left ( C _ { i n } \cdot K _ { r } \cdot K _ { c } + 1 \right ) \cdot \underbrace { N _ { r } } \,, } \, \text{No. of filters} \,, \\ \text{of the} \quad \text{where $K$} \text{ and $K$} \text{ are the kernel (filter) height and width}.$$

where K r and K c are the kernel (filter) height and width. Moreover, P r and P c are the padding sizes for height and width. Furthermore, S r and S c are the stride values for height and width. Finally, N f is the number of filters in the layer. For down-sampling the input tensor data, a pooling layer is

needed, and the number of FLOPs of such layer is expressed as:

$$M _ { \text{POOL} } = \underbrace { \left ( \frac { I _ { r } - K _ { r } } { S _ { r } } + 1 \right ) } _ { \text{Output height} } \cdot \underbrace { \left ( \frac { I _ { c } - K _ { c } } { S _ { c } } + 1 \right ) } _ { \text{Output width} } \cdot \underbrace { C _ { \text{in} } } _ { \text{Input channels} } \cdot \underbrace { \text{(and)} } _ { \text{Input channels} } \cdot \underbrace { \text{the size} } _ { \text{The cor} } \\ \text{Thus, the total number of FLOPs for the forward proposals}$$

Thus, the total number of FLOPs for the forward propagation of a single input sample in a CNN can be calculated by summing the contributions across all layer types [41]. As a result, the total FLOPs of a CNN is:

$$M _ { \text{CNN}, \text{FP} } & = N _ { \text{epochs} } \cdot N _ { \text{S}, \text{T} } \cdot \\ & \left ( \sum _ { l = 1 } ^ { L _ { c } } M _ { \text{CONV} } + \sum _ { l = 1 } ^ { L _ { p } } M _ { \text{POOL} } + M _ { \text{MLP} } \right ). \quad ( 1 5 ) \quad \text{For mo} \\ & \quad \text{dense } l _ { \text{more } g _ { \text{c} } }$$

c) KANs: KANs leverage the Kolmogorov-Arnold representation theorem to decompose complex multivariate functions into univariate sub-functions. This is achieved through the use of learnable B-spline activation functions and shortcut paths [39]. Unlike MLPs, which apply fixed activation functions to nodes, KANs feature learnable activation functions on edges, providing greater flexibility and interpretability. According to [42], the computational complexity of a single KAN layer can be computed as:

$$K A N & \, \text{layer can be computed as} \, \stackrel { \sim } { \sim } \, \stackrel { r \sim } { \sim } \, \stackrel { r \sim } { \sim } \, \text{we can} \\ M _ { \text{KAN} } = \sum _ { l = 1 } ^ { L - 1 } ( \underbrace { M _ { \text{NLF} } \cdot M _ { l - 1 } } _ { \text{From B-spline} } } _ { \text{From B-spline} } \\ + & \, ( M _ { l - 1 } \cdot M _ { l } ) \cdot \underbrace { [ 9 K \cdot ( G + 1. 5 K ) + 2 G - 2. 5 K + 3 ] ) } _ { \text{Input and output} } ( \underbrace { 1 6 } _ { \text{B-spline and grid} } ( 1 6 ) \quad \, \text{We ci process} \\ = & \, \sum _ { l = 1 } ^ { L - 1 } ( M _ { \text{NLF} } \cdot M _ { l - 1 } + ( M _ { l - 1 } \cdot M _ { l } ) \cdot M _ { \text{B} } ), \\ \text{where } M _ { \text{NLF} } \text{ denote as the number of FLOPs contributed} \quad \, r _ { \text{SPED} } \,.$$

where M NLF denote as the number of FLOPs contributed from the B-spline activation function across all input elements. Additionally, the second term accounts for the computational costs arising from the combination of input and output dimensions ( M l -1 and M l ) with the operations performed by the B-spline transformation. These operations include the iterative evaluation of spline basis functions and the associated transformations, governed by the spline order ( K ) and the number of intervals in the grid ( G ). The total FLOPs of a KANs network is calculated as the summary of all KAN layers similarly as in the case of MLPs in Eq. 12, where total L layers can be expressed as:

$$M _ { \text{KA} N, \text{FP} } = & N _ { \text{epochs} } \cdot N _ { \text{S}, \text{T} } \cdot M _ { \text{KAN} }. \quad \ \ ( 1 7 ) \ \text{els be}$$

d) Transformer: Based on the attention mechanism, the transformers [40] represent a paradigm shift in deep learning, replacing traditional architectures such as CNN with self-attention mechanisms. While enabling significant breakthroughs in AI through efficient parallel processing and robust modeling of long-range dependencies, they are relatively complex. In addition to attention layers, the transformer architecture also includes MLP, normalization, and positional encoding, making the analytic expression for the FLOP computation intractable. Following existing literature [43] and approximations 3 where C is the context length, N embed is the size of the embedding and N head is the number of heads. The complexity of the decoder in a transformer is then given by:

$$M _ { \text{TR} } = N _ { \text{decoder_blocks} } \cdot ( M _ { \text{ATT} } + \underbrace { 4 \cdot C \cdot N _ { \text{embed} } \cdot F F S } _ { \text{MLP blocks} } ), \quad ( 1 9 )$$

where N decoder blocks and FFS stand for the number of decoder blocks and the feed-forward size.

$$M _ { \text{TR}, \text{FP} } = & N _ { \text{epochs} } \cdot N _ { \text{S}, \text{T} } \cdot M _ { \text{TR} }, \text{ \quad \ \ } ( 2 0 )$$

For more specific cases, such as GPT-like architectures, a final dense layer is also needed. However, this work focuses on a more general formulation of transformer models.

## B. Model Training

To determine the computational complexity of training a model, also the backward propagation needs to be considered. The standard approximation, also confirmed in [44], is that the backward propagation requires two times the floatingpoint operations per cycle per core (FLOPs) of the forward propagation. In particular, it requires matrix multiplications for updating the weights and propagating the gradient. Thus, we can approximate the computing complexity of training a model as:

$$M _ { \text{model,tot} } \approx 3 M _ { \text{model,FP} }. \text{ \quad \ \ } ( 2 1 )$$

We can obtain the energy consumption of the entire training process as:

$$E _ { \text{train} } = \frac { 3 M _ { \text{model,FP} } [ \text{FLOPS} ] } { P U _ { \text{performance} } [ \text{FLOPs} / s / W ] }. \quad ( 2 2 )$$

From Eq. 12, 15, 17, 20, and 22 we can observe that the training data set size scales linearly with energy consumption of the model training, as it does not affect the cost of single forward/backward pass. On the other hand, training the same model on a state-of-the-art GPU with a processing power of 312 [ TFLOPs/s ] and power consumption of 400 [ W ] is approximately 28 times more energy efficient than using a CPU, which operates at 13 824 [ . TFLOPs/s ] and consumes 500 [ W ] .

Fig. 7 illustrates the relationship between the energy consumption of a model and two architectural parameters: layers and nodes in a layer. Naturally, as the architectures of the models become deeper and wider, the computational complexity

3 https://www.gaohongnan.com/playbook/training/how to calculate flops in transformer based models.html, we approach the estimation as follows:

$$\text{nrows.} \\ \text{NTT} = 2 \cdot ( \underbrace { C \cdot N _ { \text{embed} } \cdot 3 \cdot N _ { \text{embed} } } _ { K, Q, V \text{ positional embedding} } + \underbrace { C ^ { 2 } \cdot N _ { \text{embed} } } _ { \text{ Attention scores} } \\ \text{reduce} } + \underbrace { C \cdot N _ { \text{embed} } \cdot N _ { \text{embed} } } _ { \text{Projection} },$$

(a) Energy cost of training a MLP(b) Energy cost of training a CNN(c) Energy cost of training a KAN(d) Energy cost of training a transversus M l and L versus M l and L versus M l and L former versus M l and L

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00540v2-figure-da8b9d71-2dcf-4b26-b857-50e8e114e116.jpg)

Fig. 7: Energy cost of training different models with respect to the number of nodes ( M l ) and layers ( L ).

also increases, and this leads to higher energy consumption. For instance, in the case where M l = 10 and L = 5 , the energy consumption of the transformer, KAN, and CNN models are approximately 173 , 149 , and 81 times greater than that of the MLP model. However, some architectures, such as KANs, are known to perform better with fewer layers [39], therefore it is unlikely to find a KAN with L &gt; 5 . While at a first glance, it seems that MLPs may be significantly more energy efficient than KANs, the actual difference for the same performance will be less prominent for many applications. CNNs, and especially transformers are known to be computationally demanding and subsequently exhibit high energy consumption, while at the same time larger architectures proved better performance in various application areas.

## C. Model Evaluation

The model evaluation starts once training is complete. This functionality tests the performance of the model on a separate dataset. During the evaluation, the model processes the data using forward propagation without any adjustments to its parameters. Therefore, the energy consumption and energy consumption of model evaluation is expressed as:

$$E _ { \text{eval} } = \frac { M _ { \text{model} } \cdot N _ { \text{S,E} } } { P U _ { \text{performance} } }. \quad \quad ( 2 3 ) \quad \text{the } \vdots$$

It is worth noting that the total energy consumption during training and evaluation is significantly influenced by the chosen evaluation strategy. While a simple train-test split requires training the model only once, more robust techniques like k -fold cross-validation necessitate k complete training cycles. For k -fold cross-validation, the total training energy consumption becomes k times the value given in Eq. 22, as the model must be retrained from scratch for each fold. Similarly, techniques such as nested cross-validation or repeated k -fold cross-validation further multiply the required training computations and corresponding energy costs, in which we have also incorporated k -fold cross-validation considerations into our developed tool.

## VII. ENERGY COST OF INFERENCE

The complexity of making an inference depends on the number of FLOPs required for forward propagating with input sample size N I P , , i.e., N inf = M model · N I P , . For example, when we adopt the aforementioned MLP model with N I P , = 51 , the complexity is calculated as N inf = 236 · 51 = 12 036 , FLOPs. Therefore, the energy consumption of the forward propagation and the corresponding inference can be calculated as:

$$E _ { \inf } = \frac { M _ { \text{model} } \cdot N _ { \text{I,P} } } { P U _ { \text{performance} } }. \quad \quad \quad ( 2 4 )$$

We can then observe from Eq. 23 and 24 that the cost of inference is the same as in evaluation when N S E , is equal to N I P , . Thus, we can conclude that the computational complexity between finishing one training and inference is on the magnitude of 3 · N epochs · N S T , /N I P , . This indicates that the training process involves significantly more computational operations, particularly when the number of epochs or N S T , &gt;&gt; N I P , . On the other hand, the energy consumption per bit depends on the model complexity and hardware capability. Moreover, the factor of three in the energy consumption per bit during training versus inference highlights the additional computational load inherent to training.

## VIII. ECAL: THE ENERGY COST OF AIOT LIFECYCLE

In this section, we describe the end-to-end energy consumption of an AIoT system for making inferences with the aforementioned neural network architectures. First, we consider the energy consumption of developing the model illustrated on the left in Fig. 1 as follows:

$$E _ { \text{D} } = E _ { \text{DC} } + E _ { \text{pre} } + E _ { \text{train} } + E _ { \text{eval} }, \quad \ \ ( 2 5 )$$

and the energy consumption per bit that can be expressed as:

$$E _ { \text{D}, \text{b} } = \frac { E _ { \text{D} } } { f N _ { \text{S} } I _ { \text{S} } }.$$

To compare the energy consumption of all the data manipulation components required for developing and deploying an AIoT system, we assume that the models are trained and evaluated using 5-fold cross validation (as discussed in Section VI), they require 256 samples with input size of 10 , i.e., N S · I S = 2560 , collected from an IoT device over a wireless network configured with the following parameters: P T = 200 [ mW ] , P R = 5 [ mW ] , R T = R R = 10 [ Mbps ] , N , cycle ,l = 100 [ c/b ] , and P , cycle = 10 -10 [ W/c ] . We also assume the link between the IoT device and the server is

TABLE III: Total energy consumption of developing ( E D ) for different models, with contributions of individual datamanipulating components.

| Component                                         | MLP                                                                                                               | CNN                                                                                                       | KAN                                                                                                   | Transformer                                                                                             |
|---------------------------------------------------|-------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------|
| E DC [ J ] E pre [ J ] E train [ J ] E eval [ J ] | 8 . 44 × 10 - 2 ( 89 . 12% ) 1 . 54 × 10 - 6 ( 0 . 00% ) 1 . 03 × 10 - 2 ( 10 . 86% ) 1 . 72 × 10 - 5 ( 0 . 02% ) | 8 . 44 × 10 - 2 ( 12 . 37% ) 1 . 54 × 10 - 6 ( 0 . 00% 5 . 97 × 10 - 1 ( 87 . 9 . 95 × 10 - 4 ( 0 . 15% ) | 8 . 44 × 10 - 2 ( 6 . 31% ) 1 . 54 × 10 - 6 ( 0 . 00% ) 1 . 25 ( 93 . 53% ) 2 . 08 × 10 - 3 ( 0 . 16% | 8 . 44 × 10 - 2 ( 3 . 72% ) 1 . 54 × 10 - 6 ( 0 . 00% ) 2 . 18 ( 96 . 12% ) 3 . 64 × 10 - 3 ( 0 . 16% ) |
| D [ J                                             |                                                                                                                   | )                                                                                                         | )                                                                                                     |                                                                                                         |
|                                                   |                                                                                                                   | 48% )                                                                                                     |                                                                                                       |                                                                                                         |
| E ]                                               | 9 . 47 × 10 - 2                                                                                                   | 6 . 83 × 10 - 1                                                                                           | 1 . 34                                                                                                | 2 . 27                                                                                                  |

stable, and there is no retransmission needed. Additionally, each layer introduces DP and CP overhead of 10% and 5% , respectively. Furthermore, the normalization is utilized as data transformation procedure, as discussed in Section V. Finally, model training requires 10 epochs ( N epochs = 10 ) and utilizes 5 -fold validation. The energy consumption of this example is shown in Table III and reveals the contribution of each component to the total energy cost of developing models. Specifically, the data collection dominates the total energy cost for developing a very simple model based on a 3 layer 10 nodes/per layer MLP architecture. For the same parameters, the relative contribution of the data collection decreases for the KAN, CNN and transformer architectures making the energy cost of training the primary contributor. The eCAL calculator enables changing the hyperparameters of the four architectures while also adding well known CNN architectures such as ResNets [45], VGGs [46] or transformer based such as Baichuan 2 [47].

Once the model is developed, it is packaged and deployed in the AIoT system, and it can produce inference in the form of continuous or discrete outputs. During its operational lifecycle, the model will be presented with some input data and requested to produce the corresponding inference. Thus, we can express the associated energy cost as:

$$E _ { \inf, p } = E _ { \text{DC} } + E _ { \text{pre} } + E _ { \inf }. \quad \quad ( 2 7 ) \quad \ \ 1 0$$

This means that we consider the energy cost of inference before the current model needs to be updated. The corresponding energy per bit of inference, once deployed, is calculated as:

$$E _ { \inf, p, b } = \frac { E _ { \inf, p } } { f N _ { \mathbf I, P } I _ { S } }. \quad \quad ( 2 8 ) \quad \text{when}$$

Next, to obtain the entire energy cost over the lifetime of a model in the AIoT system, also referred to as eCAL abs (measured in [ J ]), which is defined as the sum of energy consumed during the model development, E D and the energy required for performing an inference E inf p , multiplied by the number of times γ the inference is performed with the currently deployed model. In addition, as edge cloud systems may use virtualization in developing and operating the models, we enable considering the impact of such technologies through a factor. As a result, eCAL abs can be expressed as:

$$\ e C A L _ { \text{abs} } = \left ( 1 + \gamma _ { \nu } \right ) \left ( E _ { \text{D} } + \gamma E _ { \text{inf,p} } \right ), \quad \left ( 2 9 \right ) \text{ that } \tt t \tt }$$

Fig. 8: Comparison of energy consumption per bit in different data manipulation components over the lifecycle of the AIoT model.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00540v2-figure-95dceb56-cf18-4ca9-9ae8-74087f37694c.jpg)

Fig. 9: Energy cost of AIoT lifecycle ( eCAL ) for different models over the number of inferences ( γ ), log scale on both axes.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00540v2-figure-7fc8c746-f7c4-4369-ad17-2e81b0946f08.jpg)

where γ v represents the contribution from the virtualization, and γ ∈ [0 , ∞ ) .

Finally, the proposed metric computing the corresponding energy consumption per bit over the lifecycle of the currently deployed AIoT model can be expressed as:

$$\ e C A L = \frac { \ e C A L _ { \text{abs} } } { f I _ { \text{S} } ( N _ { \text{S} } + \gamma N _ { \text{I}, \text{P} } ) }. \quad \quad ( 3 0 )$$

To further quantify this relationship, we first compare the outcomes of Eq. 26, Eq. 28, and Eq. 30, depicted in Fig. 8, where we assume the virtualization is contributing additional 10% to the total energy cost ( γ v = 10% ). The figure shows that the energy consumption per bit of developing a 3 layer

-10 node MLP is 2 41 . × 10 -6 [ J/b ] , which is approximately 3 7 . times higher than the energy consumption per bit of one inference ( E inf p b , , = 6 57 . × 10 -7 [ J/b ] ). Furthermore, we can observe that the energy consumption per bit decreases with more inferences, dropping from 2 11 . × 10 -6 [ J/b ] for 100 inferences to 7 73 . × 10 -7 [ J/b ] for 1000 inferences, which is a 2 73 . times of improvement on energy efficiency. Moreover, we can observe that models with architectural blocks that require higher computational complexity need significantly more energy per bit during development compared to simpler models and their energy cost per bit of one inference. For instance, the transformer consumes 3 74 . × 10 -4 [J / b] , which means it is nearly 165 and 20 7 . times less efficient than the MLP in making one inference. This is also confirmed in Fig. 9. As the number of inferences γ in the current model increases, the energy efficiency of the AIoT lifecycle improves.

## IX. CONCLUSIONS AND FUTURE WORK

In this work, we proposed a novel metric, namely, eCAL. Unlike the traditional metrics that focus only on the energy required for transmitting, compute infrastructure or for AI models, eCAL can capture the overall energy cost of generating an inference in an AIoT system during the entire lifecycle of a trained model. We propose a detailed methodology to determine the eCAL of an AIoT system by breaking it down into various data manipulation components, such as data collection, preprocessing, training, evaluation, and inference, and analyzing the complexity and energy consumption of each component. The proposed metric demonstrates that the more a model is utilized, the more energy-efficient each inference becomes. For example, considering a simple MLP architecture, the energy consumption per bit for 100 inferences is 2 73 . times higher than for 1000 inferences. Furthermore, we developed an open-source modular energy estimation tool based on our approach, enabling fine-grained analysis of energy consumption across AIoT components, allowing developers to evaluate and optimize AIoT energy efficiency in various deployment scenarios.

Through the proposed eCAL metric, this study provides foundations for understanding the energy consumption of AIoT system architectures. Future work may focus on extending the study from AIoT system to other communication system architectures that are increasingly relying on AI techniques for their automation and performance optimization, especially the emerging open radio access network (O-RAN) based cellular systems. The exploration of the relationship between model performance and eCAL for 6G verticals and related network slices may also be worthy of pursuing. Finally, the development of dynamic energy management algorithms in view of a more sustainable operation is highly relevant.

## ACKNOWLEDGEMENTS

This work was supported in part by the HORIZON-MSCAIF project TimeSmart (No. 101063721), the European Commission NANCY project (No.101096456), and by the Slovenian Research Agency under grants P2-0016 and J2-50071.

## REFERENCES

- [1] M. Arnold, S. Dorner, S. Cammerer, and S. Ten Brink, 'On deep learning-based massive MIMO indoor user localization,' in Proc. SPAWC , Jun. 2018.
- [2] H. Kong, L. Lu, J. Yu, Y. Chen, and F. Tang, 'Continuous authentication through finger gesture interaction for smart homes using wifi,' IEEE Trans. Mobile Comput. , vol. 20, no. 11, pp. 3148-3162, Nov. 2021.
- [3] J. Franco, A. Aris, B. Canberk, and A. S. Uluagac, 'A survey of honeypots and honeynets for internet of things, industrial internet of things, and cyber-physical systems,' IEEE Commun. Surveys Tuts. , vol. 23, no. 4, pp. 2351-2383, Aug. 2021.
- [4] S. H. Alsamhi, F. A. Almalki, O. Ma, M. S. Ansari, and B. Lee, 'Predictive estimation of optimal signal strength from drones over iot frameworks in smart cities,' IEEE Trans. Mobile Comput. , vol. 22, no. 1, pp. 402-416, Jan. 2023.
- [5] I. Tomar, I. Sreedevi, and N. Pandey, 'State-of-art review of traffic light synchronization for intelligent vehicles: Current status, challenges, and emerging trends,' Electronics , vol. 11, p. 465, Feb. 2022.
- [6] H. Moudoud, S. Cherkaoui, and L. Khoukhi, 'Towards a scalable and trustworthy blockchain: IoT use case,' in Proc. IEEE ICC , Jun. 2021.
- [7] L. Zhen, A. K. Bashir, K. Yu, Y. D. Al-Otaibi, C. H. Foh, and P. Xiao, 'Energy-efficient random access for LEO satellite-assisted 6G internet of remote things,' IEEE Internet of Things J. , vol. 8, no. 7, pp. 51145128, Oct. 2021.
- [8] J. Zhang and D. Tao, 'Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things,' IEEE Internet of Things J. , vol. 8, no. 10, pp. 7789-7817, Nov. 2020.
- [9] P. Dhar, 'The carbon impact of artificial intelligence,' Nature Machine Intelligence , vol. 2, no. 8, pp. 423-425, 2020.
- [10] A. S. Luccioni, S. Viguier, and A.-L. Ligozat, 'Estimating the carbon footprint of bloom, a 176B parameter language model,' Journal of Machine Learning Research , vol. 24, no. 253, pp. 1-15, 2023.
- [11] E. Garc` ıa-Mart` ın, C. F. Rodrigues, G. Riley, and H. Grahn, 'Estimation of energy consumption in machine learning,' Journal of Parallel and Distributed Computing , vol. 134, pp. 75-88, 2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0743731518308773
- [12] A. Luccioni, A. Lacoste, and V. Schmidt, 'Estimating carbon emissions of artificial intelligence [opinion],' IEEE Technology and Society Magazine , vol. 39, no. 2, pp. 48-51, 2020.
- [13] A. Faiz, S. Kaneda, R. Wang, R. C. Osi, P. Sharma, F. Chen, and L. Jiang, 'LLMCarbon: Modeling the end-to-end carbon footprint of large language models,' in The Twelfth International Conference on Learning Representations , 2024. [Online]. Available: https://openreview.net/forum?id=aIok3ZD9to
- [14] C.-J. Wu, B. Acun, R. Raghavendra, and K. Hazelwood, 'Beyond efficiency: Scaling AI sustainably,' IEEE Micro , 2024.
- [15] P. Zhang, Y. Xiao, Y. Li, X. Ge, G. Shi, and Y. Yang, 'Toward net-zero carbon emissions in network AI for 6G and beyond,' IEEE Communications Magazine , vol. 62, no. 4, pp. 58-64, Apr. 2024.
- [16] S. Savazzi, V. Rampa, S. Kianoush, and M. Bennis, 'An energy and carbon footprint analysis of distributed and federated learning,' IEEE Trans. Green Commun. Netw. , vol. 7, no. 1, pp. 248-264, Mar. 2023.
- [17] X. Hou, J. Liu, X. Tang, C. Li, J. Chen, L. Liang, K.-T. Cheng, and M. Guo, 'Architecting efficient multi-modal AIoT systems,' in Proc. 50th Annual International Symposium on Computer Architecture . New York, USA: Association for Computing Machinery, Jun. 2023. [Online]. Available: https://doi.org/10.1145/3579371.3589066
- [18] S. Zhu, K. Ota, and M. Dong, 'Energy-efficient artificial intelligence of things with intelligent edge,' IEEE Internet of Things J. , vol. 9, no. 10, pp. 7525-7532, May 2022.
- [19] B. Bertalaniˇ c, M. Meˇ za, and C. Fortuna, 'Resource-aware time series imaging classification for wireless link layer anomalies,' IEEE Trans. Neural Netw. Learn. Syst. , vol. 34, no. 10, pp. 8031-8043, 2022.
- [20] D. Trihinas, L. Thamsen, J. Beilharz, and M. Symeonides, 'Towards energy consumption and carbon footprint testing for AI-driven IoT services,' in Proc. IEEE IC2E , Sept. 2022, pp. 29-35.
- [21] S. L. Jurj, F. Opritoiu, and M. Vladutiu, 'Environmentally-friendly metrics for evaluating the performance of deep learning models and systems,' in Proc. ICONIP . Springer, 2020, pp. 232-244.

- [22] C. Naastepad and F. Com, 'Labour market flexibility, productivity and national economic performance in five european economies,' Dutch Report, Flexibility and Competitiveness: Labour Market Flexibility, Innovation and Organisation Performance, EU Commission DG Research Contract HPSE-CT-2001-00093 , 2003.
- [23] P. Rysavy, 'Challenges and considerations in defining spectrum efficiency,' Proceedings of the IEEE , vol. 102, no. 3, pp. 386-392, 2014.
- [24] J. Buberger, A. Kersten, M. Kuder, R. Eckerle, T. Weyh, and T. Thiringer, 'Total CO -equivalent 2 life-cycle emissions from commercially available passenger cars,' Renewable and Sustainable Energy Reviews , vol. 159, p. 112158, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1364032122000867
- [25] J. Diaz-De-Arcaya, A. I. Torre-Bastida, G. Z´ arate, R. Mi˜ n´n, o and A. Almeida, 'A joint study of the challenges, opportunities, and roadmap of mlops and aiops: A systematic survey,' ACM Computing Surveys , vol. 56, no. 4, pp. 1-30, 2023.
- [26] (2015) Imt vision -framework and overall objectives of the future development of imt for 2020 and beyond. [Online]. Available: https://www.itu.int/dms pubrec/itu-r/rec/m/r-rec-m. 2083-0-201509-i!!pdf-e.pdf
- [27] H. R. Chi, A. Radwan, C. Zhang, and A.-E. M. Taha, 'Managing energyexperience trade-off with AI towards 6G vehicular networks,' IEEE Communications Standards Magazine , vol. 7, no. 3, pp. 24-31, 2023.
- [28] P. Pegus, B. Varghese, T. Guo, D. Irwin, P. Shenoy, A. Mahanti, J. Culbert, J. Goodhue, and C. Hill, 'Analyzing the efficiency of a green university data center,' in Proceedings of the 7th ACM/SPEC on International Conference on Performance Engineering , ser. ICPE '16. New York, NY, USA: Association for Computing Machinery, 2016, p. 63-73. [Online]. Available: https://doi.org/10.1145/2851553.2851557
- [29] D. Barry, A. Danalis, and H. Jagode, 'Effortless monitoring of arithmetic intensity with papi's counter analysis toolkit,' in Proc. Tools for High Performance Computing 2018/2019 . Springer, 2021, pp. 195-218.
- [30] R. Pereira, M. Couto, F. Ribeiro, R. Rua, J. Cunha, J. P. Fernandes, and J. Saraiva, 'Ranking programming languages by energy efficiency,' Science of Computer Programming , vol. 205, p. 102609, 2021.
- [31] J. Baliga, R. Ayre, K. Hinton, and R. S. Tucker, 'Energy consumption in wired and wireless access networks,' IEEE Communications Magazine , vol. 49, no. 6, pp. 70-77, 2011.
- [32] R. Bolla, R. Bruschi, O. M. Jaramillo Ortiz, and P. Lago, 'The energy consumption of tcp,' in Proceedings of the Fourth International Conference on Future Energy Systems , ser. e-Energy '13. New York, NY, USA: Association for Computing Machinery, 2013, p. 203-212. [Online]. Available: https://doi.org/10.1145/2487166.2487189
- [33] T. Stefanec and M. Kusek, 'Comparing energy consumption of ap-
- plication layer protocols on iot devices,' in 2021 16th International Conference on Telecommunications (ConTEL) , 2021, pp. 23-28.
- [34] J. Schandy, L. Steinfeld, and F. Silveira, 'Average power consumption breakdown of wireless sensor network nodes using ipv6 over llns,' in 2015 International Conference on Distributed Computing in Sensor Systems , 2015, pp. 242-247.
- [35] N. Drucker, S. Gueron, and V. Krasnov, 'Making aes great again: The forthcoming vectorized aes instruction,' in 16th International Conference on Information Technology-New Generations (ITNG 2019) , S. Latifi, Ed. Cham: Springer International Publishing, 2019, pp. 37-41.
- [36] J. Farfan and A. Lohrmann, 'Gone with the clouds: Estimating the electricity and water footprint of digital data services in europe,' Energy Conversion and Management , vol. 290, p. 117225, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S019689042300571X
- [37] Z. Wang and T. Oates, 'Imaging time-series to improve classification and imputation,' arXiv preprint arXiv:1506.00327 , 2015.
- [38] K. O'Shea and R. Nash, 'An introduction to convolutional neural networks,' 2015. [Online]. Available: https://arxiv.org/abs/1511.08458
- [39] Z. Liu, Y. Wang, S. Vaidya, F. Ruehle, J. Halverson, M. Soljaˇ ci´ c, T. Y. Hou, and M. Tegmark, 'Kan: Kolmogorov-arnold networks,' 2024. [Online]. Available: https://arxiv.org/abs/2404.19756
- [40] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, 'Attention is all you need,' Advances in neural information processing systems , vol. 30, 2017.
- [41] B. Bertalaniˇ c and C. Fortuna, 'Carmel: Capturing spatio-temporal correlations via time-series sub-window imaging for home appliance classification,' Engineering Applications of Artificial Intelligence , vol. 127, p. 107318, 2024.
- [42] R. Yu, W. Yu, and X. Wang, 'Kan or mlp: A fairer comparison,' 2024. [Online]. Available: https://arxiv.org/abs/2407.16674
- [43] A. Ouyang, 'Understanding the performance of transformer inference,' Ph.D. dissertation, Massachusetts Institute of Technology, 2023.
- [44] A. G. Baydin, B. A. Pearlmutter, A. A. Radul, and J. M. Siskind, 'Automatic differentiation in machine learning: a survey,' Journal of Machine Learning Research , vol. 18, no. 153, pp. 1-43, 2018. [Online]. Available: http://jmlr.org/papers/v18/17-468.html
- [45] K. He, X. Zhang, S. Ren, and J. Sun, 'Deep residual learning for image recognition,' in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016, pp. 770-778.
- [46] K. Simonyan, 'Very deep convolutional networks for large-scale image recognition,' arXiv preprint arXiv:1409.1556 , 2014.
- [47] A. Yang, B. Xiao, B. Wang, B. Zhang, C. Bian, C. Yin, C. Lv, D. Pan, D. Wang, D. Yan et al. , 'Baichuan 2: Open large-scale language models,' arXiv preprint arXiv:2309.10305 , 2023.
Shih-Kai Chou Jernej Hribar Vid Hanžel Mihael Mohorčič Carolina Fortuna
The Energy Cost of Artificial Intelligence of Things Lifecycle
Artificial Intelligence (AI) coupled with the existing Internet of Things
(IoT) enables more autonomous operations across various economic sectors. While
this paradigm shift results in increased energy consumption it is difficult to
quantify the end-to-end energy consumption of such systems with the
conventional metrics as they either focus on the communication, the computation
infrastructure or model development. To address this, we propose a new metric,
the Energy Cost of AI lifecycle (eCAL). eCAL captures the energy consumption
throughout the architectural components and lifecycle of an AI-powered wireless
system by analyzing the complexity of data collection and manipulation in
individual components and deriving overall and per-bit energy consumption. We
show that the better a model and the more it is used, the more energy efficient
an inference is. For an example Artificial Intelligence of Things (AIoT)
configuration, eCAL for making 100 inferences is 2.73 times higher than for
1000 inferences. Additionally, we developed a modular open source simulation
tool to enable researchers, practitioners, and engineers to calculate the
end-to-end energy cost with various configurations and across various systems,
ensuring adaptability to diverse use cases.