extfieldsofstudy:
Computer Science
Mathematics

text:
Cooperative Multiple-Access Channels with Distributed State Information

This paper studies a memoryless state-dependent multiple access channel (MAC) where two transmitters wish to convey a message to a receiver under the assumption of causal and imperfect channel state information at transmitters (CSIT) and imperfect channel state information at receiver (CSIR). In order to emphasize the limitation of transmitter cooperation between physically distributed nodes, we focus on the so-called distributed CSIT assumption, i.e. where each transmitter has its individual channel knowledge, while the message can be assumed to be partially or entirely shared a priori between transmitters by exploiting some on-board memory. Under this setup, the first part of the paper characterizes the common message capacity of the channel at hand for arbitrary CSIT and CSIR structure. The optimal scheme builds on Shannon strategies, i.e. optimal codes are constructed by letting the channel inputs be a function of current CSIT only. For a special case when CSIT is a deterministic function of CSIR, the considered scheme also achieves the capacity region of a common message and two private messages. The second part addresses an important instance of the previous general result in a context of a cooperative multi-antenna Gaussian channel under i.i.d. fading operating in frequency-division duplex mode, such that CSIT is acquired via an explicit feedback of perfect CSIR. The capacity of the channel at hand is achieved by distributed linear precoding applied to Gaussian codes. Surprisingly, we demonstrate that it is suboptimal to send a number of data streams bounded by the number of transmit antennas as typically considered in a centralized CSIT setup. Finally, numerical examples are provided to evaluate the sum capacity of the binary MAC with binary states as well as the Gaussian MAC with i.i.d. fading.

Abstract-This paper studies a memoryless state-dependent multiple access channel (MAC) where two transmitters wish to convey a message to a receiver under the assumption of causal and imperfect channel state information at transmitters (CSIT) and imperfect channel state information at receiver (CSIR). In order to emphasize the limitation of transmitter cooperation between physically distributed nodes, we focus on the so-called distributed CSIT assumption, i.e., where each transmitter has its individual channel knowledge, while the message can be assumed to be partially or entirely shared a priori between transmitters by exploiting some on-board memory. Under this setup, the first part of the paper characterizes the common message capacity of the channel at hand for arbitrary CSIT and CSIR structure. The optimal scheme builds on Shannon strategies, i.e., optimal codes are constructed by letting the channel inputs be a function of current CSIT only. For a special case when CSIT is a deterministic function of CSIR, the considered scheme also achieves the capacity region of a common message and two private messages. The second part addresses an important instance of the previous general result in a context of a cooperative multi-antenna Gaussian channel under i.i.d. fading operating in frequency-division duplex mode, such that CSIT is acquired via an explicit feedback of perfect CSIR. The capacity of the channel at hand is achieved by distributed linear precoding applied to Gaussian codes. Surprisingly, we demonstrate that it is suboptimal to send a number of data streams bounded by the number of transmit antennas as typically considered in a centralized CSIT setup. L. Miretti   interference and enhance the network performance. Although the benefits of TX cooperation have been identified in terms of coverage, throughput scaling, spectral efficiency, and energy efficiency, most of the existing cooperative schemes and performance analysis build on the common assumption that perfect, or at least perfectly shared, channel state information at the transmitters (CSIT), referred to centralized CSIT, is available (see e.g. [1]- [3] and references therein). While such an assumption is convenient for analysis, it is however being challenged in a number of practical wireless scenarios. In fact, the acquisition of centralized CSIT always entails direct communication between transmitters or feedback from the receivers so that a given transmitter can collect the CSIT of other transmitters. This inevitably induces impairments and delays, which can be represented as a transmitter-specific distortion added to the channel state information. In order to capture such a limitation, we focus on the so-called distributed CSIT such that each transmitter has its own channel knowledge.
On the other hand, we assume that messages are partially or entirely shared between transmitters prior to the actual data transmission. This assumption is justified for instance in cache-aided networks where parts of delay-tolerant web content (video files) can be pre-fetched at transmitters typically during off-peak hours [4], [5]. More generally, our setup can be thought as representing service situations where the CSIT time sensitivity is high in relation to that of the data contents.
By taking into account both practical CSIT limitation and inherent message-sharing opportunity, we study a statedependent multiple access channel (MAC) illustrated in Fig. 1. Namely, two transmitters with respective state knowledge S 1 , S 2 wish to cooperatively convey a message W through inputs X 1 , X 2 to a single receiver (RX) with state knowledge S R . We do not consider direct communication links between transmitters that enable further online interactions such as conferencing [6], [7]. Rather, we aim to design the transmission strategy for a predefined CSI distribution mechanism described by the joint distribution of (S, S 1 , S 2 , S R ) and a given message cooperation defined by the rate of (W 0 , W 1 , W 2 ). More precisely, the a priori cooperation among the TXs is modeled by the following two components: • State cooperation, modeled by the distribution of the CSIT (S 1 , S 2 ). Perfect state cooperation corresponds to a centralized CSIT configuration where the TXs share the same state information, i.e., S 1 = S 2 (note that this does not necessarily correspond to perfect CSIT). • Message cooperation, modeled by splitting W into 3 sub-messages (W 0 , W 1 , W 2 ), where W 0 is the portion of W available at both TXs, and where W k , k = 1, 2, is the portion available only at TX k. Perfect message cooperation corresponds to W 0 = W or W 1 = W 2 = ∅.
Distributed CSIT gives rise to many interesting, yet challenging, problems because TXs must cooperate on the basis of uncertainties about each other's state information. There are roughly two classes of works. The first class focuses on signal processing methods [8] such as particular precoders optimization, and asymptotic ergodic rate analysis in the regime of high signal-to-noise ratio [9], [10] for cooperative multi-user networks with interference. The second class is based on the information theoretic models. These include the MAC with partial CSIT S 1 , S 2 and full CSIR S = S R = (S 1 , S 2 ) [11], the slow-fading Gaussian MAC with partial CSIT S 1 , S 2 [11], the MAC with conferencing encoders under noncausal partial CSIT S 1 , S 2 and full CSIR S = S R = (S 1 , S 2 ) [7], the MAC with partial and strictly causal state information S k at TX k and no CSIR [12], and the cooperative MAC with non-causal CSIT at one TX and strictly causal at the other [13]. Although useful system design and performance analysis are obtained from these two frameworks individually, they are disconnected each other in the sense that insights obtained from one class cannot be useful for another.
Motivated by such an observation, we wish to close the gap between these two approaches by designing a simple yet information theoretically optimal cooperative scheme under distributed channel state information. To the best of our knowledge, such a result was not reported before. In particular, we study the capacity of a common message and the capacity region of three messages over a memoryless state-dependent MAC with causal distributed CSIT. Before summarizing the main contributions of the current work in Section I-B, we first review the existing results on coding with causal CSIT under various network models.

A. Coding with causal CSIT
In [14], Shannon characterized the capacity of a memoryless state-dependent point-to-point channel with causal state knowledge at the transmitter S = S T ∈ S and no CSIR S R = ∅. The capacity of the channel at hand can be alternatively given by [15] C = max p(u), x=f (u,s) where U ∈ U is an auxiliary random variable of finite cardinality and independent of S, and f is a deterministic function. Notice that U can be seen as an index for the family of functions S → X , also known as Shannon strategies. This result has a practical impact to the design of modern wireless communication systems as it suggests that capacity is achieved by encoding the message through a function f depending only on the current CSIT.
In the following, we briefly summarize the existing results exploiting Shannon strategies. Shannon strategies were generalized to more general setups with imperfect CSIT S T and imperfect CSIR S R [16], and to particular cases of state process with memory in [16]. Shannon strategies were also extended to more general network models, including degraded broadcast channels [17], [18], degraded relay channels [18], [19], as well as multiple access channels [12], [13], [17], [18], [20]- [27]. By focusing on the MAC literature, the capacity region of the state-dependent MAC was studied by Das and Narayan in [20], where multi-letter formulas are given for very general channel and CSI models. Unfortunately the multiletter expressions provide very little insights and cannot be easily computed. In contrast, single-letter expressions on an achievable rate region (R 1 , R 2 ) have been derived for the case of a common state S 1 = S 2 = S in [18]. When the two states (S 1 , S 2 ) are independent, Shannon strategies are proved to achieve the sum capacity [21]. In practical wireless systems operating in frequency division duplexing (FDD) mode, it is typical to assume that the CSIT is a deterministic function of the CSIR as CSIT is acquired as an explicit feedback from the receiver. Under this condition, the full capacity region of the state dependent MAC has been characterized in [21] for independent states and then generalized to arbitrarily correlated states in [22]. For the case of degraded message sets as a special case of Fig. 1 when W 2 = ∅, the capacity region has been characterized for the case of one-sided CSIT Interestingly, Shannon strategies are known to be suboptimal in general state-dependent MAC. In particular, Lapidoth and Steinberg demonstrated that Shannon strategies fail to achieve some rate pairs in the state-dependent MAC with no CSIR for the case of common state [24] and the case of independent states [25], causally or strictly causally available at the encoders. This is because block-Markov encoding can help the TXs to provide some CSIR at the RX by compressing and sending past CSIT cooperatively [24] or non-cooperatively [25]. Clearly, this strategy is not useful when the CSIT is already available at the RX, as in [22] and in parts of this work. The scheme proposed in [24], [25] have been further generalized in [12], where the TXs compress and send the past codewords along with past channel states. The idea of sending the past codewords via block-Markov encoding has been proposed for the MAC with feedback [6], [28], [29], while the idea of sending the past state together with new messages was also considered in the simultaneous state and data communication (see e.g. [30] and references therein) and in the cooperative MAC with strictly causal CSIT [27].

B. Contributions and Paper Outline
This paper provides the following contributions: 1) We demonstrate that the common message capacity of the memoryless state-dependent MAC under distributed CSIT is achieved by Shannon strategies for any CSI distribution p(s, s 1 , s 2 , s R ) in Theorem 1. The exact characterization of the common message capacity complements the existing results in [21], [24], [25], which consider only two private messages, the results in [26] which assumes S 2 = ∅ and W 2 = ∅, and it extends the single-user results in [14], [16] to two distributed TXs. 2) For the special case when the CSIT of each TX k is a deterministic function q k of the CSIR, i.e., S k = q k (S R ) for k = 1, 2, we prove that Shannon strategies achieve the full capacity region on three messages (W 0 , W 1 , W 2 ) in Theorem 2. This extends the existing result [22] to the case when a common message is present. The contribution of Theorem 2 lies in our converse proof based on a standard information inequality chain, which overcomes the technique used in [21] restricted to the case of independent states while significantly simplifying the approach in [22]. 3) By specializing the model of Theorem 2, we establish the common message capacity of the multiple-input multipleoutput (MIMO) Gaussian fading channel operating in frequency-division-duplexing (FDD) mode in Theorem 3. We demonstrate that distributed linear precoding over Gaussian codewords based on Shannon strategies is optimal. The difference with respect to centralized CSIT is that each TX shall choose its precoding vector as a function of its channel knowledge, rather than global CSIT. As a key ingredient for the achievability proof, we allow the number of data streams conveying W 0 to grow large up to a given upper bound that depends on the CSIT cardinalities. Furthermore, as a non-trivial extension, Theorem 4 characterizes the entire capacity region of the aforementioned channel. The converse proof exploits the underlying channel structure and functional dependencies, while achievability builds on superposition encoding and the distributed precoding technique developed for Theorem 3. 4) By letting the number of precoded data streams be the maximum dimension, in Proposition 2 and Proposition 3 we prove that the optimal distributed precoding design, belonging to the well-known class of difficult problems called team decision problems [8], can be cast into a convex form. Moreover, in Proposition 4 and Proposition 5 we provide a more in-depth analysis on the optimal number of common data streams. Surprisingly, we prove that the common wisdom of limiting the number of precoded data streams by the number of transmit antennas is strictly suboptimal under a distributed CSIT setup. This is in sharp contrast to the case of centralized CSIT. The paper is organized as follows. In Section II, we provide the formal system model and the main results for general MAC with distributed CSIT. Section III presents the results for the specific cooperative MIMO MAC at hand. The insights given by the above sections are then further illustrated via numerical examples in Section IV. For readability purposes, most of the proofs are moved to appendices.

II. COOPERATIVE MULTIPLE-ACCESS CHANNELS WITH CAUSAL AND DISTRIBUTED CSIT
This section first provides the general channel model and the basic definitions adopted throughout this work. Then, we present general results on the cooperative multiple access channel (MAC) with causal and distributed CSIT illustrated in Fig. 1. Hereafter, we follow the standard notation of [15].
A. System Model and Problem Statement a) Channel Model: Consider the state-dependent MAC in Fig. 1, with a common message W 0 , two private messages is assumed to follow a generic memoryless law p(s, s 1 , s 2 , s R ). An nsequence of inputs, output and states is then governed by p(s n , s n 1 , s n 2 , s n R ) = n i=1 p(s i , s 1i , s 2i , s Ri ).
We assume that three messages W 0 , W 1 , W 2 are independently and uniformly distributed over the sets W j ∆ = {1, . . . , 2 nRj }, j = 0, 1, 2, where R j ≥ 0 is the rate of the message W j . All alphabets are assumed to be finite, unless otherwise stated.
b) Encoding and Decoding: A block code (2 nR0 , 2 nR1 , 2 nR2 , n) of length n with causal CSIT is defined by a set of encoding functions . . , n, yielding the transmitted symbols x ki = φ ki (w 0 , w k , s i k ), as well as a decoding function ψ : Y n × S n R → W 0 × W 1 × W 2 , yielding the decoded messages (ŵ 0 ,ŵ 1 ,ŵ 2 ) = ψ(y n , s n R ). Each encoder k = 1, 2 is subject to an average input cost constraint said to be achievable if, for the considered channel, there exists a family of block codes of length n defined as before such that the average probability of error satisfies P c) Figure of Merit: For a given cost pair (P 1 , P 2 ), the closure of the set of all achievable rates (R 0 , R 1 , R 2 ) is the capacity-cost region C (P 1 , P 2 ) of the considered channel. In this work, we are mostly interested in two operating points in C (P 1 , P 2 ). Namely, the common message capacity, defined by C 0 (P 1 , P 2 ) ∆ = max{R 0 ∈ C (P 1 , P 2 )} and the sum capacity Fig. 1.

B. General Results
As a non-trivial extension of [26,Corollary 3] with the onesided CSIT (S 2 = ∅) and of [14], [16] for the centralized CSIT case S 1 = S 2 to a general CSI structure, we provide the main result of this section. Theorem 1. The common message capacity of the channel in Fig. 1 is given by where U ∈ U is an auxiliary random variable of finite cardinality, independent of (S, S 1 , S 2 , S R ), and where f k is a deterministic functions U × S k → X k for k = 1, 2.
Proof: The proof is given in Appendix A.
The main finding of Theorem 1 is that the common message capacity (or equivalently, the sum capacity) can be achieved by Shannon strategies, i.e., by coding over the current CSIT S 1i , S 2i only while neglecting the past CSIT sequences. In fact, the converse proof also shows that providing the strictly causal sequence (S i−1 1 , S i−1 2 ) to both TXs does not increase the common message capacity.
It is also worth emphasizing the difference with respect to the centralized CSIT where both TXs share S 1 = S 2 ∆ = S T . In such case, by omitting for simplicity the input cost constraints, we recover in fact the classical result of [14], [16] Although Shannon strategies are optimal in both both distributed and centralized CSIT cases, the distributed CSIT assumption imposes the design of two different functions f 1 , f 2 depending on the local CSIT only each, rather than a single f as in the (virtually) centralized case.
In order to prove the achievability part of Theorem 1, we obtain an achievable region for the MAC with a common message and two private messages as a byproduct. Specifically, we obtain the following result by combining Slepian-Wolf coding [31] for the state-less MAC with common message and Shannon strategies [14]. Fig. 1, C (P 1 , P 2 ) includes the convex hull of all rate triples (R 0 , R 1 , R 2 ) such that

Lemma 1. For the channel in
It is well known that Shannon strategies, i.e., the scheme of Lemma 1, fail to achieve C (P 1 , P 2 ) for general p(s, s 1 , s 2 , s R ), as observed for a special case of a common CSIT S 1 = S 2 = S and no CSIR S R = ∅ in [24]. This is because block-Markov encoding enables two TXs to compress past state information and send it as a common message to provide possibly useful CSIR to the RX. Nevertheless, Theorem 1 shows that such a scheme based on block-Markov encoding is not necessary for achieving the sum capacity of the considered setup. Namely, provided that R 0 is large enough (in the worst case, equal to C sum (P 1 , P 2 )), the sum capacity is indeed achievable by the scheme in Lemma 1.
In the following we focus on the particular case where each CSIT is a deterministic function of CSIR. This assumption is highly relevant to frequency-division duplex (FDD) systems, where each transmitter acquires channel knowledge via an explicit quantized feedback from the receiver. As a straightforward extension of [22,Theorem 4] and [21,Theorem 5] restricted to two private messages 1 , we characterize the capacity region for three messages as follows.
Theorem 2. By assuming that S 1 = q 1 (S R ) and S 2 = q 2 (S R ), where q 1 , q 2 are two deterministic functions, the capacity region C (P 1 , P 2 ) of the channel in Fig. 1 is given by the convex hull of all rate triples satisfying for some pmf p(x 1 |s 1 , u)p(x 2 |s 2 , u)p(u), where U ∈ U is an auxiliary variable of finite cardinality and independent of (S, Proof: The proof is given in Appendix B. Our main contribution lies in the converse proof, which solves the issue highlighted in [21] through an appropriate identification of the auxiliary variable U , thus allowing to greatly simplify the non-traditional, yet innovative, converse proof given by [22]. Note that Theorem 2 refers to a setup where the RX is fully informed about (S 1 , S 2 ), hence there is no need for the TXs to convey (S 1 , S 2 ) through block-Markov schemes as in [24], [25]. In contrast to the general CSI setup in Lemma 1, the private messages W 1 , W 2 can be directly encoded into the input alphabets X 1 , X 2 as observed already in [21,Theorem 5]. In light of Theorem 2 we highlight the following expression for the common message capacity, which will be used to prove the main result of the second part of this paper.

Remark 2.
Under the assumption S 1 = q 1 (S R ) and S 2 = q 2 (S R ), the expression in (1) is equivalently given by where U ∈ U is an auxiliary random variable of finite cardinality, independent of (S, S 1 , S 2 , S R ), and where f k is a deterministic functions U × S k → X k for k = 1, 2.
We conclude the first part of this paper by providing the following outer bound.
is included in the convex hull of all rate triples satisfying for some auxiliary variables (U 0 , U 1 , U 2 ) ∈ U 0 × U 1 × U 2 of finite cardinality, independent of the CSI (S, S 1 , S 2 , S R ), with pmf factorizing as p(u 0 )p(u 1 |u 0 )p(u 2 |u 0 ), and for some deterministic functions f k : Proof. The proof is given in Appendix C.
Propostion 1 shows that, for (S 1 , S 2 ) conditionally independent given S R , Shannon strategies may be sub-optimal only in terms of individual rates (indeed, [25] proves that higher individual rates are achievable for independent (S 1 , S 2 ) and S R = ∅). This extends [21,Theorem 4], which considered independent (S 1 , S 2 ) and no common message. The bound on R 1 + R 2 was already reported in [23] and references therein by using the same technique as [22]. Similarly to Theorem 2, our contribution lies in a simpler converse proof.

III. FDD COOPERATIVE MIMO CHANNEL WITH FADING
In this section, we specialize the channel in Fig. 1 to a practical 2 × 2 cooperative Gaussian MIMO channel with fading operating in FDD mode, illustrated in Fig. 2. The goal of this section is to particularize the general results of Section II and derive operational rules for encoding in the Gaussian MIMO setting. We point out that the results can be generalized to more general antenna configurations and number of TXs as discussed in Section III-E.

A. Channel Model and Notation
For this second part of the article, we extend our notations to better highlight multi-dimensional quantities. In particular, we denote deterministic matrices, column vectors and scalars by X, x, and x respectively. We denote by X i,j the entry of X in the i-th column and j-th row. Random matrices, column vectors and scalars are denoted instead by X, X, and X respectively. The operators (·) H and · F denote respectively the Hermitian transpose and the Frobenius norm. We denote by e i ∈ {0, 1} d a standard column selector, i.e., with the i-th element set to 1 and all the other elements set to 0. The identity matrix of dimension n is denoted by I n , or simply by I when the dimension is clear from the context. Finally, C, R + , and S n + denote respectively the set of complex numbers, non-negative real numbers, and Hermitian positivesemidefinite matrices of dimension n.
We consider a classical Gaussian MIMO channel law p(y|x 1 , x 2 , S) and let the RX signal Y ∈ C 2×1 for a given channel use be given by: where the state S ∈ C 2×2 is a matrix of random fading coefficients, X k ∈ C is the signal transmitted by TX k, subject where the RX has perfect CSIR S R = S, and where the CSIT is a quantized version of the CSIR, i.e., If q 1 = q 2 , for example in the case of different feedback rates, we clearly fall into a distributed CSIT configuration. Note that, consistently with the definitions in Section II-A with S replaced by S, we consider coding over a large number of i.i.d. fading realizations. By using classical wireless terminology, we recall that this corresponds to the so-called fast-fading regime.

B. Optimality of Distributed Linear Precoding with an Unconventional Number of Data Streams
In what follows, we establish the capacity region of the considered distributed setting and show that distributed linear precoding over Gaussian codewords is optimal. As we will see, the main novelty lies in an unconventional joint encoding technique for the common message W 0 . For the sake of clarity, we present this technique by focusing on the common message capacity first.
Theorem 3. The common message capacity of the channel in Section III-A is given by Furthermore C 0 (P 1 , P 2 ) can be achieved by letting where U is the encoded common message.
Proof: We first apply the well-known maximum differential entropy lemma [15, p. 21] to the mutual information in (3)  For the achievability part, it suffices to show that every feasible Σ(S 1 , S 2 ) can be obtained via distributed linear precoders of dimension d = |S 1 | + |S 2 |. To this end, we first define the random vectorsX . . .
collecting the random inputs X k conditioned on each of the |S k | realizations of S k , and the covariance matrix It is easy to see that Q contains all the elements of Σ(s 1 , s 2 ), ∀(s 1 , s 2 ) ∈ S 1 × S 2 . Note that, due to the power constraint s k ∈S k E[|f k (U, s k )| 2 ]p(s k ) ≤ P k < ∞, any feasible (f 1 , f 2 , p U ) must satisfy E[|f k (U, s k )| 2 ] < ∞ ∀s k ∈ S k , hence Q has finite entries. Since Q is Hermitian positive semidefinite, there exists a square matrix F ∈ C d×d such that We denote its column vectors by Finally, simple calculations show that the scheme in (5) with distributed linear precoders designed using the above procedure, i.e., selected from (6), preserves the desired Σ(S 1 , S 2 ), and attains the maximum entropy upper bound. The main result of Theorem 3 is that distributed linear precoding [8] of shared Gaussian codewords achieves the performance limits of the considered cooperative MIMO setting. However, as a sufficient condition to prove achievability, Theorem 3 considers the transmission of possibly d = |S 1 | + |S 2 | independent data streams. This unconventional design choice appears to be in sharp contrast with the centralized CSIT configuration (i.e., S 1 = S 2 ∆ = S T ), where the capacity of the 2×2 MIMO channel is achieved by encoding d ≤ 2 streams in the presence of perfect message cooperation. In this latter case, by considering the per-antenna power constraint, the capacity takes the well-known expression given for example by [32] C 0 (P 1 , P 2 ) = max where Σ(s T ) is the conditional input covariance. Clearly, the capacity in (7) can be achieved by taking the matrix square-root G(S T ) ∆ = Σ 1 2 (S T ) ∈ C 2×2 and by letting or, in other words, by precoding d = 2 data streams only. Such an approach cannot be used for general distributed settings, as it generally leads to unfeasible linear precoders violating the functional dependencies x k = f k (u, s k ).
The proof of Theorem 3 addresses this issue by increasing the dimensionality d of the linear precoders up to d 2, i.e., beyond conventional design choices. We now show that this unconventional linear precoding technique can be applied to extend Theorem 3 to the full capacity region.
Theorem 4. The capacity region C (P 1 , P 2 ) of the channel in Section III-A is given by the union of all rate triples (R 0 , R 1 , R 2 ) such that for some g k (S k ) ∈ C d and γ k (S k ) ∈ R + such that Furthermore any point in C (P 1 , P 2 ) can be achieved by letting where (U , V 1 , V 2 ) ∼ CN (0, I d +2 ) are the encoded common and private messages, respectively.
Proof. The proof is given in Appendix D.
Theorem 4 shows that superposition of jointly and independently encoded Gaussian codes achieves the capacity region. However, while encoding of the private messages W 1 , W 2 follows traditional approaches (one power-controlled stream per TX), the joint encoding of W 0 may require a larger number of precoded data streams (beyond two streams in our case) as already seen for Theorem 3. A more detailed analysis of the role played by d in terms of optimal distributed precoding is provided in the following sections.

C. Convex Reformulation for Optimal Distributed Precoding Design
The distributed precoding design problem (4) belongs to the class of static team decision problems [8], [33], for which no efficient solutions are known in general. However, in contrast to the traditional precoding design with d ≤ N T , where N T = 2 is the total number of TX antennas, by letting d ≤ d we are able to recast the optimal precoding design problem (4) into an equivalent convex problem.
Proof: The proof follows by simply rewriting (4) in light of the technique used for the proof of Theorem 3. Specifically, we let d = d and use SΣ(S 1 , S 2 )S H = SE H (S 1 , S 2 )F H FE(S 1 , S 2 )S H = S eq QS H eq , where F is given by (6) and Q = F H F. The power constraints correspond to linear constraints on the diagonal elements of Q.
Problem (9) corresponds to the capacity of a virtual d × 2 MIMO channel with state S eq , perfect CSIR, no CSIT, and (fixed) transmit covariance Q. The capacity achieving distributed precoders for the original channel can be then designed from the optimal Q in Problem (9) as follows: An important remark here is that if the constraint d ≤ d of Problem (4) is replaced by d < d, the technique of Proposition 2 does not lead to a convex reformulation. This is because the d × d matrix F is replaced by a d × d matrix F , hence introducing a non-convex constraint rank(Q) ≤ d < d to Problem (9). However, note that if the optimal Q for the unconstrained problem has rank r < d, then we can reduce with no loss of optimality the dimensionality of g k (S k ) in (10) down to d = r.
Mirroring the previous section, the above result on common message capacity can be extended to the following weighted sum-rate maximization problem maximize (R0,R1,R2)∈C (P1,P2) where α k ≥ 0, k = 0, 1, 2 are non-negative weights identifying rate priorities. We recall that the above problem can be used to characterize the boundary of C (P 1 , P 2 ), since the weights can be interpreted as coefficients of a supporting hyperplane to such boundary (see e.g. [15], [34] and references therein). (11) is equivalent to the following convex problem

Proposition 3. Problem
where S eq is given as in Proposition 2.
Proof: The proof follows by the same technique as in the proof of Proposition 2. The details are omitted.
Problems (9) and (12) can be solved numerically via known convex optimization tools. A comprehensive discussion on the efficiency of various competing approaches is out of the scope of this work. Here, we point out two critical issues that should be taken into account in a practical system design. First, advanced stochastic optimization techniques may be required if the fading distribution p(S) is continuous 2 . Second, classical second-order methods as interior-point methods for semi-definite optimization typically scale badly with the dimension d of Q. Hence, first-order methods may be more suitable whenever the cardinality of the CSIT alphabets |S k | is large. As a result of the algorithmic complexity stemming out of the above considerations (which are still very active research topics), we envision that feasible implementations of the proposed distributed precoding design should operate in an offline fashion. Specifically, Problems (9) and (12) could be solved in a preliminary codebook design phase, while in the data transmission phase TX k simply selects the precoder from the pre-designed codebook based on the received CSIT index S k .

D. Further Comments on the Optimal Number of Data Streams
In this section we further elaborate on the optimal number of data streams d by addressing two important questions left open by Section III-B. Theorem 3 shows that using a number of precoded data streams d = d is a sufficient condition for achievability of the common message capacity. However, we know that for some CSIT configurations (e.g., for centralized CSIT, as already discussed), d > 2 is not necessary. A first crucial question is whether there exists some distributed CSIT configuration for which such a condition is indeed necessary. In the next proposition we answer positively to this question.
Proposition 4. For some p(S, s 2 , s 2 ) and power constraints (P 1 , P 2 ), restricting d ≤ N T in problem (4), where N T = 2 is the total number of TX antennas, leads to strictly suboptimal rates.
Proof: The proof is given in Appendix E, by showing the existence of a CSI distribution p(S, s 2 , s 2 ) with binary CSIT |S 1 | = |S 2 | = 2 such that d ≥ 3 is necessary for achieving C 0 (P 1 , P 2 ).
A second natural question is whether the developed upper bound d ≤ d = |S 1 | + |S 2 | is tight, for some p(S, s 1 , s 2 ). In the following we answer negatively to this question, by showing that indeed we can consider a slightly tighter upper bound. However, we firstly remark that obtaining tighter bounds is not trivial and is in fact related to the well-known low-rank matrix completion problem [35]. Let us consider the matrix Q ∈ S d + defined in the proof of Theorem 3, or equivalently in Proposition 2, and its partition into blocks Informally, we recall that Q collects the elements of the conditional input covariances Σ(S 1 , S 2 ) for all realizations of (S 1 , S 2 ). By direct inspection of the capacity expression (4), or equivalently of the objective in Problem (9), we observe that the off-diagonal elements of the sub-matrices Q k do not contribute to the achievable rate, since they do not correspond to any element of any realization of Σ(S 1 , S 2 ). Hence, by lettingQ be any optimal solution of (9), the solution Q of the (non-convex) problem is also an optimal solution of (9), but where the off-diagonal elements of Q k have been optimized such that the rank is minimized. Since we have seen that the rank r ≤ d of Q corresponds to the dimension d of optimal distributed precoders (see Section III-C), establishing a tighter upperbound on d can be cast into finding an upper-bound on the solution of (14), which is an instance of a low-rank (semidefinite) matrix completion problem (see, e.g., [35]).
To the best of the authors knowledge, non-trivial upperbounds to problems of the type (14) remain elusive. Nevertheless, in the following proposition we provide a simple result showing the existence of a tighter upper-bound than d = |S 1 | + |S 2 |. Proposition 5. The common message capacity C 0 (P 1 , P 2 ) (resp. capacity region C (P 1 , P 2 )) given by Theorem 3 (resp. Theorem 4) is also achievable by letting Proof. The proof is given in Appendix F. The above result is by no means satisfactory, since the dimensionality reduction is marginal for large CSIT alphabets. Informally, the main limitation of the above bound is that the proof optimizes only one of the (coupled) variables in (14). However, note that the above bound is tight for the toy example considered in the proof of Proposition 4.

E. Extension to Arbitrary Users and Antenna Configurations
Theorem 3 can be readily extended to K TXs and arbitrary antenna configuration. By letting N k and M be respectively the number of antennas at the k-th TX and at the RX, and by considering a fading state matrix S ∈ C M ×( K k=1 N k ) and distributed CSIT (S 1 , . . . , S K ), S k = q k (S) ∈ S k , |S k | < ∞, it can be shown that the common message capacity is given by C 0 (P 1 , . . . , where The formal proof of the above statement follows from the same lines as for the proof of Theorem 3, by considering K functions f k : U × S k → C N k ×1 . The detailed steps do not provide additional intuitions, hence they are omitted. The convex reformulation of Proposition 2 can be also similarly extended. Finally, as for Theorem 4, the full-capacity region for K private messages and a single common message can be achieved by superimposing standard MIMO MAC codes for W 1 , . . . , W K to the non-traditional distributed precoding technique achieving (15).

A. Channel with Additive Binary Inputs and State
We consider the following channel with binary inputs and state, i.e., X 1 = X 2 = S = {0, 1}, and where Y = {0, 1, 2, 3}. We do not consider input cost constraints. We further assume S ∼ Bernoulli(q), no CSIR (S R = ∅), and distributed CSIT p(s 1 , s 2 |s) = p(s 1 |s)p(s 2 |s), where p(s k |s) is a binary symmetric channel with transition probability k ∈ [0, 0.5]. Under the above model, the common message capacity (which coincides with the sum capacity) is given by A (non-scalable) method for optimally solving the above optimization problem is to adapt to the considered distributed setting the original idea of coding over the alphabet of Shannon strategies [14], [15], combined with classical results on the computation of the capacity of point-to-point channels [36]. More precisely, we proceed as follows: 1) We build the alphabet of distributed Shannon strategies by enumerating all the functions where each function is indexed by U . There are |U| = |X 1 | |S1| |X 2 | |S2| = 16 such functions. 2) We set x k = f k (u, s k ) = t k,u (s k ) and compute the equivalent state-less point-to-point channel 3) We run the Blahut-Arimoto algorithm for computing the capacity of the equivalent channel p(y|u) [36]. Note that the above procedure is similar to the one outlined in [37] for centralized settings. Furthermore, it can be readily generalized to arbitrary CSIR by simply considering an augmented outputỸ ∆ = (Y, S R ). In Fig. 3 we plot the capacity C 0 versus the CSIT quality at TX 2, for various choices of CSIT quality at TX 1, and for q = 0.5. Note that k = 0 and k = 0.5 model respectively perfect and no CSIT at the k-th TX. Interestingly, the capacity of the system decreases with 2 down to a flat regime in which any further decrease in quality does not matter, and the turning point depends on 1 . This can be interpreted as a regime in which the quality at one TX is so degraded that, although some CSIT is available, it does not allow for proper coordination with the better informed TX. Intuitively, it is important for the less informed TX to not act as unknown noise for the other TX. In fact, in the aforementioned regime it turns out that the optimal scheme at the less informed TX is to throw away completely its CSIT, making its behaviour not adaptive to the channel conditions but completely predictable by the more informed TX.

B. Cooperative AWGN MIMO with Rayleigh Fading and Quantized Feedback
In this section we simulate a practical cooperative MIMO channel with Rayleigh fading and with limited feedback rates. In particular, we let each element of S to be i.i. d. CN (0, 1), and we set for simplicity P 1 = P 2 ∆ = SNR. The distributed CSIT configuration p(S, s 1 , s 2 ) is given by two random quantizers with different rates β 1 , β 2 .
More precisely, let S k = {1, . . . , 2 β k } be the index set of a codebook {Ŝ k,i } 2 β k i=1 of randomly and independently generated codewords distributed as p(S). We then let q k (S) to be a simple nearest neighbour vector quantizers in the Frobenius norm, i.e., q k (S) = arg min i∈S k S −Ŝ k,i F . This scenario corresponds to an error-free feedback link from the RX to the k-th TX with limited rate of β k bits per channel realization. We set β 1 = 4 and β 2 = 3, which implies d = |S 1 | + |S 2 | = 24. We recall that the RX is assumed to have perfect CSIR.
We approximately solve Problem (9) through an off-theshelf numerical solver for convex problems, by substituting p(S, s 1 , s 2 ) with its empirical distributionp(S, s 1 , s 2 ) ∆ = 1 L L l=1 1[(S, s 1 , s 2 ) = (S l , q 1 (S l ), q 2 (S l ))] obtained from L = 1000 i.i.d. samples {S l } L l=1 . This allows us to replace the expectation in (9) with a finite sum of L convex functions. The capacity obtained is exact for a channel with state distribution equal to the empirical distributionp(S, s 1 , s 2 ), and approximates the capacity for p(S, s 1 , s 2 ) as L grows large. Furthermore, we repeat the above simulations by considering instead a single antenna at the RX, a setting denoted here as cooperative MISO.
In Fig. 4 we plot the common message capacity versus SNR of a given instance of the considered channel model. We also plot the common message capacity for perfect CSIT at both TXs We recall that these CSIT configurations are equivalent to a centralized 2 × 2 MIMO system, hence we can simply use the classical MIMO results summarized e.g. in [32], adapted to a per-antenna power constraint. For a fair comparison, these capacities are computed over the same empirical marginal distributionp(S). As expected, for the MIMO case, the capacity gain given by distributed CSIT w.r.t no CSIT follows the wellknown beamforming gain trend of the perfect CSIT case, i.e., it vanishes in the high SNR regime. Similarly, for the MISO case, this gain converges to a constant power offset.

V. CONCLUSION
In this paper, we studied a two-user memoryless statedependent multiple access channel under the assumption that causal and distributed CSIT is available and messages can be partly or entirely shared prior to the data transmission. We characterized the common message capacity of this channel and demonstrated that it is optimal to encode the message as a function of current CSIT only based on Shannon strategies. We provide an insightful example over an additive binaryinput quaternary-output channel with binary states showing that, interestingly, in some cases there is a threshold in terms of CSIT quality below which one encoder shall not use its channel knowledge. For a special case when CSIT is a deterministic function of CSIR, the full capacity region of a common message and two private messages is also characterized. This last result is specialized to a practically relevant cooperative MIMO fading channel operating in FDD mode such that CSIT is acquired via an explicit feedback from the receiver. The cooperative MIMO example surprisingly reveals that in a distributed CSIT setup the optimal number of data streams shall not be restricted to the minimum number of transmit antennas. This is in contrast to the classical MIMO design under the centralized CSIT assumption.
Interesting open problems include the evaluation of the minimum message cooperation (i.e., the minimum rate R 0 ) required such that the sum capacity is achievable via Shannon strategies, and the extension of the coding ideas derived for the cooperative MIMO case to systems with multiple receivers.

APPENDIX
A. Proof of Theorem 1 Converse: Let us define U i = (W 0 , S i−1 1 , S i−1 2 ). We construct an upper-bound by assuming that past CSIT realizations (S i−1 1 , S i−1 2 ) are available at both encoders. Hence, we assume that X 1i and X 2i are functions of (W 0 , S i . We then have: where (a) follows from Fano's inequality (lim n→∞ n = 0), (b) follows from the Markov chainỸ i−1 → (W 0 , S i−1 1 , S i−1 2 ) →Ỹ i , and (c) is because S Ri is independent of U i . The code must also satisfy the input cost constraints We combine the bounds in (16) and (17) by means of a timesharing variable Q uniformly distributed in {1, . . . , n} and independent of everything else, and by letting U = (U Q , Q), Note that the resulting distribution p Y,X1,X2,S,S1,S2,S R ,U factors as p(y|x 1 , an indicator function. With these identifications, we obtain Hence, we finally have R 0 ≤ C 0 (P 1 , P 2 ) + n .
Achievability: Achievability follows from standard arguments, hence a formal proof is omitted. An informal yet intuitive proof can be obtained by the classical physical device argument of Shannon [14], [15]. More precisely, by fixing the functions f k (u k , s k ), we can consider a new state-less and memoryless MAC with messages W 0 , W 1 , W 2 , inputs U 1 , U 2 , and output (Y, S R ). For a given f k (u k , s k ), the capacity region of this auxiliary channel is simply achievable by Slepian-Wolf coding for the MAC with common and independent messages [31], which gives the achievable region in Lemma 1.
The finite cardinality of U 1 , U 2 follows directly by Shannon argument, which states that using Shannon strategies corresponds to coding over an augmented input alphabet of functions S k → X k of size |X k | |S k | , indexed by U k [15]. Hence we can consider |U k | ≤ |X k | |S k | . The finite cardinality of U 0 follows by a simple application of the support lemma [15,Appendix C] applied to the Slepian-Wolf region of the auxiliary channel, which gives [15, p. 344] Finally, the expression in Theorem 1 can be obtained by specializing the proof of Lemma 1 to the transmission of a common message only, i.e. by letting R 1 = R 2 = 0, and by identifying U 0 = U 1 = U 2 ∆ = U .

B. Proof of Theorem 2
Achievability: The proof follows the same lines as in [21]. Achievability builds on Lemma 1, where we rewrite the mutual information terms as follows. By focusing first on the sumrate, we observe that and (c) because of the Markov chain (S 1 , S 2 , U 1 , U 2 ) → (X 1 , X 2 , S R ) → Y . Similarly, one can show Finally, by the functional representation lemma [15,Appendix B] and since U 1 and U 2 do not appear in the bounds given by (2) and proven above, designing where 1[·] is an indicator function, is equivalent to designing p(u)p(x 1 |s 1 , u)p(x 2 |s 2 , u) (replacing U 0 with U ).
Converse: Let us define U i = (W 0 , S i−1 1 , S i−1 2 ). We construct an outer bound by assuming that past CSIT realizations (S i−1 1 , S i−1 2 ) are available at both encoders. Hence, we assume that X 1i and X 2i are functions of (W 0 , where (a) follows from Fano's inequality (lim n→∞ n = 0), (b) from the independence of W 1 and S n R , (c) from (S i1 , S 2i ) being a function of S Ri , and (d) from the Markov chain The code must also satisfy the input cost constraints We combine all the bounds by means of a time-sharing variable Q uniformly distributed in {1, . . . , n} and independent of everything else, and by letting U = (U Q , Q), X 1 = X 1Q , Note that the resulting distribution p Y,X1,X2,S,S1,S2,S R ,U factors as as required. With these identifications, we readily obtain

C. Proof of Proposition 1
Let us define U 0i := (W 0 , S i−1 R ), U 1i := (W 1 , S i−1 1 , U 0i ), and U 2i := (W 2 , S i−1 2 , U 0i ). Note that X 1i and X 2i are functions of (U 1i , S 1i ) and (U 2i , S 2i ) respectively, and, due to the Markov chain S 1i → S Ri → S 2i , we also have U 1i → U 0i → U 2i as required. By Fano's inequality (lim n→∞ n = 0), and by following similar steps as in the previous sections, we obtain where the last inequality comes from the memoryless property of the channel. Following similar lines one can prove which can be combined with the bound on R 1 + R 2 and the power constraints by means of the usual time-sharing step.

D. Proof of Theorem 4
We construct an outer bound C o (P 1 , P 2 ) by following similar steps as in [34], but starting from the single-letter formulation of C (P 1 , P 2 ) given by Theorem 2 extended to continuous alphabets similarly to [15], [16], [32]. We consider the following applications of the maximum differential entropy lemma [15, p. 21], adapted to the complex field. From the rightmost bound in [15, Eq. 2.6], we have By the structure of the input distribution, we now observe that Γ(S 1 , S 2 ) = diag(γ 1 (S 1 ), γ 2 (S 2 )) and that, ∀(s 1 , s 2 ) ∈ S 1 × S 2 , where we define the functions By following similar steps for h(Y |X 1 , U, S = S) and h(Y |X 2 , U, S = S), and by applying the resulting bounds to the mutual information terms in Theorem 2, we obtain , R 1 + R 2 ≤ E log det I + Sdiag(γ 1 (S 1 ), γ 2 (S 2 ))S H , The outer bound C o (P 1 , P 2 ) is then established by taking the convex hull of the union of all rate triples (R 0 , R 1 , Similarly to the proof of Theorem 3, it can be now shown that every Σ(S 1 , S 2 ) as in (18) induced by any p(x 1 |s 1 , u)p(x 2 |s 2 , u)p(u) can also be obtained by the scheme in (8), i.e., via superposition of linearly precoded Gaussian codes. The key point is showing that the second term in the RHS of (18) can be obtained via distributed linear precoders of dimension d. This follows by the same technique used in the proof of Theorem 3, by simply replacing the functions f k (u, s k ) with µ k (u, s k ). Finally, since the inputs are conditionally Gaussian, standard arguments [15] show that (8) attains the bound C o (P 1 , P 2 ), without time sharing (i.e., we can omit the convex hull operation).

E. Proof of Proposition 4
The proof is split for the sake of clarity in the following three steps: 1) We fix a specific conditional input covariance matrix Σ (S 1 , S 2 ), and we show that it is achievable via distributed linear precoding if and only if d > 2. 2) We construct a specific p(S, s 1 , s 2 ) such that Σ (S 1 , S 2 ) is the unique optimal solution to Problem (4). 3) We combine the above steps to show that there exists a channel for which d ≤ 2 leads to strictly suboptimal rates.

Proof:
The main idea is to build such CSI distribution by "reversing" a spatio-temporal water-filling algorithm which gives as unique optimal solution the conditional input covariance Σ (S 1 , S 2 ). We now provide the details.
Define a uniformly distributed random state S taking values in the finite alphabet S = {S 1 , S 2 , S 3 , S 4 }, and let the CSIT be given by the functions ≤ max Σ∈P E log det I + SΣ(S 1 , S 2 )S H , ≤ max Σ∈P E log det I + SΣ(S 1 , S 2 )S H , where P ∆ = {Σ(S 1 , S 2 ) ∈ S 2 + | tr{E[Σ(S 1 , S 2 )]} ≤ 2}, is the set obtained by relaxing the per-TX power constraint P to a total power constraint (P ⊆ P ). Inequalities (24) and (25) are obtained respectively by relaxing the achievability via distributed linear precoding and the power constraint.
Problem (25) turns out to be an instance of a classical (centralized) MIMO capacity problem, where the optimal solution is given by the well-known spatio-temporal waterfilling algorithm. More precisely, let us rewrite (25) as max Σ(S1,S2)∈P where we defined Σ i ∆ = Σ(q 1 (S i ), q 2 (S i )), and wherẽ A well-known application of the Hadamard's inequality gives the following upper bound in terms of the channel eigen- where the optimal ξ i,k are given by the water-filling conditions and where equality is achieved for Consider the conditional covariance Σ (S 1 , S 2 ) given by (20). Note that Σ (S 1 , S 2 ) ∈ P , i.e., it satisfies the total power constraint. We wish to construct S = {S i } such that Σ (S 1 , S 2 ) is the unique optimal solution for (25). This can be done by "reversing" the MIMO water-filling algorithm described above. More precisely, let us consider Σ i ∆ = Σ (q 1 (S i ), q 2 (S i )) and their eigen-decompositions We construct now S by letting where the eigenvalues Λ i = diag(λ i,1 , λ i,2 ) are given by , i = 1, . . . , 4, k = 1, 2, and any choice of ∞ > ν > max i,k ξ i,k = 1.8. By construction, Σ (S 1 , S 2 ) is an optimal solution for (25). Uniqueness of the solution can be proven by contradiction as in [38, Section III.A], or directly by the strict concavity of 0}, which is a direct consequence of the strict concavity of log det(A) in A 0 and of the positive definiteness of S i by construction. Finally, since Σ (S 1 , S 2 ) ∈ P ∩ G(4), (25) and (24) are satisfied with equality.

F. Proof of Proposition 5
The proof follows by manipulating an off-diagonal entry of the sub-matrices Q 1 , Q 2 given by (13) until Q becomes rank-deficient. We recall that varying these entries has no influence on the achievable rates, provided that the positive semi-definiteness of Q is maintained. Consider the symmetric matrixQ Let λ min : S d → R be the minimum eigenvalue of a Hermitian symmetric matrix (not necessarily positive semi-definite). By definition, λ min (Q(0)) ≥ 0. In addition, ∃t 1 > 0 such that λ min (Q(t 1 )) < 0. Furthermore, by the continuity of the map λ min in the matrix entries [39,Theorem 5.2], λ min (Q(t)) is a continuous function of t. Hence, by the intermediate value theorem, ∃t 0 ∈ [0, t 1 ] such that λ min (Q(t 0 )) = 0, i.e., such thatQ(t 0 ) is positive semi-definite, low rank, and achieves the same rates as Q.


Characterizing Technical Debt and Antipatterns in AI-Based Systems: A Systematic Mapping Study

Background: With the rising popularity of Artificial Intelligence (AI), there is a growing need to build large and complex AI-based systems in a cost-effective and manageable way. Like with traditional software, Technical Debt (TD) will emerge naturally over time in these systems, therefore leading to challenges and risks if not managed appropriately. The influence of data science and the stochastic nature of AI-based systems may also lead to new types of TD or antipatterns, which are not yet fully understood by researchers and practitioners. Objective: The goal of our study is to provide a clear overview and characterization of the types of TD (both established and new ones) that appear in AI-based systems, as well as the antipatterns and related solutions that have been proposed. Method: Following the process of a systematic mapping study, 21 primary studies are identified and analyzed. Results: Our results show that (i) established TD types, variations of them, and four new TD types (data, model, configuration, and ethics debt) are present in AI-based systems, (ii) 72 antipatterns are discussed in the literature, the majority related to data and model deficiencies, and (iii) 46 solutions have been proposed, either to address specific TD types, antipatterns, or TD in general. Conclusions: Our results can support AI professionals with reasoning about and communicating aspects of TD present in their systems. Additionally, they can serve as a foundation for future research to further our understanding of TD in AI-based systems.

I. INTRODUCTION
Artificial Intelligence (AI) covers different technologies for searching, reasoning, planning, problem solving, and learning with the overall aim of "automating intellectual tasks normally performed by humans" [1]. Its rise in popularity in recent years is mostly due to advancements in Machine Learning (ML), an area of AI focusing on algorithms and systems to identify rules and patterns in data based on statistical modeling techniques.
With more and more companies offering AI-powered products and using AI techniques to improve their internal processes, there is a need to build large, complex AI systems in a costeffective and manageable way. At the surface level, this may not seem like a new problem: AI systems are software systems too, so we can use well-known, established software engineering principles, practices, and processes to build such systems (e.g., separation of concerns, component-based encapsulation, and agile delivery). However, recent studies show that the AI/ML * All authors contributed equally to this study. domain possesses characteristics that make it distinct from other software application domains, such as an increased importance of data quality and management, unclear abstraction boundaries for complex models, and challenges in the customization and reuse of AI/ML components [2][3][4]. Such characteristics seem to necessitate adaptations of principles, practices, and processes successfully used in other domains, or even the adoption of new, AI-specific ones [5].
A successful practice when building software systems in an iterative fashion is the awareness and management of technical debt (TD) [6]. TD is a metaphor used to describe design or implementation constructs that may be expedient in the short term, but can make future changes more costly or even impossible [7]. Looking at the differences between AI/ML and other application domains from the TD perspective, researchers from Google proposed in 2015 new TD instances that are specific to the development of AI-based systems [4]. Since this seminal paper, various research works from both academia and industry followed up with the documentation of additional TD items and antipatterns in AI/ML systems [8][9][10].
Despite these efforts, there is still no comprehensive conceptual overview of TD in AI-based systems. It is unclear, for instance, whether these systems accrue more "traditional" TD than other types of systems, such as code, architecture, or documentation debt. It is also important to understand if AI-specific TD types emerge and what their characteristics, associated antipatterns, and proposed solutions are. Gaining such an overview would provide a foundation for future research, and support practitioners to better manage the maintenance and evolution of AI-based systems. To characterize TD in AI-based systems, we therefore conducted a systematic mapping study (SMS), and collected and analyzed relevant papers on the topic. Grounded in 21 primary studies, our contribution with this paper is the thorough analysis and discussion of the concepts of TD and antipatterns in AI-based systems.

A. Technical Debt
Since its formulation by Cunningham in 1992 [6], the definition of technical debt has continuously evolved and broadened in scope. Nowadays, it encompasses a vast range of concepts, artifacts, and processes [11]. Among the current definitions of TD, a widely adopted one was formulated during Dagstuhl seminar 16162 [7]. Simply referred to as the 16162 definition, it specifies TD as "design or implementation constructs that are expedient in the short term, but set up a technical context that can make a future change more costly or impossible". To structure the knowledge on TD, different TD types have been described, e.g. architectural debt, requirements debt, and test debt, allowing researchers and practitioners to effectively focus on specific technical issues where the debt metaphor applies [12]. Instances of such types vary considerably in nature, from suboptimal reuse of architectural components [13], to deferred testing [14], or the involvement of certain development communities [15]. In this research, we embrace the 16162 definition of TD, and build upon the TD type taxonomies presented by Li et al. [12] and Rios et al. [16].

B. Antipatterns
While design patterns constitute proven solution blueprints for specific problems, there is also the inverse concept of antipatterns, i.e. frequently occurring suboptimal solutions [17]. Developers may choose these solutions under time pressure, but antipatterns often appear due to insufficient expertise. They can have immediate negative effects on quality attributes such as maintainability, performance efficiency, or reliability, but may also hinder the sustainable evolution of a system, leading to the accumulation of TD. Antipatterns can exist at different levels of abstraction, such as code antipatterns, architectural antipatterns, or even project management antipatterns.
There is also the related concept of bad smells, e.g. code smells [18] or architectural smells [19]. Some authors keep these terms strictly separated [20], i.e. software smells are seen as potential indicators of bad quality that may require further investigation, while an antipattern is always supposed to be a bad practice that should be avoided. However, similar to patterns, many antipatterns can also be context-sensitive and may be perceived as "bad" only in specific cases. When collecting archetypes of suboptimal software practices, a clear distinction between the two concepts becomes less important and several studies have handled them uniformly [21,22]. For the purpose of this paper, we therefore do not differentiate between the terms antipattern and smell, i.e. we collect both concepts under the same umbrella. In this sense, we treat e.g. code smells as antipatterns on the implementation level.

C. Related Work
Technical debt and antipatterns have been the target of numerous reviews in different SE subfields and domains. However, to the best of our knowledge, there exists no comprehensive secondary study focusing on these concepts in the area of AI-based systems. TD has been studied in the context of databases [23] and data-intensive systems [24], but without a clear focus on AI or ML, as with Sculley et al. [4].
Nonetheless, several studies focus on general software quality aspects in AI-based systems. Humbatova et al. [25] conceptualized a fault taxonomy for deep learning systems by analyzing GitHub repositories, StackOverflow posts, and conducting interviews. While faults are not in the scope of our study, they can be related to TD and antipatterns in some cases, e.g. as symptoms of their existence. Concerning the quality assurance of AI-based systems, several position papers discuss differences compared to "traditional" systems and highlight the need for adapted quality assurance techniques [26,27]. Some empirical studies also went further than this and distilled effective techniques for ML system engineering, e.g. Serban et al. [28] derived general AI system development best practices, and Siebert et al. [29] provided guidelines for the quality assurance of such systems. Lastly, the study that comes closest to the goal of our own research is a preliminary multivocal literature review by Washizaki et al. [30]: they collected design patterns and antipatterns for ML systems from both white and grey literature. However, they only identified eight antipatterns, seven from Sculley et al. [4] and one from a company blog post, and did not provide any insights on how TD is characterized in AI-based systems. With our study, we therefore fill this gap by providing a detailed characterization of TD and antipatterns in AI-based systems.

III. METHODOLOGY
In this section, we document the research design, which was rigorously adhered to during study execution. We primarily followed the guidelines for conducting systematic literature studies in software engineering research by Kitchenham [31].

A. Research Objective and Questions
The aim of this research is to further the understanding of technical debt and antipatterns in AI-based software systems. To refine this goal, we derived the following research questions (RQs), which guided our mapping study:

B. Research Process
An overview of the research process followed is depicted in Fig. 1. The process started with the execution of a conservative automated search query via Google Scholar, followed by an iterative forward-and backward-snowballing process, until theoretical saturation was reached. Following the methodology by Wohlin et al. [32], we based our search on a start set obtained via an automated search query executed on Google Scholar. This set was then used for exhaustive bidirectional snowballing. The use of Google Scholar allowed us to avoid Google Scholar 5 Step 2 Application of Selection Criteria

90
Step 1 Automated Initial Search Step 6 Report Step 5 Data Synthesis Fig. 1. Systematic mapping study process overview bias in favor of any specific publisher [32]. Further details of each research step are reported in the following subsections.

1) Step 1: Automated Initial Search
To gather an initial set of potentially relevant studies, we executed a conservative automated query on the Google Scholar literature indexer. The title-focused query was designed to encompass related literature focusing specifically on the topic under investigation, and is formulated as follows: Listing 1. Automated search query 1 ALLINTITLE: ("technical debt" OR "antipatterns" OR 2 "antipattern" OR "anti patterns" OR "anti pattern" OR 3 "smell" OR "smells") AND ("artificial intelligence" OR 4 "machine learning" OR "deep learning" OR "intelligent" 5 OR "smart" OR "AI" OR "ML" OR "DL") This query identifies literature containing in their title keywords referring either to "technical debt", "antipatterns", or "smells" (Listing 1, Lines 1-3) and keywords referring to AI, or related synonyms and acronyms (Listing 1, Lines 3-5). The automated query was executed mid-June 2020, and yielded 90 potentially relevant studies. As we were not interested in publications regarding a specific timeframe, the publication date was purposely left unbounded in the query.

2) Step 2: Application of Selection Criteria
After identifying the set of potentially relevant studies via the automated query, we conducted a manual selection process. During this step, we evaluated the initial pool of studies based on pre-defined selection criteria. A paper was selected as a primary study if it satisfied all inclusion criteria and none of the exclusion ones. We used the following criteria: I1-Publications reporting technical debt, antipatterns, or suboptimal software engineering practices I2-Publications focusing on AI-based systems E1-Non-English publications E2-Publications for which the full text is not available to us E3-Duplicates or extensions of already included publications E4-Secondary or tertiary studies E5-Publications in the form of editorials, tutorials, books, extended abstracts, etc. E6-Non-scientific publications (i.e. grey literature) The two inclusion criteria (I1, I2) were formulated to ensure that primary studies focused on the investigated topic, namely TD and antipatterns in AI-based systems, and hence provided relevant data to answer our RQs. The exclusion criteria instead were designed to guarantee that data could be extracted from papers (E1, E2), without duplication or redundancy (E3, E4), and consisted of scientific literature (E5, E6).
Given the fast pace at which the investigated topic evolves, we purposely included preprints during the selection process. However, preprints needed to possess a sufficient level of quality (reviewed by all three researchers), to have already been cited by high-quality academic literature, and to be from reputable authors who published other studies in the field. 1 During the selection, adaptive reading depth [33] was used to efficiently assess potentially relevant studies. To mitigate subjective bias, all authors independently applied the selection criteria for the 90 candidate studies. Differences were jointly discussed until consensus was reached. This led to the selection of five primary studies, i.e. the snowballing start set.

3) Step 3: Snowballing
To obtain a sound and encompassing set of primary studies, the automated search was complemented by recursive backward and forward snowballing [32]. During this step, all studies either citing or cited by the primary studies were examined. Similar as for the initial selection, the snowballing process was conducted by three researchers: in each round, all researchers independently suggested new primary studies to be included, i.e. studies which fulfilled the selection criteria. Divergences were jointly discussed and resolved, after which the next iteration started. Overall, it took four rounds of backward and forward snowballing until no new studies were identified. Snowballing led to the inclusion of 16 studies, i.e. our SMS selection process led to the identification of 21 primary studies.

4) Step 4: Data Extraction
In the next step, we systematically analyzed the primary studies and extracted data related to our RQs. To gain a preliminary understanding, a data extraction pilot with four papers was conducted independently by all researchers. Subsequently, the extracted data was jointly discussed, leading to the extraction framework used in this study. Two researchers were randomly assigned to each primary study. They independently extracted the data and agreed on the final extractions per paper in a consensus meeting, with the intervention of the third researcher when required. The extraction framework is divided into one part for each specific RQ of our study (see Section III-A).
To characterize TD in AI-based systems (RQ1), data needed to be extracted according to its four sub-questions. For the recurrence of established TD types in AI-based systems (RQ1.1), we identified types based on the TD taxonomies of Li et al. [12] and Rios et al. [16] (e.g. code, test, or architectural debt). Using the same taxonomies, we also extracted variations of established TD types (RQ1.2), i.e. if the nature or scope of TD types changed in AI-based systems. An example for this is test debt, as it extends in AI-based systems to testing models and data. We also analyzed the primary studies for new TD types in AI-based systems (RQ1.3), i.e. debt types which (i) are documented in the context of AI-based systems and (ii) cannot be traced back to established TD types. Lastly, we extracted quality attributes affected by TD in these systems (RQ1.4), which was based on ISO/IEC 25010 [34], with the possibility to extend it with additional identified attributes.
To answer RQ2 (antipatterns), the primary studies were analyzed for recurrent suboptimal solutions in AI-based systems. Such suboptimal solutions could be explicitly referenced as "antipatterns" (e.g. correction cascades [4]), or reported as root causes of TD (e.g. unstable data dependencies [10]).
Finally, to answer RQ3 (solutions), we extracted the solutions proposed to mitigate or resolve TD and antipatterns in AI-based systems. Such solutions could be specific to a certain TD type or antipattern (e.g. model isolation to resolve entanglements) or general best practices to mitigate or prevent the introduction of TD in AI-based systems (e.g. periodically assessing assumptions during ML model evolution).
Note that the extractions did not have to explicitly mention "debt", "antipattern", or a specific quality attribute. The decision if a passage implicitly describing these concepts warranted extraction was up to the researchers' interpretation. 5) Step 5: Data Synthesis As a final step, the extracted data was harmonized (e.g. merging identical or very similar antipatterns), and then analyzed to derive answers to the research questions. This analysis relied on open coding [35] to systematically identify recurrent concepts. Further axial coding [35] was required to reduce the growing complexity of some emerging concepts (e.g. antipattern subcategories). During the coding, emerging results were continuously discussed among the authors to keep codes and their abstraction level consistent. Finally, summary statistics were created to discuss general findings and their potential implications. For the sake of transparency and reproducibility, we make all study artifacts publicly available online 2 .

IV. RESULTS
Our results are extracted from 21 primary studies [P1]- [P21], which were published in conferences (12/21), workshops (5/21), journals (3/21), or distributed as white papers (1/21). Since the appearance of the first paper focusing on TD in AI-based systems in 2015 [P3], we observed a growing publication trend until 2020 3 . Interestingly, a large number of primary studies were co-authored by at least one industrial practitioner (13/21), including nine papers authored exclusively by practitioners. Google is the most recurrent company [P3-P7], while other prominent examples include Microsoft [P1], Amazon [P8], and IBM [P9]. The considerable involvement of industrial parties displays the industrial relevance of the topic, which has still to gain traction in academic environments.
In the remainder of this section, we present the results of our study, according to the four RQs guiding the investigation.

A. Characteristics of TD in AI-based Systems (RQ1)
This section reports the results for the sub-questions of RQ1, aiming to characterize the nature of TD in AI-based systems. An overview of the recurrence of all identified TD types is reported in Fig. 2 (both established and new types). Following we discuss the distribution of established TD types plus their variations, new types emerging in AI-based systems, and finally affected quality attributes. Mentioned antipatterns are explained in more detail in Section IV-B.  We observed that this distribution is mostly due to new engineering challenges related to AI-based systems. Our primary studies link these systems to an inherently experimental development process, deep entanglement between architectural components and utilized data, and necessary data transformation steps between components. Coping with such difficulties often leads to the introduction of TD, e.g. the evolutionary composition of AI pipelines may result in generalpurpose components being precariously stitched together via glue code [P3]. Additionally, the data-driven nature of AI algorithms introduces novel difficulties for quality assurance, often manifested as suboptimal testing practices, such as undertested or ill-tested functionalities.
Moreover, AI-specific variations or extensions of TD types, rather than their established scope [12,16], are often used in our primary studies. Specifically, out of the 61 established TD type extractions, 37 are such variations. While some TD types occur unchanged in AI-based systems, re-interpreting several existing debt types is necessary to accommodate the new characteristics of these systems. The most recurrent established debt types, namely infrastructure, architectural, code, and test debt, are also the types which frequently exhibit an extended or augmented scope in our primary studies.
Regarding infrastructure debt, this TD type is extended with deficiencies related to the implementation and operation of AI pipelines as well as to the management of AI models. In AIbased systems, this TD type often manifests in form of complex infrastructure comprising various AI pipelines [P3], suboptimal allocation of resources to train/test AI models [P2], and weak AI monitoring and debugging capabilities [P1], leading to major operations and reproducibility issues.
Architectural debt variations instead reflect the emphasis in AI-based systems on data, leading to a deep entanglement of architecture components with their underlying data. This may introduce debt items such as complex and non-deterministic dependencies between architectural components and datasets [P2], hard to assess compositions of architectural elements [P10], or undeclared consumers of AI models [P11].
Similarly, test debt extends to data testing, ranging from naive omission of basic sanity checks to the lack of more sophisticated tests to assess data quality or distributions [P3]. In addition, new facets include suboptimal practices in testing AI models and pipelines. Their deep connection to training data and the stochastic nature of some AI algorithms [P4,P6] make these artifacts increasingly complex to evaluate.
Code debt is shaped by the experimental nature of AI model development. This frequently emerges in form of dead experimental code paths in production code [P12] and the suboptimal refactoring of experimental models into deployable software [P5]. The algorithmic complexity of these systems also increases the likelihood of certain code deficiencies [P13].
For versioning debt as a less referenced TD type, we exclusively identified the AI-centric usage of the term. In AIbased systems, this now includes the versioning of AI models and training/testing data, which is often done in suboptimal fashion, if at all [P4,P8,P12].
Finally, documentation, people, requirements, build, defect, design, and process debt mostly appear in our studies according to their established scope, i.e. while such debt types also appear in AI-based systems, characteristics of AI do not have a prominent impact on their manifestation. This highlights that numerous commonalities are shared with software systems not employing AI. Missing documentation, insufficient developer skills, and unclear system requirements are all examples of TD which also frequently occur in non-AI software. Slight variations for some of these TD types are the extension of documentation debt to features and assumptions on the used data [P14], of build debt to suboptimal dependencies of internal and external AI models [P10], and of defect debt to ignored issues related to the quality of model predictions [P4].
2) New TD types in AI-based systems (RQ1.3) With this RQ, we wanted to synthesize new TD types important for AI-based systems, i.e. types not included in the taxonomies of Li et al. [12] and Rios et al. [16]. Specifically, we found four such types of debt: data debt, model debt, configuration debt, and ethics debt, which we further describe below.
Data debt. The most recurrent new TD type regards suboptimal constructs around the data used in AI-based systems (16/21). Specifically, this TD type refers to deficiencies related to the collection, management, and usage of data, both for training and production [P7,P15,P16]. In addition to causing immediate issues, this TD type can also be latent, i.e. not manifesting itself immediately, but rather posing a risks for the long-term evolution of systems. Commonly referenced instances of data debt are data quality issues, unmanaged data dependencies and anomalies, or poor data relevance. Given their heavy reliance on data, this TD type can strongly impact AI systems, including reduced classification effectiveness, data loss due to premature aggregation [P15], and compatibility issues.
Model debt. The second most referenced new TD type is model debt (15/21). This AI-specific debt type regards suboptimal practices in the design, training, and management of AI models [P3,P4,P8]. As such, model debt manifests itself as deficiencies occurring exclusively in model-related constructs. Most prominently, model debt originates from suboptimal feature selection processes, neglected hyperparameter tuning, and poorly engineered model deployment strategies. Recurrent items of model debt are feature entanglement [P3], hidden feedback loops [P17], unrecognized model staleness [P8], and substantial differences between training and production performance, i.e. training/serving skew [P3]. As models constitute the logic kernel of AI-based systems, this TD type can have serious consequences, ranging from major challenges in maintaining a model, to severe deterioration of model accuracy.
Configuration debt. This debt type (8/21) describes deficiencies around the configuration mechanisms of AI-based systems [P3,P12,P17]. Often, configuration debt arises when the complexity of e.g. dynamic feature selection, hyperparameter tuning, and data pre-and post-processing makes it difficult to efficiently outsource machine-and human-readable configuration files for these activities. This encourages AI engineers to take shortcuts and to only consider the clean-up, restructuring, and commenting of configuration files as an afterthought. While the lines of configuration for AI-based systems may even exceed the lines of source code [P3], configurations are frequently not given the same level of quality control as code, e.g. reviews or tests. Prominent instances of configuration debt include massive/complex, poorly documented (or simply undocumented), unversioned, or untested configuration files. As such, configuration debt may have some touch points with e.g. infrastructure or documentation debt, but its explicit description in the context of AI still warrants its own TD type.
Ethics debt. One less referenced new debt type is ethics debt (6/21), which comprises deficiencies around ethical aspects of AI-based systems, such as algorithmic fairness, prediction bias, or a lack of transparency and accountability [P14,P18,P19]. Specifically, ethics debt arises when socio-ethical concerns are deliberately or inadvertently neglected during the design or training phase of AI-based systems. While this can go hand in hand with reduced model accuracy, the resulting systems may also satisfy all technical requirements while leaving one or more ethical concerns unaddressed. This debt type can lead to ethical fallacies so deeply embedded into an AI-based system that they usually cannot be resolved with only minor data or software changes. Instead, they may require a complete restructuring and retraining of AI models or major source code updates. Depending on the relevant regulations, ethics debt can also have legal consequences.

3) Affected quality attributes (RQ1.4)
We extracted a total of 12 unique quality attributes impacted by TD in AI-based systems, for which the recurrence in our 21 primary studies is depicted in Fig. 3. Overall, functional suitability is the most referenced quality attribute (18/21). Specifically, the introduction of TD in AIbased systems leads to issues in the functional correctness sub-characteristic (18/21), which, given the data-driven nature of AI-based systems, can be cumbersome to detect and resolve. Among the most mentioned reasons for this is diminished model accuracy, e.g., due to training/serving skew [P4]. Maintainability is the second most mentioned quality attribute (16/21), with emphasis on the sub-characteristics modifiability (14/21), testability (13/21), and reusability (11/21). In this case, we noted how TD frequently leads to quality issues specific to or caused by AI models, such as difficult model re-training or reuse, ripple effects on changes through harmful dependencies, or complex collections of scripts and pipelines that are hard to analyze. Observability (8/21) as additional quality attribute represents the degree to which the runtime behavior of a deployed AI-based system can be monitored. This quality attribute can be severely impacted by infrastructure and architecture debt. Security and reliability are referenced equally often (7/21). Security issues in AI-based systems often concern the sub-characteristics accountability (5/21) or confidentiality (4/21). Reliability, by contrast, is most often related to maturity of AI-based systems, i.e. the degree to which the system meets expectations under normal operation. Other quality attributes, such as performance efficiency (5/21), compatibility (4/21), and usability (1/21), are mentioned less frequently, i.e. are most likely less impacted by TD in AI-based systems. Similarly, new quality attributes emerging in AI-based systems such as reproducibility (4/21), fairness (4/21), transparency (4/21), and responsibility (3/21) are also not frequent. We believe the growing attention to topics related to these quality attributes in academia and industry will probably lead to a higher recurrence in the near future. Finally, the only attribute from ISO/IEC 25010 not mentioned at all is portability.
Main findings (RQ1): Infrastructure debt (12) is the most recurrent established TD type in AI-based systems, followed by architectural, code, and test debt (8). We identified four new debt types emerging in AI-based systems, namely data, model, configuration, and ethics debt. Functional suitability (18) and maintainability (16) are the most impacted quality attributes, followed by observability (8), security (7), and reliability (7).

B. Antipatterns (RQ2)
From the 21 primary studies, we extracted a total of 72 unique antipatterns for AI-based systems. To organize this large collection, we formed six categories (with an additional level of subcategories for the larger ones), where each antipattern is assigned to exactly one category. These categories, and associated number of antipatterns, are displayed in Table I,  while Table II lists the 14 most prominent antipatterns mentioned in at least three publications. The largest categories are model antipatterns (29) and data antipatterns (22), which together account for ∼70% of all identified antipatterns. The four remaining categories -design & architecture antipatterns (8), code antipatterns (5), infrastructure antipatterns (4), and ethics antipatterns (4) -are all of similar size. In the following subsections, we briefly present each category with some antipattern examples.
Model Antipatterns. The largest group of identified antipatterns (29) describes deficiencies or suboptimal practices with AI models, mostly in the context of machine learning. Often, these refer to the training, validation, or management of models, i.e. to specific activities in the model life cycle. A concrete example is the training antipattern direct feedback loops [P3,P15,P20], which describes the unwanted state that a model impacts  its own future training data selection. This self-sustaining relationship may lead to wrong classifications and biased decisions, especially if feedback loops remain hidden or are not managed appropriately. An example for a model validation antipattern is offline/online proxy metric divergence [P4,P20]. Effectiveness of a production system is usually evaluated with metrics like user engagement or revenue (online), while models of AI components in such a system are validated with e.g. accuracy or mean squared error (offline). If offline metrics are not sufficiently aligned with the online metrics, e.g. via correlation, system effectiveness may be severely impacted.
Lastly, a general model management antipattern is the absence of versioning and version control systems specifically for the model (no version control for models [P4]), which hinders sustainable evolution and potential rollbacks. A different type of antipatterns in this category focuses on model features and their relationships. An example of this is feature entanglement [P3,P11,P17,P20], which refers to the interdependence of different model features. Adding, removing, or changing the distribution of one feature often has an impact on other features, therefore hindering incremental system improvement. This has also been described as the CACE principle [P3], i.e. "changing anything changes everything". A second example are epsilon features [P4,P17], which are features only leading to negligible model improvement. Since every feature comes with costs for maintenance and evolution (especially when considering feature entanglement), feature inclusion should be carefully considered based on merit.
Finally, a frequently mentioned type of model antipatterns is related to training/serving skew [P2,P4,P12,P15,P19,P20]. This is the most recurrent unique antipattern and, in general, describes substantial model accuracy divergences in the production system when compared to the training accuracy. Reasons for this can be different code paths to compute features in production, but also a non-representative or non-exhaustive training data set. More specialized variants of this antipattern have been called distribution skew or scoring/serving skew [P6]. The divergence between training and serving accuracy can also emerge slowly over time, which is called data drift [P15,P20] and leads to stale models [P4,P14].
Data Antipatterns. The second largest antipattern group (22) is related to deficiencies or suboptimal practices around the data of AI-based systems. Since data is the foundation for machine learning models, such antipatterns can substantially diminish system effectiveness. Many instances in this area are related to data quality or the existence of data anomalies. Examples of bad quality are data duplication [P7,P9,P15,P16], null/missing data values [P7,P9,P15,P16], or data miscoding smells [P3,P7,P15,P21], where an attribute is represented with an unsuitable data type or format. Similarly, examples for anomalies in machine learning data can be an unnormalized feature [P7,P21], where values exhibit large variance, or very few extreme outliers [P7], which may distort important aggregate values used by models. While of both these subcategories can also be important for data-intensive non-AI systems, these antipatterns have been specifically described in the AI context. Data relevance is another mentioned property that can be subject to antipatterns. Selected examples are the emphasis on available data [P14] or the usage of overcurated data [P19], both of which can lead to training/serving skew.
One of the larger subcategories focuses on data management. Since it can be quite complex in some cases, an undocumented data collection process [P16] may negatively affect the longterm evolution of an AI-based system. Similarly, premature data aggregation [P15] during this collection process can destroy important data points that cannot be retrieved again. A third example, which is frequently mentioned, is unsound/missing metadata [P6,P15,P16], i.e. a suboptimal or absent documentation and schema to describe the used data.
Lastly, a smaller subcategory is related to data dependencies. An example here is unstable data dependencies [P3,P10,P11]. Consuming data from other systems as input signals for model features may initially speed up development. However, if the external data is unstable and changes over time, these dependencies can have negative and hard to diagnose effects on the related ML component.
Design & Architecture Antipatterns. We also extracted eight antipatterns related to the design and architecture of AI-based systems. While it is one of the smaller categories, it contains several frequently mentioned antipatterns. An example related to modularity is undeclared consumers [P3,P11,P20], i.e. if the results of an AI component serve as input for a broad range of other systems or components. These undeclared or silent consumers constitute hidden coupling, which can have negative and obscure side effects during software evolution. Similarly, an antipattern where, instead of the output, the complete model is reused and slightly altered is called correction cascades [P3,P11,P14]. Changes in the original model then may lead to unintended ripple effects cascading to the "corrected" downstream models. In [P14], this is also described with the improper reuse of complete AI components or pipelines.
A second type of antipatterns is concerned with software integration. The prime example here is glue code [P2,P3,P17]. AI-based systems are often built with many generic packages or components, which are then connected with custom code, e.g. for data transformation or reading and writing data. This not only makes it difficult to keep an overview of the system but glue code may also tightly couple the system to specific external libraries. In the area of data collection and preparation, a specialized version of glue code is called pipeline jungles [P3,P17], i.e. the same stitching together but more on an architectural level and with ML pipelines.
Finally, a small subcategory regards technology adoption. An example, not fully AI-specific, but still mentioned as a consequence of the nature of AI systems, is the multiplelanguage smell [P3,P8,P17]. While using Python or R for ML models, and other languages for non-ML components, may enable using the best frameworks or libraries for the task at hand, it also entails disadvantages in maintaining, testing, or handing over a component to colleagues.
Code Antipatterns. We generally identified two subcategories of code antipatterns: those specific to AI-based systems and generic ones that occur more frequently in these systems. A frequently mentioned AI-specific example is dead experimental codepaths [P3,P12,P17]. The influence of data science leads to an iterative and experimental development process for AI components, where several conditional branches exist, which increase complexity and may also be forgotten, resulting in dead code. Examples of generic code antipatterns which occur more frequently in AI software are long lambda functions or long ternary conditional expressions [P13].
Infrastructure Antipatterns. We also identified a small number of antipatterns related to the infrastructure of AI-based systems. The most prominent example from this category is weak or missing monitoring [P1,P2,P4]. Using AI components leads to additional observability requirements, e.g. monitoring data sources or model accuracy to detect training/serving skew. Moreover, the black-box nature of AI components can make it difficult to perform root cause analysis without specialized tooling. Another infrastructure antipattern is hence weak or missing debugging [P1]. As a last example, inadequate configuration management [P12] describes missing or suboptimal tooling mechanisms to manage important model configuration, e.g., features, preprocessing settings, or hyperparameters.
Ethics Antipatterns. The last smaller category of antipatterns is concerned with ethical issues in AI-based systems. The obvious example are biased models [P4,P17], i.e. models that have been created based on incomplete or irrelevant data or with a prejudice-inducing algorithm or process. Such models not only produce inaccurate but also unfair results, which depending on the use case can have substantial negative societal effects, e.g. with predictive policing or recidivism models. For such usage scenarios, it is especially important to measure and control the consequences of the respective AI system. Failing to do so is described by the antipattern unmanaged social impact [P14]. A final example in this category is undefined human accountability [P14], i.e. when the role and responsibility of humans in AI-supported decisions is not clearly documented, allowing people to hide behind a machine.
Main findings (RQ2): We extracted 72 unique antipatterns in six categories. Largest categories are model (29) and data (22). Design & architecture only consists of eight antipatterns, but many of them occurred several times. The antipattern which was mentioned the most is training/serving skew (6).

C. Solutions (RQ3)
From the 21 primary studies, we identified 46 unique instances of solutions. Out of these, about a third of the instances explicitly mentions a TD type, another third mentions an antipattern, while the remaining third does not mention any specific TD type or antipattern addressed. In particular, the last group contains advices of broad and generic nature (e.g. perform extensive testing [P4]) and specific methods typically implemented in a tool or framework (e.g. Data Quality Advisor [P9]). Given the difficulty in categorizing solutions using a single dimension (TD type, antipattern, or specificity level), we focus instead on discussing the five solutions that are most referenced in our primary studies (Table III).
Manage model configuration prescribes that configuration changes in AI applications should be tracked, reviewed, and possibly tested in the same way as code [P12]. In this line, a suggested good practice is to externalize the configuration options from the code and to maintain them in human-and machine-readable files [P13]. Use clear component and code APIs relates instead to reducing design and architectural debt by encapsulating AI functionality in software components with clear required and provided interfaces [P5,P17]. At the code level, wrapping black-box packages into custom APIs can address the glue code antipattern [P3]. Remove unnecessary features prescribes to reduce model debt by periodically examining if all input features of a model are still needed [P3], e.g. with so-called "leave-one-feature-out evaluations". Additionally, new features should not be introduced if they do not significantly contribute to the prediction performance [P17].
Refactor the code is a straightforward and generic solution to deal with design and code debt, e.g. in the form of coderelated antipatterns [P13]. While this is a general best practice, AI-based systems require collaboration with expert developers when refactoring is needed, e.g. to boost performance or reimplement complex ML algorithms [P10]. Finally, monitor deployed models suggests that ML models and their prediction performance should be closely monitored after deployment, e.g. to identify and address training/serving skew [P1,P8].

V. THREATS TO VALIDITY
Several limitations have to be mentioned for our study. Internal validity is influenced by the applied scientific rigor and potentially hidden confounding factors, both of which may impact the consistency and correctness of the results. Since selection, extraction, and synthesis activities of an SMS may rely partially on subjective interpretation, they may be prone to researcher bias. Although we diligently designed and adhered to our SMS protocol and always assigned at least two researchers to each paper, other researchers may have achieved slightly different results with our protocol. External validity is concerned with the generalizability of the results. With 21 final papers, our SMS can be regarded as comparatively small, which indicates that research on this topic is just getting started. Moreover, many of our primary studies directly reference Sculley et al. [P3] and build on their findings. The majority of publications from industry is also from large software enterprises like Google, Amazon, or Microsoft. Several results of our study are therefore heavily skewed towards Internet-scale ML systems. As a consequence, reported facets of TD types or the relevance of certain antipatterns may slightly differ in other AI contexts.

VI. CONCLUSION
In this paper, we aimed at characterizing the notions of TD and antipatterns for AI-based systems by performing a systematic mapping study. Our research questions focused on both established and new types of TD in these systems, but also on reported antipatterns and solutions. We identified four new TD types emerging in AI-based systems (data, model, configuration, and ethics debt) and observed that several established types (e.g. infrastructure, architectural, code, and test debt) are frequently occurring, although their scope was extended to include AI-specific aspects, e.g., the management and monitoring of both AI pipelines and models to mitigate infrastructure debt. We also identified and categorized 72 unique antipatterns, the majority of which relate to data and models. Finally, we identified 46 solutions that can be used to reduce or prevent debt accumulation in AI-based systems.
For industry, our results can support AI/ML professionals to better communicate aspects of TD present in their systems, to raise awareness for common antipatterns, and to identify solutions to address both. From a research perspective, our contribution provides an encompassing overview and characterization of TD and antipatterns that can emerge in the development of AI-based systems. This study may also serve as a foundation for future research that both deepens our understanding of particular AI debt types and proposes more elaborate solutions to address them. In this respect, we see potential for follow-up grey literature or interview studies in this area, as well as the development of tools and techniques to identify, address, or avoid specific AI antipatterns.