## Adaptive Quantum Generative Training using an Unbounded Loss Function

Kyle Sherbert Department of Physics Virginia Tech Blacksburg, VA kyle.sherbert@vt.edu

Jim Furches ∗ Department of Physics Virginia Tech Blacksburg, VA jfurches@vt.edu

Karunya Shirali ∗ Department of Physics Virginia Tech Blacksburg, VA karunyashirali@vt.edu

Sophia E. Economou Department of Physics Virginia Tech Blacksburg, VA

economou@vt.edu

Carlos Ortiz Marrero AI &amp; Data Analytics Division Pacific Northwest National Laboratory Seattle, WA carlos.ortizmarrero@pnnl.gov

Abstract -We propose a generative quantum learning algorithm, R´ enyi-ADAPT, using the Adaptive Derivative-Assembled Problem Tailored ansatz (ADAPT) framework in which the loss function to be minimized is the maximal quantum R´ enyi divergence of order two, an unbounded function that mitigates barren plateaus which inhibit training variational circuits. We benchmark this method against other state-of-the-art adaptive algorithms by learning random two-local thermal states. We perform numerical experiments on systems of up to 12 qubits, comparing our method to learning algorithms that use linear objective functions, and show that R´ enyi-ADAPT is capable of constructing shallow quantum circuits competitive with existing methods, while the gradients remain favorable resulting from the maximal R´ enyi divergence loss function.

Index Terms -Quantum Algorithm, Qubit, Circuit Synthesis, Logic Gates restricted to a specific architecture [10]-[12] but none provide a generic way to train quantum models that is guaranteed to avoid these no-go results. Recent work by Kieferov´ a et al. [13] suggests the use of a Quantum Divergence, an unbounded , nonlinear loss function with simple gradients, in place of a linear objective function, when performing generative training to address the barren plateau problem. The standard arguments for barren plateau theorems do not apply to divergences because these experience a logarithmic divergence when the two states are nearly orthogonal. This causes the gradients of the divergence between nearly orthogonal quantum states to be large, and thereby provides a workaround for all known barren plateau results.

## I. INTRODUCTION

Many promising quantum machine learning algorithms almost exclusively use a linear bounded operator as their objective functions during training [1]-[6]. These loss functions are typically estimated by measuring the expectation values of Hermitian operators directly during training, computing their gradients, and updating the parameters. Despite optimism, training using linear objective functions has proved computationally difficult due to a concentration of measure phenomenon known as a barren plateau that leads to exponentially small gradients as we scale the system size of our problem [7]. These results paint a bleak picture for the future of quantum machine learning, and finding a scalable approach to train generic parametrized models has become a central problem in the field. Existing approaches to overcome barren plateaus are based on empirical evidence [8], [9] or are

This work was funded by grants from the US Department of Energy, Office of Science, National Quantum Information Science Research Centers, Co-Design Center for Quantum Advantage under contract number DESC0012704.

* These authors contributed equally to this work

In our work, we extend Kieferov´ a et al. [13] method and propose an adaptive training algorithm to optimize the maximal quantum R´ enyi divergence between two quantum states, that is, the output of a quantum circuit and some quantum data state. We provide evidence showing how our adaptive algorithm performs compared to other generative training algorithms. In particular, we explore the gradient evolution of each method and argue that our R´ enyi-ADAPT has favorable gradients at scale, when compared to methods such as Overlap-ADAPT [6] and ADAPT-VQE-Gibbs [14].

## II. ADAPT-VQE ALGORITHM

Variational quantum eigensolvers (VQEs) [15]-[17] employ parametrized quantum circuits to divide the computational task of optimizing an objective function f ( θ , θ 1 2 , . . . θ n ) (most often the ground state energy of a physical system, estimated as the expectation value of the system Hamiltonian ⟨ H ⟩ ) between quantum and classical processors, in which the quantum processor performs measurements of the objective function and the classical processor updates the variational parameters { θ i } at each iteration. The choice of parametrized circuit (the 'ansatz') is critical to the performance of the algorithm, and the limited coherence times available on current quantum hardware prompt the use of shallow circuits. In general, a good ansatz needs to be sufficiently expressive to represent the solution accurately, but also have a low circuit depth.

The Adaptive Derivative-Assembled Problem Tailored ansatz (ADAPT)-VQE algorithm [18] employs an adaptive strategy to construct the variational ansatz dynamically using operators in a predefined operator pool, starting from an initial state | ψ ref ⟩ . Defining the variational parameters θ ( k ) = ( θ , . . . , θ 1 k ) and the operator pool A = { A (1) , A (2) , . . . A ( N ) } , the ansatz in iteration k +1 of the algorithm may be written as

$$| \psi _ { k + 1 } ( \theta ^ { ( k + 1 ) } ) \rangle = e ^ { - i \theta _ { k + 1 } A _ { k + 1 } } | \psi _ { k } ( \theta ^ { ( k ) } ) \rangle. \quad ( 1 )$$

In (1), the ansatz at iteration k is grown by appending operator A k +1 with coefficient θ k +1 ; the operator is chosen by measuring the energy gradients ∣ ∣ ∣ ∂ H /∂θ ⟨ ⟩ k +1 | θ k +1 =0 ∣ ∣ ∣ for each operator in the pool and selecting the one with the largest gradient. For this step, it can be shown that

$$\left | \partial \langle H \rangle / \partial \theta _ { k + 1 } | _ { \theta _ { k + 1 } = 0 } \right | = \left | \langle \psi _ { k } ( \theta ^ { ( k ) } ) | \left [ A _ { k + 1 }, H \right ] | \psi _ { k } ( \theta ^ { ( k ) } ) \rangle \right |, \quad \text{stereg} \quad \text{(2)} \ \text{Gibbs st}$$

where the right hand side of (2) can be efficiently measured on a quantum processor. The pool operator gradient-measurement step is followed by a convergence check: if the pool operator gradient norm is smaller than a threshold ε , the calculation is terminated; if not, the ansatz is grown as in (1). The ansatz-growing step is followed by a VQE optimization of all variational parameters, where the parameters ( θ , . . . , θ 1 k ) take their previously optimized values, and the newly added parameter is initialized as 0 .

In this work, we are interested in training mixed states, so we extend ADAPT following the strategy in [14], in which we use (1) to prepare a pure state on a set of visible and hidden (ancilla) qubits. The trial mixed state is taken as the state on the visible qubits when the hidden qubits are traced out:

$$\sigma _ { k } ( \theta ^ { ( k ) } ) = \text{Tr} _ { H } \left | \psi _ { k } ( \theta ^ { ( k ) } ) \right \rangle \left \langle \psi _ { k } ( \theta ^ { ( k ) } ) \right | \quad \ \ ( 3 ) \quad \ \ e ^ { - \beta H }.$$

We hereafter write σ k ( θ ( k ) ) as σ θ ( ) for notational simplicity, or even simply as σ when context allows.

The ADAPT-VQE algorithm was shown to reach arbitrarily accurate energies and yield shallow ans¨ atze when applied to small molecules, and inspired adaptive techniques for studying other problems, including time evolution and dynamics [19], [20], excited-state preparation [21], and optimization [22], [23]. In particular, Overlap-ADAPT-VQE [6], a method that we use in this work, dynamically constructs an ansatz by maximizing the overlap between a trial state and the target state. It was found to produce ultracompact ans¨ atze, thereby providing accurate reference states for subsequent ADAPT runs, and resulting in shallower circuit depths of the final ansatz compared to regular ADAPT.

The loss function in Overlap-ADAPT-VQE [6], formulated for pure states, is

$$\mathcal { L } _ { O } ( \theta ) = \langle \Psi _ { \text{target} } | \psi ( \theta ) \rangle.$$

For mixed states ρ and σ , the fidelity has the following generalization:

$$F ( \rho, \sigma ) = \text{Tr} \sqrt { \sqrt { \rho } \, \sigma ( \theta ) \, \sqrt { \rho } }$$

In this work, we extend Overlap-ADAPT-VQE to prepare mixed states, using the loss-function 1 -F ρ, σ θ ( ( )) 2 , where ρ is the target state and σ θ ( ) is the trial state.

$$\mathcal { L } _ { O } ( \rho, \sigma ( \theta ) ) = 1 - \left ( \text{Tr} \sqrt { \sqrt { \rho } \, \sigma ( \theta ) \, \sqrt { \rho } } \right ) ^ { 2 }. \quad ( 6 )$$

## III. THERMAL STATE PREPARATION USING THE ADAPT-VQE-GIBBS ALGORITHM

An adaptive protocol for the preparation of Gibbs thermal states ρ G = e -βH /Z was proposed in [14] in which an easily measurable customized objective function C σ ( ) was introduced; it replaced the Gibbs free energy F σ ( ) = E σ ( ) -k B TS σ ( ) as the objective function to minimize. Using this objective function, in combination with the ADAPT-VQE strategy of dynamically constructing ans¨ atze from a predefined operator pool, it was shown that accurate representations of the Gibbs state could be obtained across a range of temperatures using low-depth circuits. The protocol makes use of a purifying ancilla system A in addition to the data system D to generate a pure state | ψ ⟩ ∈ H D ⊗H A , where the Gibbs state ρ G of system D is obtained by tracing out the ancillary system (the visible and hidden qubits in this work correspond to the data and ancilla qubits, respectively, in ADAPT-VQE-Gibbs). The modified objective function

$$C ( \sigma ( \theta ) ) = - \text{Tr} ( \rho _ { G } \sigma ) + \frac { 1 } { 2 } \text{Tr} ( \sigma ^ { 2 } ), \text{ \quad \ \ } ( 7 )$$

where σ = Tr A | ψ ⟩⟨ ψ | is the variational trial state, and ρ G is an approximation of the target Gibbs state. Using C σ ( ) as the objective function is convenient because it circumvents the need to measure the von Neumann entropy, which is hard to measure on current quantum hardware. The approximation of ρ G in (7) is obtained via a truncated Taylor expansion of e -βH /Z .

In our calculations, we truncate the Taylor expansion of e -βH at order m = 5 , i.e., set e -βH ≈ ∑ 5 n =0 1 n ! ( -β ) n H n , which was found to lead to Gibbs states of fidelities comparable to those obtained when using the exact operator produced by summing the infinite series [14].

## IV. R´ ENYI-ADAPT

The loss function which we are interested in comparing is a generalization of the quantum relative entropy known as the quantum R´ enyi divergence or 'sandwiched' R´ enyi relative entropy [24]-[26]. The Quantum R´ enyi divergence inherits many of the mathematical properties of the R´ enyi divergence, and it reduces to the quantum relative entropy (quantum analog of the KL-divergence) and the limit where the dominant parameter of the loss function goes to 1. We are particularly interested in using an upper bound to the quantum R´ enyi divergence called the maximal R´ enyi divergence of order 2,

$$\widetilde { D } _ { 2 } ( \sigma ( \theta ) | \rho ) = \log \left ( \text{Tr} \left ( \sigma ^ { 2 } \rho ^ { - 1 } \right ) \right ). \quad \quad ( 8 )$$

̸

Here, ρ is the density of the training data state and σ θ ( ) corresponds to the density of the output state of a parameterized quantum circuit, where θ are the circuit parameters. These loss functions have not been widely studied in a quantum computing context and were introduced in [13] as a way to do quantum generative training. The main reason for using ˜ D 2 ( σ ρ | ) as a loss function is that it upper-bounds the quantum relative entropy and its gradients are considerably simpler to measure on quantum hardware than those of the ordinary R´ enyi divergence and quantum relative entropy [13]. Please consult Appendix B for more information on the gradients. Note that, in general, ˜ D 2 ( ρ σ | ) = ˜ D 2 ( σ ρ | ) . However, if both ρ and σ are full rank, ˜ D 2 ( ρ σ | ) = ˜ D 2 ( σ ρ | ) = 0 if and only if ρ = σ [27]. This makes ˜ D 2 ( σ θ ( ) | ρ ) is a reasonable loss function to consider for generative training.

As described in Section II, the ADAPT algorithm selects an operator from a pool with the largest energy gradient at each iteration. We can apply a similar logic, but instead use the gradient of the maximal R´ enyi divergence to select operators from a pool { H ,. . . , H 1 k } to form a trial state σ θ ( ) following the logic outlined in equations (1) and (3), which we can then optimize at each iteration. This technique is what we call R´ enyi-ADAPT.

## V. EXPERIMENTS

Following the notation of [13], we denote the n -qubit target thermal state as ρ , the controllable parameters as θ , and the trial state as σ θ ( ) . In the context of the algorithm, the trial state starts as a pure state acting on a register of n V visible qubits and n H hidden (ancilla) qubits, within a full Hilbert space of H σ = H V ⊗ H H . The final (mixed) state output of σ θ ( ) is obtained by tracing out the hidden register. Unless otherwise noted, all experiments were carried out with n H = n V , and we use the letter n to denote both. All multivariate optimization steps were performed using the BFGS algorithm, as implemented in the Optim.jl Julia package [28], [29].

## A. Adaptive optimization

In order to demonstrate that the adaptive optimization process works, we ran ADAPT on a single random two-local thermal state ρ of n = 3 qubits. We trained ans¨ atze with three different loss functions (6,7,8) to learn ρ . For the operator pool, we include all one- and two-local Paulis acting on any of the six qubits (three visible and three hidden). There are a total of ( ) 6 1 · 3 1 = 18 one-local operators and ( ) 6 2 · 3 2 = 135 two-local operators, for a total of 153 operators. In addition to the three ADAPT variants investigated, we ran VQE with a fixed-structure ansatz ('VQE') consisting of all 153 Pauli rotations (with arbitrary ordering) for all three loss functions, to compare against previous work [13].

Like the other adaptive methods, R´ enyi-ADAPT successfully learned the target state (Fig. 1). Because the VQE ans¨ atze start out sufficiently expressive to reach the target state with just one optimization, they required drastically fewer iterations to converge compared to the adaptive methods, implying the fixed-structure approach would require drastically fewer circuit

Fig. 1. Loss curves to train a random two-local thermal state on 3 system qubits and 3 ancillae, using the maximal Renyi divergence (8), the overlap (6), and the modified objective in ADAPT-VQE-Gibbs (7) as cost functions. Note that the latter has its minimum at a non-zero value, so we plot the difference between the loss of the trial state and the target state. Markers indicate where individual optimizations completed and a new Pauli rotation was added in the ADAPT runs. The dotted lines give the loss curve when running a single VQE optimization on an ansatz consisting of all 153 Pauli rotations.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00218v1-figure-f84d1308-ea9b-402a-bb1c-7217d8eb364d.jpg)

TABLE I PARAMETER COUNTS OF EACH ALGORITHM

| Algorithm       |   Parameters |
|-----------------|--------------|
| VQE             |          153 |
| Overlap-ADAPT   |           52 |
| ADAPT-VQE-Gibbs |           57 |
| R´nyi-ADAPT e   |           61 |

Fig. 2. ADAPT convergence to train a random two-local thermal state, for each choice of loss function. Different line styles show results for different system sizes n . The results shown are worst-case infidelities collected from up to 20 trials of randomly sampled target states and reference states (due to time constraints, we completed only one successful trial with n = 4 from each method).

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00218v1-figure-f8f431cd-b1f3-4a3f-bccb-49ceda359f83.jpg)

evaluations in a quantum experiment. In contrast, adaptive methods achieved excellent loss values with at least a 60% reduction in parameter count (Table I) compared to the VQE ans¨ atze. This translates to a commensurate reduction of circuit depth in a quantum experiment, making the adaptive approach the preferred method for devices with limited coherence times.

## B. Increasing system size

Next, we investigate the robustness of our method for larger system sizes. Similarly to the previous experiment, each algorithm was tested in 20 random thermal states, where the number of visible qubits now varied in the range n ∈ [1 , 4] . For each method, we recorded the worst-case infidelity over all instances after each ADAPT iteration, enabling a direct comparison of compactness between the three loss functions.

For all system sizes, overlap-ADAPT achieved a given infidelity threshold with the fewest parameters, but performance was competitive among the loss functions (Fig. 2). At the largest system size of n = 4 , ADAPT-VQE-Gibbs and R´ enyi-ADAPT stopped near infidelities of 10 -3 while overlapADAPT reached 10 -4 . However, we were only able to run 1 trial at this size due to computational constraints, and so this behavior may not be representative. Interestingly, all methods required exponentially deeper circuits as n increased due to the larger Hilbert space dimension and the locality of the operator pool. These simulations suggest that the R´ enyi divergence does not provide an advantage in terms of circuit depth.

Fig. 3. Largest pool gradient in the first ADAPT selection, as a function of system size n . Error bars show the full range of values from up to 20 trials of randomly sampled target states and reference states. As n increases, all pool gradients for the overlap and for ADAPT-VQE-Gibbs vanish exponentially, indicating the presence of barren plateaus. For large enough systems, the gradients will be too small to resolve the optimal choice of operator, and ADAPT will fail to start. However, gradients for the R´ enyi divergence are nearly constant.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00218v1-figure-25ce420a-7900-4e97-ac1e-4f80f542883c.jpg)

It is worth mentioning here that the loss functions for R´ enyi divergence ˜ D 2 ( ρ σ θ ∥ ( )) (8) and overlap L O ( ρ, σ ( θ )) (6) were evaluated using the exact thermal state ρ , while the loss of ADAPT-VQE-Gibbs C σ θ ( ( )) (7) was calculated using a truncated Taylor expansion of the thermal state, thus limiting the fidelity it could achieve (although it still achieves &gt; 99 8% . fidelity for the n = 4 trial shown). This approximation was intentional; in most quantum machine learning applications, the target state ρ is known, whereas in ADAPT-VQE-Gibbs, which was designed for quantum chemical and condensed matter systems, ρ is the desired output for which a priori estimates are not guaranteed, necessitating the need for an approximation.

## C. Gradient results

The unbounded nature of the maximal R´ enyi divergence lends it resilience to barren plateaus, as discussed by Kieferova et al. [13], providing better trainability for variational circuits. We investigated this claim by analyzing the gradients of the loss functions.

In a similar setup as before, using 20 random combinations of the target state ρ and the reference state σ 0 , we calculated the initial pool gradients used to select the first operator. We denote the magnitude of the largest gradient as || g || ∞ , which we refer to as the L ∞ norm of the pool gradient. Because this did not require the full ADAPT procedure, we were able to run experiments on a system size of n = 6 (12 total qubits, with n H = n V ) . We expected that barren plateaus would manifest as an exponential decay of || g || ∞ as n increased, and therefore hypothesized that the R´ enyi divergence would not decay.

The results are shown in Fig. 3 with the lines indicating the median of the initial pool gradient ∞ -norm over the trials and the shaded regions depicting the full range of values. The

TABLE II DECAY OF INITIAL POOL GRADIENT

| Loss    | Fitted curve                      | Predicted failure at 10 - 5   |
|---------|-----------------------------------|-------------------------------|
| Overlap | || g || ∞ = 1 . 644 × 2 . 162 - n | n = 16                        |
| Gibbs   | || g || ∞ = 1 . 648 × 2 . 354 - n | n = 14                        |
| R´nyi e | || g || ∞ = 1 . 676 × 1 . 198 - n | n = 67                        |

R´ enyi divergence exhibited larger, nearly constant gradients as the system size n increased, while the overlap and Gibbs loss functions decayed exponentially at rates similar to each other.

Table II contains exponential functions fitted to the medians of each loss function with respect to n . While R´ enyi-ADAPT still suffers exponential decay in the large n limit, the decay is significantly slower. To illustrate by how much slower, let us imagine that the gradient measurement has a resolution of 10 -5 , such that when all operators have gradients below this threshold (i.e. || g || ∞ &lt; 10 -5 ), the ADAPT protocol cannot meaningfully begin. We denote this in Table II as the 'predicted failure'. Using the loss functions for either overlap-ADAPT or ADAPT-VQE-Gibbs, the algorithm would fail to start at n ≈ 15 qubits (16 and 14, respectively), while with the R´ enyi divergence, this threshold is not reached until n = 67 , a 4x improvement. For our lower-precision criteria of || g || ∞ = 10 -3 , we expect that overlap-ADAPT and ADAPT-VQE-Gibbs will become untrainable at system sizes of n ≈ 10 . Note that our simulations ( n ≤ 6) are well below this regime, such that this advantage of R´ enyi-ADAPT is not reflected in the experiments whose results are shown in Figs. 1 and 2.

## D. Initial state

The choice of reference state is important for the performance of ADAPT, with good reference states being close to the target [6]. For poor reference states, ADAPT may require many more operators, removing its compact-circuit advantage, or even fail to converge altogether. Some examples of good reference states include the Hartree-Fock state for the ground state of molecular electronic structure problems [18] and the N´ eel antiferromagnetic state for the ground state of lattice spin systems [30]. For overlap-ADAPT and R´ enyi-ADAPT, the ideal reference state is, of course, σ 0 = ρ .

An interesting possibility of the R´ enyi divergence is that it could avoid concentrating (or flattening out) far away from ρ . If true, this would improve the trainability of circuits and reduce the need to choose a good reference state, which would be helpful when one is either hard to produce or completely unknown for the given problem.

After processing the data in Fig. 2, we calculated the fidelities F ρ, σ ( 0 ) between the random target and reference states. The initial pool gradient || g || ∞ is shown in Fig. 4 as a function of F . We observed that further away from ρ , the gradient landscapes for ADAPT-VQE-Gibbs and overlapADAPT decayed exponentially, suggesting that trainability could become an issue. In contrast, R´ enyi-ADAPT again demonstrated favorable near-constant gradients (indicative of

Fig. 4. Largest pool gradient in the first ADAPT selection, as a function of fidelity between the reference state σ 0 and the target thermal state ρ . Markers for smaller systems are plotted with more transparency. When the fidelity starts very small, its gradient is also very small, indicating the presence of a barren plateau in the cost-function landscape far from the target state. By contrast, the Renyi divergence exhibits large gradients throughout.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00218v1-figure-40661b40-f3dc-491c-b420-fc7a381135a8.jpg)

Fig. 5. Largest pool gradient element over the course of twenty ADAPT runs for n = 3 . Completion is calculated per run as the number of ansatz parameters divided by the final ansatz length, which was defined to terminate once the pool gradient fell below 10 -3 . On average, gradients decline exponentially (linear on the log plot) as ADAPT progresses. However, once an ansatz is sufficiently expressive, no further improvements can be made and the gradients drop off rapidly. For a threshold of ε = 10 -3 , only R´ enyi-ADAPT reaches this regime (albeit with a longer ansatz, as can be seen Figs. 1 and 2).

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00218v1-figure-696caa1b-0c0e-445c-8495-21a420b5bb5f.jpg)

the loss landscape not flattening), supporting the idea that it functions effectively with a wider range of reference states.

## E. Completion

ADAPT-VQE experiments require a specific choice of the value ε to take as the gradient norm threshold. In this work, we adopted the somewhat arbitrary choice of ε = 1 e -3 . To assess the degree to which this choice was appropriate, we examine the L ∞ norm of the pool gradient || g || ∞ for each ADAPT iteration, for all 20 trials where n = 3 qubits. As a general trend, pool gradients tend to fall exponentially as each parameter increases the expressivity of the ansatz. However, when no further improvements can be made to the trial state, all pool gradients will rapidly drop to zero. To highlight this behavior, we plot in Fig. 5 the L ∞ norm of the pool gradient as a function of the percent completion of the ADAPT run, where a 'completed' run is one where the final || g || ∞ &lt; ε .

Our results show that, on average, || g || ∞ values for both ADAPT-VQE-Gibbs and overlap-ADAPT decay only exponentially all the way through the || g || ∞ &lt; ε stopping condition. Meanwhile, the R´ enyi-ADAPT curves exhibit superexponential decay roughly 80% of the way to the same stopping condition, followed by several runs dropping from || g || ∞ &gt; 1 e -3 to || g || ∞ &lt; 1 e -4 in a single ADAPT iteration. We reiterate that R´ enyi-ADAPT tended to require deeper circuits to attain this convergence (as observed in Figs. 1 and 2), so this observation does not necessarily indicate an advantage for R´ enyi-ADAPT, especially in terms of nearterm viability. However, this does suggest that R´ enyi-ADAPT requires a lower minimum resolution when measuring pool gradients to attain a region of strong convergence, implying that each pool operator could be selected in R´ enyi-ADAPT using a lower shot count than in the other methods, helping to offset the major cost of measurement overhead incurred by the ADAPT protocol.

## VI. DISCUSSION

In this work, we characterize R´ enyi-ADAPT as a method resistant to barren plateaus for preparing thermal states. It combines the trainability advantages of the maximal R´ enyi divergence [13] with the adaptive optimization procedure of ADAPT-VQE [18]. We did not observe the gradients obtained in R´ enyi-ADAPT to decay in our experiments. We conclude with a discussion of our results and how they relate to the existing literature on barren plateaus.

A recent review of barren plateaus by Larocca et al. [7] summarizes the types and origins of barren plateaus. They can arise from highly expressive circuits, poor choice of initial state, global measurements, and noise. The relationship between these is studied for 2-designs in [31], [32].

Expressiveness By iteratively constructing circuits, ADAPT balances the trade-off between constructing a sufficiently expressive circuit to reach the objective minimum and overparameterizing the circuit, which would induce barren plateaus.

Our results in Figs. 1 and 2 show overlap-ADAPT producing the most compact circuits in terms of parameter count. On devices where producing the most compact circuits is a priority (like near-term devices with limited coherence times), we surmise that the overlap would be the best choice of the algorithms we tested, provided a suitable reference state.

Note that the expressiveness of the circuit is characterized by the size of its dynamical Lie algebra (DLA) [7], [31], [32]. Since ADAPT can, in principle, append a previouslyadded operator (which would not increase the DLA), the DLA may be smaller than expected by assuming unique operators for each parameter. However, similar to the existing ADAPT literature [6], [14], [18], [30], we use parameter counts as a proxy for circuit expressiveness and leave the quantification of DLA to future work.

Measurement The compactness of ADAPT comes at the considerable expense of measuring the operator pool at each iteration. The 1- and 2-local pool (which has 3( n V + n H ) + 9 · ( n V + n H 2 ) operators) we used is complete. However, by using these small, local operators, constructing a large unitary requires many more iterations. We could alternatively choose a minimal operator pool, which has been shown to have a number of operators linear in the number of qubits. [33], [34] This minimizes the measurement overhead in each ADAPT iteration, at the cost of requiring an even longer circuit to reach arbitrary states.

On the other hand, by expanding the pool to include all k -local Paulis, we can reduce the number of parameters required in the circuit, at the cost of significantly more measurements required to measure the full pool gradient. According to current barren plateau research, the increased globality of the measurements could be problematic, suggesting there is a trade-off between including larger pool operators and trainability. We did not investigate this in our work.

Initial State Choosing a random reference state increases the chances of a barren plateau, following the curse of dimensionality [7]. As detailed in Appendix A, we had to choose a random reference state to allow ADAPT to construct mixed states.

Our experiments show that R´ enyi-ADAPT has larger gradients for our reference states both with respect to increasing system size (Fig. 3) and larger distances from the objective (Fig. 4). Therefore, at least in the absence of a good reference state, overlap-ADAPT and ADAPT-VQE-Gibbs should have to make exponentially more measurements of the gradient, or they might possibly not start at all, a reason to favor R´ enyiADAPT.

As we used a state vector simulator with infinite shots, this was not shown in our experiments. Additionally, our simulations did not reach the scale where we anticipate overlap-ADAPT and ADAPT-VQE-Gibbs to start below the convergence criteria.

Noise Noise-induced barren plateaus are perhaps the most significant obstacle to leveraging near-term quantum devices that lack error correction. Longer circuits are generally noisier, which is the original motivation behind ADAPT-VQE [18].

We did not test with a noisy quantum simulator, which we leave to future work. Our noiseless results show overlapADAPT producing more compact circuits, which one would expect to have less noise. However, as noise concentrates the loss function, this restricts the space of good reference states, which we found to be a potential obstacle to the scalability of overlap-ADAPT. Investigating this interplay would be interesting. Additionally, since R´ enyi-ADAPT was more trainable (initially) with poor reference states, one could imagine that a hybrid algorithm where R´ enyi-ADAPT prepares a reference state for overlap-ADAPT might perform well, combining the advantages of each algorithm we observed.

## VII. DATA AVAILABILITY

The code and data used for the simulations and figures can be found at https://github.com/kmsherbertvt/RenyiADAPT.

## ACKNOWLEDGMENT

The authors would like to thank Ada Warren for her helpful comments and discussion on the ADAPT-VQE-Gibbs code. This research was supported by the DOE Office of Science, National Quantum Information Science Research Centers, Codesign Center for Quantum Advantage (C2QA) under contract number DE-SC0012704.

## REFERENCES

- [1] Alberto Peruzzo, Jarrod McClean, Peter Shadbolt, Man-Hong Yung, Xiao-Qi Zhou, Peter J. Love, Al´ an Aspuru-Guzik, and Jeremy L. O'Brien. A variational eigenvalue solver on a photonic quantum processor. Nature Communications , 5(1):4213, Jul 2014.
- [2] Edward Farhi, Jeffrey Goldstone, and Sam Gutmann. A quantum approximate optimization algorithm. arXiv preprint arXiv:1411.4028 , 2014.
- [3] Vojtˇ ech Havl´ ıˇek, c Antonio D C´ orcoles, Kristan Temme, Aram W Harrow, Abhinav Kandala, Jerry M Chow, and Jay M Gambetta. Supervised learning with quantum-enhanced feature spaces. Nature , 567(7747):209212, 2019.
- [4] Maria Schuld, Alex Bocharov, Krysta M Svore, and Nathan Wiebe. Circuit-centric quantum classifiers. Physical Review A , 101(3):032308, 2020.
- [5] Jack Ceroni, Torin F Stetina, Maria Kieferova, Carlos Ortiz Marrero, Juan Miguel Arrazola, and Nathan Wiebe. Generating approximate ground states of molecules using quantum machine learning. arXiv preprint arXiv:2210.05489 , 2022.
- [6] C´ esar Feniou, Muhammad Hassan, Diata Traor´ e, Emmanuel Giner, Yvon Maday, and Jean-Philip Piquemal. Overlap-adapt-vqe: practical quantum chemistry on quantum computers via overlap-guided compact ans¨ atze. Communications Physics , 6(1), July 2023.
- [7] Martin Larocca, Supanut Thanasilp, Samson Wang, Kunal Sharma, Jacob Biamonte, Patrick J. Coles, Lukasz Cincio, Jarrod R. McClean, Zo¨ e Holmes, and M. Cerezo. A review of barren plateaus in variational quantum computing, 2024.
- [8] Edward Grant, Leonard Wossnig, Mateusz Ostaszewski, and Marcello Benedetti. An initialization strategy for addressing barren plateaus in parametrized quantum circuits. Quantum , 3:214, 2019.
- [9] Andrea Skolik, Jarrod R McClean, Masoud Mohseni, Patrick van der Smagt, and Martin Leib. Layerwise learning for quantum neural networks. Quantum Machine Intelligence , 3(1):1-11, 2021.
- [10] Arthur Pesah, M Cerezo, Samson Wang, Tyler Volkoff, Andrew T Sornborger, and Patrick J Coles. Absence of barren plateaus in quantum convolutional neural networks. arXiv preprint arXiv:2011.02966 , 2020.
- [11] M Cerezo, Akira Sone, Tyler Volkoff, Lukasz Cincio, and Patrick J Coles. Cost-function-dependent barren plateaus in shallow quantum neural networks. arXiv preprint arXiv:2001.00550 , 2020.
- [12] Kunal Sharma, M Cerezo, Lukasz Cincio, and Patrick J Coles. Trainability of dissipative perceptron-based quantum neural networks. arXiv preprint arXiv:2005.12458 , 2020.
- [13] Maria Kieferova, Carlos Ortiz Marrero, and Nathan Wiebe. Quantum generative training using r´nyi e divergences. arXiv preprint arXiv:2106.09567 , 2021.
- [14] Ada Warren, Linghua Zhu, Nicholas J. Mayhall, Edwin Barnes, and Sophia E. Economou. Adaptive variational algorithms for quantum gibbs state preparation, 2022.
- [15] M. Cerezo, Andrew Arrasmith, Ryan Babbush, Simon C. Benjamin, Suguru Endo, Keisuke Fujii, Jarrod R. McClean, Kosuke Mitarai, Xiao Yuan, Lukasz Cincio, and Patrick J. Coles. Variational quantum algorithms. Nature Reviews Physics , 3(9):625-644, Sep 2021.
- [16] Jules Tilly, Hongxiang Chen, Shuxiang Cao, Dario Picozzi, Kanav Setia, Ying Li, Edward Grant, Leonard Wossnig, Ivan Rungger, George H. Booth, and Jonathan Tennyson. The variational quantum eigensolver: A review of methods and best practices. Physics Reports , 986:1-128, 2022. The Variational Quantum Eigensolver: a review of methods and best practices.
- [17] Kishor Bharti, Alba Cervera-Lierta, Thi Ha Kyaw, Tobias Haug, Sumner Alperin-Lea, Abhinav Anand, Matthias Degroote, Hermanni Heimonen, Jakob S. Kottmann, Tim Menke, Wai-Keong Mok, Sukin Sim, LeongChuan Kwek, and Al´ an Aspuru-Guzik. Noisy intermediate-scale quantum algorithms. Rev. Mod. Phys. , 94:015004, Feb 2022.
- [18] Harper R. Grimsley, Sophia E. Economou, Edwin Barnes, and Nicholas J. Mayhall. An adaptive variational algorithm for exact molecular simulations on a quantum computer. Nature Communications , 10(1):1-9, July 2019.
- [19] Feng Zhang, Niladri Gomes, Yongxin Yao, Peter P. Orth, and Thomas Iadecola. Adaptive variational quantum eigensolvers for highly excited states. Phys. Rev. B , 104:075159, Aug 2021.
- [20] Niladri Gomes, Anirban Mukherjee, Feng Zhang, Thomas Iadecola, CaiZhuang Wang, Kai-Ming Ho, Peter P. Orth, and Yong-Xin Yao. Adaptive variational quantum imaginary time evolution approach for ground state preparation. Advanced Quantum Technologies , 4(12):2100114, 2021.
- [21] Yong-Xin Yao, Niladri Gomes, Feng Zhang, Cai-Zhuang Wang, KaiMing Ho, Thomas Iadecola, and Peter P. Orth. Adaptive variational quantum dynamics simulations. PRX Quantum , 2:030307, Jul 2021.
- [22] Linghua Zhu, Ho Lun Tang, George S. Barron, F. A. Calderon-Vargas, Nicholas J. Mayhall, Edwin Barnes, and Sophia E. Economou. Adaptive quantum approximate optimization algorithm for solving combinatorial problems on a quantum computer. Phys. Rev. Res. , 4:033029, Jul 2022.
- [23] Yanzhu Chen, Linghua Zhu, Chenxu Liu, Nicholas J. Mayhall, Edwin Barnes, and Sophia E. Economou. How much entanglement do quantum optimization algorithms require?, 2023.
- [24] Mark M Wilde, Andreas Winter, and Dong Yang. Strong converse for the classical capacity of entanglement-breaking and hadamard channels via a sandwiched r´ enyi relative entropy. Communications in Mathematical Physics , 331(2):593-622, 2014.
- [25] Martin M¨ uller-Lennert, Fr´ ed´ric e Dupuis, Oleg Szehr, Serge Fehr, and Marco Tomamichel. On quantum r´ enyi entropies: A new generalization and some properties. Journal of Mathematical Physics , 54(12):122203, 2013.
- [26] D´ enes Petz. Quasi-entropies for finite quantum systems. Reports on mathematical physics , 23(1):57-65, 1986.
- [27] Tim Van Erven and Peter Harremos. R´ enyi divergence and kullbackleibler divergence. IEEE Transactions on Information Theory , 60(7):3797-3820, 2014.
- [28] Patrick Kofod Mogensen and Asbjørn Nilsen Riseth. Optim: A mathematical optimization package for Julia. Journal of Open Source Software , 3(24):615, 2018.
- [29] Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral B. Shah. Julia: A Fresh Approach to Numerical Computing. SIAM Review , 59(1):65-98, January 2017. Publisher: Society for Industrial and Applied Mathematics.
- [30] John S. Van Dyke, Karunya Shirali, George S. Barron, Nicholas J. Mayhall, Edwin Barnes, and Sophia E. Economou. Scaling adaptive quantum simulation algorithms via operator pool tiling. Phys. Rev. Res. , 6:L012030, Feb 2024.
- [31] Michael Ragone, B. Bakalov, Fr´ ed´ eric Sauvage, Alexander F. Kemper, Carlos Ortiz Marrero, Mart´ ın Larocca, and M. Cerezo. A Unified Theory of Barren Plateaus for Deep Parametrized Quantum Circuits. arXiv , 2023.
- [32] Enrico Fontana, Dylan Herman, Shouvanik Chakrabarti, Niraj Kumar, Romina Yalovetzky, Jamie Heredge, Shree Hari Sureshbabu, and Marco Pistoia. The Adjoint Is All You Need: Characterizing Barren Plateaus in Quantum Ans¨ atze. arXiv , 2023.
- [33] Ho Lun Tang, V.O. Shkolnikov, George S. Barron, Harper R. Grimsley, Nicholas J. Mayhall, Edwin Barnes, and Sophia E. Economou. Qubitadapt-vqe: An adaptive algorithm for constructing hardware-efficient ans¨ atze on a quantum processor. PRX Quantum , 2:020310, Apr 2021.
- [34] V. O. Shkolnikov, Nicholas J. Mayhall, Sophia E. Economou, and Edwin Barnes. Avoiding symmetry roadblocks and minimizing the measurement overhead of adaptive variational quantum eigensolvers. Quantum , 7:1040, June 2023.

| Hyperparameter                     | Value    |
|------------------------------------|----------|
| Target qubits n                    | 1-6      |
| Hidden qubits n H                  | Set to n |
| Overlap convergence ||∇ F || ∞     | 10 - 3   |
| Gibbs convergence ||∇ C || ∞       | 10 - 3   |
| R´nyi e convergence ||∇ ˜ D 2 || ∞ | 10 - 3   |

Operator pool

P

(2

n

)

∪ P

(2

n

)

TABLE III

1

2

EXPERIMENT HYPERPARAMETERS

## APPENDIX

## A. Hyperparameters

We give the hyperparameters of our experiments in Table III. P ( n ) k denotes the set of all k -local Pauli operators that act on n qubits. The gradient convergence expressions refer to the gradient of the operator pool.

The Hamiltonians used in the experiments were generated by assigning random coefficients drawn from a normal distribution with zero mean and unit variance to one- and two-local Pauli operators on the n V visible qubits. To ensure that the energy scale remained the same with different system sizes, the Hamiltonians ˆ H = ∑ i c P i i (where each P i is oneor two-local) were subsequently normalized such that c i → c i || c || . Twenty such Hamiltonians were generated to sample various two-local models. The operator pool consisted of all oneand two-local Pauli operators on the total number of qubits n V + n H .

The reference states used were produced by generating states that are partially-entangled between the visible and hidden qubits. As discussed in previous work [14], reference states without any entanglement between the visible and hidden qubits maximize the purity Tr ( ρ 2 ) of ρ = Tr A | ψ ⟩⟨ ψ | . Nonzero gradients of the ADAPT-VQE-Gibbs objective function are then obtained only from the first term in (7); the structure of the pool operators then implies that the operator with the largest gradient will always be local to the visible system qubits, without the ability to generate entanglement between it and the hidden system. This would result in the true Gibbs state (a mixed state) being unattainable. A similar argument can be made against the use of maximally mixed reference states. Correspondingly, for R´ enyi-ADAPT, it was found that for product reference states, individual Pauli operators were incapable of generating entanglement between the visible and hidden qubits, and that operators consisting of linear combinations of diagonal and off-diagonal Pauli operators ('diagonal' here refers to operators whose eigenvectors the product reference is composed of) would be necessary in order for such entanglement to be created.

In order to overcome the constraint on the pool operators arising from using product states, partially-entangled reference states were generated by first applying random single-qubit R y rotations to each qubit in the state | 0 ⟩ ⊗ n V + n H , with the angle being drawn from [ -π, π ) , followed by a layer of CNOT gates between each visible-hidden qubit pair, with the control being on the hidden qubits. Twenty such reference states were generated to sample various initializations. We leave the investigation of operator pools consisting of linear combinations of Pauli operators to future work.

The convergence criterion for ADAPT was set such that the calculation was ended when the gradient norm of the pool operators was below 1 e -3 .

## B. Gradients

In order to select operators and conduct gradient descent, it is essential to have a formula for gradients that can be measured in a quantum computer. Below we provide a discussion of the closed form expression of the gradients of each loss function that we considered in our work:

1) Overlap-ADAPT: The expression for the pool operator gradients at step k +1 may be written as [6]

$$\begin{array} { c } o \, \text{the} \quad & \frac { \partial } { \partial \theta _ { k + 1 } } \, \langle \Psi _ { \text{target} } \left | e ^ { - i \theta _ { k + 1 } A _ { k + 1 } } \psi _ { k } ( \theta ^ { ( k ) } ) \rangle \right | _ { \theta _ { k + 1 } = 0 } \\ \text{first} _ { \cdot } \quad & = \langle \Psi _ { \text{target} } \left | i A _ { k + 1 } \psi _ { k } ( \theta ^ { ( k ) } ) \rangle. \quad ( 9 ) \\ \cdot \quad \end{array}$$

2) ADAPT-VQE-Gibbs: In the case where the ansatz is composed of operators e -iθG whose Hermitian generators G have exactly two distinct eigenvalues ϵ 0 and ϵ 1 , the exact gradients of the objective function can be efficiently obtained using the parameter-shift rule. Defining an auxiliary function

$$\tilde { C } ( \theta, \phi ) = - \text{Tr} ( \rho _ { G } \sigma ( \theta ) ) + \frac { 1 } { 2 } \text{Tr} ( \sigma ( \theta ) \sigma ( \phi ) ), \quad ( 1 0 )$$

the gradient of C σ θ ( ( )) may be written as

$$\text{and} \intertext {acc}$$

where r = 1 2 ( ϵ 1 -ϵ 0 ) .

3) R´ enyi-ADAPT: This divergence has closed-form gradients that can be efficiently computed on a Quantum Computer. Define a parameterized quantum circuit by σ v := Tr h [ Π N j =1 e -iH θ j j | ψ ref ⟩ ⟨ ψ ref | Π 1 j = N e iH θ j j ] , where we measure the output density by taking a partial trace, denoted by Tr h , over a hidden subspace of some set of ancilla qubits. In this case, the gradients of equation 8 take the form,

$$\int _ { \text{nce} } \int _ { \text{nce} } \int _ { \text{that} } \int _ { \text{$\dots$} } \int _ { \text{$\dots$} } \int _ { \text{$\dots$} } \int _ { \text{$\dots$} } \int _ { \text{$\dots$} } \int _ { \text{$\dots$} } \int _ { \text{$\dots$} } \int _ { \text{$\dots$} } \int _ { \text{$\dots$} } \int _ { \text{$\dots$} } \int _ { \text{$\dots$} } \int _ { \text{$\dots$} } \int _ { \text{$\dots$} } \int _ { \text{$\dots$} } \int _ { \text{$\dots$} } \end{array}$$

where ˜ H k = ∏ k -1 j =1 e -iH θ j j H k ∏ 1 j = k -1 e iH θ j j . See Ref. [13] for more details on the algorithms and access model needed for this calculation.
Kyle Sherbert Jim Furches Karunya Shirali Sophia E. Economou Carlos Ortiz Marrero
Adaptive Quantum Generative Training using an Unbounded Loss Function
We propose a generative quantum learning algorithm, R\'enyi-ADAPT, using the
Adaptive Derivative-Assembled Problem Tailored ansatz (ADAPT) framework in
which the loss function to be minimized is the maximal quantum R\'enyi
divergence of order two, an unbounded function that mitigates barren plateaus
which inhibit training variational circuits. We benchmark this method against
other state-of-the-art adaptive algorithms by learning random two-local thermal
states. We perform numerical experiments on systems of up to 12 qubits,
comparing our method to learning algorithms that use linear objective
functions, and show that R\'enyi-ADAPT is capable of constructing shallow
quantum circuits competitive with existing methods, while the gradients remain
favorable resulting from the maximal R\'enyi divergence loss function.