extfieldsofstudy:
Computer Science
Mathematics
Engineering

text:
On data-driven control: informativity of noisy input-output data with cross-covariance bounds

In this paper we develop new data informativity based controller synthesis methods that extend existing frameworks in two relevant directions: a more general noise characterization in terms of cross-covariance bounds and informativity conditions for control based on input-output data. Previous works have derived necessary and sufficient informativity conditions for noisy input-state data with quadratic noise bounds via an S-procedure. Although these bounds do not capture cross-covariance bounds in general, we show that the S-procedure is still applicable for obtaining non-conservative conditions on the data. Informativity-conditions for stability, $\mathcal{H}_\infty$ and $\mathcal{H}_2$ control are developed, which are sufficient for input-output data and also necessary for input-state data. Simulation experiments illustrate that cross-covariance bounds can be less conservative for informativity, compared to norm bounds typically employed in the literature.

distributed controller design [7] and dissipativity analysis [8], [9] from data, and represent, for example, magnitude, energy and variance bounds on the noise.
In this paper, we consider the problem of determining informativity of input-output and input-state data for control with prior knowledge of the noise in the form of a sample cross-covariance type bound with respect to a user-chosen instrumental signal. Bounds on the sample cross-covariance were introduced in [10] as an alternative to magnitude bounds in parameter bounding identification, given its overly conservative noise characterization, cf. [11] for a comparison of instantaneous and (quadratic) energy type bounds for datadriven control. Our approach to data-driven control extends existing frameworks in two relevant directions: a more general noise characterization in terms of cross-covariance bounds with practical relevance and informativity conditions for control based on input-output data. We provide sufficient conditions for informativity for stabilization, H ∞ and H 2 control, which are also necessary for input-state data.

A. Cross-covariance noise bounds
Consider the sample cross-covariance with respect to the noise e ∈ R p and a variable r ∈ R M , given by The variable r is instrumental and can be specified by the user (as discussed at the end of this subsection), i.e., it is a given variable in the upcoming analysis. We assume prior knowledge on the noise of the form where H u is an upper-bound on the squared sample cross- In generalized form, we write with Q 22 ≺ 0. For Q 11 = N H u , Q 12 = 0 and Q 22 = −I, the bound (5) is recovered. Note that (6) can be rewritten as the bound in [4, [4]. This means that the data informativity results of [4] cannot be used to establish data informativity for general cross-covariance noise bounds. However, the matrix S-lemma in [4,Theorem 13] can still be exploited to obtain necessary conditions for informativity of input-state data with crosscovariance bounds, as shown in Proposition 1 of this paper.
In the state-space representation (3), the matrices A z and B z contain unknown parametersĀ,B and B 0 . Write and so that Λ e and B e are unknown parameter matrices, concatenated with zero rows, and J 1 and J 2 are binary matrices. The set of all pairs (Λ e , B e ) that are compatible with the data is and consider Proof: The first statement follows from (4), (6), and the definition of Σ RQ (U,Y ) . Full-column rank of R − [Z ⊤ − U ⊤ − ] and (9) imply that the last ml + p(l − 1) rows of [Λ e B e ] = col(M 1 , 0) are zero and We denote all (A z , B z ) that are compatible with the data bȳ We have provided a parametrization of (a superset of) Σ RQ (U,Y ) based on the data equation (4). One can equivalently parametrizeΣ RQ (U,Y ) on the basis of the state data equation This leads to an equal setΣ RQ (U,Y ) , but the 'repeated' data in the parametrization contained in Z + := ζ(1) · · · ζ(N ) , would render the evaluation numerically sensitive. Design methods in [4], [6] can yield a parametrizationΣ RQ (U,Y ) based on this state data equation, but with limited applicability to cross-covariance bounds (6), i.e., only if the dimension of r satisfies M ≥ N .
Existing guidelines [10] recommend choosing an instrumental variable r that is correlated with the input u, but uncorrelated with the noise e. Hence, this suggests the choice of (filtered/delayed versions) of the input for r in an open-loop case for data collection, and an external reference/dithering signal for r in a closed-loop case. Moreover, Lemma 1 provides an additional guideline for the choice of r to reduce conservatism in the case of input-output data, has full column rank only if the number of instrumental variables M satisfies M ≥ pl + m(l + 1).

B. Output-feedback control
Consider a (dynamic) output feedback controller described by the difference equation of the form [5] with C(ξ) ∈ R m×m [ξ] and D(ξ) ∈ R m×p [ξ] polynomial matrices given by C(ξ) = I + C 1 ξ + C 2 ξ 2 + · · · + C n ξ l and D(ξ) = D 1 ξ + D 2 ξ 2 + · · · + D n ξ l . We define a state ζ c for (10) as ζ c := col(u(t−1), . . . , u(t−l), y(t−1), . . . , y(t−l)), yielding a state-space representation for the controller: Hence, the closed-loop system described by (3) and (11) has a representation With K := DC , the closed-loop system matrix A cl satisfies , we say that the controller (10) stabilizes (1) if the closed-loop system (12) is stable, i.e., if all eigenvalues of A z +B z K are in the open unit disk, since this implies stability of the closedloop system (1) and (10). The notion of stabilization with respect to the state-space representation (3) was introduced in [5] for data-driven stabilization. We note that in the singleinput-single-output case, (A z , B z ) is controllable if and only if A(ξ) and B(ξ) are coprime [5].

III. INFORMATIVITY FOR STABILIZATION
A. Informativity of input-output data Definition 1 The data (U, Y ) are said to be informative for quadratic stabilization by output-feedback controller (10) if there exist a K and P ≻ 0 so that (7)-(8), we find that the existence of K and P ≻ 0 so that is equivalent to the existence of K and P ≻ 0 such that (13) holds true. Now, for the data (U, Y ) to be informative for quadratic stabilization, we require the existence of K and P ≻ 0 so that (13) . This is precisely a problem that can be solved by the S-procedure; more specifically, by the matrix-valued S-lemma [4].
We remark that if there is a Z so thatZ ⊤ ΛZ ≻ 0 with Z := col(I, Z), called the generalized Slater condition [4], then (14) is also a necessary condition for informativity of input-output data for quadratic stabilization, if R − [Z ⊤ − U ⊤ − ] has full column rank (Lemma 1). Unlike in the case of inputstate data, which will be discussed next, we note that the generalized Slater condition can in general not hold true in the input-output case if l ≥ 1, since the noise affects a subspace of the extended state space, yielding a degenerate matrix Λ. The combination of noisy and noiseless states in ζ suggests that necessity could potentially be proven in general by a 'fusion' of the matrix S-lemma and matrix Finsler's lemma [12].

B. Informativity of input-state data
We will now consider a special case, where input-state data is available instead of input output data. That is, we measure a state y(t) = x(t) and the class of systems considered is with the corresponding data equation All systems that explain the data (U − , X) for some E − satisfying the cross-covariance bound (6) are in the set (6) and (16) hold}.
By (16), the set of feasible systems is Σ RQ  (6), we observe that for this special choice R − = I, we recover the set of feasible systems in [4], and, hence, the informativity conditions in [4].

Definition 2
The data (U − , X) are said to be informative for quadratic stabilization by state feedback if there exist a feedback gain K and P ≻ 0 so that We will now provide a necessary and sufficient condition for informativity of input-state data for quadratic stabilization, given prior knowledge on the cross-covariance (6). Consider the generalized Slater condition Proposition 1 Suppose that there exists a Z so that (18) holds true. Then the data (U − , X) are informative for quadratic stabilization if and only if there exist L ∈ R m×n , P ≻ 0, α ≥ 0 and β > 0 so that  Proof: (⇐) This is proven by the same argument as in the proof of Theorem 1. (⇒) Let the data be informative for quadratic stabilization, i.e., there exist K and P ≻ 0 so that, with Π defined in (13) with J 1 = 0, J 2 = 0: We will now show that ker Λ X 22 ⊆ Λ X 12 , such that necessity follows by the matrix S-lemma [4]. First, notice that ker Λ was chosen arbitrary, this shows that ker Λ X 22 ⊆ Λ X 12 . By ker Λ X 22 ⊆ Λ X 12 and (18), there exist α ≥ 0 and β > 0 so that, by [4,Theorem 13]: which is equivalent to (19) for L := KP by the Schur complement. This completes the proof.

IV. INCLUDING PERFORMANCE SPECIFICATIONS
We will now consider the problem of finding a controller (10) for which the closed-loop system achieves an H ∞ or H 2 performance bound from the input-output data (U, Y ). Consider the performance output z, given by z(t) = C z ζ(t) + D z u(t). For any pair (A z , B z ), the controller (10) yields the closed loop system Hence, the transfer matrix from e to z is given by for which the H ∞ and H 2 norm are denoted T H∞ and T H2 , respectively.
For given K, the H ∞ norm of T is less than γ, T H∞ < γ, if and only if there exists X ≻ 0 such that [13, p. 125 where A K := A z + B z K and C K := C z + D z K.

Definition 3
The data (U, Y ) are said to be informative for common H ∞ control by output-feedback controller (10) with performance γ if there exist a K and X ≻ 0 so that Theorem 2 The data (U, Y ) are informative for common H ∞ control with performance γ if there exist L ∈ R m×n , P ≻ 0, α ≥ 0 and β > 0 so that (21) holds true.
Proof: By a congruence transformation of (20) with diag(P, I, P, I) with P := X −1 and the application of the Schur complement (twice), the existence of K and X ≻ 0 so that (20) holds, is equivalent to the existence of P and L so that P ≻ 0 and Hence, the data (U, Y ) are informative for common H ∞ control with performance γ if and only if there exist P ≻ 0 and L such that P − γ −1 F ⊤ F ≻ 0 and (23) holds for all (Λ e , B e ) ∈ Σ RQ (U,Y ) . By assumption, there exist P ≻ 0, L, α ≥ 0 and β > 0 such that (21) holds true. By the Schur complement, (21) is equivalent to Π H∞ − αΛ βI 0 0 0 , which implies that (23) holds for all (Λ e , B e ) ∈ Σ RQ (U,Y ) . The conditions (21) are linear with respect to P , L, α and β. By a straightforward additional application of the Schur complement, (21) can also be made linear with respect to γ.
For a given controller parameter matrix K, the H 2 norm of T is less than γ, T H2 < γ, if and only if there exists X ≻ 0 such that [14, Proposition II.1], cf. [4] trace H ⊤ z XH z < γ 2 and X ≻ A ⊤ K XA K + C ⊤ K C K . (24) Definition 4 The data (U, Y ) are said to be informative for common H 2 control by output-feedback controller (10) with performance γ if there exist a K and X ≻ 0 so thatΣ RQ (U,Y ) ⊆ {(A z , B z ) | (24) holds true}.
Theorem 3 The data (U, Y ) are informative for common H 2 control with performance γ if there exist L ∈ R m×n , symmetric Z, P ≻ 0, α ≥ 0 and β > 0 so that trace Z < γ 2 , (25) holds true, Proof: By a congruence transformation of (24) with P := X −1 and the Schur complement (in both directions), it follows that (24) is equivalent to P − F ⊤ F ≻ 0, and trace H ⊤ z P −1 H z < γ 2 . Now, we can rewrite (27) as  which, by (7)- (8), holds if and and only if  There exist P ≻ 0, L so that (28), P − F ⊤ F ≻ 0 and trace H ⊤ z P −1 H z < γ 2 if and only if there exist P ≻ 0, L, Z so that (28), P − F ⊤ F ≻ 0, Z − H ⊤ z P −1 H z 0 and trace Z < γ 2 . Indeed, for Z := H ⊤ z P −1 H z we infer trace Z < γ 2 . Sufficiency follows from H ⊤ z P −1 H z Z ⇒ trace H ⊤ z P −1 H z ≤ trace Z. Hence, the data (U, Y ) are informative for common H 2 control with performance γ if and only if there exist P ≻ 0, L, Z so that Z −H ⊤ z P −1 H z 0, trace Z < γ 2 , P − F ⊤ F ≻ 0 and (28) hold for all (Λ e , B e ) ∈ Σ RQ (U,Y ) . By assumption, trace Z < γ 2 is satisfied, and consider a performance output z(t) = [0 0 1]x(t). The objective is to determine if input state data collected from the system are informative for common H 2 control. We consider a noise signal e(t) with a uniform distribution, taking values from the closed ball {e ∈ R 3 | e 2 2 ≤ 0.35}. First, we consider this noise bound to be known, represented by the noise model [4,Section VI.A]. This can be represented by the noise model (5) with R − = I, cf. [4,Equation (5)]. We consider the informativity analysis for various data lengths N ranging from N = 2 to N = 250. For each data length N , we generate 50 data sets. Given the bound on E − , we can verify informativity for common H 2 control via Theorem 17 in [4]. We find that the generalized Slater condition [4,Equation (16)], holds true for all data sets, thus the data are informative for common H 2 control with performance γ if and only if the condition [4, Equation (H 2 )] is feasible. The relative number of data sets for which this necessary and sufficient condition is feasible for some γ > 0 is visualized in Figure 1a for each data length N , in red. Naturally, if the condition is not feasible for any γ > 0, the data are actually not informative for feedback stabilization, although the true system is stable. Now, we consider the quadratic cross-covariance bound (5) for the noise. We choose an instrumental variable that contains lagged versions of the input: r(t) := col(u(t), u(t − 1), u(t − 2), . . . , u(t − 8), u(t − 9)). We assume prior knowledge in the sense that where H u is taken as H u = I, independent of N . The cross-covariance bounds hold true for all generated data sets. We verify that there exists some Z so that (18) holds true for all data sets. Hence, by Remark 2, the data are informative for common H 2 control with performance γ if and only if the conditions in Theorem 3 are feasible. The relative number of data sets for which this necessary and sufficient condition is feasible for some γ > 0 is visualized in Figure 1a for each data length N , in blue. For N ≥ 20, all data sets are informative for common H 2 control. For these data sets, the smallest H 2 norm upper bounds γ 2 are visualized in Figure 1b, where the median performance is indicated by a solid line and the shaded area is bounded by the 25 th and 75 th percentiles. In comparison, the H 2 norm that can be achieved by a state feedback controller with knowledge of (A 0 , B 0 ) is equal to 1.000, which therefore is a benchmark that cannot be outperformed by any data-based controller. Now, consider that noisy output measurements are available instead of state measurements. Consider system (1) with A(q −1 ) and B(q −1 ) such that T 0 (q −1 ) = A −1 (q −1 )B(q −1 ) with T 0 := C 0 (qI −A 0 ) −1 B 0 , where C 0 is the output matrix. We consider three cases: For each choice of output, we generate 50 data sets for data lengths ranging from N = 2 to N = 250. We choose an instrumental signal containing lagged input signals as before, which is therefore independent on the choice of output. The upper-bound is chosen H u = 0.3, which holds for all data sets. By Theorem 3, feasibility of the conditions for informativity for H 2 control for some γ > 0 is verified for each data set. The results are depicted in Figure 2. We observe that the data sets are not informative for low data lengths, which can be expected. For increasing data length, informativity becomes dependent on the choice of output. For N = 30, for example, 90% of the data sets yielded feasible informativity conditions for the choice of C 0 = [1 0 0], compared to less than 50% of the data sets for the other two choices for C 0 .

VI. CONCLUDING REMARKS
We have considered the problem of informativity of inputoutput data for control, with prior knowledge of the noise in the form of quadratic sample cross-covariance bounds. Sufficient informativity conditions for stabilization, H ∞ and H 2 control via dynamic output feedback were derived, which are also necessary if the state is measured. We have provided a numerical case study where data-informativity can be concluded with cross-covariance bounds, while the data are concluded to be non-informative with magnitude bounds. Finally, we have illustrated how the choice of output affects the informativity of input-output data via a numerical example.


In vivo human whole-brain Connectom diffusion MRI dataset at 760 µm isotropic resolution

We present a whole-brain in vivo diffusion MRI (dMRI) dataset acquired at 760 μm isotropic resolution and sampled at 1260 q-space points across 9 two-hour sessions on a single healthy participant. The creation of this benchmark dataset is possible through the synergistic use of advanced acquisition hardware and software including the high-gradient-strength Connectom scanner, a custom-built 64-channel phased-array coil, a personalized motion-robust head stabilizer, a recently developed SNR-efficient dMRI acquisition method, and parallel imaging reconstruction with advanced ghost reduction algorithm. With its unprecedented resolution, SNR and image quality, we envision that this dataset will have a broad range of investigational, educational, and clinical applications that will advance the understanding of human brain structures and connectivity. This comprehensive dataset can also be used as a test bed for new modeling, sub-sampling strategies, denoising and processing algorithms, potentially providing a common testing platform for further development of in vivo high resolution dMRI techniques. Whole brain anatomical T1-weighted and T2-weighted images at submillimeter scale along with field maps are also made available.

Background & Summary
Diffusion magnetic resonance imaging (dMRI) 1 has proven to be an invaluable tool in the study of brain structural connectivity and complex tissue microarchitecture. It has also offered the ability to investigate a variety of neurological, neurodegenerative and psychiatric disorders [2][3][4][5][6][7][8][9][10] . In recent years, a number of dMRI data repositories [11][12][13][14][15][16][17][18] have been published to enhance the understanding of human brain, including large-scale population study projects launched across the world, such as the Human Connectome Project (HCP) [19][20][21][22] and the UK Biobank project 23 . However, most of these studies acquired in vivo dMRI datasets at a relatively low spatial resolution of 1.25-3 mm, due to technical difficulties in achieving higher spatial resolution that include: i) the inherent low SNR of dMRI acquired with smaller voxels; ii) the distortion/blurring artifacts from longer EPI readouts; and iii) the inevitable participant motion during long scan time required for obtaining multiple diffusion directions. Submillimeter isotropic resolution dMRI datasets have therefore been mostly confined to ex-vivo studies [24][25][26] where there are no scan time and motion constraints. While ex vivo dMRI plays an important role in studying brain structures with its higher achievable resolution, the alteration of tissue properties due to fixation may fundamentally change the dMRI measurement and affect its ability to reflect accurate values in the living human brain. In vivo dMRI serves as a non-invasive probe of tissue structure in different brain pathologies, and can also be combined with functional and behavioral data for further investigation of structure-function relationships. The availability of submillimeter resolution in vivo dMRI datasets would enable more detailed examination of the brain's fine-scale structures, such as probing short-association fibers at the gray-white matter boundary 27 or studying the cytoarchitecture of thin cortical layers (<1-4 mm in thickness) 26,28 . Submillimeter resolution in vivo dMRI would also facilitate the development and optimization of new sampling, modeling and processing methods specifically targeted at recovering information about tissue microstructure from such high resolution in vivo dMRI datasets.
With the advent of advanced hardware 20,29,30 , acquisition and reconstruction strategies [31][32][33][34][35][36] , submillimeter isotropic resolution dMRI is becoming increasingly feasible [37][38][39][40][41] , presenting new opportunities for studying brain structure at the mesoscopic scale in vivo. In this work, we aim to create the first publicly available in vivo whole brain dMRI reference dataset acquired at submillimeter isotropic resolution of 760 μm, i.e. at 7.7 times smaller voxel size than conventional dMRI acquisition at ~1.5 mm resolution. To enable high-SNR at this extreme resolution, state-of-the-art hardware and acquisition techniques were used along with a carefully designed diffusion protocol targeting at a single-participant scan across 9 two-hour sessions. The data were acquired on the MGH-USC 3 T Connectom scanner equipped with high-strength gradient (G max = 300 mT/m) 20,29,42 and a custom-built 64-channel phased-array coil 30 , using the recently developed SNR-efficient simultaneous multi-slab imaging technique termed gSlider-SMS 39,40 . To mitigate potential image artifacts and loss in resolution, a custom-made form-based headcase (https://caseforge.co) that precisely fit the shape of participant's head and the coil was also used to ensure minimal motion contamination, consistency of participant positioning and same field shim settings across sessions. Advanced parallel imaging reconstruction with ghost-reduction algorithm (Dual-Polarity GRAPPA 43,44 ) and reversed phase-encoding acquisition were also performed to mitigate ghosting artifacts and image distortion. We acquired a total of 1260 q-space points including 420 directions at b = 1000 s/ mm 2 and 840 directions at b = 2500 s/mm 2 across the 9 sessions to ensure high angular resolution. An optimized preprocessing pipeline for the multi-session data was employed afterwards to correct for intensity drifts, distortions and motion while preserving as much high-resolution information as possible.
We envision that this reference dMRI dataset will have a broad range of investigational, educational, and clinical applications that will advance the understanding of human brain structure. In particular, the data should be useful in the exploration of mesoscale structures revealed through submillimeter in vivo dMRI, and in the evaluation of structural brain connectivity that should improve with reduced intravoxel fiber complexity using smaller voxels and with higher intravoxel fiber crossing resolution using high angular resolution [45][46][47] . The data may also serve as a test bed for new modeling, data sub-sampling, denoising and processing algorithms [48][49][50] , potentially providing a common testing platform for further technical development of in vivo high resolution dMRI.

participant.
The participant was a young healthy Caucasian man (born in 1989). He was fully instructed about the scans and gave written informed consent for participation in the study. The experiments were performed with approval from the institutional review board of Partners Healthcare. Data acquisition. All data were acquired in 9 two-hour scan sessions on the MGH-USC 3 T Connectom scanner equipped with a maximum gradient strength of 300 mT/m and a maximum slew rate of 200 T/m/s 20,29,42 , using a custom-built 64-channel phased-array coil 30 . A custom-designed personalized motion-robust stabilizer (https://caseforge.co), that was precisely fit to the shape of the participant's head and the inside of the coil, was used to ensure the same participant positioning across sessions and minimal motion contamination within a session (Fig. 1a). Field shim settings were kept identical across sessions to minimize differences in image distortion, with the goal of minimizing reliance on image post-processing that could introduce blurring.
To obtain high spatial resolution dMRI data, the gSlider-SMS 39 sequence (Fig. 1a) was used in our study. gSlider-SMS is a recently developed self-navigated simultaneous multi-slab acquisition that combines blipped-CAIPI 51 controlled aliasing with a slab-selective radio frequency encoding, which has been demonstrated to provide high SNR-efficiency for high resolution dMRI. The gSlider-SMS protocol was constructed to achieve a whole brain isotropic submillimeter acquisition at high SNR with low reconstruction errors and limited distortion and blurring, while still keeping the scan time tolerable. Motivated by this goal, the decision of the final imaging parameters was made by evaluation of data quality of pilot scans and conclusions from prior studies. The 0.76-mm isotropic resolution was chosen as it provides a good trade-off in term of resolution and scan time to achieve reasonable SNR. A gSlider encoding factor of 5 and a multi-band acceleration factor of 2 were used to achieve high SNR efficiency by acquiring 10 slices per shot across two RF-encoded slabs, where each slab contains 5 slices that can be resolved through 5 gSlider-encoded repetitions. In-plane acceleration of 3 was selected to mitigate EPI distortion and blurring by reducing the effective echo spacing from 1.02 ms to 0.34 ms, while ensuring good reconstruction performance with low noise amplification when the reconstruction was performed with Dual polarity GRAPPA to also mitigate ghosting artifacts. Despite the use of in-plane acceleration and shorter readout length, a small level of image blurring still exists along phase-encoding (PE) direction due to signal decay across EPI readout, characterized by a ~7-15% increase in the full width at half maximum of the point spread function along PE direction for normal gray and white matter (T 2 around 60-100 ms, and T 2 * around 40-80 ms) based on theoretical simulation analysis. Before diffusion acquisition, a calibration scan was acquired using a multi-shot mono-polar SE-EPI to provide fully-sampled data, with the same effective echo spacing as the in-plane accelerated data and in both readout polarities, to calculate the dual-polarity GRAPPA kernel 43,44 . The detailed imaging parameters are listed in Fig. 2a. Using an actual maximum gradient amplitude of 180 mT/m and a slew rate of 125 T/m/s in our acquisition, a shorter TE (75 ms) was achieved at b = 2500 s/mm 2 on Connectom scanner than other state-of-art clinical scanners (e.g., TE = 87 ms using 3 T Prisma) with all the same parameters but a lower achievable maximum gradient amplitude of 80 mT/m. Noted that the maximum gradient strength (300 mT/m) and the maximum slew rate (200 T/m/s) were not applied in Connectom scanner to avoid strong eddy current and exceeding peripheral nerve stimulation limit. The TEs and TRs of b = 1000 s/mm 2 and b = 2500 s/mm 2 were matched. The TR of each RF encoded volume was 3.5 s, and the use of 5 RF encoded volumes resulted in an effective volume TR of 17.5 s.
To achieve good SNR and high angular resolution, a total of 2808 volumes of dMRI data were acquired across the 9 sessions, consisting of 144 b = 0 s/mm 2 images, 420 b = 1000 s/mm 2 diffusion-weighted images (DWIs) and 840 b = 2500 s/mm 2 DWIs, along with their paired reversed phase-encoding volumes. The two shells at b = 1000 s/mm 2 and 2500 s/mm 2 are shown in Fig. 2b, where the 420 directions at b = 1000 s/mm 2 represented a subset of the 840 directions at b = 2500 s/mm 2 . The diffusion vectors were designed using the algorithm described in Emmanuel Caruyer et al. 52 (publicly available tool at http://www.emmanuelcaruyer.com/q-space-sampling.php), which not only provides uniform angular coverage for each shell and for overall multishell sampling, but is also able to provide uniform incremental angular distribution. In each session, 312 dMRI volumes were acquired, Two acquisition blocks with paired reversed phase encoding composed a data subset with a unique set of diffusion directions. 6 data subsets were acquired at b = 1000 s/mm 2 in the first 3 sessions, and 12 data subsets at b = 2500 s/mm 2 in the remaining 6 sessions. (c) Data reconstruction workflow. dMRI data were reconstructed using Dual Polarity GRAPPA, partial Fourier reconstruction, real value diffusion and gSlider slab-to-slice reconstruction, while field maps and structural images were reconstructed using standard GRAPPA reconstruction implemented on the scanner. Minimally preprocessed data were then provided after the gradient non-linearity correction (Data output A). (d) dMRI preprocessing with one-step resampling. The preprocessed data were obtained after intensity normalization to correct for signal drift, along with susceptibility-induced and eddy current-induced distortion correction, gradient non-linearity correction and motion correction, all of which were performed together using a one step resampling process to minimize interpolation-induced image blurring (Data output B).
www.nature.com/scientificdata www.nature.com/scientificdata/ including 280 DWIs (140 unique diffusion directions with reversed phase-encoding acquisition) and 32 b = 0 s/ mm 2 images (interleaved with every 10 diffusion images). These data were further divided and acquired in four acquisition blocks, so the calibration data could be updated every ~25 mins and the paired reversed phase-encoding volumes could be interleaved.
In each session, a low-resolution field map and a structural scan (T 1 or T 2 -weighted image) were also acquired as time permitted. The low-resolution field maps were acquired using a two-TE gradient echo field mapping sequence: TR = 351 ms; TEs = 4.14, 6.6 ms; FOV = 220 × 220 mm 2 ; voxel size = 2 × 2 mm 2 ; number of slices = 38; slice thickness = 2.5 mm; slice gap = 1.3 mm; flip angle = 45°; bandwidth = 500 Hz/Pixel; total acquisition time = 79 s. The T 1 -weighted images were acquired with the Multi-echo Magnetization-Prepared Rapid Acquisition Gradient Echo (MEMPRAGE) 53  total acquisition time = 8.38 minutes. T 2 -weighted data were acquired with a T 2 -SPACE 54 sequence at 0.7-mm isotropic resolution with the following parameters: TR/TE = 3200/563 ms; FOV = 224 × 224 × 180 mm 3 ; bandwidth = 745 Hz/Pixel; GRAPPA factor along PE = 2; total acquisition time = 8.4 minutes. All T 1 -and T 2 -weighted data were acquired with image readout in the head-foot (HF) direction, PE in the anterior-posterior (AP) direction, and partition in the left-right (LR) direction. One 1-mm isotropic resolution MEMPRAGE, three 0.7-mm isotropic resolution MEMPRAGE and three 0.7-mm isotropic resolution T 2 -SPACE images were acquired in session 3-9, respectively. Data reconstruction. The image reconstruction for dMRI was performed on raw k-space data in MATLAB (Mathworks, Natick, MA, USA) (Fig. 1c). First, dual-polarity-GRAPPA 43,44 (DPG) was used for parallel imaging reconstruction of both in-plane and multi-band accelerations. DPG has been validated to reduce Nyquist ghost artifacts of EPI effectively by correcting for higher-order phase errors 43,44 . Unaliased multi-channel images were combined using SENSE 1 complex coil combination method 55 , where the sensitivity maps were estimated from the calibration data using ESPIRiT 56 . POCS 57 was used for partial Fourier reconstruction to recover high spatial information from partially acquired k-space data. Diffusion background phase-corruptions were further removed using the real-valued diffusion algorithm to provide real-value data and avoid magnitude bias in subsequent post-processing steps 58 . Finally, gSlider's slab-to-slice reconstruction 39,40 was performed to reconstruct thick RF-encoded slabs (3.8 mm) into thin slices (0.76 mm), where T 1 recovery was incorporated into the reconstruction model to mitigate slab-boundary artifacts 59 . The slab-to-slice reconstruction has been demonstrated to maintain image resolution with a sharp point spread function (small side lobe of 7.5%) and to provide robust performance in the presence of B 1 and B 0 inhomogeneity 39 .
The image reconstruction of the T 1 -and T 2 -weighted images and field maps was performed by the vendor's (Siemens Healthcare, Erlangen, Germany) online reconstruction (including vendor's implementation of parallel imaging reconstruction, fast Fourier transformation, coil combination, root mean square combination of different echo images and calculation of field maps) and was finally stored in Digital Imaging and Communications in Medicine (DICOM) format.

Data preprocessing.
Preprocessing corrections across the reconstructed image series were performed to correct for signal drift, susceptibility, eddy currents, and gradient nonlinearity induced distortions, and participant motion. An optimized pipeline was used for the multi-session data, where the warp fields for susceptibility, eddy-current distortion, gradient nonlinearity and motion were estimated, combined, and applied to each www.nature.com/scientificdata www.nature.com/scientificdata/ image volume in a one-step resampling process to ensure minimal smoothing effects due to interpolation 60,61 . The FMRIB Software Library (FSL) 62-64 was used to perform these estimations and corrections. The preprocessing pipeline is shown in Fig. 1d and described in detail below: i) First, intensity normalization and signal drift 65,66 correction were performed. The non-DW images (b = 0 s/ mm 2 ) were used to normalize the signal intensity of the data acquired within and across the different acquisition blocks (~25 minutes each). The signal drift within each 25-minute acquisition block was corrected using scaling factors obtained from fitting a second-order polynomial on the mean signals of the non-DW images as a function of the scanned non-DW and DW images. Specifically, there were 8 non-DW images interleaved throughout each 25-minute acquisition block (one non-DW volume every 10 DW volumes, 3.2 minutes). The mean signals of the non-DW volumes were used to fit polynomial coefficients using MATLAB's polyfit function, which were then fed into the polyval function to obtain scaling factors for DW volumes in between those non-DW volumes. The signal intensity differences across acquisition blocks were normalized to the mean signal intensity of the first session. ii) Susceptibility-induced and eddy current-induced distortion estimations and corrections were performed using topup 62,67 and the eddy-current correction toolbox [68][69][70] in the FSL [62][63][64] . The warp fields for each volume were obtained and stored for later use in the combined one-step resampling correction. Note that topup and eddy-current corrections were performed for each individual data subset (2 acquisition blocks with paired reversed phase encoding block), such that the field map can be updated across scans to account for possible field changes even with the use of motion stabilizer and matched B 0 shimming. In addition to updating the field map, the interaction between susceptibility-induced field changes and participant motion within each data block was also considered by using a dynamic correction algorithm 70 as implemented in the FSL eddy toolbox [68][69][70] . iii) The warp field for gradient nonlinearity correction 60,71 was calculated for the diffusion data. Here, the gradient nonlinearity correction was performed after topup and eddy correction 60 . This is because of the consideration that the voxel displacements due to susceptibility and eddy current distortions are much larger than those due to within-block rigid motion. Therefore, to obtain better estimation of these larger voxel displacements, the original phase-encoding direction was maintained prior to the estimation and correction for susceptibility and eddy current-induced distortions 60 . The effects of performing gradient-nonlinearity at a later step on the accuracy of within-block rigid motion estimation were considered to be small, because we assume that the confined motion within each acquisition block would only cause small differences in image deformation caused by gradient non-linearity. iv) The next step estimated larger motion parameters across different data subsets such that the images from different acquisition blocks and sessions could be registered and aligned. This was performed using FLIRT [72][73][74] in FSL 62-64 with 6 degrees of freedom (DOF) rigid motion. v) Finally, the warp fields for susceptibility, eddy-current distortion, gradient nonlinearity and motion from steps ii-iv were combined and applied to each image volume in a one-step resampling process to ensure minimal smoothing effects due to interpolation 60,61 . The Jacobian matrix computed from the combined warp field was used to correct for the signal intensity affected by the transformation. The b-vectors were reoriented for each DW image after motion correction 75 .
All 18 data subsets from 9 sessions and their corresponding b-value and b-vector (reoriented) files were stored in compressed Neuroimaging Informatics Technology Initiative (NIfTI) format for downloading (Fig. 1d, data output B) and further analysis. Freesurfer 76 functions were used to load and save NIfTI data.
To provide flexibility for the end users to use their own preprocessing pipeline of choice, minimally preprocessed data are also made available for download in compressed NIfTI format (Fig. 1c, data output A). With the minimally preprocessed data, only the gradient nonlinearity correction has been performed without any other preprocessing corrections. The rationale in performing this correction is that the gradient nonlinearity coefficients needed in this correction are considered proprietary information by Siemens, and are not made publicly available. Because of this, the correction for spatial variation of b-matrices due to gradient nonlinearity [77][78][79][80][81] was not performed in this work. T 1 and T 2 -weighted images are available for download in compressed NIfTI format (Fig. 1c, data output A). After online reconstruction, the DICOM data were converted to NIfTI format using dcm2nii 82 tool of MRIcron (http://www.mricro.com), and MRI gradient nonlinearity correction was performed. Subsequently, face and ear regions were masked to de-identify the high-resolution structural images using the defacing tool provided by FSL [62][63][64] . The averaged images of the three 0.7-mm T 1 and T 2 -weighted images are also provided after registration, averaging and de-identification. The low-resolution field maps are provided after gradient nonlinearity correction.

Data Records
All data records listed in this section are publicly available in dryad 83,84 . The raw k-space data (~7 TB) can also be made available upon request. dMRI data.
i) The preprocessed dMRI data are saved in zip files with a prefix of 'processedDWI_' 83 , encoded by labels identifying scan session 1-9 and data subset number 01-18. The images of different data subsets are saved in separate files in the compressed NIfTI format, each including 78 volumes with PE in AP and another www.nature.com/scientificdata www.nature.com/scientificdata/ 78 volumes in PA. The corresponding b-value and b-vector (reoriented after motion correction) files are also provided. Data subset 01-06 were acquired at b = 1000 s/mm 2 , and data subset 07-18 were acquired at b = 2500 s/mm 2 .
ii) The minimally preprocessed dMRI data are saved in zip files with a prefix of 'reconDWI_' 84 . They are encoded and organized in the same format as the preprocessed dMRI data mentioned above.

Structural data.
i) T 1 -weighted images are saved under 'T1.zip' 83 (~ 350 MB) in compressed NIfTI format, containing one 1-mm MEMPRAGE image (echo combined), three 0.7-mm MEMPRAGE images (echo combined) and their average image. Supplementary to the above data, a readme text file 83 is included in the repository with more information of the datasets.
technical Validation assessment of DW image quality and head motion. The single and mean DWIs at b = 1000 s/mm 2 and b = 2500 s/mm 2 in three orthogonal views are shown in Fig. 3a, which highlight the high image quality of this isotropic resolution dataset in all three spatial orientations with minimal reconstruction and ghosting artifacts. The high isotropic spatial resolution of the data reveals detailed structures of the brain (e.g., thalamus, globus pallidus, cerebellum) and sharp delineation of the gray-white matter boundaries (apparent in mean DWIs).
Participant motion between the 2808 dMRI volumes across the 9 two-hour sessions was evaluated in Fig. 3b. The motion parameter estimates from different sessions acquired on different dates were concatenated together in a chronological order and separated by dashed lines. Within each two-hour session, the translations along the LR and the HF directions as well as the rotations around the AP (roll) and the HF axes (yaw) were all confined within a ± 1 mm/degree range. The rotations around the LR axis was slight larger but was still within a ± 2-degree range. The estimates of translational motion along the AP direction appeared to be the highest and contained Fig. 4 Comparison of FA maps and crossing fiber analysis results using data from 1, 3 and 9 acquisition sessions. On the left, colored FA maps in three orthogonal views are shown for data acquired in 1 session (top), 3 sessions (middle) and 9 sessions (bottom). The 9-session data provides high SNR FA and reveals exquisite fine-scale structures as shown in the zoomed-in areas (yellow boxes), including (A) the radiating fiber bundles within the corpus callosum; (B) the 1-voxel-width line of reduced-FA separating the corticospinal tract from an adjacent tract, highlighting the low level of blurring and high spatial accuracy of this data; and (C) the gray matter bridges that span the internal capsule. On the right, zoomed-in fiber orientations (three fiber compartments per voxel) within centrum semiovale (yellow box) overlaid on gray-scale maps of the total anisotropic volume fraction are shown for those data as well. In addition, the volume fraction of the third fiber are shown in red-yellow on the top left of each fiber orientation map to visualize the sensitivity of solving 3-way crossing fibers. Both fiber orientation and third-fiber volume fraction are shown when the respective volume fraction is higher than 5%. The denser q-space sampling and higher SNR of the 9-session data increases its ability to solve three-way crossing fibers and to reveal complex intermingling fiber bundles within the centrum semiovale.
fluctuations and large jumps between the 25-minute acquisition blocks with opposing phase-encoding directions. This likely resulted from the inclusion of some eddy-current induced translation along the phase-encoding direction, which is known to be difficult to differentiate from actual participant motion since they affect the data in the same way. However, whether this translation is included in the eddy current-induced estimate or the participant movement-induced estimate will not affect the image motion correction. Across different scan sessions, the abrupt changes in motion parameters represented differences in head position placements across sessions. With the use of the personalized head motion stabilizer, the position changes across different sessions were minimized to within ± 2 mm/degrees. assessment of diffusion modelling outcomes of the multi-session dataset. The quality of the multi-session dataset was also assessed by evaluating the diffusion modelling outcomes. Specifically, the diffusion tensor (DT) model and a multi-shell extension of the ball & stick model [85][86][87] implemented in the bedpostx tool in FSL were used. The necessity and benefits of the multi-session data acquisition were illustrated by comparing the above diffusion outcomes obtained using data from 1, 3 and 9 sessions (total dMRI data acquisition time of 1.6, 4.8 and 14.5 hours, with 140, 420 and 1260 q-space sampling points respectively). As shown in Fig. 4, increasing the amount of data significantly increased the SNR and quality of the fractional anisotropy (FA) maps. The FA maps obtained using data from all 9 sessions revealed exquisite detail within fine-scale structures, including the radiating fiber bundles within the corpus callosum, highlighted as red stripes on the FA map (Fig. 4, zoomed-in area, arrow-A); the ~1-voxel-width line of reduced-FA separating the corticospinal tract from an adjacent tract (Fig. 4, zoomed-in area, arrow-B), highlighting the low level of blurring and high spatial accuracy of this data acquired across 9 two-hour sessions; and the gray matter bridges that span the internal capsule, giving rise to the characteristic stripes seen in the striatum (Fig. 4, zoomed-in area, arrow-C). In addition, the increased SNR and the denser q-space sampling from the multi-session data dramatically increased the ability to resolve three-way crossing fibers (see Fig. 4 for increased volume fraction of the third fiber shown in red-yellow on top left) and revealed complex intermingling fiber bundles within the centrum semiovale (see Fig. 4 fiber orientation maps modelling 3 fiber compartments per voxel), where the callosal fibers, the corticospinal tract and the superior longitudinal fasciculus intersect. www.nature.com/scientificdata www.nature.com/scientificdata/ assessment of spatial resolution. A comparison of diffusion results at different spatial resolutions was performed to validate the benefit of using high spatial resolution acquisition. The dMRI data at 0.76 mm isotropic resolution was compared with downsampled data at 1-mm, 1.25-mm and 1.5-mm isotropic resolution (by cropping in the k-space of the dMRI data along all three dimensions). Figure 5 shows the colored-FA maps of four zoomed-in areas (yellow squares) at those spatial resolutions. Improved delineation of fiber bundles and more detailed structures can be captured under higher spatial resolution. Figure 6 shows the comparison of DTI tensors at 1.5 mm and 0.76 mm isotropic resolution, which consistently highlights the improved capability of the submillimeter dMRI data to visualize subcortical white matter as it turns into the cortex. A clear dark band with reduced FA at the gray-white junction can be seen at 0.76-mm resolution (red arrows, top and middle rows). In addition, 0.76-mm data detected more sharp-turning fibers such as those connecting cortical regions between adjacent gyri (red arrows, bottom row). These features of high spatial resolution could potentially help improve the accuracy of tractography results (e.g., reducing gyral bias).

Code availability
The MATLAB code for the preprocessing steps described above are included in the released dataset 83 . The code make use of the functions in the FSL 62-64 toolbox and Freesurfer 76 that are freely available for download at https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FslInstallation and https://surfer.nmr.mgh.harvard.edu/fswiki/ DownloadAndInstall.

Fig. 6
Comparison of DTI principal directions at 1.5-mm and 0.76-mm resolution. Zoomed-in FA and DTI principal vector maps are shown in the area highlighted by yellow boxes. High resolution dMRI data shows improved ability to visualize subcortical white matters as it turns into the cortex. A clear dark-band with reduced FA at the grey-white junction can be seen (red arrows, top and middle rows) at 0.76 mm resolution, and more sharping-turning fibers such as those connecting cortical regions between adjacent gyri (red arrows, bottom row) can be detected.


Design and Implementation of Smart Parking System Using Location-Based Service and Gamification Based on Internet of Things

Information on the number of available parking slot capacity and trip routes to the destination parking area, and motivation in choosing a parking area location are parameters that can help two-wheeled vehicle users choose the right parking area location. The three parameters that determine the accuracy of selecting a parking area location are implemented in an Internet of Things (IoT) based smart parking system. This system can provide information about the maximum number of slot capacities and the number of available slot capacities at the parking area. Two-wheeled riders are given information about which route to take to the destination parking area by utilizing the Location-Based Service (LBS). These two features are then supported by applying the gamification method to motivate two-wheeled riders to choose the right parking area. The smart parking system is tested with considered Quality of Service (QoS) parameter and black box testing. The results of testing the smart parking system produce QoS performance on the Line of Sight (LOS) test, with an average delay is 71.66 ms, average jitter is 107.59 ms, and throughput is 23 kbps. Meanwhile, in the non-LOS test, the average delay is 132.88 ms, the average jitter is 200.84 ms, and the throughput is 12 kbps. Overall system performance obtained the percentage of reliability is 99.65 %, and availability is 99.65 %. In black-box testing, LBS and gamification methods can implement according to application requirements specifications.

INTRODUCTION
The population of Indonesia has increased in the last ten years. Starting from 2010 to 2020, the population in Indonesia has increased by 32.56 million people, or an average increase of 3.26 million people each year [1]. As the population increases, the demand for motorbikes also increases by 6.2% per year [2]. The use of the increasing number of motorbikes has made parking areas a necessity [3]. However, the limited location of the parking area can cause twowheelers to spend time and fuel in finding the location of the designated parking area [4], [5], [6]. This problem is followed by a lack of information regarding the maximum number of parking slot capacities and the number of available parking slot capacities. Twowheeled vehicle users also need to know the route they will trip to the desired parking area. In addition, there needs to be additional motivation for two-wheeled riders to balance the available parking area [7], [8].
Previous research by Eka Santi Wahyuningtyas, Rendy Munadi, and Sussi has made a smart parking application that can provide information about the parking capacity of two-wheeled vehicles. This application is integrated with the Radio Frequency Identification (RFID) sensor, placed at the entrance and exit gates of the two-wheeled vehicle parking area at Telkom University [9]. The weakness of this research is that the number of parking areas that are informed is limited to only one parking area, and the application used cannot show the route taken to the parking lot desired by two-wheeled riders. The use of a parking area limited to one location makes the Copyright  The second research conducted by Ginanjar Wiro Sasmito and Fuad Hadiansah has made an application that has a feature to inform about tourist objects in the Tegal area, Central Java. This application utilizes the Location Based Service (LBS) to implement the Tegal tourist attraction route which can provide information on the location of tourist objects, coordinate points , and support information about tourist objects in Tegal [10]. The drawback of this research is that this application cannot display the number of tourists in a tourist attraction in the Tegal area.
The third research was conducted by Kavisha Duggal, Lovi Raj Gupta, Ph.D., and Sravani Ammu in a paper entitled "Gamification Leads to Smart Parking", a research was conducted on the application of smart parking using the gamification method to motivate motorists to park their vehicles in the right way. The gamification method used in this smart parking system utilizes the point acquisition system obtained by the rider if it meets the criteria given. The criteria given are based on the location of the parking area, the accuracy of the parking position of the vehicle in the available parking slot, and how much time it takes the rider to park his vehicle in the slot provided [7]. The drawback of this research is that the smart parking system is only implemented on fourwheeled drivers.
Based on the existing problems, this research creates a smart parking system by utilizing Internet of Things (IoT). That can be accessed through an Android-based application used to provide information about the maximum number of slot capacities and slot capacities that are still available in a parking area. And can provide information about the route taken by two-wheeled vehicle users to the desired parking area. Data of the available capacity are obtained from the RFID reader sensor. Meanwhile, information on the route to the designated parking area was obtained using LBS [11]. The two features in this application are then supported by using the gamification method, which functions to motivate two-wheeled riders to be more precise in choosing the desired parking area [12], [13]. The smart parking system that was created measured concerning parameter Quality of Service (QoS) such as delay, jitter, throughput, reliability, and availability to determine network performance on the communication of RFID system device to Firebase. And also, the tested carry out with black-box testing to testing the functionality of parking slot capacity information, LBS, and gamification on the android application.

A. Smart Parking
Smart parking is one of the many technologies implemented in a smart city concept [14]. All devices in the parking area are integrated with electronic devices. This integrated device is then connected to internet connectivity to form an IoT [15].

B. Location-Based Service (LBS)
LBS is an information service with software orientation provided by mobile devices to perform control operations using location-based data [16]. This information service can be accessed using a mobile device via the internet network and can display the geographic position of the mobile device [10]. In this research, the implemented LBS service is information about the route taken by two-wheeled riders to the parking area.

C. Gamification
Gamification is a method of applying game techniques, game thinking, and game design to nongame activities. The purpose of this method is to motivate someone to achieve their goals [7]. Gamification consists of 4 semantic components, namely games, elements, design, and non-game context. Games are systems in which players engage in artificial conflicts, are determined by rules, and produce measurable results [8].
Gamification elements can be interpreted as elements that are characteristic of games found in many games and are significant to the meaning of games [8]. Gamification applies several elements in the game: points, leaderboard, achievement, level, story, clear goals, feedback, reward progress, and challenges [17]. This research applied the game elements are points and achievements. Point is a game element whose application occurs when the user completes certain activities in a game. Meanwhile, the achievement is a game element whose application occurs when the user reaches a particular achievement in a game.
Design is a component that distinguishes game design elements from game-based technology. Whereas technology in games includes technological aspects such as game engines or controllers, the definition of gamification refers explicitly to the deliberate design process [8].
The non-game context is an area where it is impossible to apply the game concept. This concept can be a potential opportunity to apply gamification to that area. The only context excluded by definition is game design elements within the game itself or in the game design process [8].

D. Internet of Things (IoT)
IoT is a node with intelligence and interoperability that is interconnected in a dynamic global network infrastructure. IoT applies any concept of connectivity from anywhere and anytime. Simply put, IoT is the ability to make everything around us intelligently connected to the internet and take into account the existence of a type of autonomy and privacy [18]. Something or object referred to in this discussion is a 65 Jurnal Infotel Vol. 13  physical object or virtual object [19]. Physical objects are represented in virtual objects to facilitate IoT application to society, especially in terms of conveying information data [20].  can communicate with each other either through a gateway or without going through a gateway. In this research, IoT implementation can be done in an application that informs the maximum slot capacity and available slot capacity in the parking area based on the number of two-wheeler riders who enter and leave using the RFID system. Figure 2 explained this research is implemented in two different parking area locations, so two RFID system devices are placed at each parking area location. Each RFID system device is placed at the entrance gate and exit gate of the parking area location. The four RFID system devices are connected in realtime to the Firebase realtime database via the internet network.  Figure 3 describes the Firebase console that can use as a database for stores data on the amount of available parking area slot capacity. This data is then sent to the application to find out information on the available parking area slot capacity. The application can inform the user about the destination parking area using a polyline and voice direction using LBS. In addition, the application also applies gamification elements consisting of points and achievement [12].  Figure 4 is the overall view from the hardware of the RFID system device that carries out in parking areas A and B. Figure 5 describes the device user needs to be connected to a power source to activate the components on the device. The device also requires an internet connection to transmit data to Firebase. The RFID tag is used to enter input data in the form of a User Identification (UID) read by the RFID reader. The data read on the RFID reader is continued to the microcontroller. In the microcontroller, coding has been done to identify which UID users are allowed to enter the parking area and which are not allowed.  The LCDs a description of whether or not users are allowed to enter the designated parking area. In addition, the input from the RFID tag is then processed by the microcontroller to make the process of adding or reducing the number of available parking slot capacities. After the process on the microcontroller is complete, the data is stored in the Firebase realtime database. The process of increasing the number of available parking slot capacity is carried out every time there is tapping by the user at the exit gate of the parking area. In contrast, decreasing the number of available parking slot capacity is carried out every time there is tapping by the user at the entrance gate of the parking area. Figure 6 explains that the smartphone used needs an internet connection and can use GPS service access because both are components needed to implement LBS on a smartphone [21]. With the fulfillment of these two components, users can get trip route information to the location of the selected parking area. Route information consists of 2 implementations. The first route information is implemented in a polyline direction, from the user's initial location to the destination parking area. They are parsing the JSON data implemented on these polyline lines.. JSON data is obtained based on the latitude and longitude of the user's initial position and the location of the intended parking area. In addition to polyline direction, users can also take advantage of voice direction, which is a route of the trip, by implementing the Intent class to move to the Google Maps application.

H. Gamification Design
The gamification method consists of implementing the point and achievement elements. Acquisition of points in the parking area can be divided into four conditions. The first condition, if parking area A has an available capacity of less than 30 and parking area B has an available capacity of more than 30. Parking area B is worth 50 points. Meanwhile, parking area A is worth 0 points. The second condition, if parking area B has an available capacity of less than 30 and parking area A has an available capacity of more than 30, then parking area A is worth 50 points.
Meanwhile, parking area B is worth 0 points. The third condition, if the two parking areas have available capacities above 30, then the two parking areas are worth 0 points. The fourth condition, if the two parking areas have an available capacity of below 30, then the two parking areas are worth 0 points. Users who have reached specific points get an achievement in the form of a pop-up window with a star image on it. Achievement appears in 3 different conditions. The user can achieve if the user has reached 1000 points, 2000 points, and 3000 points, as in Fig.7.

I. Application Design
The application called Parking Points is designed to contain several functionalities that users can use to determine the location of the designated parking area. The application is connected with several other support systems such as the Firebase realtime database, the Google Maps API, and the Google Maps application. Figure 8 explains the relationship between actors and systems contained in the use case diagram. Based on the use case diagram, the Parking Points application has four actors, each connected to the use case on the system. As the first actor, the user has connected to the use case sign in, sign up, check the acquisition of points, confirm the point and select the location of the parking area. By connecting the user to the use case, the user can run the Parking Points application functionality based on the connected use case. As the second actor, Firebase is connected to use cases related to account authentication functionality, account registration, point storage, and parking slot capacity information. Meanwhile, the Google Maps API and Google Maps application actors are related to using cases related to the implementation of LBS in the Parking Points application, namely information on route polyline direction and voice direction.  An application consists of several activities that users use to interact with the application. The following are some of the components in the application design used in this research. a) Splash screen activity Figure 9 describes user interface of splash screen activity that functions as the initial display for the user before entering the next activity. This activity has a duration of 2 seconds. After the duration of this activity is complete, another activity is displayed. b) The sign-in activity Figure 10 describes the user interface of the sign-in activity that functions to users to sign-in to their account before accessing application services. In this activity, users need to fill in the email and password that were previously registered. c) The sign-up activity Figure 11 describes the user interface of the sign-up activity that functions to register for an account if the user does not have an account to access the application service. Registration is carried out related to filling in the username, email, and password.  Figure 12 describes the user interface of the main activity that functions to select the location of the parking area. As information for the route, the user takes to the destination parking area. The route for this activity is informed in the form of a polyline direction. The number of available parking slot capacities and the maximum parking slot capacities are displayed in this activity. The amount of capacity can display by clicking on the marker, which is the location of the parking area. The marker color can also change according to the number of available parking slot capacities. e) The user profile activity Figure 13 describes the user profile activity that display user profile data such as username, email, and points obtained. The Pop-up window is implemented as an achievement to the user. This pop-up window appears three times depending on the achievement of the points earned by the user. Figure 14 describes user that has reached 1000 points can earn achievement with 1 star. The user that has reached 2000 points can earn achievement with two stars describes in Fig.15. Figure 16 describes user that has reached 3000 points can earn achievement with three stars.
g) The Google Maps application Figure 17 describes the Google Maps application that functions to provide voice direction services. This application is implemented by utilizing the Intent class in the Parking Points application to call the Google Maps application when voice direction services are needed. The Google Maps application in Fig.17

J. Quality of Service (QoS)
QoS is the overall characteristics of telecommunication services related to its ability to meet the needs of service users, both expressed and implied [22]. In this research, the measurement observes the packets sent from the system and measures several performance parameters such as delay, jitter, throughput, reliability, and availability.
According to the Telecommunications and Internet Protocol Harmonization Over Networks (TIPHON), the specifications for delay and jitter are presented in Table 1 and The throughput parameter measured by paying attention to the package received, and the total amount of delivery time, these two variables can be formulated as (1),

Throughput =
Packets received Total delivery time .
(1) A smart parking system needs to pay attention to two parameters in reviewing the system's success rate in carrying out its functions. The two parameters are reliability and availability. Reliability is the probability of the system carrying out the required functions under certain conditions and periods so that the system is run must allow it to be free from failure [24] Reliability = (Uptime-Downtime) Availability is the system's ability to perform the functions required during the system's life, or it says that availability is the probability that the system will not fail or is not being repaired when the system is needed [24] Availability = Uptime (Uptime + Downtime) × 100 %.
In (2) and (3), there are two variables determining the value of availability and reliability, namely uptime and downtime. Uptime is the total time the system 71 Jurnal Infotel Vol. 13  performs its functions. Meanwhile, downtime is the total time the system does not perform its function.

K. Black Box Testing
Black box testing or functional testing is a technique in software testing designed to perform functionality testing based on information from the software specifications being tested. This test does not focus on the source code mechanism of software but only focuses on the output produced in response to the selected input [25]. In this research, black box testing is used to test the application's performance in receiving input from users so that the output produced is by the specifications of the application requirements.

L. Hardware and Software
Implementing a smart parking system needed some hardware to develop an RFID system device that explained in Table 3. In addition, it also needs some software that is explained in Table 4. Table 3. Hardware Function.

Development board based Chip ESP8266 ESP-12F
As the main control center in the RFID system device RFID tag As an identifier when user tapping in RFID system device RFID reader Read UID of RFID tag when user tapping in RFID system device

Jumper cable
Connecting some of the hardware in RFID system device so that the hardware is integrated LCD 16x2 Displays some of the sentence based on condition when the user exit and enter from the parking area gate after the user tapping on the RFID system device Lithium-ion battery As the central resource in running the RFID system device Battery holder As the place put the lithium-ion battery in the RFID system device

M. Method of Testing System
System testing aims to test the fundamental research to see that the system is operating properly and correctly. In this research, the system is tested as follows.

A. Hardware Testing
The hardware in the RFID system tested consists of an RFID reader, a microcontroller, a lithium-ion battery, and a 16x2 LCD.  Table 5 explains every component contained in the RFID system device can function properly. The RFID tag detection test on the RFID reader can run as needed when viewed from the specified indicators. This condition cannot separate from the RFID tag detection by distance parameter on the RFID reader. Table 6 explains the maximum distance that the RFID reader can detect is 3 cm, while at a distance of 4 cm and 5 cm, the RFID reader cannot detect the RFID tag.

B. Testing of Parking Slot Capacity Information
The testing using black-box techniques on the Parking Points application. This test is done by testing the functionality of the Parking Points application [25].  Table 7 explains that every click made on a parking area location marker can display an info window informing the available parking slot capacity and maximum parking slot capacity. And Table 8 explains the color change from green which indicates a parking slot capacity of more than 200 slots, to a color change to red, indicating a parking slot capacity of fewew than 30 slots.

C. Testing of Location-Based Service (LBS)
The testing using black-box techniques on the Parking Points application. This test is done by testing the functionality of the Parking Points application [25].  Table 9 explains if the user chooses the location of the parking area with a distance of the user from the location of the selected parking area of more than 75 meters for parking area A and more than 50 meters for parking area B, then the route information is displayed in the form of a polyline direction. The user can use voice direction services by clicking the button provided. Meanwhile, suppose the user chooses the location of the parking area with the user's distance from the location of the selected parking area less than 75 meters for parking area A and less than 50 meters for parking area B. In that case, the route information is not displayed.

D. Testing of Gamification
The testing using black-box techniques on the Parking Points application. This test is done by testing the functionality of the Parking Points application [25]. Table 10 explains if one of the parking areas is in a condition where there are less than 30 available parking slots. And another parking area is in a condition that the parking slot capacity is more than 30 slots, then 50 points are obtained if the user chooses a parking area with a capacity of more than 30 slots. The points obtained for selecting the two parking areas can be worth 0 points if the condition of the two parking areas has an available parking slot capacity of fewer than 30 slots or more than 30 slots. Table 11 explains that the user can confirm the acquisition of points because the user is less than 10 meters from the location of the intended parking area. If the user is more than 10 meters from the designated parking area, the user cannot confirm the point acquisition.

Input Expected Output Output Produced
The point is 1000 points The star image in the pop-up window is one Appropriate The point is 2000 points The star image in the pop-up window is two Appropriate The point is 3000 points The star image in the pop-up window is three Appropriate

E. Quality of Service (QoS) Testing
The tested QoS parameters consist of delay, jitter, throughput, reliability, and availability. Parameters of delay, jitter, and throughput were tested using the Wireshark application by observing packet traffic filtered based on source and destination IP addresses with port 443 on communication between RFID system device to Firebase and vice versa. Meanwhile, testing reliability and availability were tested by measuring uptime and downtime using a stopwatch. a) Delay Testing Figure 18, if referring to Table 1, the average delay results resulted from the LOS test can be classified into the excellent category.  Figure 19, if referring to Table 2, the results of the average jitter resulted from the LOS test can be classified into the not good category. c) Throughput Testing Figure 20 describes tests in the LOS test that resulted in a higher throughput value than the non-LOS test.

d) Testing of Reliability and Availability
In Table 13, if referring to equations 2 and 3, the percentages for reliability and availability of the whole system are 99.65% and 99.65%.  IV. DISCUSSION The result that tested and measured then analyzed the result of the implementation of the design. The smart parking system consists of an RFID system device as a hardware component and Parking Points application as an Android application. Both of them have functionality and are needed. It can see from hardware testing that properly, this is because each function of the component is under the specified indicators. Such as on the RFID reader testing obtained distance variety for detect RFID reader, this is due to the use of the antenna on the RFID, tag which affects its operating distance. And application testing consists of parking slot capacity information, LBS, and gamification testing. Based on software testing obtained, the result can match the expected output because the application designed before the test was carried out has been adjusted to the expected application needs.
The QoS testing obtained results for the delay, jitter, and throughput parameter had better results in the LOS scenario than the non-LOS scenario. This scenario is influenced by the absence of obstacles during the LOS testing. The chance of a decrease in signal quality caused by the obstacle can be reduced [26]. Apart from the obstacle effect, fluctuating internet connections from time to time also affects these results. As well as, on the parameter of reliability and availability obtained a downtime of 300 seconds in 86700 seconds total tested. Downtime is obtained from replacing a lithium-ion battery with an exhausted electric charge with a fully charged lithium-ion battery.

V. CONCLUSION
Based on the test result, the smart parking system can display information on the maximum slot capacity and available parking slot capacity based on the tapping input made by the user on the RFID system when entering or exiting the parking area gate. The smart parking system can display trip route information taken by the user in a polyline direction and voice direction by utilizing the LBS service in the Parking Points application. The smart parking system can apply the gamification method by using point and achievement elements in the Parking Points application. Measurements of QoS obtained on the LOS scenario can generate better results in parameters delay, jitter, and throughput. The measurement of the whole system had resulted in reliability and availability above 99% for implementation in the parking area.


Evolution of the Agile Scaling Frameworks

Over the past decade, agile methods have become the favored choice for projects undertaken in rapidly changing environments. The success of agile methods in small, co-located projects has inspired companies to apply them in larger projects. Agile scaling frameworks, such as Large Scale Scrum and Scaled Agile Framework, have been invented by practitioners to scale agile to large projects and organizations. Given the importance of agile scaling frameworks, research on those frameworks is still limited. This paper presents our findings from an empirical survey answered by the methodologists of 15 agile scaling frameworks. We explored (i) framework evolution, (ii) main reasons behind their creation, (iii) benefits, and (iv) challenges of adopting these frameworks. The most common reasons behind creating the frameworks were improving the organization’s agility and collaboration between agile teams. The most commonly claimed benefits included enabling frequent deliveries and enhancing employee satisfaction, motivation, and engagement. The most mentioned challenges were using frameworks as cooking recipes instead of focusing on changing people’s culture and mindset.

Introduction
Ever since the creation of the Agile Manifesto in 2001, practitioners and academics have devoted a great deal of attention to agile software development methods [1]. Initially, they were designed for small, co-located, and selforganizing teams that develop software in close collaboration with business customers using short iterations [2]. Hence, agile methods have been primarily applied to projects within the so-called 'agile sweet spot', i.e., small and colocated teams of less than 50 persons with easy access to the user and business experts and that develop non-life-critical software [3]. Given the successful adoption of agile methods in small organizations and projects, also many large software organizations have begun to adopt these methods [4]. However, the adoption of agile methods outside the agile sweet spot poses significant challenges to organizations, such as coordination challenges in multi-team environments [5]. To resolve issues associated with the adoption of agile methods in large-scale organizations and projects, several agile scaling frameworks, such as Large Scale Scrum (LeSS) 1 and Scaled Agile Framework (SAFe) 2 , have been created both by some custodians of existing agile methods and by others who have worked with companies to scale agile methods to their settings [4,6,7]. As large organizations face growing pressures and expectations to become more agile, and the agile scaling frameworks claim to provide off-the-shelf solutions to scaling, their adoption has rapidly increased in industry, as confirmed by the yearly non-scientific survey on the state of agile development conducted by VersionOne [6][7][8].
Not only is there a growing interest in adopting agile scaling frameworks from an industrial perspective [8], but there is also a growing academic interest to study the adoption of these frameworks [6]. A systematic mapping study by Uludag et al. [6] uncovered the topic of agile scaling frameworks as a major research stream in the field of large-scale agile development, with a total of 16% of all published studies related to large-scale agile development. The existing literature on the scaling frameworks mainly investigates how individual frameworks are adopted based on case studies (cf. [9]) followed by a comparison of the frameworks based on their underlying characteristics based on literature reviews (cf. [10]). However, the existing literature on agile scaling frameworks disregards the following topics: (i) providing a comprehensive overview of agile scaling frameworks and their evolution, (ii) studying the reasons behind creating these frameworks, and (iii) investigating the benefits and (iv) challenges of adopting these frameworks. To address this research gap, we conducted a survey with the creators/methodologists of known agile scaling frameworks and aim to answer the following research questions (RQs): -RQ1: How did the agile scaling frameworks evolve over the years? -RQ2: What are key reasons behind creating of agile scaling frameworks? -RQ3: What are the claimed benefits of adopting agile scaling frameworks? -RQ4: What are the claimed challenges of adopting of agile scaling frameworks?
The remainder of this paper is structured as follows. In Sect. 2, we provide an overview of related work. In Sect. 3, we portray the research design of our paper. Section 4 presents the result of our survey. In Sect. 5, we discuss our main findings and limitations and conclude our study with a summary of our results and remarks on future research.

Background and Related Work
The successful adoption of agile methods in small teams ignited a new passion among firms to start using agile methods in large projects, even beyond software development, across the enterprise [11]. This phenomenon is often referred as 'large-scale agile development' [12]. In line with Dikert et al. [5], we understand the term 'large-scale agile development', as the application of agile methods in large multi-team settings consisting of 50 persons or more, or at least six teams.
Over the past two decades, software engineers and researchers have devoted a great deal of attention to agile software development [13]. Within few years, various agile methods appeared on the landscape, such as Extreme Programming and Scrum, to name a few [1]. Figure 1 presents the various agile methods, their interrelationships, and their evolutionary paths [13].  Agile methods adhering to varying degrees to the tenets of the Agile Manifesto 3 share some common characteristics, e.g., iterative and incremental development and focus on small releases [1]. The ideal context of applying agile methods in software projects lies within the so-called 'agile sweet spot', i.e., small and co-located teams of less than 50 persons with easy access to the user and business experts and that develop non-life-critical software [3]. However, applying agile methods both for larger projects or in larger companies [5], i.e., scaling agile methods, involves two significant challenges. First, the scaling of agile methods entails additional scaling and complexity factors that summon 'bitter spot' conditions for agile methods, such as a large number of teams, geographical distribution, entrenched culture, or formal governance structures [14]. Second, present agile methods do not provide sufficient guidance on dealing with these scaling and complexity factors [15]. Thus, custodians of existing agile methods and consultants that have worked with companies in scaling agile to their settings have proposed several agile scaling frameworks over the last years to address the limitations of the agile methods in large organizations and projects [6,7,10]. These frameworks incorporate predefined workflow patterns to deal with issues related to large number of teams, inter-team coordination, and customer involvement [10,16].
Due to the importance of this topic to companies, researchers have started to study the frameworks' adoption [6]. Based on a structured literature review, Uludag et al. [17] identified 20 different agile scaling frameworks presented in Table 1 4 . Secondary studies on the scaling frameworks compare some of them based on different criteria. For instance, Alqudah and Razali [10] juxtapose DAD, LeSS, Nexus, RAGE, SAFe, and Spotify based on, e.g., team size, available training and certificates, and the underlying agile methods and practices. Diebold et al. [18] provide a map visualizing underlying agile practices of different frameworks, such as DAD, LeSS, and Nexus, to support organizations in the selection of appropriate frameworks. Based on 13 agile transformation cases, Conboy and Carroll [16] provide nine challenges and a set of recommendations associated with agile scaling frameworks, such as LeSS, Nexus, S@S, and Spotify.
Although agile scaling frameworks have received some attention from academics [6], to the best of our knowledge, there is no other work that provides an overview of agile scaling frameworks, their evolution, and reasons, as well as the benefits and challenges of these frameworks.

Research Methodology
Survey Design. To answer the research questions, we created a survey following the guidelines suggested by Linåker et al. [19]. We opted to conduct a survey as it often aims to provide a state-of-the-art overview on particular methods [20], such as agile scaling frameworks. As a large part of our survey consists of closedended questions to quantitatively analyze the agile scaling frameworks, we used a survey as it is a suitable means to provide a quantitative description of the data [20]. The questionnaire consisted of four sections with a total of 22 questions 5 . The first section included questions on the framework background, e.g., reasons Table 1.
Overview of agile scaling frameworks based on [17]  behind the framework creation and the claimed benefits and challenges. The second section presented questions about framework evolution, e.g., the framework version history. In the third section, we aimed to capture the lean and agile foundations behind the framework, e.g., agile practices adopted to develop the framework. In the final section, we collected information on compatibility between the frameworks. The questions were compiled based on previous studies [8,17] and the Ask Matrix 6 .
Survey Validation. Two experienced researchers validated the questionnaire from the software engineering research group at TU Munich. Their suggestions on length, language, and the order of questions were incorporated.
Data Collection. We collected data between August 2017 and September 2019 using the online tool Unipark 7 . We used various approaches to reach out to the inventors or organizations, i.e., methodologists, that created the frameworks shown in Table 1. First, we sent out the questionnaire link to 22 methodologists by email. Second, we contacted some of the methodologists in two of the leading agile conferences: XP 2019 8 and Agile 2019 9 , and emailed them the survey link. Third, we reached a few methodologists via LinkedIn 10 by sending a personal message with the survey link. We received responses from 15 creators.
Data Analysis. We imported the survey data related to our four research questions to excel sheets. The first two authors analyzed data for all research questions individually by following Corbin and Strauss's coding guidelines [21]. We started with breaking down the data into meaningful entities, i.e., open codes. Later, based on the constant comparison of similarities and differences, we grouped the open codes into higher categories of codes called axial codes. Finally, both authors had a few discussions to compare the open and axial codes from their analysis. The majority of the codes matched between the two authors, and only a few adjustments were made by mutual agreement.  According to our survey data, Crystal is the first created agile scaling framework which development started in 1997. Nexus, eScrum, and S@S were also relatively early designed compared to most other agile scaling frameworks. However, it took the methodologists almost ten years to publish these frameworks, e.g., by publishing their official guides. Although nine frameworks were created before 2010, only three of them went public before 2010. Whereas, between 2011 and 2018, twelve frameworks were published. None of the methodologists indicated stopping the further development of their frameworks. Most frameworks have multiple versions, whereas four frameworks have only one version, namely Nexus, LeSS, Spotify, and XSCALE. Gill was initially created as the ASSF framework (2005-2008), which then evolved into Gill in 2012. The methodologists of Mega indicated that Mega 1.0 was a derivative of SoS. They also stated that Mega 2.0 was influenced by Spotify including the idea to extend the adoption of agile practices to other parts of the organization. The methodologists of Spotify found inspiration from Craig Larman's and Bass Vodde's two books (cf. [22,23]), that later became LeSS. Spotify was also influenced by the Program Increment Planning events of SAFe (cf. [24]). Table 2 presents 12 reasons behind creating scaling frameworks based on our survey. These reasons were grouped into four categories: complexity, customer, market, and organization. The most commonly stated reasons were: improving the agility/adaptability of the organization, improving the collaboration of agile teams working on same product, improving the coordination of agile teams working, and improving the synchronization of agile teams working on same product.  Table 3 presents 30 claimed benefits of adopting scaling frameworks based on our survey. These benefits were grouped into two categories, namely: business/product and organization/culture. The most commonly mentioned benefits were: enabling frequent product deliveries, enhancing employee satisfaction/motivation/engagement, improving software quality, providing customer/ business value, improving the collaboration of agile teams working on same product, improving the coordination of agile teams working on same product, improving the synchronization of agile teams working on same product. Table 4 presents 22 challenges of adopting scaling frameworks based on our survey. These challenges were grouped into three categories: implementation, organization/culture, and scope. The most commonly mentioned challenges were: using frameworks as cooking recipes and using frameworks without understanding for what reasons they should be applied.

RQ1: How did agile scaling frameworks evolve over the years?
By comparing the evolution map of agile scaling frameworks in Fig. 2 with the evolutionary map of agile methods by Abrahamsson et al. [13], we observed two notable parallels. First, similar to the movement of agile methods, the movement to agile scaling frameworks emerged from parallel innovation both by some inventors of existing agile methods and by consultants who supported organizations in scaling the agile methods. Second, likewise to agile methods, agile scaling frameworks have been continuously emerging and evolving after the movement started. This trend will likely continue as the methodologists of agile scaling frameworks seem to be committed to improving them in the future. Although the evolution map visualizes several agile scaling frameworks, users have concentrated on a few frameworks [25], particularly on SAFe and SoS [8]. The most recent State of Agile survey [8] confirms this by stating that 35% of their respondents adopted SAFe and 16% used SoS. A similar observation can be made for agile methods, as 58% of the respondents of the State of Agile survey use Scrum, making it the most commonly used agile method [8].

RQ2: What are key reasons behind creating of agile scaling frameworks?
In total, we found 12 reasons behind the creation of 15 agile scaling frameworks. The reasons identified in our survey fall into either the category of improving the current state of the organization or dealing with the organization's prevalent challenges. Both look similar to reasons that trigger an organizational change [26]. Several reasons, e.g., improving the collaboration and coordination agile teams working on same product and dealing with changing environments were found in previous studies on large-scale agile development [27,28]. Other reasons related to the scaling of agile methods, such as dealing with increased complexity and scaling agile to more people, were also reported in [9,[29][30][31]. However, to our knowledge, two reasons found in our survey related to descaling large product organizations into smaller independent entities and improving customer involvement were not reported by the extant literature on agile development. Surprisingly, several popular reasons for agile, e.g., improving productivity, improving visibility, and improving predictability, were not reported as reasons [8]. As the questionnaire's question was about the main reasons of creating a framework, these earlier mentioned reasons can be some of the implicit reasons behind the creation of the 15 agile scaling frameworks.

RQ3: What are the claimed benefits of adopting agile scaling frameworks?
In total, we identified 30 claimed benefits. The majority of these claimed benefits were similar to the benefits of agile adoption in general found from recent studies on agile method, e.g., State of Agile survey [8]. However, the most common benefit of agile, namely improved productivity [8], was not mentioned by any methodologists. We also identified benefits related to reducing headcount and fostering servant leadership, which were not found in the previous literature on large-scale agile development. More research on benefits is needed to establish scientific evidence of using these frameworks in the industry. It is also crucial to understand which practices have contributed to these benefits.

RQ4: What are the challenges of adopting agile scaling frameworks?
We identified 22 challenges from 15 scaling frameworks. To our knowledge, none of the framework's official websites has given information related to the difficulties encountered while adopting these frameworks. The most common challenges identified in our study, i.e., using frameworks as cooking recipes and using frameworks without understanding for what reasons they should be applied, were not reported by previously published empirical studies. The majority of the challenges found in our study, e.g., change resistance, moving away from agile, implementation is difficult due to remaining power structures, and lack of management buy-in, were already reported in previously published studies on scaling frameworks [16,[32][33][34] and large-scale agile development [5,35]. The challenges look similar to agile transformation challenges in general. Hence, using an agile scaling framework is not a silver bullet for scaling agile in large organizations, but a starting point for an agile transformation [33]. Several methodologists mentioned that leaders and change agents should focus on changing people's culture and mindset, rather than using frameworks only as cooking recipes.

Limitations
We discuss the limitations of our study through the threats, as suggested by Wohlin et al. [36].
Construct Validity. This threat is concerned whether the questions presented in the questionnaire represent the attributes being measured. Two survey experts thoroughly checked the questionnaire and evaluated its' understandability, clarity, and readability to counteract this threat. Moreover, the questions were compiled based on previously published studies in the realm of agile software development.

External Validity.
This threat is about the generalizability of the results. We aimed to collect responses from all existing scaling frameworks. Out of 22 frameworks, we received responses from 15 methodologists. We could not get responses from the methodologists of seven frameworks despite contacting them several times via email. Thus, this threat could not be completely mitigated. However, we received responses from the most widely adopted scaling frameworks, such as SAFe, LeSS, DAD, and Spotify [8].
Internal Validity. This threat is concerned with factors that can affect the relationship between the research process and survey results, i.e., the cause and effect relationship. We contacted the methodologists via emails found from the frameworks' official websites. We received confirmation from most methodologists after they filled in the survey, which ensured that the right persons answered the survey. We also met some methodologists during the agile conferences personally and asked them to answer the survey.

Conclusion
Validity. This threat deals with the ability to conclude from survey data. The data was coded independently by two researchers. Both researchers compared the codes and drew conclusions together to avoid misinterpretation and misunderstanding of the data.

Conclusions
Large-scale agile development has received significant interest by practitioners and academics over the last years [37]. As organizations are driven by pressures to scale and to react fast, agile scaling frameworks are increasingly prevalent in contemporary software organizations [7,8], sparking a growing academic interest in studying the adoption of these frameworks [6]. Although there is a body of knowledge on agile scaling frameworks, less research has been conducted to provide an overview of these frameworks and their evolution, study the reasons behind creating these frameworks, and investigate the benefits and challenges of adopting these frameworks. We surveyed the methodologists behind the agile scaling frameworks to address this research gap.
Our study provides an overview of 22 agile scaling frameworks of which 15 were covered by our survey. Our study extends extant literature by providing a map on agile scaling frameworks with their evolutionary paths. Although many methodologists started creating their first frameworks between 2001 and 2011, most guides on these frameworks were published later on. Our findings show a cluster of framework publications between 2011 and 2018, confirming the rising industry interest in scaling the agile methods. We identified 12 reasons behind the creation of the agile scaling frameworks. We revealed two new reasons which were not reported by the existing literature on agile development: descaling large product organizations into smaller independent entities and improving customer involvement. Further, the methodologists claimed 30 different benefits of adopting their frameworks related to business, product, organizational, and cultural aspects. The methodologists also reported two new benefits which were not described in the previous literature: reducing headcount and fostering servant leadership. The methodologists recognized 22 challenges in the adoption of the frameworks of which two were newly discovered in our study, i.e., using frameworks as cooking recipes and using frameworks without understanding for what reasons they should be applied.
We encourage researchers to investigate further how contextual factors, such as complexity, multi-product development, or agile maturity, impact a scaling framework's selection. We call for cross-case analyses to compare the adoption of agile scaling frameworks based on common comparison characteristics.