CMB constraints on $\beta$-exponential inflationary models

We analyze a class of generalized inflationary models proposed in Ref. [1], known as $\beta$-exponential inflation. We show that this kind of potential can arise in the context of brane cosmology, where the field describing the size of the extra-dimension is interpreted as the inflaton. We discuss the observational viability of this class of model in light of the latest Cosmic Microwave Background (CMB) data from the Planck Collaboration through a Bayesian analysis, and impose tight constraints on the model parameters. We find that the CMB data alone prefer \textit{weakly} the minimal standard model ($\Lambda$CDM) over the $\beta$-exponential inflation. However, when current local measurements of the Hubble parameter, $H_0$, are considered, the $\beta$-inflation model is {\it moderately} preferred over the $\Lambda$CDM cosmology, making the study of this class of inflationary models interesting in the context of the current $H_0$ tension.

I. INTRODUCTION
The inflationary scenario -the current paradigm of early universe cosmology -offers an elegant theoretical framework which is able to explain the large size and entropy of the current universe, its spatial flatness and, most importantly, the causal origin of the primordial cosmological perturbations. In the simplest models, inflation is driven by a single minimally-coupled scalar field φ rolling down a smooth potential V (φ), which generates a primordial scalar perturbation with nearly scaleinvariant power spectrum. This framework seems to agree with the most recent cosmic microwave background observations [2,3], and thanks to the accuracy of these data, it has also been possible to test the observational viability of a wide range of inflationary models (see, e.g., [4] and references therein). However, no compelling statistical evidence has been found for a specific inflationary model and, therefore, an important task nowadays is to examine the theoretical predictions of different classes of scenarios in the light of current observational data (we refer the reader to [5][6][7][8] for different points of view of the current observational status of inflation).
In this work we study theoretical and observational predictions of a class of potentials of the type * aparecida.fisica@dfte.ufrn.br † micolbenetti@on.br ‡ alcaniz@on.br § fabrito@df.ufcg.edu.br ¶ raimundosilva@dfte.ufrn.br named β-exponential potential, which was introduced in Ref. [1] as a generalization of the usual inflationary exponential potential (see [9][10][11] and references therein). The β-exponential function, exp 1−β (f ), is defined as above for positive values of the term between brackets and zero otherwise, and satisfies the inverse identity exp 1−β [ln 1−β (f )] = f , where ln 1−β (f ) = (f β − 1)/β is the β-logarithmic function [1]. Although in the limit β → 0 all the above expressions reproduce the usual exponential and logarithm properties, this is not the case for the observational predictions of the potential (1), as discussed in Ref. [12]. Actually, β-exponential potentials present a number of cosmological solutions for a large interval of values of β.
The β-exponential potential was originally proposed as a purely phenomenological model. In Sec. II, however, we show how this class of potentials can arise in the context of braneworld cosmology, where the field describing the size of the extra-dimension is interpreted as the inflaton. In Sec. III we derive the observational quantities of the β-exponential inflation and discuss some prior constraints on the parameters β and λ. A detailed observational analysis of this class of models in light of the latest CMB data provided by the Planck Collaboration [2,3] is presented in Sec. IV along with a Bayesian model comparison with respect to the standard Λ Cold Dark Matter Inflationary (ΛCDM) cosmology. We discuss the main results of our analysis in Sec. V and summarise our main conclusions in Sec. VI. Throughout this paper we work in units such that M Pl = (8πG) −1/2 = c =h = 1.

II. β-POTENTIALS FROM BRANE INFLATION
In what follows we show how the β-exponential potential of Eq. (1) can appear in the context of brane cosmology, where the radion (a field describing the size of the arXiv:1710.09808v1 [astro-ph.CO] 26 Oct 2017 extra-dimension) is interpreted as the inflaton. A stabilization radion mechanism is required in order to have static extra-dimension consistent with the equations of motion. This is achieved as the radion reaches the vacuum expectation value of the radion potential [13][14][15][16].
The action of thick branes version of such scenario can be written as which in general is a D-dimensional gravitational theory coupled to i = 1, 2, ..., N scalar fields, where M = 0, 1, ..., D − 1. Below we shall focus on D = 5 dimensions and N = 2 scalars. The five-dimensional geometry is assumed to have the general form with Poincaré invariance along the four-dimensional worldvolume of the 3-brane embedded in a five-dimensional bulk whose fifth coordinate is r, i.e., In a supergravity inspired action, the equations of motion can be solved by the following set of first-order differential equations for the scalar fields φ i (r) and warp factor exp (2A(r)) for the scalar potential given in terms of the superpotential W in the form [17,18] More specifically in [19] it was addressed the scenario with dilatonic scalar potential in terms of the superpotential The Bogomol'nyi-Prasad-Sommerfield (BPS) solutions of the first-order differential equations (4) are with A(r) = ln(1 + c 1 r), for r > 0 and A(r) = ln(1 − c 1 r) for r < 0. These are dilatonic solutions [20] which are also known as scaling solutions [21,22]. Now by properly combining background BPS solutions under the transformation φ 1 (r) → φ 1 (r − L), which means A(r) → A(|r − L|) to patch together the solutions for r < L and r > L along with a thin brane located at r = L, and φ 2 (r) → −φ 2 (r) into the Lagrangian and considering that the fields have only implicit time dependence via the radion field L ≡ L(t) we find At the 'thin wall limit' we can identify delta functions as follows. This regime can be easily satisfied for c 1 = sufficiently large, where 1/λ has dimension of energy. For the sake of simplicity, we assume b 1 = b 2 , such that As 1 the Lagrangian (9) becomes is the brane tension. The four-dimensional action is then given by where V eff (L) = V 0 (1 + c 1 L) The induced four-dimensional metric g 4µν is obtained from the five-dimensional metric g M N as follows: where L is the position of the brane in relation to r = 0. Particularly in the model presented above, to ensure localization of four-dimensional gravity, L r c , where r c is the crossover scale [19] for further details see [23,24].
The brane inflation scalar potential can be readily found from the effective potential into (11) and can be written as the β-exponential potential of Eq. (1), with . This agrees with the phenomenologically favored values of the parameter β and with the fact that our brane scenario admits fourdimensional gravity in a limited scale L r c which is a consequence of the geometry having infinite volume for arbitrary large values of the fifth coordinate r. Then the radion (inflaton) cannot stabilize at infinity as the exponential potential at the limit β → 0 would require. The finite vacuum expectation values where the radion stabilizes is given by the zeroes of (1), i.e., at φ 0 = 1/βλ. For λ = 1 and β = 1/2, 1/4 we find the well-known quadratic and quartic potentials which are usually employed in chaotic inflation. In Fig. 1 we show the behaviour of the potential (1) as a function of the field φ for selected values of the parameters β and λ.

III. β-EXPONENTIAL INFLATION
In this section, we discuss some of the theoretical predictions of the class of potential (1) derived in the previous section. As is well known, the slow-roll inflationary regime is characterized by parameters which depend on the form of the potential and its derivative with respect to the field φ [25]. For the β-exponential potential, the slow-roll parameters are written as In this regime, the end of the inflationary phase is expected to happen at φ e , where the condition (φ e ) ∼ 1 is satisfied. From the above equation, we find The primordial power spectrum of the curvature perturbation is given by where ( * ) refers to pivot scale, i.e., when the CMB mode exits from horizon at the scale φ * . The value of P R (k * ) is set by the COBE normalization to the value 2.2×10 −9 for the pivot choice k * = 0.05 Mpc −1 [2]. Now, by inverting the Eq. (14), we can write the value of the amplitude V 0 as Noteworthy is the strict dependence of V 0 with both the λ and β parameters, or rather the degeneracy of such parameters in the amplitude potential value. This will be of crucial importance for the analysis in the next section. The observable field value φ * can be related to the number of e-folds at which the pivot scale crossed out the Hubble radius during inflation, defined From the above equations, one finds or still, Note that, assuming a positive inflationary field, the slow-roll conditions are fully met by the β-exponential potential for values of β ≥ 0 and 0 ≥ λ ≥ √ 2. Finally, the primordial spectral index, n s , and the tensor-to-scalar ratio, r, can be expressed respectively as n s −1 = −4 +2η and r = 16 [25]. Here, these parameters take the form and the relation between n s and r is given by The contours correspond to 68% and 95% (C.L.) obtained from the Planck(2015)+BICEP2/Keck Array data [3]. In an opposite way as the prediction of the spectral index n s , we note that the higher the values of β the smaller the model prediction of r, which is in agreement with observations. At the same time, the values of the parameter β must obey the constraint φ * ≥ φ ini for a fixed value of N * , implying in a more restrict interval of β (see Eq. 17) * . This conclusion changes if lower values of the parameter λ is assumed, as can be seen from a direct comparison of the left and right panels of Fig. (2).

IV. METHOD AND ANALYSIS
We perform a Bayesian model comparison analysis considering three models, namely, the standard ΛCDM scenario (as reference model) and the β-exponential inflation model varying both the β and λ parameters and fixing λ to a proper value, hereafter "model-1" and "model-2", respectively.
In order to perform our analysis, we use the Cos-moMC code [26] and the MultiNest algorithm [27- * The initial scale of the slow roll inflation is conventionally assumed to occur at N * = 70. Using such a value in Eq. (17) it is possible to calculate φ ini . 29], necessary to resolve the Boltzmann equations, explore the cosmological parameter space and make a Bayesian models selection. Two main modifications to the most recent CosmoMC release are performed. The first is in the Code for Anisotropies in the Microwave Background (CAMB) [30], already included in the CosmoMC, since in its basic realization it assumes a power-law parametrization for the primordial perturbation spectrum as P R = A s (k/k * ) ns−1 . Instead, in this work we intend to use the β-inflationary primordial potential form given by Eq. (1), and we need to compute the dynamics and perturbations of this model to construct the primordial power spectrum. In this context, we propose a modification in CAMB following the lines of the ModeCode [31,32] adapted for the our primordial potential choice. This latter code is able to compute the CMB anisotropies spectrum solving numerically the inflationary mode equations, i.e., solving the Friedmann and Klein-Gordon equations as well as the Fourier components of the gauge-invariant quantity u for an exact form of the single field inflaton potential V (φ). By integrating these equations it is possible to obtain H and φ as a function of time and the solution u k for the mode k. Therefore, following these steps, the code can compute the power spectrum of the curvature perturbation P R by P R = k 3 2π 2 u k z 2 , evaluated when the mode crosses the horizon.
The second main modification is made in the Cos-moMC source, i.e., by implementing the nested sampling of the code MultiNest [27][28][29] to achieve our Bayesian analysis of the model. The code MultiNest is able to accurately analyse models with high number of parameters and non-gaussian density distributions and/or pro- nounced degeneracies. It also calculates the Bayesian evidence of the model, allowing the Bayesian model comparison. In this model selection, the "best" model is the one that achieves the best compromise between quality of fit of the data and predictivity, which means that the model that better fits the data thanks to many free parameters must be weighed with its added complexity (we refer the reader to [33][34][35][36][37][38][39][40] for some recent applications of Bayesian model selection in cosmology). In our analysis, we vary the usual cosmological parameters, namely, the physical baryon density, Ω b h 2 , the physical cold dark matter density, Ω c h 2 , the ratio between the sound horizon and the angular diameter distance at decoupling, θ, the optical depth, τ and the parameters β and λ. We also vary the nuisance foreground parameters [41] and consider purely adiabatic initial conditions. The sum of neutrino masses is fixed to 0.06 eV, and we limit the analysis to scalar perturbations with k * = 0.05 Mpc −1 . We perform our analysis assuming the priors on the cosmological param-eters shown in Tab. I. The values of the parameters β and λ are chosen from the considerations made in the previous section -in particular we refer to the observational predictions of Fig. 2 and also recall the correlation between the parameters λ and β, as shown in Eq. (15). Mainly for this reason, we perform two kind of analysis, leaving both the parameters free to vary (model-1) and fixing the λ value to a proper value (model-2). In particular, the λ value we consider for the model-2 is the best fit obtained of such parameter in the model-1 analysis. For both analyses, it is assumed the arbitrary value of the number of e-folds, i.e., N * = 55. We use the second release of Planck data [41] (hereafter TT+lowP), namely, the high-Planck temperature data (for 30 < < 2508) from the 100-,143-, and 217-GHz half-mission TT cross-spectra and the low-P data by the joint TT, EE, BB and TE likelihood (in the range of 2 < < 29). For the second analysis, we also consider the Riess et al. results on the local expansion rate, H 0 = 73.24 ± 1.74 km.s −1 .Mpc −1 (68% C.L.), based on direct measurements made with the Hubble Space Telescope [42]. This measurement is used as an external Gaussian prior and we refer to this joint data set as TT+lowP+HST.
It is worth mentioning that for our results we use the most accurate Bayesian Importance Nested Sampling (INS) [29,43] instead of the vanilla Nested Sampling (NS), requiring INS Global Log-Evidence error < 0.1. In order to rank the models of interest, we use the scale in terms of the evidence strength of the chosen reference model [34]: ln B ij = 0−1 , ln B ij = 1−2.5, ln B ij = 2.5−5, and ln B ij > 5 indicate, respectively, an inconclusive, weak , moderate and strong preference of the model i with respect to the reference model j. Note that negative values of ln B ij mean support in favour of the reference model.

V. RESULTS
The main quantitative results of our analysis using the TT+lowP data are shown in Tab. II, where we report the constraints on the cosmological and primordial pa-rameters for the three analysed scenarios. Note that the results for the β-inflation models agree with the ΛCDM predictions at 1σ, as also shown in Fig. 3. In Fig. 4 we plot the best fit curves for the analysed models in comparison with the ΛCDM cosmology. In the last line of Tab. II we can see that the β-inflation model-1 is strongly disfavoured with respect to the reference model. At the same time, in Tab. III we report the results using the TT+lowP+HST data. We can see that the choice of fixing the λ value (model-2) allows to better constrain the β value, decreasing the model comparison to weakly disfavoured. This is basically due to the parameter degeneracy between β and λ, discussed in the previous sections.
An important aspect that is worth mentioning concerns the predictions on the spectral index, n s , of βinflation model (see Fig. 2). This class of models predicts a higher value of n s with respect to the ΛCDM model for the β values constrained by the data. In particular, the derived value of the spectral index is n s ∼ 0.976 for the model-1 and n s ∼ 0.977 for the model-2 (see Eq. 18). As well known (for a comprehensive reading, see Ref. [44]), the higher the value of n s the higher the values of Ω b h 2 and τ and, consequently, the higher value of H 0 . Therefore, this inflationary model naturally leads to higher val- ues of H 0 , which is in better agreement with the local expansion rate based on direct measurements made with the Hubble Space Telescope [42].
As mentioned earlier, we also perform an analysis of the β-inflation model in light of the joint TT+lowP+HST data set. The results are shown in Tab. III. We note that the prior on H 0 modifies the previous results for the evidence of the standard model and makes the β-inflation moderately preferred (ln B ij = 2.6) over the reference ΛCDM cosmology. It is important to emphasize that these results depend on the choice of the e-fold number used for the analysis, i.e, N * = 55. For a lower value of N * , one may obtain a better agreement with the ΛCDM model, which would be in accordance with the results shown in Fig. (2).

VI. CONCLUSIONS
Cosmic Microwave Background data are one of the most powerful tools to study the early universe physics. In particular, the past two decades have witnessed a great improvement in the measurements of the CMB fluctuations, which are now able to test the observational viability or even rule out different classes of inflationary models as well as some their alternatives.
In this paper, we have analyzed theoretical and observational aspects of a particular class of inflationary models proposed in Ref. [1], whose field potential is given by Eq. (1). First, we have shown that this kind of potential can arise in the context of brane cosmology, where the radion is interpreted as the inflaton field. The observational viability of this class of models have been studied through a Bayesian analysis using the latest Planck (2015) data. As shown in Tables II and III, tight constraints on the parameter β have been derived. Apart from the value of the spectral index n s , our analysis shows that the predictions of the β-inflation model are very similiar to the ones of the ΛCDM model (they agree at 68.3% C.L.). Considering only the TT+lowP CMB data, we have shown that the minimal standard model is weakly preferred over the β-exponential inflation. However, this result changes when we also consider the most recent HST measurements of the Hubble parameter, as reported in Ref. [42]. In this case (TT+lowP+HST data), the β-inflation model becomes moderately preferred over the ΛCDM cosmology, with ln B ij = 2.6.
Finally, it is important to mention that stringy and brane inspired potentials have been assumed to be the ultimate potentials to accomplish unique inflaton potentials from fundamental theories. But one usually faces problems with the comparison of their theoretical predictions and observational data. In the present study, we have shown that the β-exponential potential shows to be in agreement with the current observational data at the same time that it can be derived from a fundamental theory such as supergravity with dilatonic braneworld solutions. Another important aspect that is worth emphasizing in the present scenario is the fact that large values of β are in agreement with both observational data and


Socio-Economic Constraints Affecting Sustainable Rural Livelihood

The present study about “Constraints of rural livelihood with reference of services providers and rural female in Mansehra” was conducted with objectives to determine the constraints of rural livelihood and the role of services providers i.e. government and NGOs and the rural female in enhancing rural livelihood. A sample size 210 respondents out of 460 was selected through random sampling procedure. The study concluded that a highly significant association was found between rural livelihood and finance is the core problem for all categories of entrepreneurs in rural areas was found highly significant (p=0.000), poor health issues block sustainability of rural livelihood (p=0.000), prevalence of diseases in livestock and crops contribute income decline in rural areas (p=0.003), marketing problem is a major obstacle in rural economy production (p=0.003), land shortage affects rural economy (p=0.005) and rural livelihoods affect adversely by any kind of shock (p=0.007) while Illiteracy affects negatively sustainability of rural livelihood (p=0.014), conflict leads decline in rural livelihoods (p=0.021). In addition, highly significant relationship was found (p=0.000) between rural livelihood and through NGOs efforts people adopted ability for livelihood choices. Moreover, skill development training on male side contributed good results in enhancing livelihood (p=0.018), microfinance service of government/NGOs plays a pivotal role in the development of rural economy (p=0.031),through rural livelihood programs, the NGO is promoting livelihood and food security activities (p=0.025), protective interventions are considered vital to facilitating livelihood recovery (p=0.016) and between livelihood promotion through irrigation development and introducing of productive yield is a good step by the government (p=0.024) had significant relation with rural livelihood. Furthermore, significant relation was found between absence of basic livelihood assets for female makes unstable livelihood (p=0.003), kitchen gardening, poultry and livestock rearing are major livelihood activities of rural female (p=0.004), female make contribution to run livelihood system to contribute household expenses (p=0.029) and rural livelihood. In light of the study it was suggested to policy makers to provide assistance in term of material and capacity building i.e. easy credit system, agriculture, livestock, health, illiteracy interventions and NGOs and rural female participation in sustaining rural livelihood to minimize the risk of rural livelihood vulnerability. Citation: Sajid A, Ayatullah, Khan NA, Iqbal S, Abbas S, et al. (2018) Socio-Economic Constraints Affecting Sustainable Rural Livelihood. Arts Social Sci J 9: 324. doi: 10.4172/2151-6200.1000324

Introduction
Rural livelihood is defined by Robert Chambers and Gordon Conway as livelihood comprises the capabilities, assets and activities required for a means of living, a livelihood is sustainable which can cope with and recover from stress and shocks, maintain or enhance its capabilities and assets, and provide sustainable livelihood opportunities for the next generation [1]. Livelihood is also defined as adequate stocks and flows of food and cash to meet basic needs. Sustainable refers to the maintenance or enhancement of resource productivity on a long term basis so that a family or communities cope with chronic situation. A household may be enables to gain sustainable livelihood in many ways i.e. through ownership of land, livestock or trees, right to grazing, fishing, hunting or gathering through stable employment with adequate remuneration or through varied range of activities. Perret et al. define that the term "livelihood" is used rather than "job" or even "source of income" [2]. First, most rural people work in agriculture as farmers or farm workers or get non-farm job opportunities only seasonally and often part time. Second, individuals and households create a living from various sources i.e. farming, local craftwork, small-scale industries, own labour and trading, this form the backbone of rural people's livelihood in the world. ZimVac mentioned that different socioeconomic factors lead to decline in rural livelihood. Khatun and Roy stated that rural livelihoods are affected by socioeconomic constrains and calamities [3].
The Zimbabve Vulnerability assessment Committee, ZimVac after a detail study of Rural Livelihood in different rural areas of Zimbabve presented a crystal clear report indicating the rural livelihoods and the vulnerability of rural households affecting the sustainability of livelihood resulting extreme poverty. The report showed the reality of rural livelihood vulnerability with clear cut mentioning the socioeconomic factors as poverty, poor roads, transport, infrastructure and communication, water shortage, poor access to education, inadequate health facilities, poor water and sanitation, unavailability of agriculture inputs, poor rainfall , poor market and prices, food insecurity, unemployment, lack of capital, livestock diseases, wild animals, lack of NGOs and government interventions, land shortage and power shortage. The report contains suggestions and recommendations to handle the vulnerable situation for ensuring sustainable rural livelihood. WFP et al. stated that because of a number of reasons the livelihood system could not get sustainability in both rural and urban areas of Darfur [4]. The major socio-economic challenges indicated in the study by FAO and WFP include limited water resources for crops and livestock, declining soil fertility, lack of extension services for farmers and livestock keepers, high level of crop pest infestation, use of poor quality seeds, unstable product prices, conflicts over land and water resources, prevalence of heavy livestock diseases, drought and insecurity of livestock, lack of adequate agricultural and livestock marketing, infrastructure, limited capacity of agriculture and livestock research centers, poor financial resources of farmers and livestock keepers, poor roads infrastructure, high number of IDPs. Shonia finance plays a vital role in the sustainability of any kind of business. Poor population of Bangladesh mostly faces the problem of financial and productive assets. Because of low income, no livelihood asset can be made [1,5,6]. Alam mentioned that insufficient income always creates problem to generate and continue income generation activity. Rahman et al stated that finance is a major obstacle for any kind of small scale business in rural areas.

Universe of the study
The universe of the study was District and Tehsil Mansehra. Study area more specifically consisted of three villages i.e. Hilkot, Malookra and Balimang were randomly selected as a universe of the study for data collection.

Sampling procedure and sample size
A simple stratified random sampling methodology was used to select sample respondents for the purpose of data collection. A total sample of 210 respondents was selected for the total population size of 460 according to the method devised by Sekaran. The formula of proportion allocation sample size was applied to select sample size from each village as given below:

Data collection
A well-established closed ended interview schedule was devised that encompass all the objectives and parameters of the study that helped in collecting of data from the respondents to understand their attitudes and obtaining desired data. It helped in analyzing collected data in statistical form. And the anonymity of participants was kept secret [7][8][9].

Data processing and analysis
The entering of data and its analysis was done through SPSS 20 version (Statistical Package for Social Sciences). In first phase of analysis, uni-variate analysis will be displayed for frequency and percentage distribution, while the second phase, Bi-variate analysis will be displayed for the association of independent variables and dependent variable.
Chi-square test: Chi-Square test statistics will be applied as a statistical tool to ascertain the relationship. To find the value of chisquare test, the formula will be as under:

Rural livelihoods
To measure the significance of rural livelihood, few statements regarding rural livelihood were asked from the respondents given in Table 1, which shows that that majority 85.7% of respondents were agreed about the statement that agriculture is the main source of livelihood of the people in rural areas, moreover 12.9% of the whole sample size were disagreed with statement and 1.4% did not favor yes or no. Indira Nair stated the idea that agriculture is the major way of livelihood of rural population. For income and industrial production, agriculture output is of prime value, the same idea mentioned in a paper "Rural livelihoods, importance and definitions Unit 01. Similarly, a majority 88.1% of respondents favored the statement that rural livelihoods are not static they adopt change, moreover 11.9% of respondents did not favored the statement. The idea is mentioned in Unit 01 of rural livelihoods, importance and definition. In the same way, big section 94.8% respondents clicked "Yes" to show their positive attitude about the statement that sustainable livelihoods enhance female participation in livelihood activities, while 11.9% sample respondents clicked "No" and 1.4% came under section "Don't know. The idea confirmed by Doctor Rerum and Mündlichen Prüfung [10,11].
Nevertheless, a huge portion 90.5% of respondents showed positive attitude regarding the idea that livelihood satisfies the basic needs of family members, moreover 8.1% respondents negated the idea and 1.4 remained uncertain. The same idea stated by the UNDP, ISDR and IRP in "Guidance notes on recovery livelihoods, page 07. In response of a statement that rural livelihoods sustainability maintains the means of living of people, a majority 84.3% respondent accepted, moreover 11.4% negated and 4.3% of sample respondents did not know about the statement. Similarly, majority 90.5% of sample respondents were agreed with statement that rural employment is also a part of rural livelihoods, moreover, 9.5% did not agreed about the statement. WaqasQayyum mentioned in "The Pakistan development review, part ii" that rural employment is a major way of making sustainable livelihood but absence of employment effect of household economy [12].
Furthermore, a heavy segment 85.7% respondent replied "Yes" of households. Adversely effect on rural livelihood because of socioeconomic constrains is a universal phenomenon. Weakness of rural livelihoods and socioeconomic factors are related as a number of studies have proved. The complex association between rural livelihoods and socioeconomic condition is an observable fact. A few statements were developed to review that association. Respondents and their attitudes regarding the socioeconomic constrains are given in Table 2. Finance is the core problem for all categories of entrepreneurs in rural areas was found highly significant (p=0.000) with rural livelihoods. Rahman et al. confirmed that finance is the core problem for expansion of activities in all categories of entrepreneurs in rural areas. Sheheli has stated the finance as core problem of rural poor. Moreover, poor health issues block sustainability of rural livelihood was also found highly significant (p=0.000). These findings are in line with the findings of Zimbabve, ZimVac that inadequate health facilities affect rural livelihoods [14].
In the same way, the statement that prevalence of diseases in livestock and crops contribute income decline in rural areas was found significant (p=0.003) with rural livelihoods. WFP and FAO mentioned in their study that high level of crop pest infestation and prevalence of livestock diseases are obstacles of rural livelihood [4,5].
Moreover, the idea that marketing problem is a major obstacle in rural economy production was also found significant (p=0.003) with rural livelihoods. The result supported by earlier study Zimbabve Vulnerability Assessment Committee. Furthermore, the idea that land shortage affects directly rural economy was found significant (p=0.005) with rural livelihoods. These results are similar with the findings of Jennifar.
Furthermore, a significant relatiion (p=0.007) was found between rural livelihoods affect adversely by any kind of shock and rural livelihoods. These findings are similar with the study of FAO [3,5]. Similarly a significant relation was found (p=0.014) between Illiteracy affects negatively sustainability of rural livelihood and rural livelihoods. The idea conformed Khatun and Roy in a study 'Rural Livelihood diversification, determination and constraints [3].
In the same way, significant relation (p=0.021) was found between the statement that conflict leads decline in rural livelihoods and rural livelihoods. World Food Program, WFP, and Food and Agriculture Organization, FAO, also stated that conflicts over different resources make livelihood unstable [4,5]. However, a non-significant relation (p=0.354) was found between the idea that unstable product prices to support the statement that skills trainings contributes in rural livelihoods, moreover 9.5% respondents replied with "No" and 4.8% of respondents did not reply. Ali Shah and Shahbaz have presented the same idea that capacity building and skill development training on male side also contributed good results which enhances and reconstructed their livelihood as restarting business and earning income. In the same track, majority 80.5% of respondents thought that rural livelihoods also contain small scale businesses, moreover 10.0% did not think the statement was true and 9.5% were unaware of the statement. Perret et al. stated the same idea in a study "Poverty and livelihoods in rural South Africa" [2]. Similarly, the great portion 87.1% respondents were in view that NGOs interventions in rural livelihood lead to sustainable rural livelihood, moreover 8.1% respondents showed their view against the statement and 4.8% did not share any view.
Additionally, the whole sample population agreed that rural livelihoods are affected by socioeconomic constrains and calamities. The idea strongly confirmed by Khatun and Roy [3]. Further a very proportion 93.8% pointed that sustainable livelihood provides livelihood opportunities to next generation, moreover 1.8% opposed the idea and 1.4% did not response [13]. Robert Chambers and Gorden Conway have support the same idea in a paper "sustainable rural livelihood" [6]. Likewise, majority 82.4% respondents supported the view that sustainable livelihood approach improves the condition of poor, moreover 9.0% did not agree with majority and 8.6% did not share any idea. The idea confirmed Sheial Meikem Tamsin Ramasut and Julian Walker and UNDP.
Furthermore, a big portion 83.3% agreed with statement that rural livelihoods also contain nonagricultural activities in rural areas, moreover 11.4 were not agrees and 5.2% did not answer. Perret et al. also stated that rural people are engaged in agriculture activities as farmers and also carry out non-farm activities, also in rural area people are involved in production includes farming, local craft, and small level business, and trading is also an important source of rural livelihood [2]. Similarly the big part 82.9% respondents were of the opinion that sustainable livelihood does not adversely affect environment, moreover 11.9% did not of the same opinion and 6.2% did not share any opinion. The same idea also mentioned by Robert Chamber and Gorden Conway in a paper "sustainable rural livelihood" (Table 1) [6].

Association between socioeconomic constraints and rural livelihoods
Sustainable rural livelihood depends on socioeconomic condition results poor income and rural livelihoods. WFP and FAO stated the unstable product price is one of issues in rural livelihoods [4,5]. Similarly, a non-significant relation (p=0.892) was found between difficulty in obtaining raw material create hindrance for rural livelihood and rural livelihoods. The findings confirmed by Marof Redzuan and Fariborz Aref in a study conducted in underdeveloped region of Malaysia.
In addition, non-significant relation (p=0.055) was found between the poor infrastructure always negatively affects rural economy and rural livelihoods. The result is similar with the findings of Okechukwu Agwu that poor infrastructure always negatively affects rural livelihoods. Similarly, a non-significant relationship (p=0.280) was found between power shortage always jeopardy rural livelihood and rural livelihoods.
In the same row, non-significant relationship (p=0.935) was found between the idea that heavy interest on loan discourages rural livelihood and rural livelihoods. Likewise, result illustrate that a nonsignificant relation (p=0.068) was found between low income cause to crush livelihood sustainability in rural area and rural livelihoods. The statement confirmed WFPU, FAO, Marof Redzuan and FariborzAref low income is one of major issues of rural livelihoods [4,5].

Conclusions
This study focused on the investigation into the factors behind rural livelihood. It was concluded that rural livelihood was a general phenomenon throughout the world. Rural livelihood mostly consisted


extfieldsofstudy:
Mathematics
Computer Science

text:
Multi-armed Bandits with Constrained Arms and Hidden States

The problem of rested and restless multi-armed bandits with constrained availability of arms is considered. The states of arms evolve in Markovian manner and the exact states are hidden from the decision maker. First, some structural results on value functions are claimed. Following these results, the optimal policy turns out to be a \textit{threshold policy}. Further, \textit{indexability} of rested bandits is established and index formula is derived. The performance of index policy is illustrated and compared with myopic policy using numerical examples.

I. INTRODUCTION
Multi-armed bandits are among commonly used models for solving sequential decision making problems, [1], [2]. In the multi-armed bandit problem, there are N arms and each arm can be in one of a finite set of states. The decision maker plays M arms, (M < N) at every time instant and collects rewards from the played arms. Reward from each played arm depends on the state of that arm. The state of an arm changes according to a stochastic process associated with that arm. The decision maker's aim is to maximize the long-term expected discounted reward. The state evolution may be action dependent and based on that there are two types of bandits, rested and restless bandits. In a rested bandit, the state evolves only for the arm which is played while states of other arms do not change. For a restless bandit, the states of all arms evolve even when they are not played. In this setting, each arm can be considered as a Markov decision process (MDP) with finite states and two actions (play or not to play) in each state. As a model choice, states may assumed to be either observable by decision maker or hidden to it. Now, the multi-armed bandit problem can be looked as a set of MDPs coupled together with constraints.
A rested multi-armed bandit problem was first introduced in the seminal work of [1], where the author proposed an index based policy. In such policies, state of each arm is mapped to an index, i.e., real valued number. At each time instant arms with the highest indices are played. This policy is known as Gittins index policy. Later, a generalization of the rested multi-armed bandit problem was devised in [3], where a restless multi-armed bandit was introduced and again index based policy proposed. The index policy for restless bandits is now referred to as Whittle index policy.
Recently, restless bandits have been studied when state of the arms are not observable but feedback signal is observable. The decision maker estimates the state from this feedback. This is called the hidden Markov bandit. For a hidden Markov bandit, each arm can be modeled using partially observable Markov decision processes (POMDP). An index policy The work of Varun Mehta and Kesav Kaza was done in SPANN Lab at IIT Bombay. The work of Rahul Meshram was carried out in the Bharti Centre for Communications at IIT Bombay.
To use index policy in rested and restless bandit, an approach is to first consider the singlearmed bandit problem and show that the optimal policy is of a threshold type. Using this result one can show that arm is indexable and later index can be derived. While analyzing a single-armed bandit model, structural results of POMDP can be used for hidden Markov bandits. Some structural results for POMDP have been extensively studied in [9], [10], [11], [12].
All of the above works on bandits assume that every arm is available for decision maker at each time instant to play. The decision maker determines whether to play or not play the arms using index policy. But this may not be feasible in some scenarios. For example, in a machine-repair problem one may not able to schedule a task on some of the machines due to machine breakdown. Such consideration has been made in [13]. In queuing systems, the controller may not be able to schedule jobs to some servers due to server breakdown, [14], [15]. In these examples a machine or server is available to the decision maker intermittently. In this work, we consider rested and restless bandits with arm availability constraints where arms may not be available to play at some time instants and these are called as constrained bandits. It is a generalization of the classical rested and restless multi-armed bandit problems. Usually when arm is not available, we consider a substitute arm which yield low reward compare to the arm when it is available.
In constrained bandits [13], [14], [15], each state is defined as a pair (X(t), Y (t)), where X(t) represents the state of arm and Y (t) represents availability of an arm at time t. Time is discretized in [13] while it is continuous in the models of [14], [15]. The state (X(t), Y (t)) is assumed to be observable. Under some assumptions on model parameters the index policy is analyzed in [13], [14], [15]. In this paper we consider a hidden Markov model, where state X(t) of the arm is not observable but the availability of the arm is observable.
The paper is organized as follows. In next section, we describe the hidden Markov model for multi-armed bandit with constraints. We later consider single armed bandit problem in Section III. We analyze structural results for single-armed bandit in Section IV. Section V we compute the index for hidden Markov rested bandit with availability constraints on arm. We also illustrate the performance of the index policy and compare it with that of myopic policy in Section VI. We finally conclude in Section VII and discuss some of open issues.
II. PRELIMINARIES AND MODEL DESCRIPTION Consider a multi-armed bandit with N independent arms. Each arm can be in one of two states, 0 and 1,. The system is time slotted and it is indexed by t. Let X n (t) denote the state of arm n at beginning of time slot t, X n (t) ∈ {0, 1}. Each arm has availability constraints i.e. it is intermittently available. Let Y n (t) ∈ {0, 1} represent the availability of arm n in time slot t and Y n (t) = 1 if arm n is available, 0 if arm n is not available.
When arm n is not available in slot t we will assume that , the arm n is replaced by substitute arm which yield low reward after play. A n (t) ∈ {0, 1} is the action in slot t with the following interpretation.
A n (t) = 1 if arm n is played in slot t, 0 otherwise. 3 Exactly one arm is to be played in each time slot. Arm n changes state at the end of time slot t according to transition probabilities that depend on A n (t), Y n (t) and it is defined as follows.
Pr{X n (t + 1) = j | X n (t) = i, Y n (t) = y, A n (t) = a} = P n ij (y, a) In every slot t, a binary signal Z y n (t) is observed from the arm n that is played. There is no observation from the arms that are not played. Thus Z y n (t) = 1 play of arm n is successful 0 otherwise.
Pr (Z y n (t) = 1 | X n (t) = i, Y n (t) = y, A n (t) = 1) = ρ n (i, y). Also, R a n (i, y) is the reward obtained from playing arm n given that, X n (t) = i, Y n (t) = y, A n (t) = a. Let Further, we will suppose that 0 ≤ η n,0 < r n,0 < η n,1 < r n,1 ≤ 1 for all n.
Remark 1: • The observation variable Z y n (t) may have different meanings in different applications. In communication systems, Z y n (t) = 1 may mean an acknowledgement (ACK) of a successful transmission over a given link, [5]. For a recommendation system, it may correspond to click or like by the user over a recommended item, see [6]. • Notice that η n,i = 0; this means there is a non-zero reward for playing an arm even when it is not available. This captures application scenarios where broken (not available) arms can be repaired by playing them and paying a penalty from the reward. The decision maker cannot directly observe states of the arms, and hence it does not know the states at the beginning of each time slot. But decision maker knows the probability of availability θ a n (i, y) of arm n, at the beginning of next time slot t + 1; it is as follows θ a n (i, y) = Pr (Y n (t + 1) = 1|X n (t) = i, Y n (t) = y, A n (t) = a).
However, the decision maker maintains a belief π n (t) about the state of arm n. It is the probability that the arm is in state 0 given all past availability, actions, observations. This is given as follows.
π n (t) = Pr X n (t) = 0 | (Y n (s) = y s , A n (s), Z ys n (s)) t−1 s=1 . Let H t denote the history, H t := (Y n (s) = y s , A n (s), Z ys n (s)) 1≤n≤N,1≤s<t . We can describe the state of arm n at time t by S n (t) = (π n (t), Y n (t)) ∈ [0, 1] × {0, 1}. (S 1 (t), · · · S N (t)) is the state information of the arms at the beginning of time slot t. Further, we can rewrite θ a n (i, y) as function of π in following form.
Hence the expected reward from playing arm n at time t given that Y n (t) = y is R 1 n (π n (t), y) = π n (t)R 1 n (0, y) + (1 − π n (t)R 1 n (1, y). In each slot, exactly one arm is to played. Let φ(t) is the policy by the decision maker such that φ(t) : H t → {1, · · · , N} maps the history to one of the arm at slot t. Let We are now ready to define the infinite horizon discounted reward under policy φ for initial state information (π, y), π = (π 1 (1), · · · , π N (1)) and y = (y 1 (1), · · · , y N (1)). It is given by Here, β is discount parameter, 0 < β < 1. The goal is to find a policy φ that maximizes V φ (π, y) for given π ∈ [0, 1] N , y ∈ {0, 1} N . The optimization problem (1) is a multiarmed bandit problem with availability constraints. This is generalized version of multi-armed bandits, where it has partially observable states and availability constraints. In general, this problem is known to be PSPACE-hard, [16]. Index based policies are developed in [2], [3] for rested and restless multi-armed bandits. To study such index policies, a Lagrangian relaxed version of problem (1) is analysed. In this relaxed problem, complexity of problem reduced as it separates the solving one multi-armed bandit problem to N single-armed bandit problems.
Thus it reduces to calculating the index for each arm separately. The arm with highest index is played in each time slot. We next analyze the single-armed bandit problem in next section.
III. SINGLE-ARMED BANDIT PROBLEM For notational convenience, we will drop the subscript n, i.e., the sequence number of the arm. As a widely used method for solving the single arm bandit problem, a subsidy w is assigned for not playing the arm [3]. In that case, optimization problem (1) can be rewritten as follows.
where action A(t) under policy φ is The objective is to find a policy φ that maximizes V φ (π, y).
Recall that the state evolution of arms may be action dependent. Based on this, we can have two different types of bandits, rested and restless single-armed bandit. In rested singlearmed bandit, state evolves for the arm that is played and state of other arms do not change. For restless bandit model, state of all arms changes at each time slot.
From [17], we know that the π(t) captures the information about the history H t , and it is a sufficient statistic. It suggests that the optimal policies can be restricted to stationary Markov policies. In this, one can obtain the optimum value function by solving suitable dynamic program, it will be given in later part of this section.
Let us define the value function under initial action A 1 and availability Y 1 1 But in general, transition probabilities for available and unavailable arms could be different.
We can write the following.
IV. STRUCTURAL RESULTS We now begin with some of structural results on value functions, showing convexity and threshold type policy.
2) For a fixed π, V (π), V S (π), V N S (π), V (π), V T (π) and V N S (π) are non decreasing and convex in w. A sketch of the proof is in Appendix VII-A. We first define a threshold or monotone policy for the single armed bandit problem and then prove that the optimal policy is of this kind under some restriction on model parameters.
Definition 1: (Threshold type policy) A policy is said to be threshold type, if one of the following is true.
1) The optimal action is to play the arm ∀π.
2) The optimal action is to not play the arm ∀π.
3) There exists a threshold π * such that ∀π ≤ π * the optimal action is to play the arm and not to play the arm otherwise.
A. Threshold structure of optimal policy (case µ 0 > µ 1 ) The following lemma provides sufficient conditions for monotonicity of the optimal value function.
Remark 2: The lemma says that if the rewards, observation and transition probabilities follow certain order than the optimal value functions are monotone with belief π. This result can be utilized to prove that optimal policy is a monotone policy. A monotone policy is one where the actions are monotone over state space.
To have monotone optimal policy, we first prove that the difference between the value functions V S (π) and V N S (π), is monotonic in π. Similarly, we prove this for V S (π) and V N S (π).
Lemma 3: (Isotone difference property) For fixed w and conditions of Lemma Then the following theorem gives monotone optimal policy on belief π.
Theorem 1: (Monotone optimal policy) 1) If the value function V : S 1 × A → R has isotone difference on S 1 × A then there exists a non increasing optimal policy a * : S 1 → A on belief S 1 . 2) If the value function V : S 0 × A → R has isotone difference on S 0 × A then there exists a non increasing optimal policy a * : S 0 → A on belief S 0 . Proof: From Lemma 1, the value functions V (π), V (π) are convex and monotone in π. From Lemma 2, V (π), V (π) has isotone difference property. This implies, there exists a * (π) ∈ {0, 1} that is non increasing in π.
Remark 3: Here, we observe that the optimal actions are ordered on belief space. This indeed is a threshold type policy by Definition 1. Note that a monotone policy is a threshold policy for two actions. Thus isotone difference property implies a threshold policy result.
B. Threshold structure of optimal policy (case µ 0 < µ 1 ) For µ 0 < µ 1 , different proof technique is necessary to To claim a threshold type optimal policy. Here, we will assume θ a (π, y) = θ a (y), i.e. independent of π.
We first argue that difference between the value functions V S (π) and V N S (π), is monotonic in π for special cases. Similarly, difference between V S (π) and V N S (π) is monotone in π.
Lemma 4: For fixed w and β, and We describe sketch of the proof in Appendix VII-D.
Remark 4: The proof of this Lemma is different from the earlier Lemma 2 because here we are not assuming monotonicity of value functions. Instead here we use the Lipschitz properties of value functions with respect to π, i.e., the value functions, V (π), V (π) have following property Theorem 2: For fixed w and β, and 0 ≤ µ 1 − µ 0 ≤ 1 3 , 1) The optimal policy is threshold type for V T (π) and V N S (π). That is, either V (π) = V S (π) for all π ∈ [0, 1] or V (π) = V N S (π) for all π ∈ [0, 1] or there exists π * such that for π ≤ π * , V N S (π) for π ≥ π * .

Remark 5:
• The proof of Theorem 2 is analogous to the Theorem IV-A. • In Section VI, we will present few numerical examples to illustarte a threshold-type policy for general case, where we do not make any restriction on θ and model parameters µs.
V. INDEX POLICY FOR SINGLE-ARMED BANDIT Recall that our interest here is to seek an index-type policy. We now define indexability of an arm and then its index. Let G(w) be the subset of state space S = [0, 1] × {0, 1} in which it is optimal to not play the arm with subsidy w, it is given as follows.
Using set G(w), indexability and index are defined as follows. Definition 2: An arm is indexable if G(w) is increasing in subsidy w, i.e.,
Remark 6: • Note that we can rewrite definition of set G(w) in the following way.
where π L := min{π ∈ [0, 1] : V S (π) = V N S (π)}, and π L := min{π ∈ [0, 1] : V S (π) = V N S (π)}. • If the optimal policy is of threshold type, then π * = π L and π = π L . • To claim indexability, we require to show that as subsidy w increases, π L (w) and π L (w) are non-increasing in w. • In general, it is difficult to show indexability and obtain index because there is difficulty in proving a threshold type policy. We next show the indexability and compute the closed form expression for the index of a single-armed rested bandit. The proof of index computation is along the lines of [13].

A. Rested single-armed bandit
We further simplify the rested single-armed bandit problem and make following assumptions on transition probabilities.
where δ ij equals to 1 if i = j and 0 otherwise. Also, p 00 = µ 0 , and p 10 = µ 1 . This indicates that state of the arm changes if arm is available and does not change when arm is unavailable. Further we assume θ 0 (π, 0) = 0. We now present a few preliminary results which are used to derive the index. These results make use of the definition of set G(w) and obtain value function expressions.
Proof: 1. State of the arm does not change when arm is unavailable and not played. Therefore if (π, 0) ∈ G(w), then it is always optimal to not play the arm and the expected total discounted reward starting in state (π, 0) is V (π, w) = w 1−β . 2. If (π, 1) ∈ G(w), then, the arm may visit (π, 0) state if it goes unavailable in between. Therefore, the arm is in either (π, 1) or (π, 0) state. In this case, two optimal policies are possible (a) never play the arm, (b) do not play the arm when it is in state (π, 1) and play the arm when it is in (π, 0). The expected total discounted reward for policy (a) is w 1−β and for policy (b) is given in (11).
If subsidy w is higher than η 0 , then, set G(w) can be non-empty. We will provide sufficient condition on subsidy w for G(w) to be non-empty. Also, if set G(w) is nonempty then we give lower bound on subsidy w. This is given in the next Lemma.
Lemma 7: (π, y) ∈ G(w) if and only if for τ > 0. Proof: We first assume that (π, y) ∈ G(w). We want to prove Eqn. (13). We know from Lemma 5 that if (π, 0) ∈ G(w), then V (π, w) = w 1−β and if (π, 1) ∈ G(w), then V (π, w) = w 1−β . This is true for w ≥ R 1 (π, 0). This suggests that the optimal action is not to play the arm for all time slots. The optimization problem in 2 reduces to optimal stopping problem, where arm is played until stopping time τ − 1 and not played since τ. Thus the expected discounted reward is This expected reward is upper bounded by w 1−β because not playing arm is always optimal for (π, y) ∈ G(w) as shown earlier. Hence We assume that w is lower bounded and Eqn. (13) holds true. Then, it is easy to verify that (π, y) ∈ G(w). To see this, make use of the optimal stopping time policy and Eqn. (14).

Theorem 3:
The arm is indexable and index w(π, y) is Here, τ is optimal stopping time, it is time until which arm is played.
Proof: Note that Eqn. (13) is true for every stopping time τ > 0. That implies not playing the arm is optimal. Further, the following is true.
In order to show indexability, we need to prove that G(w) set is monotone in w. From Lemma 6, we know that there is w for which set G(w) is empty. As w increases this set becomes non-empty. This is clear from Lemma 7. As subsidy w increases, Eqn. (13) continues to hold for larger subset of S = [0, 1] × {0, 1}. Thus indexability holds true by definition and index can be computed using (16).
VI. NUMERICAL RESULTS We first present few numerical examples to illustrate threshold type optimal policy for a restless single-armed bandit. We later demonstrate the performance of our index policy for rested multi-armed bandit.
In Fig. 1-a), we plot V S (π) and V N S (π) as function of π. Similarly, in Fig. 1-b), we plot value functions V S (π) and V N S (π). These plots suggest that the optimal policy is of a threshold type.
In this case, we have µ 1 − µ 0 = 0.8. But to prove analytically a threshold policy result, we have assumed 0 < µ 1 − µ 0 < 1/3, see Section IV-B. This is a limitation from analysis because it is very difficult to evaluate closed form expressions for value functions or introduce monotonicity of value functions.

B. Performance of index policy
We now present few numerical examples to illustrate the performance of index policy and compare this with that of myopic policy. This is done for rested single-armed bandit. Note that this is different from standard rested bandits because here arms are available probabilistically in each time slot. Recall that in an index policy, the arm with highest index is played in given time slot. In myopic policy, the arm with highest immediate expected reward is played at each time slot.
We We also set ρ 0 = r 0 , ρ 1 = r 1 , initial belief and availability vector of arms is We further have two sets of examples, in first set of examples we assume that the probability of availability is identical for all the arms, i.e., θ a n (π, y) = θ a (π, y). In second set of examples, each arm has different probability of availability.
1) Arms with identical probability of availability: Here, θ a n (π, y) = θ a (π, y). But we assumed different reward and transition probabilities. We consider four examples as given below.
In above first 3 examples we considered θ 1 (π, 1) > θ 1 (π, 0), see Table I-III. This implies that the probability that the arm is available in next slot given that it is not available and played in current time slot is smaller that the probability of availability in next slot given the arm is available and played. On the other hand we consider example of θ 1 (π, 1) < θ 1 (π, 0) in Table IV, which means playing an arm when it is not available leads to better chance of it being available in the next slot than playing when it is available. we observe similar performance to that of example 3.
For a rested single-armed bandit with availability constraints, we have shown that the arm is indexable and derived a formula for index. The index can also be calculated by the value iteration algorithm. From numerical examples, we observed that index policy performs better than myopic policy for some cases. This suggests that, index policy accounts for the future availability of arms and hence gives better performance. In future we seek to obtain some numerical scheme to compute the index for restless bandits with constrained arms.

APPENDIX
A. Proof of Lemma 1 1. In this part, We prove V (π) is convex by induction and use that to show other value functions are also convex. Let Now define clearly, V 1 (π) is linear and hence convex. If V n (π), V n (π) is convex in π then we can write From [18][Lemma 2], we can argue that V n+1,S (π) is convex in π. Similarly, we can show this for other value functions.
2. In this part, We can rewrite (17), in form of V n+1,S (π, w) and V n+1,N S (π, w) as function of w. We can see that V 1 (π, w) is monotone non decreasing and convex in w. V n+1,S (π, w) is constant plus a convex sum of four non decreasing convex function of w. V n+1,N S (π, w) is the sum of three non decreasing function of w. The convexity is preserved under max operation so V n+1 (π, w) is also non decreasing and convex in w and using induction, all V n (π, w) follows the same. As V n (π, w) → V (π, w) and this complete the proof for V (π). Similarly, we can show this for other value functions.

B. Proof of Lemma 2
The proof can be done via induction technique. The basic intuition behind ordering rewards, transition and observation probabilities on belief π is to get monotone decreasing value functions over π.

C. Proof of Lemma 3
From Lemma 2 V S (π) is strictly decreasing in π and V N S (π) is nonincreasing in π. Let f (π) = V S (π) − V N S (π) and f (π) is decreasing in π, i.e f (π) < f (π ′ ) for π > π ′ . This implies that we need to show Rearranging 18 we need to show Rested bandit: Right hand side of (19) is 0. We know V S (π) is decreasing, hence our claim follows. Restless bandit: When ρ 0 = 0, ρ 1 = 1 similar argument holds and claim follows. But in other cases, the claim holds under some restrictions on β and to prove this one required to use Lipschitz properties (8) of value functions.

D. Proof of Lemma 4
As before f (π) = V S (π) − V N S (π). In order to prove that f (π) is decreasing, we need to show that its partial derivative w.r.t. π is negative.